system

Abstract

We provide the system. [Solution] A means for inputting user information, A means of receiving the transmitted user information and storing it in a database, The means of selecting a field of study, A means for generating learning scenarios related to the selected field, A means of conducting interactive learning based on a learning scenario, A means of analyzing user responses and generating feedback, A system that includes means for providing generated feedback to the user.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional new employee education systems, knowledge supplementation and vocabulary correction are not sufficiently carried out, and efficient education is often difficult. Also, there is a problem that the burden on the education staff increases in order for new employees to acquire the necessary skills before going to the field. Furthermore, when educating a large number of new employees at once, there is a problem that it is difficult to provide individual feedback and flexible learning progress.

Means for Solving the Problems

[0005] The present invention provides a means for inputting user information and storing that information in a database. It also provides a means for generating a learning scenario related to the selected field, which includes a means for the user to select a field to learn. Based on this learning scenario, it provides a means for interactive learning and a means for analyzing the user's responses and generating feedback. It includes a means for providing the generated feedback to the user, a means for taking an exam, a means for analyzing the exam results and generating an evaluation, and a means for suggesting supplementary learning content based on the evaluation results. Furthermore, it provides a system that includes a means for analyzing the user's language use and providing feedback on correct word usage, and a means for presenting the next learning step according to the learning progress. This enables efficient and individualized feedback, reduces the burden on trainers, and realizes effective training of new employees.

[0006] "User information" refers to personal information such as the user's name, email address, department, and areas of interest.

[0007] A "database" refers to a data storage system used to systematically store and manage various types of data, such as user information, learning progress, and feedback.

[0008] A "learning scenario" refers to a series of educational programs or interactive question-and-answer flows that specifically demonstrate learning content related to the field selected by the user.

[0009] "Interactive learning" refers to a learning format in which the user and system interact with each other in turn, providing appropriate feedback according to the user's level of understanding and responses.

[0010] "Analysis" refers to the process of evaluating user responses and language usage using natural language processing techniques and other analytical methods to generate appropriate feedback.

[0011] "Feedback" refers to information including evaluations, advice, and corrections provided based on the user's responses and learning progress.

[0012] "Exam" refers to a test that includes questions and assignments designed to assess the user's understanding of the learning material.

[0013] "Evaluation" refers to the results of quantitatively or qualitatively analyzing test results and learning progress to indicate the user's level of mastery.

[0014] "Supplementary learning" refers to additional learning content or practice tasks proposed based on the evaluation results.

[0015] "Language usage" refers to the appropriateness and accuracy of the specific words and expressions used by the user.

[0016] A "learning step" refers to the specific learning content or activity that the system provides next, depending on the user's learning progress. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]Shows an emotion map to which a plurality of emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Modes for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention provides a system for efficiently supplementing knowledge and correcting appropriate language use in the training of new crew members. The system consistently handles everything from user information input to the selection of training programs, interactive learning, testing, evaluation, and suggestions for supplementary learning.

[0039] User registration and information entry

[0040] Upon initial access, the user enters necessary information such as their name, email address, department, and desired field of study. The terminal checks this information and sends it to the server. The server stores the received user information in its database and displays a registration completion message to the user. A unique user ID is generated during this process.

[0041] Selection of an educational program

[0042] The terminal prompts the user to select a subject they wish to learn. If the user selects, for example, "Business Etiquette," this selection is sent to the server. The server searches its database for relevant educational programs and returns a list to the terminal. The terminal then presents the user with multiple educational program options.

[0043] Implementation of interactive learning

[0044] When a user selects a specific educational program, the device sends that information to the server. The server generates a learning scenario based on the selected educational program and sends it to the device. Based on the learning scenario, the device presents the user with interactive questions and scenarios. This allows the user to deepen their knowledge through interactive learning.

[0045] Feedback and corrections

[0046] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the answer and generates appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The device then presents the generated feedback to the user and proceeds to the next question or scenario.

[0047] Tests and evaluations

[0048] Once the user has reached a certain stage of learning, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates the test results and evaluation. This includes suggestions for supplementary learning if needed. The device then displays the test results and evaluation to the user.

[0049] Supplementary learning and the next steps

[0050] The server has a means of suggesting supplementary learning content based on the evaluation results. For example, if supplementary learning on specific vocabulary is needed, the system will automatically present that content. The terminal then presents the user with the next learning step and supplementary learning content, realizing a consistent educational cycle.

[0051] Specific example

[0052] For example, when a user learns how to use polite language, the following specific actions are taken:

[0053] 1. User: "I would like to learn how to use polite language correctly."

[0054] 2. Terminal: Requests the relevant basic scenario from the server.

[0055] 3. Server: Generates the scenario and sends it to the terminal.

[0056] 4. Terminal: Displays the question, "How would you greet your boss for the first time?"

[0057] 5. User: "Thank you in advance."

[0058] 6. Terminal: Sends the response to the server.

[0059] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[0060] 8. Device: Show this feedback to the user.

[0061] As described above, this system supports user learning in a consistent step-by-step manner, providing effective education while reducing the burden on the field staff.

[0062] The following describes the processing flow.

[0063] Step 1:

[0064] When a user first accesses the system, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this user input to ensure there are no errors.

[0065] Step 2:

[0066] The terminal converts the verified user information into JSON format and sends it to the server. The server stores the received user information in a database and generates a unique user ID.

[0067] Step 3:

[0068] The server sends a registration confirmation email to the user's email address. The terminal displays a registration completion message to the user.

[0069] Step 4:

[0070] The device prompts the user to select a field of study, asking, "Please choose the field you would like to learn about."

[0071] Step 5:

[0072] The user selects the subject they want to learn about (e.g., "Business Etiquette"). The device converts the selection into JSON format and sends it to the server.

[0073] Step 6:

[0074] The server searches the database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal presents the program list to the user, allowing them to choose from multiple options.

[0075] Step 7:

[0076] The user selects a specific educational program. The device then sends this selection information to the server.

[0077] Step 8:

[0078] The server generates a learning scenario related to the selected educational program and sends it to the terminal in JSON format. The terminal then presents the user with interactive learning based on the learning scenario.

[0079] Step 9:

[0080] When a user answers a scenario-based question, the device sends that answer to the server.

[0081] Step 10:

[0082] The server analyzes the received responses using natural language processing techniques and generates appropriate feedback. This feedback includes whether the response is correct or incorrect, as well as suggestions for improving the wording.

[0083] Step 11:

[0084] The device presents the generated feedback to the user and offers further questions or scenarios.

[0085] Step 12:

[0086] Once the user has completed a certain amount of learning, the device will notify them that an exam is about to be held.

[0087] Step 13:

[0088] The user starts the test and answers the test questions. The device sends the answers to the server.

[0089] Step 14:

[0090] The server analyzes the test results and generates scores and evaluations. The evaluation results include suggestions for supplementary learning.

[0091] Step 15:

[0092] The terminal displays the generated test results and evaluations to the user.

[0093] Step 16:

[0094] The server records the evaluation results and the user's learning progress in a database and suggests the next learning steps and supplementary learning content.

[0095] Step 17:

[0096] The device presents the user with the next learning step and supplementary learning content, encouraging the user to continue learning.

[0097] (Example 1)

[0098] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In training new crew members, there is a need for a system that efficiently supplements their knowledge and corrects their language use. However, traditional training systems have limited functionality and have difficulty consistently providing feedback on user progress and language use. Furthermore, the training process is fragmented across multiple methods, resulting in a lack of efficiency.

[0100] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0101] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for using a generative AI model to generate a learning scenario related to the selected field, means for obtaining the user's learning request using prompt statements, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, and means for providing the generated feedback to the user. This makes it possible to support the user's learning in a consistent step-by-step manner and provide effective education while reducing the burden on the field staff.

[0102] "User information" refers to personal identifying information, including the user's name, email address, department, and areas of study they wish to pursue.

[0103] A "database" is a system for efficiently storing, managing, and retrieving data, and examples include MySQL (registered trademark).

[0104] A "learning scenario" is a sequence of specific educational content and question formats related to the selected learning area.

[0105] A "generative AI model" is a model that uses generative artificial intelligence techniques to generate text and scenarios, and includes, for example, GPT-3 (registered trademark).

[0106] A "prompt message" is a text input used to generate a specific output when inputted into a generative AI model.

[0107] "Interactive learning" is a learning format that progresses through dialogue between the user and the system.

[0108] "Feedback" refers to evaluations and suggestions for improvement provided in response to user responses.

[0109] A "test" is a set of questions designed to assess the level of understanding of the learned material, and includes questions that the user answers.

[0110] "Evaluation" refers to information about the user's learning progress and level of understanding, generated by analyzing test results.

[0111] "Supplementary learning" refers to additional learning content proposed based on the evaluation results.

[0112] This invention provides a system for efficiently supplementing knowledge and correcting appropriate language use in the training of new crew members. The system consistently performs tasks from user information input to the selection of training programs, interactive learning, testing, evaluation, and suggestions for supplementary learning. Each step utilizes a generative AI model and prompt sentences.

[0113] User registration and information entry

[0114] When a user first accesses the system, they enter information such as their name, email address, department, and desired field of study into the terminal. The terminal verifies the format and required fields of this information to ensure its accuracy. After verification, the terminal sends the information to the server. The server stores the received information in a database (e.g., a MySQL database) and generates a unique user ID. The server then generates a registration completion message and sends it to the terminal, which displays the message to the user.

[0115] Selection of an educational program

[0116] The terminal displays a menu prompting the user to select a subject they wish to learn. When the user selects an option such as "Business Etiquette," the terminal sends that selection to the server. The server searches its database for relevant educational programs and sends a list of found programs to the terminal. The terminal then presents this list to the user, prompting them to select a specific program.

[0117] Implementation of interactive learning

[0118] When a user selects a specific educational program, the device sends that information to the server. The server generates a learning scenario using a generative AI model (e.g., GPT-3) and sends it to the device. The device then presents the user with interactive questions and scenarios based on the learning scenario. This allows the user to deepen their knowledge through interactive learning.

[0119] Feedback and corrections

[0120] When a user answers a question in a learning scenario, the device sends the answer to the server. The server uses a generative AI model to analyze the answer and generate appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The server sends the generated feedback to the device, which then displays it to the user. The device then presents the user with the next question or scenario.

[0121] Tests and evaluations

[0122] Once the learning process reaches a certain stage, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server uses a generative AI model to analyze the answers and generate test results and an evaluation. The evaluation also includes suggestions for areas where supplementary learning is needed. The server sends the test results and evaluation to the device, which then displays them to the user.

[0123] Supplementary learning and the next steps

[0124] The server automatically generates supplementary learning content based on the evaluation results. It generates prompt messages based on the evaluation results and sends them to the terminal. The terminal presents the user with the next learning step and supplementary learning content, and the user proceeds with the learning accordingly. This ensures a consistent educational cycle.

[0125] Specific example: Learning how to use honorific language

[0126] For example, when a user is learning how to use polite language, the following prompt might be used:

[0127] 1. User: "I would like to learn how to use polite language correctly."

[0128] 2. Terminal: Requests relevant basic scenarios from the server.

[0129] 3. Server: Generates scenarios using the generated AI model and sends them to the terminal.

[0130] 4. Terminal: Displays the question, "How would you greet your boss for the first time?"

[0131] 5. User: Responds with "Thank you in advance."

[0132] 6. Terminal: Send the answer to the server.

[0133] 7. Server: The AI ​​model analyzes the response and generates feedback such as, "That's correct, but it would be better if you added a more polite 'Thank you for your assistance. I look forward to working with you.'"

[0134] 8. Device: Display feedback to the user.

[0135] In this way, this system can support user learning through a consistent process, providing effective education while reducing the burden on staff.

[0136] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0137] Step 1: Enter user information

[0138] Upon initial access, the user enters information such as their name, email address, department, and desired field of study into the terminal. The terminal verifies the format and required fields of this information to ensure its accuracy (input: user information, output: verified user information). The terminal then sends the verified information to the server (operation: data verification and transmission).

[0139] Step 2: Saving User Information

[0140] The server saves the received user information to a database (e.g., MySQL) and generates a unique user ID (input: verified user information, output: user information and unique user ID stored in the database). The server then generates a registration completion message and sends it to the terminal (action: database save, user ID generation, message generation).

[0141] Step 3: Display of registration completion message

[0142] The terminal displays a registration completion message to the user (Input: Registration completion message, Output: Message displayed to the user) (Action: Message display).

[0143] Step 4: Selecting an Educational Program

[0144] The terminal displays a menu prompting the user to select a subject they wish to study (Input: None, Output: Displayed menu). The user selects an option such as "Business Manners" and inputs it into the terminal (Input: User selection, Output: Selected subject). The terminal sends the selection to the server (Action: Retrieve and send user selection).

[0145] Step 5: Searching for and presenting educational programs

[0146] The server searches the database for relevant educational programs and sends a list of found programs to the terminal (Input: User-selected field, Output: Program list). The terminal then presents the user with a selection of educational programs (Action: Database search, Program list transmission).

[0147] Step 6: Select and submit your educational program.

[0148] The user selects a specific educational program and enters the information into the terminal (input: selected educational program, output: information about the selected program). The terminal then sends this information to the server (action: educational program selection, information transmission).

[0149] Step 7: Generating the learning scenario

[0150] The server uses a generation AI model (e.g., GPT-3) to generate a learning scenario based on the transmitted program information (input: selected program information, output: generated learning scenario). The server then sends the generated learning scenario to the terminal (operation: scenario generation and transmission by the AI ​​model).

[0151] Step 8: Start interactive learning

[0152] The device presents the user with interactive questions and scenarios based on the learning scenario (input: generated learning scenario, output: presented interactive questions). The user answers the presented questions (action: scenario output and question presentation).

[0153] Step 9: Submit User Response

[0154] The user answers questions in the learning scenario and inputs the answers into the device (input: user answers, output: user answer data). The device then sends the answers to the server (action: retrieval and transmission of user answers).

[0155] Step 10: Analyze responses and generate feedback

[0156] The server uses a generative AI model to analyze user responses and generate appropriate feedback (input: user response data, output: generated feedback). This feedback includes not only whether the response is correct or incorrect, but also suggestions for improving the wording (operation: AI model analyzes responses and generates feedback).

[0157] Step 11: Displaying Feedback

[0158] The terminal displays the generated feedback to the user (Input: Generated feedback, Output: Feedback displayed to the user) (Action: Display feedback). The terminal then presents the user with further questions or scenarios (Action: Present next scenario).

[0159] Step 12: Notification and administration of the exam

[0160] Once the user has reached a certain stage of learning, the terminal notifies the user that an exam will be held (Input: Learning progress information, Output: Exam notification). When the user starts the exam, the terminal displays the exam questions and sends the user's answers to the server (Input: User answers, Output: Submission of exam answers) (Action: Display of exam questions and submission of answers).

[0161] Step 13: Analysis and generation of test results

[0162] The server uses a generative AI model to analyze test responses and generate test results and evaluations (input: test response data, output: test results and evaluations). This also includes suggestions for when supplementary learning is needed (operation: AI model analyzes test results and generates evaluations).

[0163] Step 14: Display of evaluation and suggestion of supplementary learning

[0164] The server sends the test results and evaluation to the terminal (input: generated evaluation, output: evaluation displayed to the user). The terminal displays the evaluation to the user and suggests supplementary learning content (action: sending and displaying evaluation, suggesting supplementary learning).

[0165] Step 15: Conduct supplementary learning

[0166] The terminal presents the user with the next learning step and supplementary learning content, and the user proceeds with the learning accordingly (Input: Supplementary learning content, Output: Presented supplementary learning) (Operation: Display of the next step and start of learning). The server records the progress of supplementary learning in a database, thereby ensuring a consistent educational cycle (Operation: Recording of supplementary learning progress).

[0167] (Application Example 1)

[0168] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0169] Traditional training systems face the challenge of effectively educating new crew members and customer support staff on improving their communication skills and language use. Furthermore, providing real-time feedback and supplementary learning is difficult, resulting in low learning effectiveness. This invention aims to solve these problems, streamline the staff training process, and rapidly improve communication skills.

[0170] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0171] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for performing interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, means for inputting prompt sentences into a generating AI model and generating interactive learning scenarios and feedback, and means for suggesting scenarios to improve customer support response skills. This enables customer support staff to effectively improve their response skills.

[0172] "Means for entering user information" refers to the means by which users enter necessary information such as their name, email address, department, and the field they wish to study.

[0173] "Means for receiving transmitted user information and saving it to a database" refers to the means by which information entered by a user is sent to a server, received, and saved to a database.

[0174] "Means for selecting a field of study" refers to the means by which users can choose the field of study they wish to pursue.

[0175] "Means for generating learning scenarios related to selected fields" refers to means for creating learning content and scenarios based on the fields selected by the user.

[0176] "Means for conducting interactive learning based on learning scenarios" refers to means for advancing learning in an interactive format with the user based on a generated learning scenario.

[0177] "Means for analyzing user responses and generating feedback" refers to methods for analyzing responses provided by users during interactive learning and creating feedback such as correctness, errors, and areas for improvement.

[0178] "Means of providing generated feedback to the user" refers to means of displaying the analyzed feedback to the user.

[0179] "A means of inputting prompt sentences into a generative AI model to generate interactive learning scenarios and feedback" refers to a means of using a generative AI model to input prompt sentences and automatically create learning scenarios and feedback.

[0180] "Methods for proposing scenarios to improve customer support response skills" refers to methods for proposing scenarios to improve the response skills necessary for customer support staff.

[0181] "Means for conducting tests on customer support response skills and providing evaluation and supplementary learning" refers to means for conducting tests to measure the response skills of customer support staff and providing evaluation and supplementary learning based on the results.

[0182] "A means of providing real-time feedback and supplementary learning on response skills using a generative AI model" refers to a means of providing users with immediate feedback and supplementary learning on response skills using a generative AI model.

[0183] This invention is a system aimed at the efficient training of customer support staff. It is designed to enable new staff to quickly acquire appropriate language and communication skills. The system consists of the following main components:

[0184] Entering user information

[0185] Upon initial access, users enter necessary information such as their name, email address, department, and desired learning area into their device (smartphone or head-mounted display). The device sends this information to the server, which then stores the received user information in a database.

[0186] Selection of an educational program

[0187] The terminal prompts the user to select a field of study they wish to pursue. If the user selects, for example, "customer service skills," this selection is sent to the server. The server then searches its database for relevant educational programs and returns a list to the terminal. The terminal then presents the user with multiple educational program options.

[0188] Implementation of interactive learning

[0189] When a user selects a specific educational program, the device sends that information to the server. The server uses a generative AI model (e.g., GPT-4®) to generate a learning scenario based on the selected educational program and sends it to the device. Based on the learning scenario, the device presents the user with interactive questions and scenarios. Through interactive learning, the user can deepen their communication skills.

[0190] Feedback and corrections

[0191] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the answer and uses a generative AI model to generate appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The device then presents the generated feedback to the user.

[0192] Tests and evaluations

[0193] Once the user has reached a certain stage of learning, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates the test results and evaluation. This includes suggestions for supplementary learning if needed. The device then displays the test results and evaluation to the user.

[0194] Supplementary learning and the next steps

[0195] The server has a mechanism to suggest supplementary learning content based on the evaluation results. For example, if supplementary learning on customer service skills is needed, the content will be automatically presented. The terminal presents the user with the next learning step and supplementary learning content, realizing a consistent educational cycle.

[0196] Specific examples of components and data processing

[0197] This system uses smartphones and head-mounted displays as hardware, with React Native as the front-end software and Flask (Python) and SQLite as the back-end. It also includes a program that uses a generative AI model to analyze prompt messages and generate feedback and scenarios.

[0198] As a concrete example, if a user wants to learn how to use polite language, they would input the following prompt sentence into the AI ​​model:

[0199] "When a customer asks, 'Please tell me more about this product,' generate an appropriate response."

[0200] By providing real-time feedback and supplementary learning using generative AI models, users can efficiently and effectively learn the skills they need in the field.

[0201] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0202] Step 1:

[0203] Entering user information

[0204] Input: The user enters their name, email address, department, and the field they wish to study into the terminal.

[0205] Processing: The terminal transfers the entered user information to the server for transmission to the database.

[0206] Output: The server saves the received user information to the database and generates a unique user ID.

[0207] Step 2:

[0208] Selection of an educational program

[0209] Input: The user selects the subject they want to study on their device.

[0210] Processing: The terminal sends information about the selected field to the server. The server searches the database for relevant educational programs and sends the list back to the terminal.

[0211] Output: The terminal presents the user with multiple educational program options.

[0212] Step 3:

[0213] Scenario generation for interactive learning

[0214] Input: The user selects a specific educational program.

[0215] Processing: The terminal sends the selection information to the server. The server inputs prompt messages into the generating AI model and generates a learning scenario based on the selected educational program.

[0216] Output: The server sends the generated training scenario to the terminal, and the terminal displays it.

[0217] Step 4:

[0218] Implementation of interactive learning

[0219] Input: The user answers questions or scenarios in an interactive format.

[0220] Processing: The terminal sends the user's response to the server. The server analyzes the response using a generative AI model.

[0221] Output: Generates feedback based on the analysis results and sends it to the terminal. The terminal displays the feedback to the user.

[0222] Step 5:

[0223] Implementation of tests and evaluations

[0224] Input: Users who have made a certain level of progress in their learning will take the test.

[0225] Processing: The terminal displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates evaluation results.

[0226] Output: The evaluation results and necessary supplementary learning content are sent to the terminal, which then displays them to the user.

[0227] Step 6:

[0228] Supplementary learning and suggested next steps

[0229] Input: Test results and evaluation results.

[0230] Processing: Based on the evaluation results, the server automatically suggests the necessary supplementary learning content.

[0231] Output: The terminal presents the user with the next learning step and supplementary learning content.

[0232] As a concrete example, for a user learning "how to use polite language," the prompt sentence entered into the generative AI model would be as follows:

[0233] "When a customer asks, 'Please tell me more about this product,' generate an appropriate response."

[0234] This allows the system to effectively improve users' customer service skills.

[0235] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0236] This invention is a system that efficiently provides knowledge supplementation and corrects appropriate language use in the training of new crew members. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more precise feedback and enables effective learning.

[0237] User registration and information entry

[0238] When a user accesses the system for the first time, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this information to ensure it is complete and accurate. The verified user information is converted to JSON format and sent to the server. The server receives this information, stores it in its database, and generates a unique user ID.

[0239] Selection of an educational program

[0240] The terminal asks the user, "Please select the field you would like to learn about." When the user selects a learning field such as "Business Etiquette," that information is converted to JSON format and sent to the server. The server searches its database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal then presents the list of educational programs to the user, allowing them to choose from multiple options.

[0241] Implementation of interactive learning

[0242] When a user selects a specific educational program, that selection information is sent from the device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario.

[0243] Feedback and corrections

[0244] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving wording. The device then presents the generated feedback to the user and offers the next question or scenario.

[0245] Introducing emotion recognition

[0246] While the user is learning, the emotion engine acquires emotional information. The device sends the user's facial expressions, voice tone, etc., to the emotion engine for emotional analysis. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[0247] Tests and evaluations

[0248] Once learning reaches a certain stage, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with the user's sentiment information to the server. The server receives this information, analyzes the test results, and generates a score and evaluation. The evaluation results include suggestions for supplementary learning.

[0249] Supplementary learning and the next steps

[0250] The server suggests supplementary learning content based on the evaluation results. For example, it may provide supplementary learning on specific word usage or learning scenarios adjusted according to the user's stress level. The terminal then presents the user with the next learning step and supplementary learning content, supporting continuous learning.

[0251] Specific example

[0252] For example, when a user learns how to use polite language, the following specific actions are taken:

[0253] 1. User: "I would like to learn how to use polite language correctly."

[0254] 2. Terminal: "Request a basic scenario regarding the use of honorific language from the server."

[0255] 3. Server: Generates a learning scenario and sends it to the terminal.

[0256] 4. Terminal: The user is shown the message, "How would you greet your boss for the first time?"

[0257] 5. User: "Thank you in advance."

[0258] 6. Terminal: Send the response to the server.

[0259] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[0260] 8. Device: Display feedback to the user.

[0261] Furthermore, the emotion engine recognizes the user's emotions during the learning process. For example, if it determines that the user is "feeling stressed," it displays feedback and encouraging messages to help them relax.

[0262] This system supports user learning in a consistent step-by-step manner, provides individualized feedback, and reduces the burden on staff. Furthermore, by understanding the user's state in real time through emotion recognition and dynamically adjusting the learning content, more effective education becomes possible.

[0263] The following describes the processing flow.

[0264] Step 1:

[0265] When a user first accesses the system, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this information to ensure there are no errors. The verified user information is then converted to JSON format and sent to the server.

[0266] Step 2:

[0267] The server receives user information and stores it in the database. It generates a unique user ID and sends a registration confirmation email to the user. The terminal displays a registration completion message to the user.

[0268] Step 3:

[0269] The terminal asks the user, "Please select the field you would like to learn about." The user selects a learning field, such as "Business Etiquette." The terminal converts the selection into JSON format and sends it to the server.

[0270] Step 4:

[0271] The server searches the database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal presents the program list to the user, allowing them to choose from multiple options.

[0272] Step 5:

[0273] The user selects a specific educational program and sends this selection information from their device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format.

[0274] Step 6:

[0275] The device presents the user with interactive learning based on a learning scenario. It displays questions such as, "Which is the correct expression to use when thanking your boss?"

[0276] Step 7:

[0277] When a user answers a scenario-based question, the device sends the answer to the server. The server analyzes the answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving the wording.

[0278] Step 8:

[0279] The device presents the generated feedback to the user and offers further questions or scenarios. The feedback includes specific examples such as, "That's correct, but it would be better if you added a more polite phrase like, 'Thank you for your assistance. I appreciate your help.'"

[0280] Step 9:

[0281] During training, the emotion engine acquires data to analyze the user's facial expressions, voice tone, and other information. The device then sends this data to the emotion engine.

[0282] Step 10:

[0283] The emotion engine analyzes the user's emotions and generates emotion recognition results such as "the user is feeling stressed." The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[0284] Step 11:

[0285] When the user is feeling stressed, for example, the server generates feedback and encouragement messages that are more relaxing and sends them to the terminal. The terminal presents this to the user.

[0286] Step 12:

[0287] When the user has made some progress in learning, the terminal notifies the user to take a test. The user starts the test and answers the test questions. The terminal sends the answers to the server.

[0288] Step 13:

[0289] The server analyzes the test results and generates scores and evaluations. The evaluations also reflect the user's emotional data. For example, when the stress level is high, an evaluation that takes this into account is carried out.

[0290] Step 14:

[0291] The terminal displays the test results and evaluations to the user. The evaluation content includes suggestions for supplementary learning.

[0292] Step 15:

[0293] The server records the evaluation results and the user's learning progress in the database and proposes the next learning steps and the content of supplementary learning. For example, it includes providing relaxation content to reduce stress.

[0294] Step 16:

[0295] The terminal presents the next learning steps and the content of supplementary learning to the user and continues the user's learning. In this way, a consistent learning experience is provided to support the growth of the user.

[0296] (Example 2)

[0297] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0298] Traditional education systems struggle to efficiently assess users' learning progress and comprehension, and to provide appropriate feedback. Furthermore, they have difficulty understanding users' emotional states in real time and dynamically adjusting learning scenarios and feedback accordingly. There is also a need to provide individualized feedback on proper language use and stress management to support consistent learning.

[0299] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0300] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, and means for recognizing the user's emotions and reflecting the analysis results in the feedback. This makes it possible to efficiently evaluate the user's learning progress and understanding and provide appropriate feedback. Furthermore, the user's emotional state can be grasped in real time, and the learning scenario and feedback can be dynamically adjusted. In addition, individual feedback on correct language use and stress management can be provided to support consistent learning.

[0301] "User information" refers to a series of identifying information about an individual user, such as name, email address, department, and area of ​​study.

[0302] A "database" is a storage device used to centrally manage data such as user information, learning progress, and feedback.

[0303] A "learning scenario" is a set of specific educational content and questions generated based on the learning field selected by the user.

[0304] "Interactive learning" is a learning format in which the user and the system interactively communicate with each other while progressing.

[0305] "Feedback" refers to the evaluation and advice provided for the user's answers and actions.

[0306] An "emotion engine" is a system for analyzing emotional data such as the user's facial expressions and voice tones.

[0307] "Stress level" is an indicator showing the user's mental burden and state of tension.

[0308] "Supplementary learning" is learning content provided additionally according to the user's understanding level and progress.

[0309] "Natural language processing technology" is an artificial intelligence technology for analyzing human language and understanding its meaning.

[0310] A "test" is a series of questions and tasks for evaluating whether the user understands specific learning content.

[0311] "Evaluation" is the result representing the user's performance numerically or with comments based on tests and learning progress.

[0312] An "encouraging message" is a motivating word provided by the system to enhance the user's motivation during learning.

[0313] The present invention is a system that efficiently fills in knowledge and corrects appropriate word usage in the education of new crew members, and further provides more refined feedback and realizes effective learning by combining an emotion engine for recognizing the user's emotions.

[0314] User registration and information entry

[0315] When a user first accesses the system, they enter information such as their name, email address, department, and area of ​​study. The terminal checks this information for any errors. If there are any errors, the terminal notifies the user. The verified user information is converted to JSON format and sent to the server. The server receives this and stores it in a database to generate a unique user ID. This process uses general-purpose database technologies such as SQL or NoSQL databases.

[0316] Selection of an educational program

[0317] The device displays a prompt to the user asking, "Please select the field you would like to learn about." When the user selects a learning field, such as "Business Etiquette," the device converts that information into JSON format and sends it to the server. The server searches its database for relevant educational programs, generates a list, and sends it back to the device. The device then presents the user with the list of educational programs and allows them to choose from multiple options.

[0318] Implementation of interactive learning

[0319] When a user selects a specific educational program, that selection information is sent from the device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario.

[0320] Feedback and corrections

[0321] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving wording. The device then presents the generated feedback to the user and offers the next question or scenario. Natural language processing is performed using generative AI models and machine learning algorithms.

[0322] Introducing emotion recognition

[0323] While the user is learning, the emotion engine acquires emotional information. The device sends the user's facial expressions, voice tone, etc., to the emotion engine for emotional analysis. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback. The emotion engine uses facial recognition software and voice analysis tools.

[0324] Tests and evaluations

[0325] Once learning reaches a certain stage, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with the user's sentiment information to the server. The server receives this information, analyzes the test results, and generates a score and evaluation. The evaluation results include suggestions for supplementary learning.

[0326] Supplementary learning and the next steps

[0327] The server suggests supplementary learning content based on the evaluation results. For example, it may provide supplementary learning on specific word usage or learning scenarios adjusted according to the user's stress level. The terminal then presents the user with the next learning step and supplementary learning content, supporting continuous learning.

[0328] Specific example

[0329] For example, when a user learns how to use polite language, the following specific actions are taken:

[0330] 1. User: "I would like to learn how to use polite language correctly."

[0331] 2. Terminal: "Request a basic scenario regarding the use of honorific language from the server."

[0332] 3. Server: Generates a learning scenario and sends it to the terminal.

[0333] 4. Terminal: The user is shown the message, "How would you greet your boss for the first time?"

[0334] 5. User: "Thank you in advance."

[0335] 6. Terminal: Send the response to the server.

[0336] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[0337] 8. Device: Display feedback to the user.

[0338] Through this process, users can learn effectively while receiving real-time feedback. Furthermore, the emotion engine understands the user's emotional state and provides appropriate feedback, resulting in a more consistent learning experience.

[0339] Example of a prompt

[0340] "Please generate a learning scenario for someone who wants to learn how to use honorific language correctly."

[0341] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0342] Step 1: User registration and information entry

[0343] The user accesses the system and enters information such as their name, email address, department, and the field they wish to study.

[0344] The terminal checks the entered information to ensure there are no errors.

[0345] Input: User's name, email address, department, and information on the field of study.

[0346] Output: User information with no errors (JSON format)

[0347] Specific operation: The terminal receives the information, converts it to JSON format, and sends it to the server.

[0348] Step 2: Saving User Information

[0349] The server stores the received user information in JSON format in a database and generates a unique user ID.

[0350] Input: User information in JSON format

[0351] Output: Unique User ID

[0352] Specific operation: The server writes the received data to the database, generates a user ID, and sends it back to the terminal.

[0353] Step 3: Choosing a field of study

[0354] The device prompts the user to "Please select the field you would like to study."

[0355] The user selects a learning area, such as "business etiquette."

[0356] Input: User-selected learning field

[0357] Output: Learning subject information in JSON format

[0358] Specific operation: The terminal receives the user's selection, converts it to JSON format, and sends it to the server.

[0359] Step 4: Search for educational programs

[0360] The server searches the database for educational programs related to the learning area selected by the user.

[0361] Input: Learning subject information in JSON format

[0362] Output: List of related educational programs (in JSON format)

[0363] Specific operation: The server executes a database query, generates a list of relevant educational programs, and sends it back to the terminal.

[0364] Step 5: View and select educational programs

[0365] The device presents the user with a list of educational programs and allows them to choose from multiple options.

[0366] The user selects a specific educational program.

[0367] Input: List of educational programs, user selection

[0368] Output: Information on the selected educational program (in JSON format)

[0369] Specific operation: The device displays a list of educational programs and accepts the user's selection.

[0370] Step 6: Generating the learning scenario

[0371] The server generates learning scenarios related to the selected educational program.

[0372] Input: Selected educational program information in JSON format

[0373] Output: Training scenario (JSON format)

[0374] Specific operation: The server generates a learning scenario based on educational program information and sends it to the terminal.

[0375] Step 7: Provide interactive learning

[0376] The device presents the user with interactive learning based on a learning scenario.

[0377] The user answers questions within the learning scenario.

[0378] Input: Learning scenario, user responses

[0379] Output: User response (JSON format)

[0380] Specific operation: The device displays the learning scenario, receives the user's responses, and sends them to the server.

[0381] Step 8: Analyze responses and generate feedback

[0382] The server analyzes the received user responses using natural language processing technology and generates appropriate feedback.

[0383] Input: User's response (JSON format)

[0384] Output: Feedback (JSON format)

[0385] Specific operation: The server analyzes the response, generates feedback including correct answers and areas for improvement, and sends it to the terminal.

[0386] Step 9: Provide feedback

[0387] The device presents the generated feedback to the user and offers further questions or scenarios.

[0388] Input: Feedback (JSON format)

[0389] Output: Next learning scenarios and questions

[0390] Specific action: The device displays feedback and presents the following question.

[0391] Step 10: Acquisition and analysis of emotional information

[0392] The emotion engine analyzes the user's facial expressions, voice tone, and other factors to acquire emotional information.

[0393] Input: User facial expression data, voice data

[0394] Output: Emotion information (JSON format)

[0395] Specific operation: The emotion engine analyzes the data and recognizes emotions.

[0396] Step 11: Using emotional information

[0397] The server receives analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[0398] Input: Sentiment information (JSON format)

[0399] Output: Adjusted learning scenarios and feedback

[0400] Specific operation: The server analyzes emotional information and adjusts the learning content and feedback accordingly.

[0401] Step 12: Notification of Exam Implementation

[0402] The device will notify the user of the exam after they have completed the specified learning content.

[0403] Input: Learning progress information

[0404] Output: Notification of test implementation

[0405] Specific operation: The device detects the completion of the learning process and notifies the user to start the test.

[0406] Step 13: Conduct the test and collect responses.

[0407] The user answers the test questions.

[0408] The device converts the response into JSON format and sends it to the server along with sentiment information.

[0409] Input: Exam questions, user responses, sentiment information

[0410] Output: Responses and sentiment information (JSON format)

[0411] Specific operation: The device displays the test questions, collects the answers and sentiment information, and sends it to the server.

[0412] Step 14: Analysis and generation of test results

[0413] The server analyzes the received test results and generates scores and evaluations.

[0414] Input: Responses and sentiment information (JSON format)

[0415] Output: Exam score and evaluation (JSON format)

[0416] Specific operation: The server analyzes the test results and calculates a score and evaluation.

[0417] Step 15: Presentation of evaluation results and suggestion of supplementary learning

[0418] The device displays the evaluation results to the user and suggests supplementary learning content.

[0419] Input: Exam score and evaluation (JSON format)

[0420] Output: Proposed supplementary learning

[0421] Specific operation: The device displays the evaluation results and suggests supplementary learning.

[0422] Step 16: Implementing and managing supplementary learning

[0423] The server suggests supplementary learning content based on the evaluation results and records the progress in a database.

[0424] Input: Proposals and implementation status of supplementary learning

[0425] Output: Updated learning progress information (JSON format)

[0426] Specific operation: The server manages the content of supplementary learning and records the progress in a database.

[0427] (Application Example 2)

[0428] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0429] Traditional new crew training systems were limited to knowledge supplementation and language correction, and were unable to dynamically adjust learning scenarios to take into account users' emotions and stress levels. They also lacked methods for providing real-time feedback and appropriate advice based on stress levels and concentration during work. As a result, the training effectiveness was limited, and it did not adequately contribute to improving users' learning efficiency or work quality.

[0430] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0431] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, means for acquiring and analyzing emotional information through analysis of the user's facial expressions and voice, means for adjusting and providing the learning scenario and feedback based on the analyzed emotional information, means for the user to take a test, means for analyzing the test results and generating an evaluation, means for suggesting supplementary learning content based on the evaluation results, means for recording the progress of supplementary learning in a database, means for analyzing the user's stress and concentration level during work using an emotion engine and providing advice as needed, means for analyzing the user's language use and providing feedback on correct word usage, means for presenting the next learning step according to the learning progress, and means for acquiring the user's visual information, analyzing it with an emotion engine, and dynamically adjusting the learning content. This makes it possible to provide more effective feedback and learning scenarios while taking into account the user's emotions and stress level.

[0432] "User information" refers to basic personal information such as the user's name, contact information, department, and field of study.

[0433] A "database" is a system for systematically storing and managing data such as user information, learning scenarios, evaluation results, and supplementary learning progress.

[0434] A "learning scenario" is a specific scenario or content that guides the user through their learning process, based on the selected learning area.

[0435] "Interactive learning" is a learning method in which the user and the system interact, and the system provides appropriate feedback to the user's responses.

[0436] "Feedback" refers to the system's response and advice, such as evaluations and instructions for correction, to a user's answers and actions.

[0437] "Emotional information" refers to data that indicates the emotional state of a user, obtained from their facial expressions, tone of voice, and other similar information.

[0438] An "emotion engine" is an analytical engine that analyzes user emotional information and reflects the results in learning scenarios and feedback.

[0439] "Test results" refer to data obtained by analyzing the responses users gave when taking a test, and form the basis for generating evaluations.

[0440] "Supplemental learning" refers to additional learning content suggested based on the user's evaluation results to encourage further learning.

[0441] "Stress and concentration" refers to the mental and emotional state of a user while they are learning or working.

[0442] "Visual information" refers to information acquired through cameras and sensors, such as the user's facial expressions and movements.

[0443] This invention is a system for efficiently training new crew members, providing feedback and learning scenarios tailored to each user's individual circumstances. Specific embodiments for carrying out this invention are described below.

[0444] User registration and information entry

[0445] When a user accesses the system for the first time, they enter basic information such as their name, email address, department, and the field they wish to study. The terminal checks this information for any errors, converts it to JSON format, and sends it to the server. The server stores the received information in a database and generates a unique user ID.

[0446] Selection of an educational program

[0447] The server generates a list of relevant educational programs based on the learning area entered by the user and sends it to the terminal. The user selects a program they wish to study from this list, and that information is sent back to the server.

[0448] Implementation of interactive learning

[0449] The server generates a learning scenario related to the selected educational program and sends it to the terminal. The terminal presents the scenario to the user in an interactive learning format, and the user answers questions.

[0450] Feedback and corrections

[0451] The user's responses are sent from the device to the server, which analyzes them using natural language processing technology. Appropriate feedback is then generated and sent back to the device for the user to receive.

[0452] Introducing emotion recognition

[0453] The device transmits the user's facial expressions and voice tone to the emotion engine, which then analyzes their emotions. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[0454] Tests and evaluations

[0455] When a user reaches a certain learning stage, the device notifies them to take a test. The server analyzes the received test answers and sentiment information, generates an evaluation, and sends it to the device.

[0456] Supplementary learning and the next steps

[0457] Based on the test results, the server suggests supplementary learning content. This provides learning scenarios tailored to specific vocabulary and stress reduction. The terminal presents the suggested content to the user, supporting continuous learning.

[0458] Specific example

[0459] For example, if a user types "I want to learn how to use polite language," the server selects a relevant learning scenario and sends it to the device. If the user responds with "Thank you in advance," the server provides feedback such as, "You should say it more politely as 'Thank you for your help. I look forward to working with you.'" Also, if the emotion engine determines from the user's facial expressions that they are feeling stressed, it sends an encouraging message such as, "Relax and try again."

[0460] Examples of prompts to input into a generative AI model:

[0461] "Please build an interactive learning system to support the training of new crew members. The system needs to include user information input, learning area selection, feedback, emotion recognition, testing and evaluation, and supplementary learning."

[0462] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0463] Step 1:

[0464] Entering user information

[0465] The user enters their name, email address, department, and area of ​​study via the terminal. The terminal checks this information to ensure there are no errors. The verified user information is converted to JSON format and sent to the server. This ensures that user information is collected accurately. The input data is name, email address, department, and area of ​​study, and the output data is user information in JSON format.

[0466] Step 2:

[0467] User information storage

[0468] The server stores the received user information in JSON format in the database. During this process, a unique user ID is generated and registered in the database. This ensures that each user is uniquely identified. The input data is user information in JSON format, and the output data is the unique user ID.

[0469] Step 3:

[0470] Selection of an educational program

[0471] The terminal displays a message to the user saying, "Please select a subject to study." Once the user selects a subject, that information is converted to JSON format and sent to the server. The server searches its database for relevant educational programs, generates a list of them, and sends it back to the terminal. This presents the user with appropriate educational programs. The input data is the subject of study selected by the user, and the output data is a list of educational programs.

[0472] Step 4:

[0473] Starting interactive learning

[0474] When a user selects an educational program, the device sends that information to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario. This allows the user to begin learning interactively. The input data is the selected educational program, and the output data is the learning scenario.

[0475] Step 5:

[0476] Analysis of user responses and provision of feedback.

[0477] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This generated feedback is then provided to the user via the device. This ensures that the user's learning is properly supported. The input data is the user's answer, and the output data is the generated feedback.

[0478] Step 6:

[0479] Analysis of emotional information

[0480] During learning, the device transmits the user's facial expressions and tone of voice to the emotion engine. The emotion engine analyzes these inputs to determine the user's emotional state. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback. This enables learning support tailored to the user's emotional state. The input data consists of the user's facial expressions and tone of voice, while the output data is the analyzed emotional information.

[0481] Step 7:

[0482] Test implementation and evaluation

[0483] Once a certain learning stage is reached, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with sentiment information to the server. The server receives this, analyzes the test results, generates an evaluation, and sends it back to the device. This evaluates the user's learning performance. The input data consists of the test answers and sentiment information, and the output data is the evaluation of the test results.

[0484] Step 8:

[0485] Proposals for supplementary learning and records of progress

[0486] Based on the test results, the server suggests supplementary learning content. Learning scenarios tailored to specific vocabulary and stress reduction are provided. The terminal presents the suggested content to the user and records the progress of supplementary learning in a database. This supports the user's continuous learning. The input data is an evaluation of the test results, and the output data is a record of the supplementary learning scenario and its progress.

[0487] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0488] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0489] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0490] [Second Embodiment]

[0491] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0492] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0493] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0494] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0495] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0496] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0497] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0498] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0499] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0500] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0501] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0502] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0503] This invention provides a system for efficiently supplementing knowledge and correcting appropriate language use in the training of new crew members. The system consistently handles everything from user information input to the selection of training programs, interactive learning, testing, evaluation, and suggestions for supplementary learning.

[0504] User registration and information entry

[0505] Upon initial access, the user enters necessary information such as their name, email address, department, and desired field of study. The terminal checks this information and sends it to the server. The server stores the received user information in its database and displays a registration completion message to the user. A unique user ID is generated during this process.

[0506] Selection of an educational program

[0507] The terminal prompts the user to select a subject they wish to learn. If the user selects, for example, "Business Etiquette," this selection is sent to the server. The server searches its database for relevant educational programs and returns a list to the terminal. The terminal then presents the user with multiple educational program options.

[0508] Implementation of interactive learning

[0509] When a user selects a specific educational program, the device sends that information to the server. The server generates a learning scenario based on the selected educational program and sends it to the device. Based on the learning scenario, the device presents the user with interactive questions and scenarios. This allows the user to deepen their knowledge through interactive learning.

[0510] Feedback and corrections

[0511] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the answer and generates appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The device then presents the generated feedback to the user and proceeds to the next question or scenario.

[0512] Tests and evaluations

[0513] Once the user has reached a certain stage of learning, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates the test results and evaluation. This includes suggestions for supplementary learning if needed. The device then displays the test results and evaluation to the user.

[0514] Supplementary learning and the next steps

[0515] The server has a means of suggesting supplementary learning content based on the evaluation results. For example, if supplementary learning on specific vocabulary is needed, the system will automatically present that content. The terminal then presents the user with the next learning step and supplementary learning content, realizing a consistent educational cycle.

[0516] Specific example

[0517] For example, when a user learns how to use polite language, the following specific actions are taken:

[0518] 1. User: "I would like to learn how to use polite language correctly."

[0519] 2. Terminal: Requests the relevant basic scenario from the server.

[0520] 3. Server: Generates the scenario and sends it to the terminal.

[0521] 4. Terminal: Displays the question, "How would you greet your boss for the first time?"

[0522] 5. User: "Thank you in advance."

[0523] 6. Terminal: Sends the response to the server.

[0524] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[0525] 8. Device: Show this feedback to the user.

[0526] As described above, this system supports user learning in a consistent step-by-step manner, providing effective education while reducing the burden on the field staff.

[0527] The following describes the processing flow.

[0528] Step 1:

[0529] When a user first accesses the system, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this user input to ensure there are no errors.

[0530] Step 2:

[0531] The terminal converts the verified user information into JSON format and sends it to the server. The server stores the received user information in a database and generates a unique user ID.

[0532] Step 3:

[0533] The server sends a registration confirmation email to the user's email address. The terminal displays a registration completion message to the user.

[0534] Step 4:

[0535] The device prompts the user to select a field of study, asking, "Please choose the field you would like to learn about."

[0536] Step 5:

[0537] The user selects the subject they want to learn about (e.g., "Business Etiquette"). The device converts the selection into JSON format and sends it to the server.

[0538] Step 6:

[0539] The server searches the database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal presents the program list to the user, allowing them to choose from multiple options.

[0540] Step 7:

[0541] The user selects a specific educational program. The device then sends this selection information to the server.

[0542] Step 8:

[0543] The server generates a learning scenario related to the selected educational program and sends it to the terminal in JSON format. The terminal then presents the user with interactive learning based on the learning scenario.

[0544] Step 9:

[0545] When a user answers a scenario-based question, the device sends that answer to the server.

[0546] Step 10:

[0547] The server analyzes the received responses using natural language processing techniques and generates appropriate feedback. This feedback includes whether the response is correct or incorrect, as well as suggestions for improving the wording.

[0548] Step 11:

[0549] The device presents the generated feedback to the user and offers further questions or scenarios.

[0550] Step 12:

[0551] Once the user has completed a certain amount of learning, the device will notify them that an exam is about to be held.

[0552] Step 13:

[0553] The user starts the test and answers the test questions. The device sends the answers to the server.

[0554] Step 14:

[0555] The server analyzes the test results and generates scores and evaluations. The evaluation results include suggestions for supplementary learning.

[0556] Step 15:

[0557] The terminal displays the generated test results and evaluations to the user.

[0558] Step 16:

[0559] The server records the evaluation results and the user's learning progress in a database and suggests the next learning steps and supplementary learning content.

[0560] Step 17:

[0561] The device presents the user with the next learning step and supplementary learning content, encouraging the user to continue learning.

[0562] (Example 1)

[0563] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0564] In training new crew members, there is a need for a system that efficiently supplements their knowledge and corrects their language use. However, traditional training systems have limited functionality and have difficulty consistently providing feedback on user progress and language use. Furthermore, the training process is fragmented across multiple methods, resulting in a lack of efficiency.

[0565] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0566] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for using a generative AI model to generate a learning scenario related to the selected field, means for obtaining the user's learning request using prompt statements, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, and means for providing the generated feedback to the user. This makes it possible to support the user's learning in a consistent step-by-step manner and provide effective education while reducing the burden on the field staff.

[0567] "User information" refers to personal identifying information, including the user's name, email address, department, and areas of study they wish to pursue.

[0568] A "database" is a system for efficiently storing, managing, and retrieving data, and MySQL is one example.

[0569] A "learning scenario" is a sequence of specific educational content and question formats related to the selected learning area.

[0570] A "generative AI model" is a model that uses generative artificial intelligence techniques to generate text and scenarios, and includes, for example, GPT-3.

[0571] A "prompt message" is a text input used to generate a specific output when inputted into a generative AI model.

[0572] "Interactive learning" is a learning format that progresses through dialogue between the user and the system.

[0573] "Feedback" refers to evaluations and suggestions for improvement provided in response to user responses.

[0574] A "test" is a set of questions designed to assess the level of understanding of the learned material, and includes questions that the user answers.

[0575] "Evaluation" refers to information about the user's learning progress and level of understanding, generated by analyzing test results.

[0576] "Supplementary learning" refers to additional learning content proposed based on the evaluation results.

[0577] This invention provides a system for efficiently supplementing knowledge and correcting appropriate language use in the training of new crew members. The system consistently performs tasks from user information input to the selection of training programs, interactive learning, testing, evaluation, and suggestions for supplementary learning. Each step utilizes a generative AI model and prompt sentences.

[0578] User registration and information entry

[0579] When a user first accesses the system, they enter information such as their name, email address, department, and desired field of study into the terminal. The terminal verifies the format and required fields of this information to ensure its accuracy. After verification, the terminal sends the information to the server. The server stores the received information in a database (e.g., a MySQL database) and generates a unique user ID. The server then generates a registration completion message and sends it to the terminal, which displays the message to the user.

[0580] Selection of an educational program

[0581] The terminal displays a menu prompting the user to select a subject they wish to learn. When the user selects an option such as "Business Etiquette," the terminal sends that selection to the server. The server searches its database for relevant educational programs and sends a list of found programs to the terminal. The terminal then presents this list to the user, prompting them to select a specific program.

[0582] Implementation of interactive learning

[0583] When a user selects a specific educational program, the device sends that information to the server. The server generates a learning scenario using a generative AI model (e.g., GPT-3) and sends it to the device. The device then presents the user with interactive questions and scenarios based on the learning scenario. This allows the user to deepen their knowledge through interactive learning.

[0584] Feedback and corrections

[0585] When a user answers a question in a learning scenario, the device sends the answer to the server. The server uses a generative AI model to analyze the answer and generate appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The server sends the generated feedback to the device, which then displays it to the user. The device then presents the user with the next question or scenario.

[0586] Tests and evaluations

[0587] Once the learning process reaches a certain stage, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server uses a generative AI model to analyze the answers and generate test results and an evaluation. The evaluation also includes suggestions for areas where supplementary learning is needed. The server sends the test results and evaluation to the device, which then displays them to the user.

[0588] Supplementary learning and the next steps

[0589] The server automatically generates supplementary learning content based on the evaluation results. It generates prompt messages based on the evaluation results and sends them to the terminal. The terminal presents the user with the next learning step and supplementary learning content, and the user proceeds with the learning accordingly. This ensures a consistent educational cycle.

[0590] Specific example: Learning how to use honorific language

[0591] For example, when a user is learning how to use polite language, the following prompt might be used:

[0592] 1. User: "I would like to learn how to use polite language correctly."

[0593] 2. Terminal: Requests relevant basic scenarios from the server.

[0594] 3. Server: Generates scenarios using the generated AI model and sends them to the terminal.

[0595] 4. Terminal: Displays the question, "How would you greet your boss for the first time?"

[0596] 5. User: Responds with "Thank you in advance."

[0597] 6. Terminal: Send the answer to the server.

[0598] 7. Server: The AI ​​model analyzes the response and generates feedback such as, "That's correct, but it would be better if you added a more polite 'Thank you for your assistance. I look forward to working with you.'"

[0599] 8. Device: Display feedback to the user.

[0600] In this way, this system can support user learning through a consistent process, providing effective education while reducing the burden on staff.

[0601] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0602] Step 1: Enter user information

[0603] Upon initial access, the user enters information such as their name, email address, department, and desired field of study into the terminal. The terminal verifies the format and required fields of this information to ensure its accuracy (input: user information, output: verified user information). The terminal then sends the verified information to the server (operation: data verification and transmission).

[0604] Step 2: Saving User Information

[0605] The server saves the received user information to a database (e.g., MySQL) and generates a unique user ID (input: verified user information, output: user information and unique user ID stored in the database). The server then generates a registration completion message and sends it to the terminal (action: database save, user ID generation, message generation).

[0606] Step 3: Display of registration completion message

[0607] The terminal displays a registration completion message to the user (Input: Registration completion message, Output: Message displayed to the user) (Action: Message display).

[0608] Step 4: Selecting an Educational Program

[0609] The terminal displays a menu prompting the user to select a subject they wish to study (Input: None, Output: Displayed menu). The user selects an option such as "Business Manners" and inputs it into the terminal (Input: User selection, Output: Selected subject). The terminal sends the selection to the server (Action: Retrieve and send user selection).

[0610] Step 5: Searching for and presenting educational programs

[0611] The server searches the database for relevant educational programs and sends a list of found programs to the terminal (Input: User-selected field, Output: Program list). The terminal then presents the user with a selection of educational programs (Action: Database search, Program list transmission).

[0612] Step 6: Select and submit your educational program.

[0613] The user selects a specific educational program and enters the information into the terminal (input: selected educational program, output: information about the selected program). The terminal then sends this information to the server (action: educational program selection, information transmission).

[0614] Step 7: Generating the learning scenario

[0615] The server uses a generation AI model (e.g., GPT-3) to generate a learning scenario based on the transmitted program information (input: selected program information, output: generated learning scenario). The server then sends the generated learning scenario to the terminal (operation: scenario generation and transmission by the AI ​​model).

[0616] Step 8: Start interactive learning

[0617] The device presents the user with interactive questions and scenarios based on the learning scenario (input: generated learning scenario, output: presented interactive questions). The user answers the presented questions (action: scenario output and question presentation).

[0618] Step 9: Submit User Response

[0619] The user answers questions in the learning scenario and inputs the answers into the device (input: user answers, output: user answer data). The device then sends the answers to the server (action: retrieval and transmission of user answers).

[0620] Step 10: Analyze responses and generate feedback

[0621] The server uses a generative AI model to analyze user responses and generate appropriate feedback (input: user response data, output: generated feedback). This feedback includes not only whether the response is correct or incorrect, but also suggestions for improving the wording (operation: AI model analyzes responses and generates feedback).

[0622] Step 11: Displaying Feedback

[0623] The terminal displays the generated feedback to the user (Input: Generated feedback, Output: Feedback displayed to the user) (Action: Display feedback). The terminal then presents the user with further questions or scenarios (Action: Present next scenario).

[0624] Step 12: Notification and administration of the exam

[0625] Once the user has reached a certain stage of learning, the terminal notifies the user that an exam will be held (Input: Learning progress information, Output: Exam notification). When the user starts the exam, the terminal displays the exam questions and sends the user's answers to the server (Input: User answers, Output: Submission of exam answers) (Action: Display of exam questions and submission of answers).

[0626] Step 13: Analysis and generation of test results

[0627] The server uses a generative AI model to analyze test responses and generate test results and evaluations (input: test response data, output: test results and evaluations). This also includes suggestions for when supplementary learning is needed (operation: AI model analyzes test results and generates evaluations).

[0628] Step 14: Display of evaluation and suggestion of supplementary learning

[0629] The server sends the test results and evaluation to the terminal (input: generated evaluation, output: evaluation displayed to the user). The terminal displays the evaluation to the user and suggests supplementary learning content (action: sending and displaying evaluation, suggesting supplementary learning).

[0630] Step 15: Conduct supplementary learning

[0631] The terminal presents the user with the next learning step and supplementary learning content, and the user proceeds with the learning accordingly (Input: Supplementary learning content, Output: Presented supplementary learning) (Operation: Display of the next step and start of learning). The server records the progress of supplementary learning in a database, thereby ensuring a consistent educational cycle (Operation: Recording of supplementary learning progress).

[0632] (Application Example 1)

[0633] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0634] Traditional training systems face the challenge of effectively educating new crew members and customer support staff on improving their communication skills and language use. Furthermore, providing real-time feedback and supplementary learning is difficult, resulting in low learning effectiveness. This invention aims to solve these problems, streamline the staff training process, and rapidly improve communication skills.

[0635] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0636] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for performing interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, means for inputting prompt sentences into a generating AI model and generating interactive learning scenarios and feedback, and means for suggesting scenarios to improve customer support response skills. This enables customer support staff to effectively improve their response skills.

[0637] "Means for entering user information" refers to the means by which users enter necessary information such as their name, email address, department, and the field they wish to study.

[0638] "Means for receiving transmitted user information and saving it to a database" refers to the means by which information entered by a user is sent to a server, received, and saved to a database.

[0639] "Means for selecting a field of study" refers to the means by which users can choose the field of study they wish to pursue.

[0640] "Means for generating learning scenarios related to selected fields" refers to means for creating learning content and scenarios based on the fields selected by the user.

[0641] "Means for conducting interactive learning based on learning scenarios" refers to means for advancing learning in an interactive format with the user based on a generated learning scenario.

[0642] "Means for analyzing user responses and generating feedback" refers to methods for analyzing responses provided by users during interactive learning and creating feedback such as correctness, errors, and areas for improvement.

[0643] "Means of providing generated feedback to the user" refers to means of displaying the analyzed feedback to the user.

[0644] "A means of inputting prompt sentences into a generative AI model to generate interactive learning scenarios and feedback" refers to a means of using a generative AI model to input prompt sentences and automatically create learning scenarios and feedback.

[0645] "Methods for proposing scenarios to improve customer support response skills" refers to methods for proposing scenarios to improve the response skills necessary for customer support staff.

[0646] "Means for conducting tests on customer support response skills and providing evaluation and supplementary learning" refers to means for conducting tests to measure the response skills of customer support staff and providing evaluation and supplementary learning based on the results.

[0647] "A means of providing real-time feedback and supplementary learning on response skills using a generative AI model" refers to a means of providing users with immediate feedback and supplementary learning on response skills using a generative AI model.

[0648] This invention is a system aimed at the efficient training of customer support staff. It is designed to enable new staff to quickly acquire appropriate language and communication skills. The system consists of the following main components:

[0649] Entering user information

[0650] Upon initial access, users enter necessary information such as their name, email address, department, and desired learning area into their device (smartphone or head-mounted display). The device sends this information to the server, which then stores the received user information in a database.

[0651] Selection of an educational program

[0652] The terminal prompts the user to select a field of study they wish to pursue. If the user selects, for example, "customer service skills," this selection is sent to the server. The server then searches its database for relevant educational programs and returns a list to the terminal. The terminal then presents the user with multiple educational program options.

[0653] Implementation of interactive learning

[0654] When a user selects a specific educational program, the device sends that information to the server. The server uses a generative AI model (e.g., GPT-4) to generate a learning scenario based on the selected educational program and sends it to the device. Based on the learning scenario, the device presents the user with interactive questions and scenarios. Through interactive learning, the user can deepen their communication skills.

[0655] Feedback and corrections

[0656] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the answer and uses a generative AI model to generate appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The device then presents the generated feedback to the user.

[0657] Tests and evaluations

[0658] Once the user has reached a certain stage of learning, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates the test results and evaluation. This includes suggestions for supplementary learning if needed. The device then displays the test results and evaluation to the user.

[0659] Supplementary learning and the next steps

[0660] The server has a mechanism to suggest supplementary learning content based on the evaluation results. For example, if supplementary learning on customer service skills is needed, the content will be automatically presented. The terminal presents the user with the next learning step and supplementary learning content, realizing a consistent educational cycle.

[0661] Specific examples of components and data processing

[0662] This system uses smartphones and head-mounted displays as hardware, with React Native as the front-end software and Flask (Python) and SQLite as the back-end. It also includes a program that uses a generative AI model to analyze prompt messages and generate feedback and scenarios.

[0663] As a concrete example, if a user wants to learn how to use polite language, they would input the following prompt sentence into the AI ​​model:

[0664] "When a customer asks, 'Please tell me more about this product,' generate an appropriate response."

[0665] By providing real-time feedback and supplementary learning using generative AI models, users can efficiently and effectively learn the skills they need in the field.

[0666] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0667] Step 1:

[0668] Entering user information

[0669] Input: The user enters their name, email address, department, and the field they wish to study into the terminal.

[0670] Processing: The terminal transfers the entered user information to the server for transmission to the database.

[0671] Output: The server saves the received user information to the database and generates a unique user ID.

[0672] Step 2:

[0673] Selection of an educational program

[0674] Input: The user selects the subject they want to study on their device.

[0675] Processing: The terminal sends information about the selected field to the server. The server searches the database for relevant educational programs and sends the list back to the terminal.

[0676] Output: The terminal presents the user with multiple educational program options.

[0677] Step 3:

[0678] Scenario generation for interactive learning

[0679] Input: The user selects a specific educational program.

[0680] Processing: The terminal sends the selection information to the server. The server inputs prompt messages into the generating AI model and generates a learning scenario based on the selected educational program.

[0681] Output: The server sends the generated training scenario to the terminal, and the terminal displays it.

[0682] Step 4:

[0683] Implementation of interactive learning

[0684] Input: The user answers questions or scenarios in an interactive format.

[0685] Processing: The terminal sends the user's response to the server. The server analyzes the response using a generative AI model.

[0686] Output: Generates feedback based on the analysis results and sends it to the terminal. The terminal displays the feedback to the user.

[0687] Step 5:

[0688] Implementation of tests and evaluations

[0689] Input: Users who have made a certain level of progress in their learning will take the test.

[0690] Processing: The terminal displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates evaluation results.

[0691] Output: The evaluation results and necessary supplementary learning content are sent to the terminal, which then displays them to the user.

[0692] Step 6:

[0693] Supplementary learning and suggested next steps

[0694] Input: Test results and evaluation results.

[0695] Processing: Based on the evaluation results, the server automatically suggests the necessary supplementary learning content.

[0696] Output: The terminal presents the user with the next learning step and supplementary learning content.

[0697] As a concrete example, for a user learning "how to use polite language," the prompt sentence entered into the generative AI model would be as follows:

[0698] "When a customer asks, 'Please tell me more about this product,' generate an appropriate response."

[0699] This allows the system to effectively improve users' customer service skills.

[0700] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0701] This invention is a system that efficiently provides knowledge supplementation and corrects appropriate language use in the training of new crew members. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more precise feedback and enables effective learning.

[0702] User registration and information entry

[0703] When a user accesses the system for the first time, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this information to ensure it is complete and accurate. The verified user information is converted to JSON format and sent to the server. The server receives this information, stores it in its database, and generates a unique user ID.

[0704] Selection of an educational program

[0705] The terminal asks the user, "Please select the field you would like to learn about." When the user selects a learning field such as "Business Etiquette," that information is converted to JSON format and sent to the server. The server searches its database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal then presents the list of educational programs to the user, allowing them to choose from multiple options.

[0706] Implementation of interactive learning

[0707] When a user selects a specific educational program, that selection information is sent from the device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario.

[0708] Feedback and corrections

[0709] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving wording. The device then presents the generated feedback to the user and offers the next question or scenario.

[0710] Introducing emotion recognition

[0711] While the user is learning, the emotion engine acquires emotional information. The device sends the user's facial expressions, voice tone, etc., to the emotion engine for emotional analysis. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[0712] Tests and evaluations

[0713] Once learning reaches a certain stage, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with the user's sentiment information to the server. The server receives this information, analyzes the test results, and generates a score and evaluation. The evaluation results include suggestions for supplementary learning.

[0714] Supplementary learning and the next steps

[0715] The server suggests supplementary learning content based on the evaluation results. For example, it may provide supplementary learning on specific word usage or learning scenarios adjusted according to the user's stress level. The terminal then presents the user with the next learning step and supplementary learning content, supporting continuous learning.

[0716] Specific example

[0717] For example, when a user learns how to use polite language, the following specific actions are taken:

[0718] 1. User: "I would like to learn how to use polite language correctly."

[0719] 2. Terminal: "Request a basic scenario regarding the use of honorific language from the server."

[0720] 3. Server: Generates a learning scenario and sends it to the terminal.

[0721] 4. Terminal: The user is shown the message, "How would you greet your boss for the first time?"

[0722] 5. User: "Thank you in advance."

[0723] 6. Terminal: Send the response to the server.

[0724] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[0725] 8. Device: Display feedback to the user.

[0726] Furthermore, the emotion engine recognizes the user's emotions during the learning process. For example, if it determines that the user is "feeling stressed," it displays feedback and encouraging messages to help them relax.

[0727] This system supports user learning in a consistent step-by-step manner, provides individualized feedback, and reduces the burden on staff. Furthermore, by understanding the user's state in real time through emotion recognition and dynamically adjusting the learning content, more effective education becomes possible.

[0728] The following describes the processing flow.

[0729] Step 1:

[0730] When a user first accesses the system, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this information to ensure there are no errors. The verified user information is then converted to JSON format and sent to the server.

[0731] Step 2:

[0732] The server receives user information and stores it in the database. It generates a unique user ID and sends a registration confirmation email to the user. The terminal displays a registration completion message to the user.

[0733] Step 3:

[0734] The terminal asks the user, "Please select the field you would like to learn about." The user selects a learning field, such as "Business Etiquette." The terminal converts the selection into JSON format and sends it to the server.

[0735] Step 4:

[0736] The server searches the database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal presents the program list to the user, allowing them to choose from multiple options.

[0737] Step 5:

[0738] The user selects a specific educational program and sends this selection information from their device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format.

[0739] Step 6:

[0740] The device presents the user with interactive learning based on a learning scenario. It displays questions such as, "Which is the correct expression to use when thanking your boss?"

[0741] Step 7:

[0742] When a user answers a scenario-based question, the device sends the answer to the server. The server analyzes the answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving the wording.

[0743] Step 8:

[0744] The device presents the generated feedback to the user and offers further questions or scenarios. The feedback includes specific examples such as, "That's correct, but it would be better if you added a more polite phrase like, 'Thank you for your assistance. I appreciate your help.'"

[0745] Step 9:

[0746] During training, the emotion engine acquires data to analyze the user's facial expressions, voice tone, and other information. The device then sends this data to the emotion engine.

[0747] Step 10:

[0748] The emotion engine analyzes the user's emotions and generates emotion recognition results such as "the user is feeling stressed." The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[0749] Step 11:

[0750] The server generates and sends relaxing feedback or encouraging messages to the device, for example, if the user is feeling stressed. The device then presents these to the user.

[0751] Step 12:

[0752] Once the user has progressed to a certain level of learning, the device notifies them to take the exam. The user starts the exam and answers the questions. The device sends the answers to the server.

[0753] Step 13:

[0754] The server analyzes the test results and generates scores and evaluations. User emotional data is also reflected in the evaluation. For example, if the user is experiencing high levels of stress, this will be taken into consideration in the evaluation.

[0755] Step 14:

[0756] The device displays test results and evaluations to the user. The evaluation includes suggestions for supplementary learning.

[0757] Step 15:

[0758] The server records evaluation results and the user's learning progress in a database and suggests the next learning steps and supplementary learning content. For example, this may include providing relaxation content to reduce stress.

[0759] Step 16:

[0760] The device presents the user with the next learning step and supplementary learning content, encouraging them to continue learning. This provides a consistent learning experience and supports the user's growth.

[0761] (Example 2)

[0762] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0763] Traditional education systems struggle to efficiently assess users' learning progress and comprehension, and to provide appropriate feedback. Furthermore, they have difficulty understanding users' emotional states in real time and dynamically adjusting learning scenarios and feedback accordingly. There is also a need to provide individualized feedback on proper language use and stress management to support consistent learning.

[0764] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0765] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, and means for recognizing the user's emotions and reflecting the analysis results in the feedback. This makes it possible to efficiently evaluate the user's learning progress and understanding and provide appropriate feedback. Furthermore, the user's emotional state can be grasped in real time, and the learning scenario and feedback can be dynamically adjusted. In addition, individual feedback on correct language use and stress management can be provided to support consistent learning.

[0766] "User information" refers to a series of identifying information about an individual user, such as name, email address, department, and area of ​​study.

[0767] A "database" is a storage device used to centrally manage data such as user information, learning progress, and feedback.

[0768] A "learning scenario" is a set of specific educational content and questions generated based on the learning area selected by the user.

[0769] "Interactive learning" is a learning format in which the user and the system interact with each other as the learning process progresses.

[0770] "Feedback" refers to evaluations and advice given in response to a user's answers or actions.

[0771] An "emotion engine" is a system that analyzes emotional data such as a user's facial expressions and tone of voice.

[0772] "Stress level" is an indicator that shows the user's mental burden and state of tension.

[0773] "Supplementary learning" refers to additional learning content provided according to the user's level of understanding and progress.

[0774] "Natural language processing technology" is artificial intelligence technology that analyzes human language and understands its meaning.

[0775] An "exam" is a series of questions or tasks designed to assess whether a user understands specific learning material.

[0776] "Evaluation" refers to the results of representing a user's performance using numbers and comments based on tests and learning progress.

[0777] "Messages of encouragement" are words of encouragement provided by the system to boost the user's motivation during learning.

[0778] This invention is a system that efficiently supplements knowledge and corrects appropriate language use in the training of new crew members. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more precise feedback and enables effective learning.

[0779] User registration and information entry

[0780] When a user first accesses the system, they enter information such as their name, email address, department, and area of ​​study. The terminal checks this information for any errors. If there are any errors, the terminal notifies the user. The verified user information is converted to JSON format and sent to the server. The server receives this and stores it in a database to generate a unique user ID. This process uses general-purpose database technologies such as SQL or NoSQL databases.

[0781] Selection of an educational program

[0782] The device displays a prompt to the user asking, "Please select the field you would like to learn about." When the user selects a learning field, such as "Business Etiquette," the device converts that information into JSON format and sends it to the server. The server searches its database for relevant educational programs, generates a list, and sends it back to the device. The device then presents the user with the list of educational programs and allows them to choose from multiple options.

[0783] Implementation of interactive learning

[0784] When a user selects a specific educational program, that selection information is sent from the device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario.

[0785] Feedback and corrections

[0786] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving wording. The device then presents the generated feedback to the user and offers the next question or scenario. Natural language processing is performed using generative AI models and machine learning algorithms.

[0787] Introducing emotion recognition

[0788] While the user is learning, the emotion engine acquires emotional information. The device sends the user's facial expressions, voice tone, etc., to the emotion engine for emotional analysis. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback. The emotion engine uses facial recognition software and voice analysis tools.

[0789] Tests and evaluations

[0790] Once learning reaches a certain stage, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with the user's sentiment information to the server. The server receives this information, analyzes the test results, and generates a score and evaluation. The evaluation results include suggestions for supplementary learning.

[0791] Supplementary learning and the next steps

[0792] The server suggests supplementary learning content based on the evaluation results. For example, it may provide supplementary learning on specific word usage or learning scenarios adjusted according to the user's stress level. The terminal then presents the user with the next learning step and supplementary learning content, supporting continuous learning.

[0793] Specific example

[0794] For example, when a user learns how to use polite language, the following specific actions are taken:

[0795] 1. User: "I would like to learn how to use polite language correctly."

[0796] 2. Terminal: "Request a basic scenario regarding the use of honorific language from the server."

[0797] 3. Server: Generates a learning scenario and sends it to the terminal.

[0798] 4. Terminal: The user is shown the message, "How would you greet your boss for the first time?"

[0799] 5. User: "Thank you in advance."

[0800] 6. Terminal: Send the response to the server.

[0801] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[0802] 8. Device: Display feedback to the user.

[0803] Through this process, users can learn effectively while receiving real-time feedback. Furthermore, the emotion engine understands the user's emotional state and provides appropriate feedback, resulting in a more consistent learning experience.

[0804] Example of a prompt

[0805] "Please generate a learning scenario for someone who wants to learn how to use honorific language correctly."

[0806] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0807] Step 1: User registration and information entry

[0808] The user accesses the system and enters information such as their name, email address, department, and the field they wish to study.

[0809] The terminal checks the entered information to ensure there are no errors.

[0810] Input: User's name, email address, department, and information on the field of study.

[0811] Output: User information with no errors (JSON format)

[0812] Specific operation: The terminal receives the information, converts it to JSON format, and sends it to the server.

[0813] Step 2: Saving User Information

[0814] The server stores the received user information in JSON format in a database and generates a unique user ID.

[0815] Input: User information in JSON format

[0816] Output: Unique User ID

[0817] Specific operation: The server writes the received data to the database, generates a user ID, and sends it back to the terminal.

[0818] Step 3: Choosing a field of study

[0819] The device prompts the user to "Please select the field you would like to study."

[0820] The user selects a learning area, such as "business etiquette."

[0821] Input: User-selected learning field

[0822] Output: Learning subject information in JSON format

[0823] Specific operation: The terminal receives the user's selection, converts it to JSON format, and sends it to the server.

[0824] Step 4: Search for educational programs

[0825] The server searches the database for educational programs related to the learning area selected by the user.

[0826] Input: Learning subject information in JSON format

[0827] Output: List of related educational programs (in JSON format)

[0828] Specific operation: The server executes a database query, generates a list of relevant educational programs, and sends it back to the terminal.

[0829] Step 5: View and select educational programs

[0830] The device presents the user with a list of educational programs and allows them to choose from multiple options.

[0831] The user selects a specific educational program.

[0832] Input: List of educational programs, user selection

[0833] Output: Information on the selected educational program (in JSON format)

[0834] Specific operation: The device displays a list of educational programs and accepts the user's selection.

[0835] Step 6: Generating the learning scenario

[0836] The server generates learning scenarios related to the selected educational program.

[0837] Input: Selected educational program information in JSON format

[0838] Output: Training scenario (JSON format)

[0839] Specific operation: The server generates a learning scenario based on educational program information and sends it to the terminal.

[0840] Step 7: Provide interactive learning

[0841] The device presents the user with interactive learning based on a learning scenario.

[0842] The user answers questions within the learning scenario.

[0843] Input: Learning scenario, user responses

[0844] Output: User response (JSON format)

[0845] Specific operation: The device displays the learning scenario, receives the user's responses, and sends them to the server.

[0846] Step 8: Analyze responses and generate feedback

[0847] The server analyzes the received user responses using natural language processing technology and generates appropriate feedback.

[0848] Input: User's response (JSON format)

[0849] Output: Feedback (JSON format)

[0850] Specific operation: The server analyzes the response, generates feedback including correct answers and areas for improvement, and sends it to the terminal.

[0851] Step 9: Provide feedback

[0852] The device presents the generated feedback to the user and offers further questions or scenarios.

[0853] Input: Feedback (JSON format)

[0854] Output: Next learning scenarios and questions

[0855] Specific action: The device displays feedback and presents the following question.

[0856] Step 10: Acquisition and analysis of emotional information

[0857] The emotion engine analyzes the user's facial expressions, voice tone, and other factors to acquire emotional information.

[0858] Input: User facial expression data, voice data

[0859] Output: Emotion information (JSON format)

[0860] Specific operation: The emotion engine analyzes the data and recognizes emotions.

[0861] Step 11: Using emotional information

[0862] The server receives analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[0863] Input: Sentiment information (JSON format)

[0864] Output: Adjusted learning scenarios and feedback

[0865] Specific operation: The server analyzes emotional information and adjusts the learning content and feedback accordingly.

[0866] Step 12: Notification of Exam Implementation

[0867] The device will notify the user of the exam after they have completed the specified learning content.

[0868] Input: Learning progress information

[0869] Output: Notification of test implementation

[0870] Specific operation: The device detects the completion of the learning process and notifies the user to start the test.

[0871] Step 13: Conduct the test and collect responses.

[0872] The user answers the test questions.

[0873] The device converts the response into JSON format and sends it to the server along with sentiment information.

[0874] Input: Exam questions, user responses, sentiment information

[0875] Output: Responses and sentiment information (JSON format)

[0876] Specific operation: The device displays the test questions, collects the answers and sentiment information, and sends it to the server.

[0877] Step 14: Analysis and generation of test results

[0878] The server analyzes the received test results and generates scores and evaluations.

[0879] Input: Responses and sentiment information (JSON format)

[0880] Output: Exam score and evaluation (JSON format)

[0881] Specific operation: The server analyzes the test results and calculates a score and evaluation.

[0882] Step 15: Presentation of evaluation results and suggestion of supplementary learning

[0883] The device displays the evaluation results to the user and suggests supplementary learning content.

[0884] Input: Exam score and evaluation (JSON format)

[0885] Output: Proposed supplementary learning

[0886] Specific operation: The device displays the evaluation results and suggests supplementary learning.

[0887] Step 16: Implementing and managing supplementary learning

[0888] The server suggests supplementary learning content based on the evaluation results and records the progress in a database.

[0889] Input: Proposals and implementation status of supplementary learning

[0890] Output: Updated learning progress information (JSON format)

[0891] Specific operation: The server manages the content of supplementary learning and records the progress in a database.

[0892] (Application Example 2)

[0893] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0894] Traditional new crew training systems were limited to knowledge supplementation and language correction, and were unable to dynamically adjust learning scenarios to take into account users' emotions and stress levels. They also lacked methods for providing real-time feedback and appropriate advice based on stress levels and concentration during work. As a result, the training effectiveness was limited, and it did not adequately contribute to improving users' learning efficiency or work quality.

[0895] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0896] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, means for acquiring and analyzing emotional information through analysis of the user's facial expressions and voice, means for adjusting and providing the learning scenario and feedback based on the analyzed emotional information, means for the user to take a test, means for analyzing the test results and generating an evaluation, means for suggesting supplementary learning content based on the evaluation results, means for recording the progress of supplementary learning in a database, means for analyzing the user's stress and concentration level during work using an emotion engine and providing advice as needed, means for analyzing the user's language use and providing feedback on correct word usage, means for presenting the next learning step according to the learning progress, and means for acquiring the user's visual information, analyzing it with an emotion engine, and dynamically adjusting the learning content. This makes it possible to provide more effective feedback and learning scenarios while taking into account the user's emotions and stress level.

[0897] "User information" refers to basic personal information such as the user's name, contact information, department, and field of study.

[0898] A "database" is a system for systematically storing and managing data such as user information, learning scenarios, evaluation results, and supplementary learning progress.

[0899] A "learning scenario" is a specific scenario or content that guides the user through their learning process, based on the selected learning area.

[0900] "Interactive learning" is a learning method in which the user and the system interact, and the system provides appropriate feedback to the user's responses.

[0901] "Feedback" refers to the system's response and advice, such as evaluations and instructions for correction, to a user's answers and actions.

[0902] "Emotional information" refers to data that indicates the emotional state of a user, obtained from their facial expressions, tone of voice, and other similar information.

[0903] An "emotion engine" is an analytical engine that analyzes user emotional information and reflects the results in learning scenarios and feedback.

[0904] "Test results" refer to data obtained by analyzing the responses users gave when taking a test, and form the basis for generating evaluations.

[0905] "Supplemental learning" refers to additional learning content suggested based on the user's evaluation results to encourage further learning.

[0906] "Stress and concentration" refers to the mental and emotional state of a user while they are learning or working.

[0907] "Visual information" refers to information acquired through cameras and sensors, such as the user's facial expressions and movements.

[0908] This invention is a system for efficiently training new crew members, providing feedback and learning scenarios tailored to each user's individual circumstances. Specific embodiments for carrying out this invention are described below.

[0909] User registration and information entry

[0910] When a user accesses the system for the first time, they enter basic information such as their name, email address, department, and the field they wish to study. The terminal checks this information for any errors, converts it to JSON format, and sends it to the server. The server stores the received information in a database and generates a unique user ID.

[0911] Selection of an educational program

[0912] The server generates a list of relevant educational programs based on the learning area entered by the user and sends it to the terminal. The user selects a program they wish to study from this list, and that information is sent back to the server.

[0913] Implementation of interactive learning

[0914] The server generates a learning scenario related to the selected educational program and sends it to the terminal. The terminal presents the scenario to the user in an interactive learning format, and the user answers questions.

[0915] Feedback and corrections

[0916] The user's responses are sent from the device to the server, which analyzes them using natural language processing technology. Appropriate feedback is then generated and sent back to the device for the user to receive.

[0917] Introducing emotion recognition

[0918] The device transmits the user's facial expressions and voice tone to the emotion engine, which then analyzes their emotions. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[0919] Tests and evaluations

[0920] When a user reaches a certain learning stage, the device notifies them to take a test. The server analyzes the received test answers and sentiment information, generates an evaluation, and sends it to the device.

[0921] Supplementary learning and the next steps

[0922] Based on the test results, the server suggests supplementary learning content. This provides learning scenarios tailored to specific vocabulary and stress reduction. The terminal presents the suggested content to the user, supporting continuous learning.

[0923] Specific example

[0924] For example, if a user types "I want to learn how to use polite language," the server selects a relevant learning scenario and sends it to the device. If the user responds with "Thank you in advance," the server provides feedback such as, "You should say it more politely as 'Thank you for your help. I look forward to working with you.'" Also, if the emotion engine determines from the user's facial expressions that they are feeling stressed, it sends an encouraging message such as, "Relax and try again."

[0925] Examples of prompts to input into a generative AI model:

[0926] "Please build an interactive learning system to support the training of new crew members. The system needs to include user information input, learning area selection, feedback, emotion recognition, testing and evaluation, and supplementary learning."

[0927] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0928] Step 1:

[0929] Entering user information

[0930] The user enters their name, email address, department, and area of ​​study via the terminal. The terminal checks this information to ensure there are no errors. The verified user information is converted to JSON format and sent to the server. This ensures that user information is collected accurately. The input data is name, email address, department, and area of ​​study, and the output data is user information in JSON format.

[0931] Step 2:

[0932] User information storage

[0933] The server stores the received user information in JSON format in the database. During this process, a unique user ID is generated and registered in the database. This ensures that each user is uniquely identified. The input data is user information in JSON format, and the output data is the unique user ID.

[0934] Step 3:

[0935] Selection of an educational program

[0936] The terminal displays a message to the user saying, "Please select a subject to study." Once the user selects a subject, that information is converted to JSON format and sent to the server. The server searches its database for relevant educational programs, generates a list of them, and sends it back to the terminal. This presents the user with appropriate educational programs. The input data is the subject of study selected by the user, and the output data is a list of educational programs.

[0937] Step 4:

[0938] Starting interactive learning

[0939] When a user selects an educational program, the device sends that information to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario. This allows the user to begin learning interactively. The input data is the selected educational program, and the output data is the learning scenario.

[0940] Step 5:

[0941] Analysis of user responses and provision of feedback.

[0942] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This generated feedback is then provided to the user via the device. This ensures that the user's learning is properly supported. The input data is the user's answer, and the output data is the generated feedback.

[0943] Step 6:

[0944] Analysis of emotional information

[0945] During learning, the device transmits the user's facial expressions and tone of voice to the emotion engine. The emotion engine analyzes these inputs to determine the user's emotional state. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback. This enables learning support tailored to the user's emotional state. The input data consists of the user's facial expressions and tone of voice, while the output data is the analyzed emotional information.

[0946] Step 7:

[0947] Test implementation and evaluation

[0948] Once a certain learning stage is reached, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with sentiment information to the server. The server receives this, analyzes the test results, generates an evaluation, and sends it back to the device. This evaluates the user's learning performance. The input data consists of the test answers and sentiment information, and the output data is the evaluation of the test results.

[0949] Step 8:

[0950] Proposals for supplementary learning and records of progress

[0951] Based on the test results, the server suggests supplementary learning content. Learning scenarios tailored to specific vocabulary and stress reduction are provided. The terminal presents the suggested content to the user and records the progress of supplementary learning in a database. This supports the user's continuous learning. The input data is an evaluation of the test results, and the output data is a record of the supplementary learning scenario and its progress.

[0952] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0953] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0954] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0955] [Third Embodiment]

[0956] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0957] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0958] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0959] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0960] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0961] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0962] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0963] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0964] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0965] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0966] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0967] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0968] This invention provides a system for efficiently supplementing knowledge and correcting appropriate language use in the training of new crew members. The system consistently handles everything from user information input to the selection of training programs, interactive learning, testing, evaluation, and suggestions for supplementary learning.

[0969] User registration and information entry

[0970] Upon initial access, the user enters necessary information such as their name, email address, department, and desired field of study. The terminal checks this information and sends it to the server. The server stores the received user information in its database and displays a registration completion message to the user. A unique user ID is generated during this process.

[0971] Selection of an educational program

[0972] The terminal prompts the user to select a subject they wish to learn. If the user selects, for example, "Business Etiquette," this selection is sent to the server. The server searches its database for relevant educational programs and returns a list to the terminal. The terminal then presents the user with multiple educational program options.

[0973] Implementation of interactive learning

[0974] When a user selects a specific educational program, the device sends that information to the server. The server generates a learning scenario based on the selected educational program and sends it to the device. Based on the learning scenario, the device presents the user with interactive questions and scenarios. This allows the user to deepen their knowledge through interactive learning.

[0975] Feedback and corrections

[0976] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the answer and generates appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The device then presents the generated feedback to the user and proceeds to the next question or scenario.

[0977] Tests and evaluations

[0978] Once the user has reached a certain stage of learning, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates the test results and evaluation. This includes suggestions for supplementary learning if needed. The device then displays the test results and evaluation to the user.

[0979] Supplementary learning and the next steps

[0980] The server has a means of suggesting supplementary learning content based on the evaluation results. For example, if supplementary learning on specific vocabulary is needed, the system will automatically present that content. The terminal then presents the user with the next learning step and supplementary learning content, realizing a consistent educational cycle.

[0981] Specific example

[0982] For example, when a user learns how to use polite language, the following specific actions are taken:

[0983] 1. User: "I would like to learn how to use polite language correctly."

[0984] 2. Terminal: Requests the relevant basic scenario from the server.

[0985] 3. Server: Generates the scenario and sends it to the terminal.

[0986] 4. Terminal: Displays the question, "How would you greet your boss for the first time?"

[0987] 5. User: "Thank you in advance."

[0988] 6. Terminal: Sends the response to the server.

[0989] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[0990] 8. Device: Show this feedback to the user.

[0991] As described above, this system supports user learning in a consistent step-by-step manner, providing effective education while reducing the burden on the field staff.

[0992] The following describes the processing flow.

[0993] Step 1:

[0994] When a user first accesses the system, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this user input to ensure there are no errors.

[0995] Step 2:

[0996] The terminal converts the verified user information into JSON format and sends it to the server. The server stores the received user information in a database and generates a unique user ID.

[0997] Step 3:

[0998] The server sends a registration confirmation email to the user's email address. The terminal displays a registration completion message to the user.

[0999] Step 4:

[1000] The device prompts the user to select a field of study, asking, "Please choose the field you would like to learn about."

[1001] Step 5:

[1002] The user selects the subject they want to learn about (e.g., "Business Etiquette"). The device converts the selection into JSON format and sends it to the server.

[1003] Step 6:

[1004] The server searches the database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal presents the program list to the user, allowing them to choose from multiple options.

[1005] Step 7:

[1006] The user selects a specific educational program. The device then sends this selection information to the server.

[1007] Step 8:

[1008] The server generates a learning scenario related to the selected educational program and sends it to the terminal in JSON format. The terminal then presents the user with interactive learning based on the learning scenario.

[1009] Step 9:

[1010] When a user answers a scenario-based question, the device sends that answer to the server.

[1011] Step 10:

[1012] The server analyzes the received responses using natural language processing techniques and generates appropriate feedback. This feedback includes whether the response is correct or incorrect, as well as suggestions for improving the wording.

[1013] Step 11:

[1014] The device presents the generated feedback to the user and offers further questions or scenarios.

[1015] Step 12:

[1016] Once the user has completed a certain amount of learning, the device will notify them that an exam is about to be held.

[1017] Step 13:

[1018] The user starts the test and answers the test questions. The device sends the answers to the server.

[1019] Step 14:

[1020] The server analyzes the test results and generates scores and evaluations. The evaluation results include suggestions for supplementary learning.

[1021] Step 15:

[1022] The terminal displays the generated test results and evaluations to the user.

[1023] Step 16:

[1024] The server records the evaluation results and the user's learning progress in a database and suggests the next learning steps and supplementary learning content.

[1025] Step 17:

[1026] The device presents the user with the next learning step and supplementary learning content, encouraging the user to continue learning.

[1027] (Example 1)

[1028] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1029] In training new crew members, there is a need for a system that efficiently supplements their knowledge and corrects their language use. However, traditional training systems have limited functionality and have difficulty consistently providing feedback on user progress and language use. Furthermore, the training process is fragmented across multiple methods, resulting in a lack of efficiency.

[1030] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1031] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for using a generative AI model to generate a learning scenario related to the selected field, means for obtaining the user's learning request using prompt statements, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, and means for providing the generated feedback to the user. This makes it possible to support the user's learning in a consistent step-by-step manner and provide effective education while reducing the burden on the field staff.

[1032] "User information" refers to personal identifying information, including the user's name, email address, department, and areas of study they wish to pursue.

[1033] A "database" is a system for efficiently storing, managing, and retrieving data, and MySQL is one example.

[1034] A "learning scenario" is a sequence of specific educational content and question formats related to the selected learning area.

[1035] A "generative AI model" is a model that uses generative artificial intelligence techniques to generate text and scenarios, and includes, for example, GPT-3.

[1036] A "prompt message" is a text input used to generate a specific output when inputted into a generative AI model.

[1037] "Interactive learning" is a learning format that progresses through dialogue between the user and the system.

[1038] "Feedback" refers to evaluations and suggestions for improvement provided in response to user responses.

[1039] A "test" is a set of questions designed to assess the level of understanding of the learned material, and includes questions that the user answers.

[1040] "Evaluation" refers to information about the user's learning progress and level of understanding, generated by analyzing test results.

[1041] "Supplementary learning" refers to additional learning content proposed based on the evaluation results.

[1042] This invention provides a system for efficiently supplementing knowledge and correcting appropriate language use in the training of new crew members. The system consistently performs tasks from user information input to the selection of training programs, interactive learning, testing, evaluation, and suggestions for supplementary learning. Each step utilizes a generative AI model and prompt sentences.

[1043] User registration and information entry

[1044] When a user first accesses the system, they enter information such as their name, email address, department, and desired field of study into the terminal. The terminal verifies the format and required fields of this information to ensure its accuracy. After verification, the terminal sends the information to the server. The server stores the received information in a database (e.g., a MySQL database) and generates a unique user ID. The server then generates a registration completion message and sends it to the terminal, which displays the message to the user.

[1045] Selection of an educational program

[1046] The terminal displays a menu prompting the user to select a subject they wish to learn. When the user selects an option such as "Business Etiquette," the terminal sends that selection to the server. The server searches its database for relevant educational programs and sends a list of found programs to the terminal. The terminal then presents this list to the user, prompting them to select a specific program.

[1047] Implementation of interactive learning

[1048] When a user selects a specific educational program, the device sends that information to the server. The server generates a learning scenario using a generative AI model (e.g., GPT-3) and sends it to the device. The device then presents the user with interactive questions and scenarios based on the learning scenario. This allows the user to deepen their knowledge through interactive learning.

[1049] Feedback and corrections

[1050] When a user answers a question in a learning scenario, the device sends the answer to the server. The server uses a generative AI model to analyze the answer and generate appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The server sends the generated feedback to the device, which then displays it to the user. The device then presents the user with the next question or scenario.

[1051] Tests and evaluations

[1052] Once the learning process reaches a certain stage, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server uses a generative AI model to analyze the answers and generate test results and an evaluation. The evaluation also includes suggestions for areas where supplementary learning is needed. The server sends the test results and evaluation to the device, which then displays them to the user.

[1053] Supplementary learning and the next steps

[1054] The server automatically generates supplementary learning content based on the evaluation results. It generates prompt messages based on the evaluation results and sends them to the terminal. The terminal presents the user with the next learning step and supplementary learning content, and the user proceeds with the learning accordingly. This ensures a consistent educational cycle.

[1055] Specific example: Learning how to use honorific language

[1056] For example, when a user is learning how to use polite language, the following prompt might be used:

[1057] 1. User: "I would like to learn how to use polite language correctly."

[1058] 2. Terminal: Requests relevant basic scenarios from the server.

[1059] 3. Server: Generates scenarios using the generated AI model and sends them to the terminal.

[1060] 4. Terminal: Displays the question, "How would you greet your boss for the first time?"

[1061] 5. User: Responds with "Thank you in advance."

[1062] 6. Terminal: Send the answer to the server.

[1063] 7. Server: The AI ​​model analyzes the response and generates feedback such as, "That's correct, but it would be better if you added a more polite 'Thank you for your assistance. I look forward to working with you.'"

[1064] 8. Device: Display feedback to the user.

[1065] In this way, this system can support user learning through a consistent process, providing effective education while reducing the burden on staff.

[1066] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1067] Step 1: Enter user information

[1068] Upon initial access, the user enters information such as their name, email address, department, and desired field of study into the terminal. The terminal verifies the format and required fields of this information to ensure its accuracy (input: user information, output: verified user information). The terminal then sends the verified information to the server (operation: data verification and transmission).

[1069] Step 2: Saving User Information

[1070] The server saves the received user information to a database (e.g., MySQL) and generates a unique user ID (input: verified user information, output: user information and unique user ID stored in the database). The server then generates a registration completion message and sends it to the terminal (action: database save, user ID generation, message generation).

[1071] Step 3: Display of registration completion message

[1072] The terminal displays a registration completion message to the user (Input: Registration completion message, Output: Message displayed to the user) (Action: Message display).

[1073] Step 4: Selecting an Educational Program

[1074] The terminal displays a menu prompting the user to select a subject they wish to study (Input: None, Output: Displayed menu). The user selects an option such as "Business Manners" and inputs it into the terminal (Input: User selection, Output: Selected subject). The terminal sends the selection to the server (Action: Retrieve and send user selection).

[1075] Step 5: Searching for and presenting educational programs

[1076] The server searches the database for relevant educational programs and sends a list of found programs to the terminal (Input: User-selected field, Output: Program list). The terminal then presents the user with a selection of educational programs (Action: Database search, Program list transmission).

[1077] Step 6: Select and submit your educational program.

[1078] The user selects a specific educational program and enters the information into the terminal (input: selected educational program, output: information about the selected program). The terminal then sends this information to the server (action: educational program selection, information transmission).

[1079] Step 7: Generating the learning scenario

[1080] The server uses a generation AI model (e.g., GPT-3) to generate a learning scenario based on the transmitted program information (input: selected program information, output: generated learning scenario). The server then sends the generated learning scenario to the terminal (operation: scenario generation and transmission by the AI ​​model).

[1081] Step 8: Start interactive learning

[1082] The device presents the user with interactive questions and scenarios based on the learning scenario (input: generated learning scenario, output: presented interactive questions). The user answers the presented questions (action: scenario output and question presentation).

[1083] Step 9: Submit User Response

[1084] The user answers questions in the learning scenario and inputs the answers into the device (input: user answers, output: user answer data). The device then sends the answers to the server (action: retrieval and transmission of user answers).

[1085] Step 10: Analyze responses and generate feedback

[1086] The server uses a generative AI model to analyze user responses and generate appropriate feedback (input: user response data, output: generated feedback). This feedback includes not only whether the response is correct or incorrect, but also suggestions for improving the wording (operation: AI model analyzes responses and generates feedback).

[1087] Step 11: Displaying Feedback

[1088] The terminal displays the generated feedback to the user (Input: Generated feedback, Output: Feedback displayed to the user) (Action: Display feedback). The terminal then presents the user with further questions or scenarios (Action: Present next scenario).

[1089] Step 12: Notification and administration of the exam

[1090] Once the user has reached a certain stage of learning, the terminal notifies the user that an exam will be held (Input: Learning progress information, Output: Exam notification). When the user starts the exam, the terminal displays the exam questions and sends the user's answers to the server (Input: User answers, Output: Submission of exam answers) (Action: Display of exam questions and submission of answers).

[1091] Step 13: Analysis and generation of test results

[1092] The server uses a generative AI model to analyze test responses and generate test results and evaluations (input: test response data, output: test results and evaluations). This also includes suggestions for when supplementary learning is needed (operation: AI model analyzes test results and generates evaluations).

[1093] Step 14: Display of evaluation and suggestion of supplementary learning

[1094] The server sends the test results and evaluation to the terminal (input: generated evaluation, output: evaluation displayed to the user). The terminal displays the evaluation to the user and suggests supplementary learning content (action: sending and displaying evaluation, suggesting supplementary learning).

[1095] Step 15: Conduct supplementary learning

[1096] The terminal presents the user with the next learning step and supplementary learning content, and the user proceeds with the learning accordingly (Input: Supplementary learning content, Output: Presented supplementary learning) (Operation: Display of the next step and start of learning). The server records the progress of supplementary learning in a database, thereby ensuring a consistent educational cycle (Operation: Recording of supplementary learning progress).

[1097] (Application Example 1)

[1098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1099] Traditional training systems face the challenge of effectively educating new crew members and customer support staff on improving their communication skills and language use. Furthermore, providing real-time feedback and supplementary learning is difficult, resulting in low learning effectiveness. This invention aims to solve these problems, streamline the staff training process, and rapidly improve communication skills.

[1100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1101] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for performing interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, means for inputting prompt sentences into a generating AI model and generating interactive learning scenarios and feedback, and means for suggesting scenarios to improve customer support response skills. This enables customer support staff to effectively improve their response skills.

[1102] "Means for entering user information" refers to the means by which users enter necessary information such as their name, email address, department, and the field they wish to study.

[1103] "Means for receiving transmitted user information and saving it to a database" refers to the means by which information entered by a user is sent to a server, received, and saved to a database.

[1104] "Means for selecting a field of study" refers to the means by which users can choose the field of study they wish to pursue.

[1105] "Means for generating learning scenarios related to selected fields" refers to means for creating learning content and scenarios based on the fields selected by the user.

[1106] "Means for conducting interactive learning based on learning scenarios" refers to means for advancing learning in an interactive format with the user based on a generated learning scenario.

[1107] "Means for analyzing user responses and generating feedback" refers to methods for analyzing responses provided by users during interactive learning and creating feedback such as correctness, errors, and areas for improvement.

[1108] "Means of providing generated feedback to the user" refers to means of displaying the analyzed feedback to the user.

[1109] "A means of inputting prompt sentences into a generative AI model to generate interactive learning scenarios and feedback" refers to a means of using a generative AI model to input prompt sentences and automatically create learning scenarios and feedback.

[1110] "Methods for proposing scenarios to improve customer support response skills" refers to methods for proposing scenarios to improve the response skills necessary for customer support staff.

[1111] "Means for conducting tests on customer support response skills and providing evaluation and supplementary learning" refers to means for conducting tests to measure the response skills of customer support staff and providing evaluation and supplementary learning based on the results.

[1112] "A means of providing real-time feedback and supplementary learning on response skills using a generative AI model" refers to a means of providing users with immediate feedback and supplementary learning on response skills using a generative AI model.

[1113] This invention is a system aimed at the efficient training of customer support staff. It is designed to enable new staff to quickly acquire appropriate language and communication skills. The system consists of the following main components:

[1114] Entering user information

[1115] Upon initial access, users enter necessary information such as their name, email address, department, and desired learning area into their device (smartphone or head-mounted display). The device sends this information to the server, which then stores the received user information in a database.

[1116] Selection of an educational program

[1117] The terminal prompts the user to select a field of study they wish to pursue. If the user selects, for example, "customer service skills," this selection is sent to the server. The server then searches its database for relevant educational programs and returns a list to the terminal. The terminal then presents the user with multiple educational program options.

[1118] Implementation of interactive learning

[1119] When a user selects a specific educational program, the device sends that information to the server. The server uses a generative AI model (e.g., GPT-4) to generate a learning scenario based on the selected educational program and sends it to the device. Based on the learning scenario, the device presents the user with interactive questions and scenarios. Through interactive learning, the user can deepen their communication skills.

[1120] Feedback and corrections

[1121] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the answer and uses a generative AI model to generate appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The device then presents the generated feedback to the user.

[1122] Tests and evaluations

[1123] Once the user has reached a certain stage of learning, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates the test results and evaluation. This includes suggestions for supplementary learning if needed. The device then displays the test results and evaluation to the user.

[1124] Supplementary learning and the next steps

[1125] The server has a mechanism to suggest supplementary learning content based on the evaluation results. For example, if supplementary learning on customer service skills is needed, the content will be automatically presented. The terminal presents the user with the next learning step and supplementary learning content, realizing a consistent educational cycle.

[1126] Specific examples of components and data processing

[1127] This system uses smartphones and head-mounted displays as hardware, with React Native as the front-end software and Flask (Python) and SQLite as the back-end. It also includes a program that uses a generative AI model to analyze prompt messages and generate feedback and scenarios.

[1128] As a concrete example, if a user wants to learn how to use polite language, they would input the following prompt sentence into the AI ​​model:

[1129] "When a customer asks, 'Please tell me more about this product,' generate an appropriate response."

[1130] By providing real-time feedback and supplementary learning using generative AI models, users can efficiently and effectively learn the skills they need in the field.

[1131] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1132] Step 1:

[1133] Entering user information

[1134] Input: The user enters their name, email address, department, and the field they wish to study into the terminal.

[1135] Processing: The terminal transfers the entered user information to the server for transmission to the database.

[1136] Output: The server saves the received user information to the database and generates a unique user ID.

[1137] Step 2:

[1138] Selection of an educational program

[1139] Input: The user selects the subject they want to study on their device.

[1140] Processing: The terminal sends information about the selected field to the server. The server searches the database for relevant educational programs and sends the list back to the terminal.

[1141] Output: The terminal presents the user with multiple educational program options.

[1142] Step 3:

[1143] Scenario generation for interactive learning

[1144] Input: The user selects a specific educational program.

[1145] Processing: The terminal sends the selection information to the server. The server inputs prompt messages into the generating AI model and generates a learning scenario based on the selected educational program.

[1146] Output: The server sends the generated training scenario to the terminal, and the terminal displays it.

[1147] Step 4:

[1148] Implementation of interactive learning

[1149] Input: The user answers questions or scenarios in an interactive format.

[1150] Processing: The terminal sends the user's response to the server. The server analyzes the response using a generative AI model.

[1151] Output: Generates feedback based on the analysis results and sends it to the terminal. The terminal displays the feedback to the user.

[1152] Step 5:

[1153] Implementation of tests and evaluations

[1154] Input: Users who have made a certain level of progress in their learning will take the test.

[1155] Processing: The terminal displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates evaluation results.

[1156] Output: The evaluation results and necessary supplementary learning content are sent to the terminal, which then displays them to the user.

[1157] Step 6:

[1158] Supplementary learning and suggested next steps

[1159] Input: Test results and evaluation results.

[1160] Processing: Based on the evaluation results, the server automatically suggests the necessary supplementary learning content.

[1161] Output: The terminal presents the user with the next learning step and supplementary learning content.

[1162] As a concrete example, for a user learning "how to use polite language," the prompt sentence entered into the generative AI model would be as follows:

[1163] "When a customer asks, 'Please tell me more about this product,' generate an appropriate response."

[1164] This allows the system to effectively improve users' customer service skills.

[1165] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1166] This invention is a system that efficiently provides knowledge supplementation and corrects appropriate language use in the training of new crew members. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more precise feedback and enables effective learning.

[1167] User registration and information entry

[1168] When a user accesses the system for the first time, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this information to ensure it is complete and accurate. The verified user information is converted to JSON format and sent to the server. The server receives this information, stores it in its database, and generates a unique user ID.

[1169] Selection of an educational program

[1170] The terminal asks the user, "Please select the field you would like to learn about." When the user selects a learning field such as "Business Etiquette," that information is converted to JSON format and sent to the server. The server searches its database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal then presents the list of educational programs to the user, allowing them to choose from multiple options.

[1171] Implementation of interactive learning

[1172] When a user selects a specific educational program, that selection information is sent from the device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario.

[1173] Feedback and corrections

[1174] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving wording. The device then presents the generated feedback to the user and offers the next question or scenario.

[1175] Introducing emotion recognition

[1176] While the user is learning, the emotion engine acquires emotional information. The device sends the user's facial expressions, voice tone, etc., to the emotion engine for emotional analysis. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[1177] Tests and evaluations

[1178] Once learning reaches a certain stage, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with the user's sentiment information to the server. The server receives this information, analyzes the test results, and generates a score and evaluation. The evaluation results include suggestions for supplementary learning.

[1179] Supplementary learning and the next steps

[1180] The server suggests supplementary learning content based on the evaluation results. For example, it may provide supplementary learning on specific word usage or learning scenarios adjusted according to the user's stress level. The terminal then presents the user with the next learning step and supplementary learning content, supporting continuous learning.

[1181] Specific example

[1182] For example, when a user learns how to use polite language, the following specific actions are taken:

[1183] 1. User: "I would like to learn how to use polite language correctly."

[1184] 2. Terminal: "Request a basic scenario regarding the use of honorific language from the server."

[1185] 3. Server: Generates a learning scenario and sends it to the terminal.

[1186] 4. Terminal: The user is shown the message, "How would you greet your boss for the first time?"

[1187] 5. User: "Thank you in advance."

[1188] 6. Terminal: Send the response to the server.

[1189] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[1190] 8. Device: Display feedback to the user.

[1191] Furthermore, the emotion engine recognizes the user's emotions during the learning process. For example, if it determines that the user is "feeling stressed," it displays feedback and encouraging messages to help them relax.

[1192] This system supports user learning in a consistent step-by-step manner, provides individualized feedback, and reduces the burden on staff. Furthermore, by understanding the user's state in real time through emotion recognition and dynamically adjusting the learning content, more effective education becomes possible.

[1193] The following describes the processing flow.

[1194] Step 1:

[1195] When a user first accesses the system, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this information to ensure there are no errors. The verified user information is then converted to JSON format and sent to the server.

[1196] Step 2:

[1197] The server receives user information and stores it in the database. It generates a unique user ID and sends a registration confirmation email to the user. The terminal displays a registration completion message to the user.

[1198] Step 3:

[1199] The terminal asks the user, "Please select the field you would like to learn about." The user selects a learning field, such as "Business Etiquette." The terminal converts the selection into JSON format and sends it to the server.

[1200] Step 4:

[1201] The server searches the database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal presents the program list to the user, allowing them to choose from multiple options.

[1202] Step 5:

[1203] The user selects a specific educational program and sends this selection information from their device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format.

[1204] Step 6:

[1205] The device presents the user with interactive learning based on a learning scenario. It displays questions such as, "Which is the correct expression to use when thanking your boss?"

[1206] Step 7:

[1207] When a user answers a scenario-based question, the device sends the answer to the server. The server analyzes the answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving the wording.

[1208] Step 8:

[1209] The device presents the generated feedback to the user and offers further questions or scenarios. The feedback includes specific examples such as, "That's correct, but it would be better if you added a more polite phrase like, 'Thank you for your assistance. I appreciate your help.'"

[1210] Step 9:

[1211] During training, the emotion engine acquires data to analyze the user's facial expressions, voice tone, and other information. The device then sends this data to the emotion engine.

[1212] Step 10:

[1213] The emotion engine analyzes the user's emotions and generates emotion recognition results such as "the user is feeling stressed." The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[1214] Step 11:

[1215] The server generates and sends relaxing feedback or encouraging messages to the device, for example, if the user is feeling stressed. The device then presents these to the user.

[1216] Step 12:

[1217] Once the user has progressed to a certain level of learning, the device notifies them to take the exam. The user starts the exam and answers the questions. The device sends the answers to the server.

[1218] Step 13:

[1219] The server analyzes the test results and generates scores and evaluations. User emotional data is also reflected in the evaluation. For example, if the user is experiencing high levels of stress, this will be taken into consideration in the evaluation.

[1220] Step 14:

[1221] The device displays test results and evaluations to the user. The evaluation includes suggestions for supplementary learning.

[1222] Step 15:

[1223] The server records evaluation results and the user's learning progress in a database and suggests the next learning steps and supplementary learning content. For example, this may include providing relaxation content to reduce stress.

[1224] Step 16:

[1225] The device presents the user with the next learning step and supplementary learning content, encouraging them to continue learning. This provides a consistent learning experience and supports the user's growth.

[1226] (Example 2)

[1227] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1228] Traditional education systems struggle to efficiently assess users' learning progress and comprehension, and to provide appropriate feedback. Furthermore, they have difficulty understanding users' emotional states in real time and dynamically adjusting learning scenarios and feedback accordingly. There is also a need to provide individualized feedback on proper language use and stress management to support consistent learning.

[1229] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1230] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, and means for recognizing the user's emotions and reflecting the analysis results in the feedback. This makes it possible to efficiently evaluate the user's learning progress and understanding and provide appropriate feedback. Furthermore, the user's emotional state can be grasped in real time, and the learning scenario and feedback can be dynamically adjusted. In addition, individual feedback on correct language use and stress management can be provided to support consistent learning.

[1231] "User information" refers to a series of identifying information about an individual user, such as name, email address, department, and area of ​​study.

[1232] A "database" is a storage device used to centrally manage data such as user information, learning progress, and feedback.

[1233] A "learning scenario" is a set of specific educational content and questions generated based on the learning area selected by the user.

[1234] "Interactive learning" is a learning format in which the user and the system interact with each other as the learning process progresses.

[1235] "Feedback" refers to evaluations and advice given in response to a user's answers or actions.

[1236] An "emotion engine" is a system that analyzes emotional data such as a user's facial expressions and tone of voice.

[1237] "Stress level" is an indicator that shows the user's mental burden and state of tension.

[1238] "Supplementary learning" refers to additional learning content provided according to the user's level of understanding and progress.

[1239] "Natural language processing technology" is artificial intelligence technology that analyzes human language and understands its meaning.

[1240] An "exam" is a series of questions or tasks designed to assess whether a user understands specific learning material.

[1241] "Evaluation" refers to the results of representing a user's performance using numbers and comments based on tests and learning progress.

[1242] "Messages of encouragement" are words of encouragement provided by the system to boost the user's motivation during learning.

[1243] This invention is a system that efficiently supplements knowledge and corrects appropriate language use in the training of new crew members. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more precise feedback and enables effective learning.

[1244] User registration and information entry

[1245] When a user first accesses the system, they enter information such as their name, email address, department, and area of ​​study. The terminal checks this information for any errors. If there are any errors, the terminal notifies the user. The verified user information is converted to JSON format and sent to the server. The server receives this and stores it in a database to generate a unique user ID. This process uses general-purpose database technologies such as SQL or NoSQL databases.

[1246] Selection of an educational program

[1247] The device displays a prompt to the user asking, "Please select the field you would like to learn about." When the user selects a learning field, such as "Business Etiquette," the device converts that information into JSON format and sends it to the server. The server searches its database for relevant educational programs, generates a list, and sends it back to the device. The device then presents the user with the list of educational programs and allows them to choose from multiple options.

[1248] Implementation of interactive learning

[1249] When a user selects a specific educational program, that selection information is sent from the device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario.

[1250] Feedback and corrections

[1251] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving wording. The device then presents the generated feedback to the user and offers the next question or scenario. Natural language processing is performed using generative AI models and machine learning algorithms.

[1252] Introducing emotion recognition

[1253] While the user is learning, the emotion engine acquires emotional information. The device sends the user's facial expressions, voice tone, etc., to the emotion engine for emotional analysis. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback. The emotion engine uses facial recognition software and voice analysis tools.

[1254] Tests and evaluations

[1255] Once learning reaches a certain stage, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with the user's sentiment information to the server. The server receives this information, analyzes the test results, and generates a score and evaluation. The evaluation results include suggestions for supplementary learning.

[1256] Supplementary learning and the next steps

[1257] The server suggests supplementary learning content based on the evaluation results. For example, it may provide supplementary learning on specific word usage or learning scenarios adjusted according to the user's stress level. The terminal then presents the user with the next learning step and supplementary learning content, supporting continuous learning.

[1258] Specific example

[1259] For example, when a user learns how to use polite language, the following specific actions are taken:

[1260] 1. User: "I would like to learn how to use polite language correctly."

[1261] 2. Terminal: "Request a basic scenario regarding the use of honorific language from the server."

[1262] 3. Server: Generates a learning scenario and sends it to the terminal.

[1263] 4. Terminal: The user is shown the message, "How would you greet your boss for the first time?"

[1264] 5. User: "Thank you in advance."

[1265] 6. Terminal: Send the response to the server.

[1266] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[1267] 8. Device: Display feedback to the user.

[1268] Through this process, users can learn effectively while receiving real-time feedback. Furthermore, the emotion engine understands the user's emotional state and provides appropriate feedback, resulting in a more consistent learning experience.

[1269] Example of a prompt

[1270] "Please generate a learning scenario for someone who wants to learn how to use honorific language correctly."

[1271] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1272] Step 1: User registration and information entry

[1273] The user accesses the system and enters information such as their name, email address, department, and the field they wish to study.

[1274] The terminal checks the entered information to ensure there are no errors.

[1275] Input: User's name, email address, department, and information on the field of study.

[1276] Output: User information with no errors (JSON format)

[1277] Specific operation: The terminal receives the information, converts it to JSON format, and sends it to the server.

[1278] Step 2: Saving User Information

[1279] The server stores the received user information in JSON format in a database and generates a unique user ID.

[1280] Input: User information in JSON format

[1281] Output: Unique User ID

[1282] Specific operation: The server writes the received data to the database, generates a user ID, and sends it back to the terminal.

[1283] Step 3: Choosing a field of study

[1284] The device prompts the user to "Please select the field you would like to study."

[1285] The user selects a learning area, such as "business etiquette."

[1286] Input: User-selected learning field

[1287] Output: Learning subject information in JSON format

[1288] Specific operation: The terminal receives the user's selection, converts it to JSON format, and sends it to the server.

[1289] Step 4: Search for educational programs

[1290] The server searches the database for educational programs related to the learning area selected by the user.

[1291] Input: Learning subject information in JSON format

[1292] Output: List of related educational programs (in JSON format)

[1293] Specific operation: The server executes a database query, generates a list of relevant educational programs, and sends it back to the terminal.

[1294] Step 5: View and select educational programs

[1295] The device presents the user with a list of educational programs and allows them to choose from multiple options.

[1296] The user selects a specific educational program.

[1297] Input: List of educational programs, user selection

[1298] Output: Information on the selected educational program (in JSON format)

[1299] Specific operation: The device displays a list of educational programs and accepts the user's selection.

[1300] Step 6: Generating the learning scenario

[1301] The server generates learning scenarios related to the selected educational program.

[1302] Input: Selected educational program information in JSON format

[1303] Output: Training scenario (JSON format)

[1304] Specific operation: The server generates a learning scenario based on educational program information and sends it to the terminal.

[1305] Step 7: Provide interactive learning

[1306] The device presents the user with interactive learning based on a learning scenario.

[1307] The user answers questions within the learning scenario.

[1308] Input: Learning scenario, user responses

[1309] Output: User response (JSON format)

[1310] Specific operation: The device displays the learning scenario, receives the user's responses, and sends them to the server.

[1311] Step 8: Analyze responses and generate feedback

[1312] The server analyzes the received user responses using natural language processing technology and generates appropriate feedback.

[1313] Input: User's response (JSON format)

[1314] Output: Feedback (JSON format)

[1315] Specific operation: The server analyzes the response, generates feedback including correct answers and areas for improvement, and sends it to the terminal.

[1316] Step 9: Provide feedback

[1317] The device presents the generated feedback to the user and offers further questions or scenarios.

[1318] Input: Feedback (JSON format)

[1319] Output: Next learning scenarios and questions

[1320] Specific action: The device displays feedback and presents the following question.

[1321] Step 10: Acquisition and analysis of emotional information

[1322] The emotion engine analyzes the user's facial expressions, voice tone, and other factors to acquire emotional information.

[1323] Input: User facial expression data, voice data

[1324] Output: Emotion information (JSON format)

[1325] Specific operation: The emotion engine analyzes the data and recognizes emotions.

[1326] Step 11: Using emotional information

[1327] The server receives analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[1328] Input: Sentiment information (JSON format)

[1329] Output: Adjusted learning scenarios and feedback

[1330] Specific operation: The server analyzes emotional information and adjusts the learning content and feedback accordingly.

[1331] Step 12: Notification of Exam Implementation

[1332] The device will notify the user of the exam after they have completed the specified learning content.

[1333] Input: Learning progress information

[1334] Output: Notification of test implementation

[1335] Specific operation: The device detects the completion of the learning process and notifies the user to start the test.

[1336] Step 13: Conduct the test and collect responses.

[1337] The user answers the test questions.

[1338] The device converts the response into JSON format and sends it to the server along with sentiment information.

[1339] Input: Exam questions, user responses, sentiment information

[1340] Output: Responses and sentiment information (JSON format)

[1341] Specific operation: The device displays the test questions, collects the answers and sentiment information, and sends it to the server.

[1342] Step 14: Analysis and generation of test results

[1343] The server analyzes the received test results and generates scores and evaluations.

[1344] Input: Responses and sentiment information (JSON format)

[1345] Output: Exam score and evaluation (JSON format)

[1346] Specific operation: The server analyzes the test results and calculates a score and evaluation.

[1347] Step 15: Presentation of evaluation results and suggestion of supplementary learning

[1348] The device displays the evaluation results to the user and suggests supplementary learning content.

[1349] Input: Exam score and evaluation (JSON format)

[1350] Output: Proposed supplementary learning

[1351] Specific operation: The device displays the evaluation results and suggests supplementary learning.

[1352] Step 16: Implementing and managing supplementary learning

[1353] The server suggests supplementary learning content based on the evaluation results and records the progress in a database.

[1354] Input: Proposals and implementation status of supplementary learning

[1355] Output: Updated learning progress information (JSON format)

[1356] Specific operation: The server manages the content of supplementary learning and records the progress in a database.

[1357] (Application Example 2)

[1358] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1359] Traditional new crew training systems were limited to knowledge supplementation and language correction, and were unable to dynamically adjust learning scenarios to take into account users' emotions and stress levels. They also lacked methods for providing real-time feedback and appropriate advice based on stress levels and concentration during work. As a result, the training effectiveness was limited, and it did not adequately contribute to improving users' learning efficiency or work quality.

[1360] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1361] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, means for acquiring and analyzing emotional information through analysis of the user's facial expressions and voice, means for adjusting and providing the learning scenario and feedback based on the analyzed emotional information, means for the user to take a test, means for analyzing the test results and generating an evaluation, means for suggesting supplementary learning content based on the evaluation results, means for recording the progress of supplementary learning in a database, means for analyzing the user's stress and concentration level during work using an emotion engine and providing advice as needed, means for analyzing the user's language use and providing feedback on correct word usage, means for presenting the next learning step according to the learning progress, and means for acquiring the user's visual information, analyzing it with an emotion engine, and dynamically adjusting the learning content. This makes it possible to provide more effective feedback and learning scenarios while taking into account the user's emotions and stress level.

[1362] "User information" refers to basic personal information such as the user's name, contact information, department, and field of study.

[1363] A "database" is a system for systematically storing and managing data such as user information, learning scenarios, evaluation results, and supplementary learning progress.

[1364] A "learning scenario" is a specific scenario or content that guides the user through their learning process, based on the selected learning area.

[1365] "Interactive learning" is a learning method in which the user and the system interact, and the system provides appropriate feedback to the user's responses.

[1366] "Feedback" refers to the system's response and advice, such as evaluations and instructions for correction, to a user's answers and actions.

[1367] "Emotional information" refers to data that indicates the emotional state of a user, obtained from their facial expressions, tone of voice, and other similar information.

[1368] An "emotion engine" is an analytical engine that analyzes user emotional information and reflects the results in learning scenarios and feedback.

[1369] "Test results" refer to data obtained by analyzing the responses users gave when taking a test, and form the basis for generating evaluations.

[1370] "Supplemental learning" refers to additional learning content suggested based on the user's evaluation results to encourage further learning.

[1371] "Stress and concentration" refers to the mental and emotional state of a user while they are learning or working.

[1372] "Visual information" refers to information acquired through cameras and sensors, such as the user's facial expressions and movements.

[1373] This invention is a system for efficiently training new crew members, providing feedback and learning scenarios tailored to each user's individual circumstances. Specific embodiments for carrying out this invention are described below.

[1374] User registration and information entry

[1375] When a user accesses the system for the first time, they enter basic information such as their name, email address, department, and the field they wish to study. The terminal checks this information for any errors, converts it to JSON format, and sends it to the server. The server stores the received information in a database and generates a unique user ID.

[1376] Selection of an educational program

[1377] The server generates a list of relevant educational programs based on the learning area entered by the user and sends it to the terminal. The user selects a program they wish to study from this list, and that information is sent back to the server.

[1378] Implementation of interactive learning

[1379] The server generates a learning scenario related to the selected educational program and sends it to the terminal. The terminal presents the scenario to the user in an interactive learning format, and the user answers questions.

[1380] Feedback and corrections

[1381] The user's responses are sent from the device to the server, which analyzes them using natural language processing technology. Appropriate feedback is then generated and sent back to the device for the user to receive.

[1382] Introducing emotion recognition

[1383] The device transmits the user's facial expressions and voice tone to the emotion engine, which then analyzes their emotions. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[1384] Tests and evaluations

[1385] When a user reaches a certain learning stage, the device notifies them to take a test. The server analyzes the received test answers and sentiment information, generates an evaluation, and sends it to the device.

[1386] Supplementary learning and the next steps

[1387] Based on the test results, the server suggests supplementary learning content. This provides learning scenarios tailored to specific vocabulary and stress reduction. The terminal presents the suggested content to the user, supporting continuous learning.

[1388] Specific example

[1389] For example, if a user types "I want to learn how to use polite language," the server selects a relevant learning scenario and sends it to the device. If the user responds with "Thank you in advance," the server provides feedback such as, "You should say it more politely as 'Thank you for your help. I look forward to working with you.'" Also, if the emotion engine determines from the user's facial expressions that they are feeling stressed, it sends an encouraging message such as, "Relax and try again."

[1390] Examples of prompts to input into a generative AI model:

[1391] "Please build an interactive learning system to support the training of new crew members. The system needs to include user information input, learning area selection, feedback, emotion recognition, testing and evaluation, and supplementary learning."

[1392] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1393] Step 1:

[1394] Entering user information

[1395] The user enters their name, email address, department, and area of ​​study via the terminal. The terminal checks this information to ensure there are no errors. The verified user information is converted to JSON format and sent to the server. This ensures that user information is collected accurately. The input data is name, email address, department, and area of ​​study, and the output data is user information in JSON format.

[1396] Step 2:

[1397] User information storage

[1398] The server stores the received user information in JSON format in the database. During this process, a unique user ID is generated and registered in the database. This ensures that each user is uniquely identified. The input data is user information in JSON format, and the output data is the unique user ID.

[1399] Step 3:

[1400] Selection of an educational program

[1401] The terminal displays a message to the user saying, "Please select a subject to study." Once the user selects a subject, that information is converted to JSON format and sent to the server. The server searches its database for relevant educational programs, generates a list of them, and sends it back to the terminal. This presents the user with appropriate educational programs. The input data is the subject of study selected by the user, and the output data is a list of educational programs.

[1402] Step 4:

[1403] Starting interactive learning

[1404] When a user selects an educational program, the device sends that information to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario. This allows the user to begin learning interactively. The input data is the selected educational program, and the output data is the learning scenario.

[1405] Step 5:

[1406] Analysis of user responses and provision of feedback.

[1407] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This generated feedback is then provided to the user via the device. This ensures that the user's learning is properly supported. The input data is the user's answer, and the output data is the generated feedback.

[1408] Step 6:

[1409] Analysis of emotional information

[1410] During learning, the device transmits the user's facial expressions and tone of voice to the emotion engine. The emotion engine analyzes these inputs to determine the user's emotional state. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback. This enables learning support tailored to the user's emotional state. The input data consists of the user's facial expressions and tone of voice, while the output data is the analyzed emotional information.

[1411] Step 7:

[1412] Test implementation and evaluation

[1413] Once a certain learning stage is reached, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with sentiment information to the server. The server receives this, analyzes the test results, generates an evaluation, and sends it back to the device. This evaluates the user's learning performance. The input data consists of the test answers and sentiment information, and the output data is the evaluation of the test results.

[1414] Step 8:

[1415] Proposals for supplementary learning and records of progress

[1416] Based on the test results, the server suggests supplementary learning content. Learning scenarios tailored to specific vocabulary and stress reduction are provided. The terminal presents the suggested content to the user and records the progress of supplementary learning in a database. This supports the user's continuous learning. The input data is an evaluation of the test results, and the output data is a record of the supplementary learning scenario and its progress.

[1417] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1418] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1419] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1420] [Fourth Embodiment]

[1421] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1422] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1423] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1424] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1425] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1426] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1427] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1428] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1429] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1430] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1431] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1432] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1433] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1434] This invention provides a system for efficiently supplementing knowledge and correcting appropriate language use in the training of new crew members. The system consistently handles everything from user information input to the selection of training programs, interactive learning, testing, evaluation, and suggestions for supplementary learning.

[1435] User registration and information entry

[1436] Upon initial access, the user enters necessary information such as their name, email address, department, and desired field of study. The terminal checks this information and sends it to the server. The server stores the received user information in its database and displays a registration completion message to the user. A unique user ID is generated during this process.

[1437] Selection of an educational program

[1438] The terminal prompts the user to select a subject they wish to learn. If the user selects, for example, "Business Etiquette," this selection is sent to the server. The server searches its database for relevant educational programs and returns a list to the terminal. The terminal then presents the user with multiple educational program options.

[1439] Implementation of interactive learning

[1440] When a user selects a specific educational program, the device sends that information to the server. The server generates a learning scenario based on the selected educational program and sends it to the device. Based on the learning scenario, the device presents the user with interactive questions and scenarios. This allows the user to deepen their knowledge through interactive learning.

[1441] Feedback and corrections

[1442] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the answer and generates appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The device then presents the generated feedback to the user and proceeds to the next question or scenario.

[1443] Tests and evaluations

[1444] Once the user has reached a certain stage of learning, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates the test results and evaluation. This includes suggestions for supplementary learning if needed. The device then displays the test results and evaluation to the user.

[1445] Supplementary learning and the next steps

[1446] The server has a means of suggesting supplementary learning content based on the evaluation results. For example, if supplementary learning on specific vocabulary is needed, the system will automatically present that content. The terminal then presents the user with the next learning step and supplementary learning content, realizing a consistent educational cycle.

[1447] Specific example

[1448] For example, when a user learns how to use polite language, the following specific actions are taken:

[1449] 1. User: "I would like to learn how to use polite language correctly."

[1450] 2. Terminal: Requests the relevant basic scenario from the server.

[1451] 3. Server: Generates the scenario and sends it to the terminal.

[1452] 4. Terminal: Displays the question, "How would you greet your boss for the first time?"

[1453] 5. User: "Thank you in advance."

[1454] 6. Terminal: Sends the response to the server.

[1455] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[1456] 8. Device: Show this feedback to the user.

[1457] As described above, this system supports user learning in a consistent step-by-step manner, providing effective education while reducing the burden on the field staff.

[1458] The following describes the processing flow.

[1459] Step 1:

[1460] When a user first accesses the system, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this user input to ensure there are no errors.

[1461] Step 2:

[1462] The terminal converts the verified user information into JSON format and sends it to the server. The server stores the received user information in a database and generates a unique user ID.

[1463] Step 3:

[1464] The server sends a registration confirmation email to the user's email address. The terminal displays a registration completion message to the user.

[1465] Step 4:

[1466] The device prompts the user to select a field of study, asking, "Please choose the field you would like to learn about."

[1467] Step 5:

[1468] The user selects the subject they want to learn about (e.g., "Business Etiquette"). The device converts the selection into JSON format and sends it to the server.

[1469] Step 6:

[1470] The server searches the database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal presents the program list to the user, allowing them to choose from multiple options.

[1471] Step 7:

[1472] The user selects a specific educational program. The device then sends this selection information to the server.

[1473] Step 8:

[1474] The server generates a learning scenario related to the selected educational program and sends it to the terminal in JSON format. The terminal then presents the user with interactive learning based on the learning scenario.

[1475] Step 9:

[1476] When a user answers a scenario-based question, the device sends that answer to the server.

[1477] Step 10:

[1478] The server analyzes the received responses using natural language processing techniques and generates appropriate feedback. This feedback includes whether the response is correct or incorrect, as well as suggestions for improving the wording.

[1479] Step 11:

[1480] The device presents the generated feedback to the user and offers further questions or scenarios.

[1481] Step 12:

[1482] Once the user has completed a certain amount of learning, the device will notify them that an exam is about to be held.

[1483] Step 13:

[1484] The user starts the test and answers the test questions. The device sends the answers to the server.

[1485] Step 14:

[1486] The server analyzes the test results and generates scores and evaluations. The evaluation results include suggestions for supplementary learning.

[1487] Step 15:

[1488] The terminal displays the generated test results and evaluations to the user.

[1489] Step 16:

[1490] The server records the evaluation results and the user's learning progress in a database and suggests the next learning steps and supplementary learning content.

[1491] Step 17:

[1492] The device presents the user with the next learning step and supplementary learning content, encouraging the user to continue learning.

[1493] (Example 1)

[1494] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1495] In training new crew members, there is a need for a system that efficiently supplements their knowledge and corrects their language use. However, traditional training systems have limited functionality and have difficulty consistently providing feedback on user progress and language use. Furthermore, the training process is fragmented across multiple methods, resulting in a lack of efficiency.

[1496] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1497] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for using a generative AI model to generate a learning scenario related to the selected field, means for obtaining the user's learning request using prompt statements, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, and means for providing the generated feedback to the user. This makes it possible to support the user's learning in a consistent step-by-step manner and provide effective education while reducing the burden on the field staff.

[1498] "User information" refers to personal identifying information, including the user's name, email address, department, and areas of study they wish to pursue.

[1499] A "database" is a system for efficiently storing, managing, and retrieving data, and MySQL is one example.

[1500] A "learning scenario" is a sequence of specific educational content and question formats related to the selected learning area.

[1501] A "generative AI model" is a model that uses generative artificial intelligence techniques to generate text and scenarios, and includes, for example, GPT-3.

[1502] A "prompt message" is a text input used to generate a specific output when inputted into a generative AI model.

[1503] "Interactive learning" is a learning format that progresses through dialogue between the user and the system.

[1504] "Feedback" refers to evaluations and suggestions for improvement provided in response to user responses.

[1505] A "test" is a set of questions designed to assess the level of understanding of the learned material, and includes questions that the user answers.

[1506] "Evaluation" refers to information about the user's learning progress and level of understanding, generated by analyzing test results.

[1507] "Supplementary learning" refers to additional learning content proposed based on the evaluation results.

[1508] This invention provides a system for efficiently supplementing knowledge and correcting appropriate language use in the training of new crew members. The system consistently performs tasks from user information input to the selection of training programs, interactive learning, testing, evaluation, and suggestions for supplementary learning. Each step utilizes a generative AI model and prompt sentences.

[1509] User registration and information entry

[1510] When a user first accesses the system, they enter information such as their name, email address, department, and desired field of study into the terminal. The terminal verifies the format and required fields of this information to ensure its accuracy. After verification, the terminal sends the information to the server. The server stores the received information in a database (e.g., a MySQL database) and generates a unique user ID. The server then generates a registration completion message and sends it to the terminal, which displays the message to the user.

[1511] Selection of an educational program

[1512] The terminal displays a menu prompting the user to select a subject they wish to learn. When the user selects an option such as "Business Etiquette," the terminal sends that selection to the server. The server searches its database for relevant educational programs and sends a list of found programs to the terminal. The terminal then presents this list to the user, prompting them to select a specific program.

[1513] Implementation of interactive learning

[1514] When a user selects a specific educational program, the device sends that information to the server. The server generates a learning scenario using a generative AI model (e.g., GPT-3) and sends it to the device. The device then presents the user with interactive questions and scenarios based on the learning scenario. This allows the user to deepen their knowledge through interactive learning.

[1515] Feedback and corrections

[1516] When a user answers a question in a learning scenario, the device sends the answer to the server. The server uses a generative AI model to analyze the answer and generate appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The server sends the generated feedback to the device, which then displays it to the user. The device then presents the user with the next question or scenario.

[1517] Tests and evaluations

[1518] Once the learning process reaches a certain stage, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server uses a generative AI model to analyze the answers and generate test results and an evaluation. The evaluation also includes suggestions for areas where supplementary learning is needed. The server sends the test results and evaluation to the device, which then displays them to the user.

[1519] Supplementary learning and the next steps

[1520] The server automatically generates supplementary learning content based on the evaluation results. It generates prompt messages based on the evaluation results and sends them to the terminal. The terminal presents the user with the next learning step and supplementary learning content, and the user proceeds with the learning accordingly. This ensures a consistent educational cycle.

[1521] Specific example: Learning how to use honorific language

[1522] For example, when a user is learning how to use polite language, the following prompt might be used:

[1523] 1. User: "I would like to learn how to use polite language correctly."

[1524] 2. Terminal: Requests relevant basic scenarios from the server.

[1525] 3. Server: Generates scenarios using the generated AI model and sends them to the terminal.

[1526] 4. Terminal: Displays the question, "How would you greet your boss for the first time?"

[1527] 5. User: Responds with "Thank you in advance."

[1528] 6. Terminal: Send the answer to the server.

[1529] 7. Server: The AI ​​model analyzes the response and generates feedback such as, "That's correct, but it would be better if you added a more polite 'Thank you for your assistance. I look forward to working with you.'"

[1530] 8. Device: Display feedback to the user.

[1531] In this way, this system can support user learning through a consistent process, providing effective education while reducing the burden on staff.

[1532] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1533] Step 1: Enter user information

[1534] Upon initial access, the user enters information such as their name, email address, department, and desired field of study into the terminal. The terminal verifies the format and required fields of this information to ensure its accuracy (input: user information, output: verified user information). The terminal then sends the verified information to the server (operation: data verification and transmission).

[1535] Step 2: Saving User Information

[1536] The server saves the received user information to a database (e.g., MySQL) and generates a unique user ID (input: verified user information, output: user information and unique user ID stored in the database). The server then generates a registration completion message and sends it to the terminal (action: database save, user ID generation, message generation).

[1537] Step 3: Display of registration completion message

[1538] The terminal displays a registration completion message to the user (Input: Registration completion message, Output: Message displayed to the user) (Action: Message display).

[1539] Step 4: Selecting an Educational Program

[1540] The terminal displays a menu prompting the user to select a subject they wish to study (Input: None, Output: Displayed menu). The user selects an option such as "Business Manners" and inputs it into the terminal (Input: User selection, Output: Selected subject). The terminal sends the selection to the server (Action: Retrieve and send user selection).

[1541] Step 5: Searching for and presenting educational programs

[1542] The server searches the database for relevant educational programs and sends a list of found programs to the terminal (Input: User-selected field, Output: Program list). The terminal then presents the user with a selection of educational programs (Action: Database search, Program list transmission).

[1543] Step 6: Select and submit your educational program.

[1544] The user selects a specific educational program and enters the information into the terminal (input: selected educational program, output: information about the selected program). The terminal then sends this information to the server (action: educational program selection, information transmission).

[1545] Step 7: Generating the learning scenario

[1546] The server uses a generation AI model (e.g., GPT-3) to generate a learning scenario based on the transmitted program information (input: selected program information, output: generated learning scenario). The server then sends the generated learning scenario to the terminal (operation: scenario generation and transmission by the AI ​​model).

[1547] Step 8: Start interactive learning

[1548] The device presents the user with interactive questions and scenarios based on the learning scenario (input: generated learning scenario, output: presented interactive questions). The user answers the presented questions (action: scenario output and question presentation).

[1549] Step 9: Submit User Response

[1550] The user answers questions in the learning scenario and inputs the answers into the device (input: user answers, output: user answer data). The device then sends the answers to the server (action: retrieval and transmission of user answers).

[1551] Step 10: Analyze responses and generate feedback

[1552] The server uses a generative AI model to analyze user responses and generate appropriate feedback (input: user response data, output: generated feedback). This feedback includes not only whether the response is correct or incorrect, but also suggestions for improving the wording (operation: AI model analyzes responses and generates feedback).

[1553] Step 11: Displaying Feedback

[1554] The terminal displays the generated feedback to the user (Input: Generated feedback, Output: Feedback displayed to the user) (Action: Display feedback). The terminal then presents the user with further questions or scenarios (Action: Present next scenario).

[1555] Step 12: Notification and administration of the exam

[1556] Once the user has reached a certain stage of learning, the terminal notifies the user that an exam will be held (Input: Learning progress information, Output: Exam notification). When the user starts the exam, the terminal displays the exam questions and sends the user's answers to the server (Input: User answers, Output: Submission of exam answers) (Action: Display of exam questions and submission of answers).

[1557] Step 13: Analysis and generation of test results

[1558] The server uses a generative AI model to analyze test responses and generate test results and evaluations (input: test response data, output: test results and evaluations). This also includes suggestions for when supplementary learning is needed (operation: AI model analyzes test results and generates evaluations).

[1559] Step 14: Display of evaluation and suggestion of supplementary learning

[1560] The server sends the test results and evaluation to the terminal (input: generated evaluation, output: evaluation displayed to the user). The terminal displays the evaluation to the user and suggests supplementary learning content (action: sending and displaying evaluation, suggesting supplementary learning).

[1561] Step 15: Conduct supplementary learning

[1562] The terminal presents the user with the next learning step and supplementary learning content, and the user proceeds with the learning accordingly (Input: Supplementary learning content, Output: Presented supplementary learning) (Operation: Display of the next step and start of learning). The server records the progress of supplementary learning in a database, thereby ensuring a consistent educational cycle (Operation: Recording of supplementary learning progress).

[1563] (Application Example 1)

[1564] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1565] Traditional training systems face the challenge of effectively educating new crew members and customer support staff on improving their communication skills and language use. Furthermore, providing real-time feedback and supplementary learning is difficult, resulting in low learning effectiveness. This invention aims to solve these problems, streamline the staff training process, and rapidly improve communication skills.

[1566] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1567] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for performing interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, means for inputting prompt sentences into a generating AI model and generating interactive learning scenarios and feedback, and means for suggesting scenarios to improve customer support response skills. This enables customer support staff to effectively improve their response skills.

[1568] "Means for entering user information" refers to the means by which users enter necessary information such as their name, email address, department, and the field they wish to study.

[1569] "Means for receiving transmitted user information and saving it to a database" refers to the means by which information entered by a user is sent to a server, received, and saved to a database.

[1570] "Means for selecting a field of study" refers to the means by which users can choose the field of study they wish to pursue.

[1571] "Means for generating learning scenarios related to selected fields" refers to means for creating learning content and scenarios based on the fields selected by the user.

[1572] "Means for conducting interactive learning based on learning scenarios" refers to means for advancing learning in an interactive format with the user based on a generated learning scenario.

[1573] "Means for analyzing user responses and generating feedback" refers to methods for analyzing responses provided by users during interactive learning and creating feedback such as correctness, errors, and areas for improvement.

[1574] "Means of providing generated feedback to the user" refers to means of displaying the analyzed feedback to the user.

[1575] "A means of inputting prompt sentences into a generative AI model to generate interactive learning scenarios and feedback" refers to a means of using a generative AI model to input prompt sentences and automatically create learning scenarios and feedback.

[1576] "Methods for proposing scenarios to improve customer support response skills" refers to methods for proposing scenarios to improve the response skills necessary for customer support staff.

[1577] "Means for conducting tests on customer support response skills and providing evaluation and supplementary learning" refers to means for conducting tests to measure the response skills of customer support staff and providing evaluation and supplementary learning based on the results.

[1578] "A means of providing real-time feedback and supplementary learning on response skills using a generative AI model" refers to a means of providing users with immediate feedback and supplementary learning on response skills using a generative AI model.

[1579] This invention is a system aimed at the efficient training of customer support staff. It is designed to enable new staff to quickly acquire appropriate language and communication skills. The system consists of the following main components:

[1580] Entering user information

[1581] Upon initial access, users enter necessary information such as their name, email address, department, and desired learning area into their device (smartphone or head-mounted display). The device sends this information to the server, which then stores the received user information in a database.

[1582] Selection of an educational program

[1583] The terminal prompts the user to select a field of study they wish to pursue. If the user selects, for example, "customer service skills," this selection is sent to the server. The server then searches its database for relevant educational programs and returns a list to the terminal. The terminal then presents the user with multiple educational program options.

[1584] Implementation of interactive learning

[1585] When a user selects a specific educational program, the device sends that information to the server. The server uses a generative AI model (e.g., GPT-4) to generate a learning scenario based on the selected educational program and sends it to the device. Based on the learning scenario, the device presents the user with interactive questions and scenarios. Through interactive learning, the user can deepen their communication skills.

[1586] Feedback and corrections

[1587] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the answer and uses a generative AI model to generate appropriate feedback. This feedback includes not only whether the answer is correct or incorrect, but also suggestions for improving the wording. The device then presents the generated feedback to the user.

[1588] Tests and evaluations

[1589] Once the user has reached a certain stage of learning, the device notifies the user to take the test. When the user starts the test, the device displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates the test results and evaluation. This includes suggestions for supplementary learning if needed. The device then displays the test results and evaluation to the user.

[1590] Supplementary learning and the next steps

[1591] The server has a mechanism to suggest supplementary learning content based on the evaluation results. For example, if supplementary learning on customer service skills is needed, the content will be automatically presented. The terminal presents the user with the next learning step and supplementary learning content, realizing a consistent educational cycle.

[1592] Specific examples of components and data processing

[1593] This system uses smartphones and head-mounted displays as hardware, with React Native as the front-end software and Flask (Python) and SQLite as the back-end. It also includes a program that uses a generative AI model to analyze prompt messages and generate feedback and scenarios.

[1594] As a concrete example, if a user wants to learn how to use polite language, they would input the following prompt sentence into the AI ​​model:

[1595] "When a customer asks, 'Please tell me more about this product,' generate an appropriate response."

[1596] By providing real-time feedback and supplementary learning using generative AI models, users can efficiently and effectively learn the skills they need in the field.

[1597] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1598] Step 1:

[1599] Entering user information

[1600] Input: The user enters their name, email address, department, and the field they wish to study into the terminal.

[1601] Processing: The terminal transfers the entered user information to the server for transmission to the database.

[1602] Output: The server saves the received user information to the database and generates a unique user ID.

[1603] Step 2:

[1604] Selection of an educational program

[1605] Input: The user selects the subject they want to study on their device.

[1606] Processing: The terminal sends information about the selected field to the server. The server searches the database for relevant educational programs and sends the list back to the terminal.

[1607] Output: The terminal presents the user with multiple educational program options.

[1608] Step 3:

[1609] Scenario generation for interactive learning

[1610] Input: The user selects a specific educational program.

[1611] Processing: The terminal sends the selection information to the server. The server inputs prompt messages into the generating AI model and generates a learning scenario based on the selected educational program.

[1612] Output: The server sends the generated training scenario to the terminal, and the terminal displays it.

[1613] Step 4:

[1614] Implementation of interactive learning

[1615] Input: The user answers questions or scenarios in an interactive format.

[1616] Processing: The terminal sends the user's response to the server. The server analyzes the response using a generative AI model.

[1617] Output: Generates feedback based on the analysis results and sends it to the terminal. The terminal displays the feedback to the user.

[1618] Step 5:

[1619] Implementation of tests and evaluations

[1620] Input: Users who have made a certain level of progress in their learning will take the test.

[1621] Processing: The terminal displays the test questions and sends the user's answers to the server. The server analyzes the answers and generates evaluation results.

[1622] Output: The evaluation results and necessary supplementary learning content are sent to the terminal, which then displays them to the user.

[1623] Step 6:

[1624] Supplementary learning and suggested next steps

[1625] Input: Test results and evaluation results.

[1626] Processing: Based on the evaluation results, the server automatically suggests the necessary supplementary learning content.

[1627] Output: The terminal presents the user with the next learning step and supplementary learning content.

[1628] As a concrete example, for a user learning "how to use polite language," the prompt sentence entered into the generative AI model would be as follows:

[1629] "When a customer asks, 'Please tell me more about this product,' generate an appropriate response."

[1630] This allows the system to effectively improve users' customer service skills.

[1631] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1632] This invention is a system that efficiently provides knowledge supplementation and corrects appropriate language use in the training of new crew members. Furthermore, by combining it with an emotion engine that recognizes user emotions, it provides more precise feedback and enables effective learning.

[1633] User registration and information entry

[1634] When a user accesses the system for the first time, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this information to ensure it is complete and accurate. The verified user information is converted to JSON format and sent to the server. The server receives this information, stores it in its database, and generates a unique user ID.

[1635] Selection of an educational program

[1636] The terminal asks the user, "Please select the field you would like to learn about." When the user selects a learning field such as "Business Etiquette," that information is converted to JSON format and sent to the server. The server searches its database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal then presents the list of educational programs to the user, allowing them to choose from multiple options.

[1637] Implementation of interactive learning

[1638] When a user selects a specific educational program, that selection information is sent from the device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario.

[1639] Feedback and corrections

[1640] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving wording. The device then presents the generated feedback to the user and offers the next question or scenario.

[1641] Introducing emotion recognition

[1642] While the user is learning, the emotion engine acquires emotional information. The device sends the user's facial expressions, voice tone, etc., to the emotion engine for emotional analysis. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[1643] Tests and evaluations

[1644] Once learning reaches a certain stage, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with the user's sentiment information to the server. The server receives this information, analyzes the test results, and generates a score and evaluation. The evaluation results include suggestions for supplementary learning.

[1645] Supplementary learning and the next steps

[1646] The server suggests supplementary learning content based on the evaluation results. For example, it may provide supplementary learning on specific word usage or learning scenarios adjusted according to the user's stress level. The terminal then presents the user with the next learning step and supplementary learning content, supporting continuous learning.

[1647] Specific example

[1648] For example, when a user learns how to use polite language, the following specific actions are taken:

[1649] 1. User: "I would like to learn how to use polite language correctly."

[1650] 2. Terminal: "Request a basic scenario regarding the use of honorific language from the server."

[1651] 3. Server: Generates a learning scenario and sends it to the terminal.

[1652] 4. Terminal: The user is shown the message, "How would you greet your boss for the first time?"

[1653] 5. User: "Thank you in advance."

[1654] 6. Terminal: Send the response to the server.

[1655] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[1656] 8. Device: Display feedback to the user.

[1657] Furthermore, the emotion engine recognizes the user's emotions during the learning process. For example, if it determines that the user is "feeling stressed," it displays feedback and encouraging messages to help them relax.

[1658] This system supports user learning in a consistent step-by-step manner, provides individualized feedback, and reduces the burden on staff. Furthermore, by understanding the user's state in real time through emotion recognition and dynamically adjusting the learning content, more effective education becomes possible.

[1659] The following describes the processing flow.

[1660] Step 1:

[1661] When a user first accesses the system, they enter information such as their name, email address, department, and the field they wish to study. The terminal checks this information to ensure there are no errors. The verified user information is then converted to JSON format and sent to the server.

[1662] Step 2:

[1663] The server receives user information and stores it in the database. It generates a unique user ID and sends a registration confirmation email to the user. The terminal displays a registration completion message to the user.

[1664] Step 3:

[1665] The terminal asks the user, "Please select the field you would like to learn about." The user selects a learning field, such as "Business Etiquette." The terminal converts the selection into JSON format and sends it to the server.

[1666] Step 4:

[1667] The server searches the database for relevant educational programs, generates a list, and sends it back to the terminal. The terminal presents the program list to the user, allowing them to choose from multiple options.

[1668] Step 5:

[1669] The user selects a specific educational program and sends this selection information from their device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format.

[1670] Step 6:

[1671] The device presents the user with interactive learning based on a learning scenario. It displays questions such as, "Which is the correct expression to use when thanking your boss?"

[1672] Step 7:

[1673] When a user answers a scenario-based question, the device sends the answer to the server. The server analyzes the answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving the wording.

[1674] Step 8:

[1675] The device presents the generated feedback to the user and offers further questions or scenarios. The feedback includes specific examples such as, "That's correct, but it would be better if you added a more polite phrase like, 'Thank you for your assistance. I appreciate your help.'"

[1676] Step 9:

[1677] During training, the emotion engine acquires data to analyze the user's facial expressions, voice tone, and other information. The device then sends this data to the emotion engine.

[1678] Step 10:

[1679] The emotion engine analyzes the user's emotions and generates emotion recognition results such as "the user is feeling stressed." The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[1680] Step 11:

[1681] The server generates and sends relaxing feedback or encouraging messages to the device, for example, if the user is feeling stressed. The device then presents these to the user.

[1682] Step 12:

[1683] Once the user has progressed to a certain level of learning, the device notifies them to take the exam. The user starts the exam and answers the questions. The device sends the answers to the server.

[1684] Step 13:

[1685] The server analyzes the test results and generates scores and evaluations. User emotional data is also reflected in the evaluation. For example, if the user is experiencing high levels of stress, this will be taken into consideration in the evaluation.

[1686] Step 14:

[1687] The device displays test results and evaluations to the user. The evaluation includes suggestions for supplementary learning.

[1688] Step 15:

[1689] The server records evaluation results and the user's learning progress in a database and suggests the next learning steps and supplementary learning content. For example, this may include providing relaxation content to reduce stress.

[1690] Step 16:

[1691] The device presents the user with the next learning step and supplementary learning content, encouraging them to continue learning. This provides a consistent learning experience and supports the user's growth.

[1692] (Example 2)

[1693] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1694] Traditional education systems struggle to efficiently assess users' learning progress and comprehension, and to provide appropriate feedback. Furthermore, they have difficulty understanding users' emotional states in real time and dynamically adjusting learning scenarios and feedback accordingly. There is also a need to provide individualized feedback on proper language use and stress management to support consistent learning.

[1695] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1696] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, and means for recognizing the user's emotions and reflecting the analysis results in the feedback. This makes it possible to efficiently evaluate the user's learning progress and understanding and provide appropriate feedback. Furthermore, the user's emotional state can be grasped in real time, and the learning scenario and feedback can be dynamically adjusted. In addition, individual feedback on correct language use and stress management can be provided to support consistent learning.

[1697] "User information" refers to a series of identifying information about an individual user, such as name, email address, department, and area of ​​study.

[1698] A "database" is a storage device used to centrally manage data such as user information, learning progress, and feedback.

[1699] A "learning scenario" is a set of specific educational content and questions generated based on the learning area selected by the user.

[1700] "Interactive learning" is a learning format in which the user and the system interact with each other as the learning process progresses.

[1701] "Feedback" refers to evaluations and advice given in response to a user's answers or actions.

[1702] An "emotion engine" is a system that analyzes emotional data such as a user's facial expressions and tone of voice.

[1703] "Stress level" is an indicator that shows the user's mental burden and state of tension.

[1704] "Supplementary learning" refers to additional learning content provided according to the user's level of understanding and progress.

[1705] "Natural language processing technology" is artificial intelligence technology that analyzes human language and understands its meaning.

[1706] An "exam" is a series of questions or tasks designed to assess whether a user understands specific learning material.

[1707] "Evaluation" refers to the results of representing a user's performance using numbers and comments based on tests and learning progress.

[1708] "Messages of encouragement" are words of encouragement provided by the system to boost the user's motivation during learning.

[1709] This invention is a system that efficiently supplements knowledge and corrects appropriate language use in the training of new crew members. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more precise feedback and enables effective learning.

[1710] User registration and information entry

[1711] When a user first accesses the system, they enter information such as their name, email address, department, and area of ​​study. The terminal checks this information for any errors. If there are any errors, the terminal notifies the user. The verified user information is converted to JSON format and sent to the server. The server receives this and stores it in a database to generate a unique user ID. This process uses general-purpose database technologies such as SQL or NoSQL databases.

[1712] Selection of an educational program

[1713] The device displays a prompt to the user asking, "Please select the field you would like to learn about." When the user selects a learning field, such as "Business Etiquette," the device converts that information into JSON format and sends it to the server. The server searches its database for relevant educational programs, generates a list, and sends it back to the device. The device then presents the user with the list of educational programs and allows them to choose from multiple options.

[1714] Implementation of interactive learning

[1715] When a user selects a specific educational program, that selection information is sent from the device to the server. The server generates a learning scenario related to the selected educational program and sends it to the device in JSON format. The device then presents the user with interactive learning based on the learning scenario.

[1716] Feedback and corrections

[1717] When a user answers a question in a learning scenario, the device sends the answer to the server. The server analyzes the received answer using natural language processing techniques and generates appropriate feedback. This feedback includes whether the answer is correct or incorrect and suggestions for improving wording. The device then presents the generated feedback to the user and offers the next question or scenario. Natural language processing is performed using generative AI models and machine learning algorithms.

[1718] Introducing emotion recognition

[1719] While the user is learning, the emotion engine acquires emotional information. The device sends the user's facial expressions, voice tone, etc., to the emotion engine for emotional analysis. The server receives the analysis results from the emotion engine and incorporates them into the learning scenario and feedback. The emotion engine uses facial recognition software and voice analysis tools.

[1720] Tests and evaluations

[1721] Once learning reaches a certain stage, the device notifies the user to take a test. When the user starts the test and answers the test questions, the device sends the answers along with the user's sentiment information to the server. The server receives this information, analyzes the test results, and generates a score and evaluation. The evaluation results include suggestions for supplementary learning.

[1722] Supplementary learning and the next steps

[1723] The server suggests supplementary learning content based on the evaluation results. For example, it may provide supplementary learning on specific word usage or learning scenarios adjusted according to the user's stress level. The terminal then presents the user with the next learning step and supplementary learning content, supporting continuous learning.

[1724] Specific example

[1725] For example, when a user learns how to use polite language, the following specific actions are taken:

[1726] 1. User: "I would like to learn how to use polite language correctly."

[1727] 2. Terminal: "Request a basic scenario regarding the use of honorific language from the server."

[1728] 3. Server: Generates a learning scenario and sends it to the terminal.

[1729] 4. Terminal: The user is shown the message, "How would you greet your boss for the first time?"

[1730] 5. User: "Thank you in advance."

[1731] 6. Terminal: Send the response to the server.

[1732] 7. Server: Generates feedback saying, "That's correct, but it would be better if you said it more politely as, 'Thank you for your assistance. I look forward to working with you.'"

[1733] 8. Device: Display feedback to the user.

[1734] Through this process, users can learn effectively while receiving real-time feedback. Furthermore, the emotion engine understands the user's emotional state and provides appropriate feedback, resulting in a more consistent learning experience.

[1735] Example of a prompt

[1736] "Please generate a learning scenario for someone who wants to learn how to use honorific language correctly."

[1737] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1738] Step 1: User registration and information entry

[1739] The user accesses the system and enters information such as their name, email address, department, and the field they wish to study.

[1740] The terminal checks the entered information to ensure there are no errors.

[1741] Input: User's name, email address, department, and information on the field of study.

[1742] Output: User information with no errors (JSON format)

[1743] Specific operation: The terminal receives the information, converts it to JSON format, and sends it to the server.

[1744] Step 2: Saving User Information

[1745] The server stores the received user information in JSON format in a database and generates a unique user ID.

[1746] Input: User information in JSON format

[1747] Output: Unique User ID

[1748] Specific operation: The server writes the received data to the database, generates a user ID, and sends it back to the terminal.

[1749] Step 3: Choosing a field of study

[1750] The device prompts the user to "Please select the field you would like to study."

[1751] The user selects a learning area, such as "business etiquette."

[1752] Input: User-selected learning field

[1753] Output: Learning subject information in JSON format

[1754] Specific operation: The terminal receives the user's selection, converts it to JSON format, and sends it to the server.

[1755] Step 4: Search for educational programs

[1756] The server searches the database for educational programs related to the learning area selected by the user.

[1757] Input: Learning subject information in JSON format

[1758] Output: List of related educational programs (in JSON format)

[1759] Specific operation: The server executes a database query, generates a list of relevant educational programs, and sends it back to the terminal.

[1760] Step 5: View and select educational programs

[1761] The device presents the user with a list of educational programs and allows them to choose from multiple options.

[1762] The user selects a specific educational program.

[1763] Input: List of educational programs, user selection

[1764] Output: Information on the selected educational program (in JSON format)

[1765] Specific operation: The device displays a list of educational programs and accepts the user's selection.

[1766] Step 6: Generating the learning scenario

[1767] The server generates learning scenarios related to the selected educational program.

[1768] Input: Selected educational program information in JSON format

[1769] Output: Training scenario (JSON format)

[1770] Specific operation: The server generates a learning scenario based on educational program information and sends it to the terminal.

[1771] Step 7: Provide interactive learning

[1772] The device presents the user with interactive learning based on a learning scenario.

[1773] The user answers questions within the learning scenario.

[1774] Input: Learning scenario, user responses

[1775] Output: User response (JSON format)

[1776] Specific operation: The device displays the learning scenario, receives the user's responses, and sends them to the server.

[1777] Step 8: Analyze responses and generate feedback

[1778] The server analyzes the received user responses using natural language processing technology and generates appropriate feedback.

[1779] Input: User's response (JSON format)

[1780] Output: Feedback (JSON format)

[1781] Specific operation: The server analyzes the response, generates feedback including correct answers and areas for improvement, and sends it to the terminal.

[1782] Step 9: Provide feedback

[1783] The device presents the generated feedback to the user and offers further questions or scenarios.

[1784] Input: Feedback (JSON format)

[1785] Output: Next learning scenarios and questions

[1786] Specific action: The device displays feedback and presents the following question.

[1787] Step 10: Acquisition and analysis of emotional information

[1788] The emotion engine analyzes the user's facial expressions, voice tone, and other factors to acquire emotional information.

[1789] Input: User facial expression data, voice data

[1790] Output: Emotion information (JSON format)

[1791] Specific operation: The emotion engine analyzes the data and recognizes emotions.

[1792] Step 11: Using emotional information

[1793] The server receives analysis results from the emotion engine and incorporates them into the learning scenario and feedback.

[1794] Input: Sentiment information (JSON format)

[1795] Output: Adjusted learning scenarios and feedback

[1796] Specific operation: The server analyzes emotional information and adjusts the learning content and feedback accordingly.

[1797] Step 12: Notification of Exam Implementation

[1798] The device will notify the user of the exam after they have completed the specified learning content.

[1799] Input: Learning progress information

[1800] Output: Notification of test implementation

[1801] Specific operation: The device detects the completion of the learning process and notifies the user to start the test.

[1802] Step 13: Conduct the test and collect responses.

[1803] The user answers the test questions.

[1804] The device converts the response into JSON format and sends it to the server along with sentiment information.

[1805] Input: Exam questions, user responses, sentiment information

[1806] Output: Responses and sentiment information (JSON format)

[1807] Specific operation: The device displays the test questions, collects the answers and sentiment information, and sends it to the server.

[1808] Step 14: Analysis and generation of test results

[1809] The server analyzes the received test results and generates scores and evaluations.

[1810] Input: Responses and sentiment information (JSON format)

[1811] Output: Exam score and evaluation (JSON format)

[1812] Specific operation: The server analyzes the test results and calculates a score and evaluation.

[1813] Step 15: Presentation of evaluation results and suggestion of supplementary learning

[1814] The device displays the evaluation results to the user and suggests supplementary learning content.

[1815] Input: Exam score and evaluation (JSON format)

[1816] Output: Proposed supplementary learning

[1817] Specific operation: The device displays the evaluation results and suggests supplementary learning.

[1818] Step 16: Implementing and managing supplementary learning

[1819] The server suggests supplementary learning content based on the evaluation results and records the progress in a database.

[1820] Input: Proposals and implementation status of supplementary learning

[1821] Output: Updated learning progress information (JSON format)

[1822] Specific operation: The server manages the content of supplementary learning and records the progress in a database.

[1823] (Application Example 2)

[1824] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1825] Traditional new crew training systems were limited to knowledge supplementation and language correction, and were unable to dynamically adjust learning scenarios to take into account users' emotions and stress levels. They also lacked methods for providing real-time feedback and appropriate advice based on stress levels and concentration during work. As a result, the training effectiveness was limited, and it did not adequately contribute to improving users' learning efficiency or work quality.

[1826] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1827] In this invention, the server includes means for inputting user information, means for receiving transmitted user information and storing it in a database, means for selecting a field to learn, means for generating a learning scenario related to the selected field, means for conducting interactive learning based on the learning scenario, means for analyzing the user's responses and generating feedback, means for providing the generated feedback to the user, means for acquiring and analyzing emotional information through analysis of the user's facial expressions and voice, means for adjusting and providing the learning scenario and feedback based on the analyzed emotional information, means for the user to take a test, means for analyzing the test results and generating an evaluation, means for suggesting supplementary learning content based on the evaluation results, means for recording the progress of supplementary learning in a database, means for analyzing the user's stress and concentration level during work using an emotion engine and providing advice as needed, means for analyzing the user's language use and providing feedback on correct word usage, means for presenting the next learning step according to the learning progress, and means for acquiring the user's visual information, analyzing it with an emotion engine, and dynamically adjusting the learning content. This makes it possible to provide more effective feedback and learning scenarios while taking into account the user's emotions and stress level.

[1828] "User information" refers to basic personal information such as the user's name, contact information, department, and field of study.

[1829] A "d...

Claims

1. Means of entering user information, A means of receiving the transmitted user information and storing it in a database, The means of selecting a field of study, A means for generating learning scenarios related to the selected field, A means of conducting interactive learning based on a learning scenario, A means of analyzing user responses and generating feedback, A system that includes means for providing generated feedback to the user.

2. The means by which users can take the exam, A means for analyzing test results and generating evaluations, A means of proposing supplementary learning content based on evaluation results, The system according to claim 1, comprising means for recording the progress of supplementary learning in a database.

3. A means of analyzing the user's language usage and providing feedback on correct word usage, The system according to claim 1, comprising means for presenting the next learning step according to the learning progress.

Images