system

Abstract

We provide the system. [Solution] A means for the user to enter authentication information and send it to the server, A means for the server to verify the received authentication information and generate a user dashboard if authentication is successful, A means for a user to select a topic they want to learn about and send that selection information to the server, A means by which the server activates the instructor AI based on the selected theme and initiates interaction with the user, The server records the content of the conversation with the instructor AI and stores the data for use in the next session. A system that includes this.

Description

Technical Field

[0004] , , , ,

[0005] , , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern learning systems, it is important to provide an environment where users can efficiently proceed with learning at any time and anywhere. However, conventional e-learning systems have limited two-way interaction and it is difficult to provide appropriate feedback in real time according to the progress and understanding level of users. In addition, users need to manage their own learning progress, and it is difficult to maintain the motivation for self-learning. Furthermore, since the learning content is fixed, there is a lack of flexible learning support tailored to the individual needs of users. It is necessary to solve such problems and provide a system that allows users to efficiently learn while interacting with an instructor like a friend.

Means for Solving the Problems

[0005] The present invention is a system that includes means for a user to input authentication information and send it to a server, means for the server to verify the received authentication information and generate a user dashboard if authentication is successful, means for the user to select a topic they wish to learn and send that selection information to the server, means for the server to activate an instructor AI based on the selected topic and start a dialogue with the user, and means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session. With this system, after authentication, the user can choose a learning topic that suits their interests and efficiently proceed with learning through two-way dialogue with the instructor AI. In addition, the instructor AI can respond to the user's questions in real time and provide appropriate learning advice, thereby improving the user's learning experience. Furthermore, by having the server record the user's learning progress and use it in the next session, a learning plan tailored to each individual user can be provided. This enables efficient learning support while maintaining the user's learning motivation.

[0006] A "user" is an individual who uses a computer system or software.

[0007] A "server" is a central control unit that receives requests from users and returns processing results.

[0008] "Authentication information" refers to data used to verify a user's identity, such as a user ID and password.

[0009] A "dashboard" is an interface that allows users to check and manage their learning progress and settings.

[0010] A "theme" is a specific area of ​​knowledge or topic that a user wants to learn about.

[0011] An "instructor AI" is a program that uses artificial intelligence to provide learning support and answer questions for users.

[0012] "Dialogue" refers to a form of communication in which the user and the instructor AI exchange messages in a two-way manner.

[0013] "Progress" refers to data that shows the content and degree to which a user has achieved during the learning process.

[0014] "Real-time" means immediate processing, where a response is returned instantly to user input.

[0015] "Recording" refers to the act of saving a user's learning content and dialogue history to a database. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system for users to learn online via an internet connection. This system allows users to learn anytime, anywhere, while interacting with an AI instructor, and to manage their progress.

[0038] This system consists of the following main components: user terminals, servers, instructor AI models, and databases. The entire system also uses secure communication protocols (e.g., HTTPS) for data exchange.

[0039] The main functions and operation of the system

[0040] 1. User Authentication

[0041] The user enters their authentication information (user ID and password) on the login screen and presses the submit button.

[0042] The device sends these authentication credentials to the server.

[0043] The server compares the received authentication information with the database, and if authentication is successful, it generates a user-specific dashboard.

[0044] 2. Theme Selection

[0045] The user selects a topic they want to learn about from the dashboard (e.g., "Python Programming").

[0046] The device sends the selected theme information to the server.

[0047] The server loads the appropriate instructor AI model based on the selected theme.

[0048] 3. Interaction with the Instructor AI

[0049] The server starts the loaded instructor AI model and generates an initial message (e.g., "Let's get started with Python programming.").

[0050] The instructor AI will send this message to the user.

[0051] Users can send questions and instructions to the instructor AI while learning.

[0052] The device sends these messages to the server, which then passes them on to the instructor AI.

[0053] The instructor AI generates answers to the user's questions and sends them to the user.

[0054] 4. Recording progress

[0055] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database.

[0056] This information will be used in the next session.

[0057] Explanation with specific examples

[0058] 1. The user logs in.

[0059] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[0060] The terminal receives this and sends it to the server.

[0061] The server verifies the authentication information against the database, and if authentication is successful, it displays a dashboard for the user.

[0062] 2. The user selects a theme.

[0063] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[0064] The device sends this selection information to the server.

[0065] The server selects and starts the appropriate instructor AI model.

[0066] 3. Start the conversation with the instructor AI.

[0067] The instructor AI sends the user a message saying, "Let's start Python programming. First, we'll learn about basic data types."

[0068] The user asks, "How do I use variables?"

[0069] The device sends the question to the server, which then passes it on to the instructor AI.

[0070] The instructor AI responds with "In Python, variables are declared like this..." and generates a message that includes an example.

[0071] 4. Progress Management

[0072] The server records the conversation content and learning progress in a database.

[0073] In the next session, the server will provide appropriate learning content based on the progress information from the previous session.

[0074] This allows users to always progress in their learning based on their latest learning progress and receive appropriate feedback from the instructor AI in real time. In this way, the system of the present invention provides efficient and effective learning support to users.

[0075] The following describes the processing flow.

[0076] Step 1:

[0077] The user accesses the login screen and enters their authentication information (user ID and password). The user then clicks the "Login" button.

[0078] Step 2:

[0079] The terminal encodes the entered authentication information and sends it to the server using a secure communication protocol (HTTPS).

[0080] Step 3:

[0081] The server compares the received authentication information with the database. If the user ID and password match, authentication is considered successful.

[0082] Step 4:

[0083] The server generates a user-specific dashboard page along with an authentication success message and sends it to the device. If authentication fails, an error message is sent.

[0084] Step 5:

[0085] The user accesses the dashboard page and selects a topic they want to learn about from the provided list of topics (e.g., "Python Programming").

[0086] Step 6:

[0087] The device sends the selected theme information to the server.

[0088] Step 7:

[0089] The server receives the theme information and loads the corresponding instructor AI model. Once loading is complete, it generates an initial message.

[0090] Step 8:

[0091] The instructor AI generates an initial message, "Let's start Python programming," and sends it to the user.

[0092] Step 9:

[0093] The user enters questions or requests during the learning process, and the device sends them to the server.

[0094] Step 10:

[0095] The server passes the user's questions and requests to the instructor AI. The instructor AI then generates answers to those questions.

[0096] Step 11:

[0097] The instructor AI returns the generated answer to the server, which then sends that answer to the user.

[0098] Step 12:

[0099] The user receives answers from the instructor AI and continues learning. If there are any additional questions or requests, they can enter them again.

[0100] Step 13:

[0101] When a learning session ends, the server records the content of the user-instructor AI conversation and the learning progress in a database.

[0102] Step 14:

[0103] The server uses the recorded progress data to prepare the content for the next learning session. In the next session, new themes and problems will be provided based on the progress made in the previous session.

[0104] The above explains the program's processing in detail, step by step. This system allows users to effectively learn by interacting with the instructor AI anytime, anywhere.

[0105] (Example 1)

[0106] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0107] Traditional online learning systems lacked sufficient interactive learning support based on themes selected by learners. Furthermore, managing learning progress and preparing future learning plans required manual intervention, highlighting the need for more efficient learning support. Additionally, there was a lack of systems that could provide appropriate real-time answers to learners' questions. Therefore, the development of a system that provides more effective and interactive learning support was essential.

[0108] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0109] In this invention, the server includes means for the user to input authentication information and transmit it to the server via a computer network; means for the server to compare the received authentication information with a database and generate a user-specific interface if authentication is successful; means for the user to select a subject they wish to learn and transmit that selection information to the server; means for the server to load an educational support AI based on the selected subject and initiate a dialogue with the user in natural language; means for the educational support AI to generate answers to the user's questions in real time and transmit them to the user; means for the server to record the content of the dialogue with the educational support AI in a database and save the data for use in the next session; and means for the server to prepare the content of the next learning session based on the previous learning progress and for the educational support AI to provide appropriate learning advice as needed. As a result, the user can always receive appropriate learning support based on their latest learning progress.

[0110] A "user" refers to an individual or group that uses the system to learn.

[0111] "Authentication information" refers to identification information used by users to access the system, such as user IDs and passwords.

[0112] A "computer network" refers to infrastructure used for data communication between computer systems, such as the internet and local area networks (LANs).

[0113] A "server" refers to a computer system that receives and processes requests from users.

[0114] A "database" refers to a system that efficiently stores large amounts of information and allows for searching and updating.

[0115] "Interface" refers to the screens and methods of operation that users use to interact with a system.

[0116] "Subject" refers to the specific learning content or theme that the user wants to learn about.

[0117] "Educational support AI" refers to programs and systems that use artificial intelligence technology to support users' learning.

[0118] "Natural language" refers to the language that humans use on a daily basis, which is converted into a format that computer systems can easily understand.

[0119] "Dialogue" refers to two-way communication between a user and a system where information is exchanged bi-directionally.

[0120] "Real-time" refers to the immediate response to user actions and questions.

[0121] "Generating answers" refers to the process of creating appropriate information or answers in response to a user's question.

[0122] "Saving data" refers to recording conversation content and learning progress for future use.

[0123] "Learning progress" refers to information indicating how far a user has progressed in their learning.

[0124] "Learning advice" refers to suggestions and instructions that help users streamline their learning and deepen their understanding.

[0125] This invention provides a system for users to learn online via an internet connection. This system allows users to learn and manage their progress anytime, anywhere, while interacting with an educational support AI. The system consists of the following main components: a user terminal, a server, an educational support AI model, and a database. Furthermore, the entire system can exchange data using a secure communication protocol (e.g., HTTPS).

[0126] Hardware and software to use

[0127] 1. User terminal

[0128] Internet-connected devices such as smartphones, tablets, and personal computers

[0129] Use an application or web browser

[0130] 2. Server

[0131] High-performance computer systems (e.g., Linux® servers)

[0132] Server-side scripts (e.g., Python, Node.js)

[0133] Secure authentication libraries (e.g., JWT)

[0134] 3. Database

[0135] Relational databases such as MySQL®, PostgreSQL, and MongoDB, or NoSQL databases.

[0136] 4. AI Models for Educational Support

[0137] Generative AI models such as GPT-3(registered trademark) and BERT

[0138] Machine learning libraries such as PyTorch and TENSORFLOW®

[0139] The main functions and operation of the system

[0140] 1. User Authentication

[0141] The user enters their authentication information (user ID and password) on the login screen and presses the submit button.

[0142] The device sends these authentication credentials to the server via HTTPS.

[0143] The server compares the received authentication information with the database, and if authentication is successful, it generates a user-specific interface.

[0144] 2. Theme Selection

[0145] The user selects a topic they want to learn from the dashboard (e.g., "Python Programming").

[0146] The terminal sends the selected subject information to the server.

[0147] The server loads and launches the appropriate educational support AI model based on the selected subject.

[0148] 3. Interaction with educational support AI

[0149] The server launches the loaded educational support AI model and generates an initial message (e.g., "Let's start Python programming.").

[0150] The educational support AI will send this message to the user.

[0151] Users can send questions and instructions to the educational support AI while learning.

[0152] The device sends these messages to the server, which then passes them on to the educational support AI.

[0153] The educational support AI generates answers to the user's questions and sends them to the user.

[0154] 4. Recording progress

[0155] The server records the content of the conversation between the educational support AI and the user, as well as the learning progress, in a database.

[0156] This information will be used in the next session.

[0157] Specific example

[0158] 1. The user logs in.

[0159] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[0160] The terminal receives this and sends it to the server.

[0161] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific interface.

[0162] Example of a prompt:

[0163] "A user is attempting to log in using the ID "user123" and password "password". Please explain the procedure."

[0164] 2. The user selects a theme.

[0165] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[0166] The device sends this selection information to the server.

[0167] The server selects and launches an appropriate AI model for educational support.

[0168] Example of a prompt:

[0169] "The user selected 'Python Programming' from the dashboard. Please describe the next server-side actions to take."

[0170] 3. Start the conversation with the educational support AI.

[0171] The educational support AI sends the user a message saying, "Let's start Python programming. First, we'll learn about basic data types."

[0172] The user asks, "How do I use variables?"

[0173] The device sends the question to the server, which then passes it on to the educational support AI.

[0174] The educational support AI responds with "In Python, variables are declared as follows..." and generates a message that includes an example.

[0175] Example of a prompt:

[0176] "The user asked, 'How do I use variables?' Generate an appropriate response that the educational support AI should provide."

[0177] 4. Progress Management

[0178] The server records the conversation content and learning progress in a database.

[0179] In the next session, the server will provide appropriate learning content based on the progress information from the previous session.

[0180] Example of a prompt:

[0181] "Please explain the procedure for recording the user's interaction with the educational support AI and their learning progress in a database."

[0182] This allows users to effectively progress in their learning based on their latest progress and receive appropriate feedback in real time from the educational support AI. In this way, the system of the present invention provides efficient and effective learning support to users.

[0183] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0184] Program processing flow

[0185] Step 1: User Authentication

[0186] 1.1 The user enters their authentication information on the login screen.

[0187] The user enters user ID "user123" and password "password" on the app's login screen and presses the login button.

[0188] Input: User ID and password

[0189] Output: Request to send input information

[0190] 1.2 The device sends authentication information to the server.

[0191] The terminal sends the entered user ID and password to the server via HTTPS.

[0192] Input: Authentication information entered by the user

[0193] Output: Authentication information is sent to the server.

[0194] 1.3 The server performs authentication in the database.

[0195] The server compares the received user ID and password with the database, and if authentication is successful, it generates a user-specific interface.

[0196] Input: Authentication information received by the server

[0197] Data processing: Database matching

[0198] SQL

[0199] SELECT FROM users WHERE userID='user123' AND password='password';

[0200] Output: Authentication results and user-specific interface

[0201] Step 2: Theme Selection

[0202] 2.1 The user selects the topic they want to learn about.

[0203] The user selects "Python Programming" from the list of themes displayed on the dashboard screen.

[0204] Input: Selected theme (Python programming)

[0205] Output: Theme selection submission request

[0206] 2.2 The device sends theme information to the server.

[0207] The terminal sends the selected subject information to the server.

[0208] Input: user selected theme information

[0209] Output: Theme information is sent to the server.

[0210] 2.3 The server loads the instructor AI model.

[0211] The server loads and launches the appropriate educational support AI model based on the selected theme.

[0212] Input: Received theme information

[0213] Data processing: Loading and initializing AI models

[0214] Python

[0215] AI_model = load_model("Python_Programming_Model")

[0216] Output: Activated educational support AI model

[0217] Step 3: Interacting with the Instructor AI

[0218] 3.1 The server starts the instructor AI model.

[0219] The server starts the loaded instructor AI model and generates the initial message "Let's get started with Python programming."

[0220] Input: Loaded AI model

[0221] Data processing: Initial message generation

[0222] Python

[0223] initial_message = AI_model.generate_initial_message()

[0224] Output: Initial message

[0225] 3.2 Instructor AI sends an initial message to the user

[0226] The instructor AI generates an initial message which is then sent to the user.

[0227] Input: Initial message

[0228] Output: Initial message sent to the user

[0229] 3.3 Users submit questions or instructions during learning

[0230] The user enters a question such as "Please tell me how to use variables" and presses the submit button.

[0231] Input: Questions or instructions

[0232] Output: Request to send questions or instructions

[0233] 3.4 The terminal sends the user's question to the server.

[0234] The terminal sends the user's question to the server.

[0235] Input: User's question

[0236] Output: Question sent to the server

[0237] 3.5 The server passes the question to the instructor AI.

[0238] The server passes the user's question received to the educational support AI.

[0239] Input: Question received from the user

[0240] Output: Questions given to the educational support AI

[0241] 3.6 Instructor AI generates answers to questions

[0242] The educational support AI generates an answer such as, "In Python, variables are declared as follows..." and creates a message that includes an example.

[0243] Input: User's question

[0244] Data processing: Answer generation

[0245] Python

[0246] response = AI_model.generate_response("Please tell me how to use the variables")

[0247] Output: Response message

[0248] 3.7 The instructor AI sends the answer to the user.

[0249] The instructor AI generates the answer and sends it to the user.

[0250] Input: Generated answer

[0251] Output: Response sent to the user

[0252] Step 4: Recording progress

[0253] 4.1 The server records the conversation content and learning progress.

[0254] The server records the content of the conversation between the educational support AI and the user, as well as the learning progress, in a database.

[0255] Input: Dialogue content and learning progress

[0256] Data processing: Saving to a database

[0257] SQL

[0258] INSERT INTO progress (userID, content, timestamp) VALUES ('user123', 'Learn how to use variables', NOW());

[0259] Output: Recorded data

[0260] 4.2 Utilize the progress made in the previous session in the next session

[0261] The server retrieves the user's previous learning progress from the database upon their next login and displays the message, "Last time, you learned how to use variables. Do you want to continue?"

[0262] Input: Previous learning progress data

[0263] Data processing: Message generation based on learning progress

[0264] SQL

[0265] SELECT FROM progress WHERE userID='user123' ORDER BY timestamp DESC LIMIT 1;

[0266] Output: Learning progress message sent to the user

[0267] The above describes the processing flow in the system of the present invention. Each step is explained in detail, including specific inputs and outputs, and the data processing and calculations based on them.

[0268] (Application Example 1)

[0269] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0270] Traditional online learning systems have presented challenges such as difficulty for users to track their progress in real time and in obtaining quick answers to questions about the learning content. Furthermore, it was difficult for users to manage their own learning progress and plan their next learning sessions. Additionally, the lack of widespread use of mobile devices such as smartphones has resulted in a lack of convenience.

[0271] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0272] In this invention, the server includes means for the user to input and send authentication information to the server; means for the server to verify the received authentication information and generate a user dashboard if authentication is successful; means for the user to select a topic they wish to learn and send that selection information to the server; means for the server to activate an instructor AI based on the selected topic and begin a dialogue with the user; means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session; and means for the user to interact with the instructor AI in real time using natural language with a smartphone and provide responses to questions using prompt sentences generated by the AI. This allows the user to grasp their learning progress in real time and obtain quick answers, improving the convenience of learning using a smartphone.

[0273] A "user" is someone who uses the system to learn online.

[0274] "Authentication information" refers to information used to identify an individual, such as a user ID and password, which a user enters when accessing a system.

[0275] A "server" is a computer system that receives authentication information from users, performs searches, and performs authentication.

[0276] The "user dashboard" is an interface generated for users who have successfully authenticated, displaying their learning progress, selected learning themes, and other relevant information.

[0277] A "theme" is information that indicates the content or field that the user wants to learn about.

[0278] "Instructor AI" is an artificial intelligence system that is activated based on the theme selected by the user, interacts with the user, and generates answers in response to questions.

[0279] "Natural language" refers to the languages that people use in daily life and are used for users to interact with the instructor AI.

[0280] "Prompt text" refers to a series of texts generated by the instructor AI in response to a user's question and provided as an answer.

[0281] "Smartphone" refers to a portable multifunctional phone that users use as a handheld device to access and interact with the system.

[0282] "Real-time" refers to the immediate state where, when a user asks a question to the instructor AI, an answer is returned immediately on the spot.

[0283] This invention is a system for users to learn online while interacting with the instructor AI in real time using a smartphone. This system consists of a user terminal, a server, an instructor AI model, and a database. As a result, users can obtain appropriate learning advice and quick responses.

[0284] Configuration and operation of the system

[0285] Use of hardware and software

[0286] User terminal: Uses a smartphone. It is a device for users to input authentication information, select a theme, interact with the instructor AI, etc.

[0287] Server: It is a central device that verifies authentication information, activates the instructor AI, records the interaction content, and prepares the next learning content.

[0288] Instructor AI model: An artificial intelligence that is activated based on the user's learning theme, interacts with the user in natural language, and generates prompt text for questions.

[0289] Database: A storage device that records user authentication information, learning progress information, dialogue content, etc.

[0290] Specific operating procedures and functions

[0291] 1. User Authentication

[0292] The user launches the app on their smartphone and enters their authentication information (user ID and password) on the login screen.

[0293] The user terminal sends this authentication information to the server, and the server performs authentication by comparing the received authentication information with the database.

[0294] If authentication is successful, the server will generate a user dashboard.

[0295] 2. Theme Selection

[0296] The user selects the topic they want to learn from the dashboard. For example, they might select "Python Programming".

[0297] The user terminal sends the selected theme information to the server, and the server loads the corresponding instructor AI model.

[0298] 3. Interaction with the Instructor AI

[0299] The server starts the loaded instructor AI model and generates an initial message. For example, it might generate a message like, "Let's get started with Python programming. We'll begin by learning basic data types."

[0300] The user sends a question to the instructor AI during the learning process (e.g., "Please explain how to use variables").

[0301] The user's terminal sends this question to the server, which then passes it on to the instructor AI.

[0302] The instructor AI generates an answer to the question (e.g., "In Python, variables are declared as follows...") and sends the answer to the user terminal.

[0303] 4. Recording of progress and preparation for the next learning

[0304] The server records the interaction content between the instructor AI and the user and the learning progress in the database.

[0305] At the next session, the server prepares appropriate learning content based on the previous progress information, and the instructor AI provides the next learning content.

[0306] Specific examples of prompt sentences

[0307] Login with user ID and password: 'user123', 'password'

[0308] Select the theme to learn: 'Python programming'

[0309] Ask a question: 'Please teach me how to use variables'

[0310] Save the progress

[0311] In this way, the present invention enables the user to efficiently and effectively proceed with learning while interacting with the instructor AI in real time using a smartphone.

[0312] The flow of the specific process in Application Example 1 will be described with reference to FIG. 12.

[0313] Step 1:

[0314] User authentication [[ID=五十一]]

[0315] The user launches the smartphone app and enters the authentication information (user ID and password).

[0316] The device sends this authentication information to the server. The entered data is the user ID and password.

[0317] The server compares the received authentication information with the database. If authentication is successful, it generates a user-specific dashboard and sends it to the user's terminal. The user's profile information is output upon successful authentication. An error message is output if authentication fails.

[0318] Step 2:

[0319] Theme Selection

[0320] The user selects a learning topic (for example, "Python Programming") from the dashboard.

[0321] The terminal sends the selected information (theme name) to the server. The entered data is the selected theme name.

[0322] The server loads and launches the appropriate instructor AI model from the database based on the selected theme. It also sends an initial message to the user's terminal based on the selected theme (e.g., "Let's start Python programming.").

[0323] Step 3:

[0324] Dialogue with the Instructor AI

[0325] The user enters and submits a question during the learning process (e.g., "Please explain how to use variables").

[0326] The terminal sends the question to the server. The input data is the user's question.

[0327] The server passes the question to the instructor AI, which then generates an answer. The answer includes a generated prompt (e.g., "In Python, variables are declared as follows...").

[0328] The server sends the instructor AI's response to the user's terminal and displays it to the user.

[0329] Step 4:

[0330] Progress log

[0331] The server records the content of the conversation between the instructor AI and the user, as well as the learning progress, in a database. The input data consists of the conversation content and learning progress information.

[0332] We will save this data to use in the next session and prepare appropriate learning content for the next session.

[0333] Step 5:

[0334] Preparation for the next lesson

[0335] The server automatically prepares the next learning content based on the learning progress information stored. The input data is past learning progress information.

[0336] Based on the learning content prepared by the server, the instructor AI generates scenarios and prompts to provide appropriate learning advice, and saves them for use in the next learning session.

[0337] In this way, users can engage in online learning more efficiently and effectively through the system.

[0338] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0339] This invention is a system for users to learn online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the learning experience. This system allows users to learn anytime, anywhere, while interacting with an instructor AI, and to manage their progress and emotions.

[0340] This system consists of the following main components: user terminals, servers, instructor AI models, emotion engines, and databases. The entire system uses secure communication protocols (e.g., HTTPS) to exchange data.

[0341] The main functions and operation of the system

[0342] 1. User Authentication

[0343] The user accesses the login screen and enters their authentication information (user ID and password). This information is then sent from the terminal to the server.

[0344] The server compares the received authentication information with the database and, if authentication is successful, generates a user-specific dashboard.

[0345] 2. Theme Selection

[0346] The user selects a topic they want to learn about on the dashboard (e.g., "Python Programming").

[0347] The device sends the selected theme information to the server.

[0348] The server receives the theme information and loads the corresponding instructor AI model.

[0349] 3. Interaction with the Instructor AI

[0350] The server starts the loaded instructor AI model and generates an initial message.

[0351] The instructor AI will send this message to the user.

[0352] Users can send questions and instructions to the instructor AI while learning.

[0353] The device sends these messages to the server, which then passes them on to the instructor AI.

[0354] The instructor AI generates answers to the user's questions and sends them to the user.

[0355] 4. Emotion recognition

[0356] The server passes user input data (e.g., text messages) to the emotion engine, which recognizes the user's emotions in real time.

[0357] The emotion engine recognizes the emotion information, which the server then provides to the instructor AI.

[0358] The instructor AI generates responses by considering emotional information and adjusts the dialogue according to the user's emotions.

[0359] 5. Record progress and emotions

[0360] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[0361] This information will be used in the next session.

[0362] Explanation with specific examples

[0363] 1. The user logs in.

[0364] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[0365] The terminal receives this and sends it to the server.

[0366] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific dashboard.

[0367] 2. The user selects a theme.

[0368] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[0369] The device sends this selection information to the server.

[0370] The server selects and starts the appropriate instructor AI model.

[0371] 3. Start the conversation with the instructor AI.

[0372] The instructor AI generates an initial message saying, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user.

[0373] The user asks, "How do I use variables?"

[0374] The device sends the question to the server, which then passes it on to the instructor AI.

[0375] The instructor AI responds with "In Python, variables are declared like this..." and generates a message that includes an example.

[0376] 4. Emotion Recognition and Adaptation

[0377] The server uses an emotion engine to recognize the user's emotions (e.g., confusion, relief) based on the content of the user's questions.

[0378] The emotion engine recognizes the emotions, which the server then provides to the instructor AI.

[0379] The instructor AI adjusts its response based on the user's emotions, saying something like, "You seem confused. Let me explain in more detail," and sends it to the user.

[0380] 5. Managing progress and emotions

[0381] The server records the conversation content, learning progress, and recognized emotion information in a database.

[0382] In the next session, dialogue will be provided that takes into account not only what was learned in the previous session, but also the recognized emotional information.

[0383] In this way, by monitoring the user's learning experience in real time and providing optimal feedback based on their emotions, the quality and efficiency of learning can be improved. With such a system, users can always receive appropriate learning support, making it easier to maintain motivation for self-study.

[0384] The following describes the processing flow.

[0385] Step 1:

[0386] The user accesses the login screen and enters their authentication information (user ID and password). The user then clicks the "Login" button.

[0387] Step 2:

[0388] The terminal encodes the entered authentication information and sends it to the server using a secure communication protocol (HTTPS).

[0389] Step 3:

[0390] The server compares the received authentication information with the database. If the user ID and password match, authentication is considered successful.

[0391] Step 4:

[0392] The server generates a user-specific dashboard page along with an authentication success message and sends it to the device. If authentication fails, an error message is sent.

[0393] Step 5:

[0394] The user accesses the dashboard page and selects a topic they want to learn about from the provided list of topics (e.g., "Python Programming").

[0395] Step 6:

[0396] The device sends the selected theme information to the server.

[0397] Step 7:

[0398] The server receives the theme information and loads the corresponding instructor AI model. Once loading is complete, it generates an initial message.

[0399] Step 8:

[0400] The instructor AI generates an initial message, "Let's start Python programming," and sends it to the user.

[0401] Step 9:

[0402] The user enters questions or requests during the learning process, and the device sends them to the server.

[0403] Step 10:

[0404] The server passes the user's questions and requests to the instructor AI. The instructor AI then generates answers to those questions.

[0405] Step 11:

[0406] The instructor AI returns the generated answer to the server, which then sends that answer to the user.

[0407] Step 12:

[0408] The server passes user input data to the emotion engine, which recognizes the user's emotions in real time.

[0409] Step 13:

[0410] The emotion engine recognizes emotions from the user's input data and provides that emotion information to the server.

[0411] Step 14:

[0412] The server passes the user's emotional information, obtained from the emotion engine, to the instructor AI. The instructor AI then adjusts its response based on that emotional information.

[0413] Step 15:

[0414] The instructor AI generates emotion-based responses, creating messages such as, "You seem confused. Let me explain in more detail," and sends them to the user.

[0415] Step 16:

[0416] Users receive emotionally sensitive responses and continue learning. If they have additional questions or requests, they can enter them and submit again.

[0417] Step 17:

[0418] When a learning session ends, the server records the user-instructor AI dialogue, learning progress, and recognized emotion information in a database.

[0419] Step 18:

[0420] The server uses recorded progress data and sentiment information to prepare the content for the next learning session. In the next session, new themes and problems will be presented, taking into account the progress and sentiment information from the previous session.

[0421] The above outlines the specific processing steps of the system based on the invention that combines an emotion engine. This system allows users to effectively learn by interacting with an instructor AI anytime, anywhere, and to receive emotion-sensitive feedback.

[0422] (Example 2)

[0423] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0424] Traditional online learning systems often struggle to provide immediate and appropriate responses to user questions, potentially leading to decreased learning efficiency. Furthermore, they fail to optimize individual learning experiences by providing uniform responses without considering user emotions. Additionally, a lack of effective methods for reflecting learning progress in subsequent sessions hinders the overall improvement of the learning experience.

[0425] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the user to input and transmit authentication information to the server, means for the server to verify the received authentication information and generate a user dashboard if authentication is successful, means for the user to select a theme they want to learn and transmit the selection information to the server, means for the server to activate an instructional AI based on the selected theme and start a dialogue with the user, means for the server to pass the user's input data to an analysis engine and recognize the user's emotions in real time, means for the server to provide the recognized emotion information to the instructor AI and generate a response, and means for the server to record the content of the dialogue with the instructional AI and save the data for use in the next session. This enables the generation of appropriate responses in real time that take into account the user's emotions and the effective preparation of the next learning content according to the learning progress.

[0426] "User authentication information" refers to information such as the user ID and password that a user uses to log in to the system.

[0427] A "server" is a computer system that receives requests from users on a network and processes them accordingly.

[0428] The "user dashboard" is an interface that users can access after logging in, where they can check their learning progress, select topics, and more.

[0429] A "theme" refers to a specific learning topic or subject that a user wants to study.

[0430] "Teaching AI" refers to a model that uses artificial intelligence to provide educational guidance to users, including natural language dialogue and learning support.

[0431] An "analysis engine" is a collection of software and hardware that analyzes user input data and identifies specific information (e.g., emotions).

[0432] "Emotional information" refers to information that indicates the user's emotional state, extracted from the user's input data by the analysis engine.

[0433] "Dialogue content" refers to the record of texts and messages exchanged between the user and the instructional AI.

[0434] A "data storage method" refers to a means by which a server records dialogue content and learning progress information and stores it for use in the next session.

[0435] Modes for carrying out the invention

[0436] This invention is a system for users to learn online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the learning experience. The main components of the system are a user terminal, a server, an instructional AI model, an analysis engine, and a database. The entire system exchanges data using a secure communication protocol (e.g., HTTPS).

[0437] System Overview

[0438] This system consists of the following main components:

[0439] 1. User terminal:

[0440] This refers to the device that the user accesses, such as a smartphone, tablet, or personal computer.

[0441] 2. Server:

[0442] This is a computer system that performs user authentication, data analysis, manages the AI ​​for instruction, and communicates with the database.

[0443] 3. Instructional AI Model:

[0444] It is an artificial intelligence that interacts with users, using natural language processing to generate responses to user questions.

[0445] 4. Analysis Engine:

[0446] This part analyzes user input data (e.g., text messages) in real time to recognize emotions and intentions.

[0447] 5. Database:

[0448] This is a storage system that stores user authentication information, learning progress, conversation content, and sentiment data.

[0449] System operation

[0450] This system operates using the following steps:

[0451] 1. User Authentication:

[0452] The user accesses the login screen and enters their user ID and password. The entered information is sent from the terminal to the server.

[0453] The server verifies the authentication information against the database, and if authentication is successful, it generates a user-specific dashboard.

[0454] 2. Theme Selection:

[0455] The user selects the topic they want to learn about on the dashboard.

[0456] The device sends the selected theme information to the server, and the server loads the corresponding instructional AI model.

[0457] 3. Interaction with the instructional AI:

[0458] The server activates the instructional AI model and generates an initial message. The instructional AI then sends this message to the user.

[0459] The user enters and sends questions or instructions. The device sends these messages to the server, which then passes them on to the instructional AI.

[0460] The instructional AI generates answers to the user's questions and sends them to the user.

[0461] 4. Emotion recognition:

[0462] The server passes user input data to an analysis engine, which recognizes the user's emotions in real time.

[0463] The emotion engine recognizes emotional information, which the server then provides to the guidance AI. The guidance AI then generates a response, taking this emotional information into consideration.

[0464] 5. Recording progress and emotions:

[0465] The server records the content of the interaction between the teaching AI and the user, as well as learning progress and recognized emotion information, in a database.

[0466] This information will be used in the next session.

[0467] Hardware and software to be used

[0468] User terminal:

[0469] Smartphones, tablets, or personal computers (e.g., iPhone®, iPad®, Windows PC)

[0470] server:

[0471] Cloud services or on-premises computer systems (e.g., AWS®, Microsoft® Azure®)

[0472] Instructional AI model:

[0473] Generative AI models that use natural language processing (e.g., GPT-3)

[0474] Analysis engine:

[0475] Software that analyzes user emotions (e.g., IBM Watson® Tone Analyzer)

[0476] Database:

[0477] SQL database or NoSQL database (e.g., MySQL, MongoDB)

[0478] Specific example

[0479] The user launches the smartphone app, enters the ID "user123" and password "password" on the login screen, and presses the login button. The device receives this information and sends it to the server. The server verifies the authentication information against the database, and if authentication is successful, displays a user-specific dashboard. The user selects "Python Programming" on the dashboard screen, and the device sends this selection information to the server. The server selects an appropriate teaching AI model and starts it. The teaching AI generates an initial message, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user. The user enters a question, "Please teach me how to use variables," and presses the send button. The device sends the question to the server, which passes it to the teaching AI. The teaching AI responds, "In Python, variables are declared as follows...", and generates a concrete example. The server uses an emotion engine to recognize the user's emotions (e.g., confusion, relief) from the user's question. The server provides the emotions recognized by the emotion engine to the teaching AI, which adjusts its response based on the emotions and sends it to the user. The server records the dialogue content, learning progress, and recognized emotion information in the database and uses this information in the next session.

[0480] In this way, by monitoring the user's learning experience in real time and providing optimal feedback based on their emotions, the quality and efficiency of learning can be improved.

[0481] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0482] Step 1:

[0483] The user enters and submits their login information (user ID and password). The terminal receives this information and sends it to the server. The input is the user ID and password, and the output is the login request sent to the server.

[0484] Step 2:

[0485] The server compares the received authentication information with the database. Data processing here involves extracting user authentication information from the database using SQL queries and comparing it with the input information. The output is whether the authentication was successful or not.

[0486] Step 3:

[0487] If authentication is successful, the server generates a user-specific dashboard and sends the information to the device. The output is data for displaying the dashboard, which is then sent to the device. The device receives this data and displays the dashboard to the user.

[0488] Step 4:

[0489] The user selects a topic they want to learn about on the dashboard and submits the topic information. The device receives this information and sends it to the server. The input is the topic selected by the user, and the output is the topic information sent to the server.

[0490] Step 5:

[0491] The server loads the corresponding instructional AI model based on the received theme information. Here, data processing involves loading the AI ​​model corresponding to the theme. The output is the initialization and startup of the AI ​​model.

[0492] Step 6:

[0493] The instructional AI generates an initial message and sends it to the terminal via the server. The input is the loaded AI model and the prompt for generating the initial message, and the output is the initial message sent to the user. The terminal displays this message to the user.

[0494] Step 7:

[0495] The user inputs questions or instructions and sends that information. The terminal receives this message and sends it to the server. The input is the user's questions or instructions, and the output is the user input data sent to the server.

[0496] Step 8:

[0497] The server passes user input data to the guidance AI, which then generates an answer. The input consists of the user's questions and instructions, while the output is the generated answer message.

[0498] Step 9:

[0499] The server sends the generated response message to the terminal, which then displays it to the user. The input is the response message from the AI, and the output is the message displayed to the user.

[0500] Step 10:

[0501] The server passes user input data to the analysis engine, which recognizes emotions in real time. The data processing here involves natural language processing for emotion analysis. The input is user input data, and the output is the recognized emotion information.

[0502] Step 11:

[0503] The emotion engine recognizes emotional information, which the server then provides to the guidance AI. Based on this information, the AI ​​generates a response. The input is emotional information, and the output is a response message that takes that emotional information into account.

[0504] Step 12:

[0505] The server records the content of the interaction between the teaching AI and the user, as well as learning progress and recognized emotion information, in a database. The input is the content of the interaction, learning progress, and emotion information, and the output is the information stored in the database.

[0506] Step 13:

[0507] In the next session, the server will have the instructional AI generate a customized learning plan based on the recorded information. The input will be the content of the previous conversation and emotional information, and the output will be the customized learning plan. The user will then start learning again based on this information.

[0508] In this way, by analyzing user sentiment information and learning progress in real time and providing appropriate feedback, the learning experience is improved.

[0509] (Application Example 2)

[0510] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0511] Conventional instructor AI systems failed to recognize the user's emotional state and could only provide uniform dialogue, resulting in a decline in the quality of the learning experience. Similarly, in factory robots, the inability to provide feedback and instructions that considered the operator's emotions posed a challenge to work efficiency and safety.

[0512] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input and transmit authentication information to the server; means for the server to verify the received authentication information and generate a user dashboard if authentication is successful; means for the user to select a theme they wish to learn and transmit the selection information to the server; means for the server to activate an instructor AI based on the selected theme and begin a dialogue with the user; means for the user to recognize their emotions and for the instructor AI to adjust the content of the dialogue based on those emotions; means installed in a robot to recognize the operator's emotional state and provide appropriate feedback and instructions; and means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session. This makes it possible to provide adjusted dialogue and instructions according to the emotional state of the user and operator.

[0513] "User authentication" is the process by which a user enters authentication information, sends it to a server, and verifies it.

[0514] A "server" is a computer system that receives user authentication information and selection information and processes it based on that information.

[0515] A "user dashboard" is a screen that the server generates upon successful authentication and allows the user to access it.

[0516] An "instructor AI" is an artificial intelligence model that interacts with users and operators, providing answers to questions and instructions.

[0517] "Emotion recognition" is the process of analyzing user or operator input data (voice, text, facial expressions, etc.) to identify emotional states.

[0518] "Dialogue content adjustment" refers to the instructor AI appropriately modifying the content of the dialogue based on recognized emotions.

[0519] A "robot" is a mechanical device that performs tasks within a factory, interacting with an operator and providing work instructions.

[0520] "Feedback" refers to the responses and instructions that the instructor AI provides to the user or operator.

[0521] A "database" is a data storage system where a server records conversation content, progress, emotional information, and other data for use in subsequent sessions.

[0522] This invention is a system for users to learn and work online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the experience. The system consists of the following main components: a user terminal, a server, an instructor AI model, an emotion engine, and a database. The entire system exchanges data using a secure communication protocol (e.g., HTTPS).

[0523] The main functions and operation of the system

[0524] 1. User Authentication

[0525] The user accesses the login screen and enters their authentication information (user ID and password). This information is then sent from the terminal to the server.

[0526] The server compares the received authentication information with the database and, if authentication is successful, generates a user-specific dashboard.

[0527] 2. Theme Selection

[0528] Users select a topic they want to learn about on the dashboard (e.g., "Python Programming").

[0529] The device sends the selected theme information to the server.

[0530] The server receives the theme information and loads the corresponding instructor AI model.

[0531] 3. Interaction with the Instructor AI

[0532] The server starts the loaded instructor AI model and generates an initial message.

[0533] The instructor AI will send this message to the user.

[0534] Users can send questions and instructions to the instructor AI while learning.

[0535] The device sends these messages to the server, which then passes them on to the instructor AI.

[0536] The instructor AI generates answers to the user's questions and sends them to the user.

[0537] 4. Emotion recognition

[0538] The server passes user input data (e.g., text messages) to the emotion engine, which recognizes the user's emotions in real time.

[0539] The emotion engine recognizes the emotion information, which the server then provides to the instructor AI.

[0540] The instructor AI generates responses by considering emotional information and adjusts the dialogue according to the user's emotions.

[0541] 5. Record progress and emotions

[0542] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[0543] This information will be used in the next session.

[0544] Hardware and software to be used

[0545] Hardware:

[0546] Camera (e.g., Logitech C920): Captures the user's facial expressions.

[0547] Microphone: Captures the user's voice.

[0548] software:

[0549] OpenCV: Used for inputting and processing camera images.

[0550] Emotion Recognition Library (e.g., emotion_recognition): Analyzes the user's emotions.

[0551] AI Model: As an instructor AI, it is a model that generates responses based on the task content (e.g., natural language processing model ChatGPT®).

[0552] Database (e.g., MySQL): Manages login information, work details, and sentiment information.

[0553] Specific example

[0554] Specific example 1:

[0555] The user launches the smartphone app, enters the ID "user123" and password "password" on the login screen, and presses the login button. The device receives this information and sends it to the server. The server verifies it against the database, and if authentication is successful, displays a user-specific dashboard.

[0556] The user selects "Python Programming" in the dashboard and sends this selection information to the server. The server receives this information and activates the instructor AI.

[0557] The instructor AI generates an initial message saying, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user. When the user asks, "How do I use variables?", the terminal sends the question to the server, and the instructor AI generates an answer.

[0558] The server passes the user's question to the emotion engine, which recognizes the user's emotions. If the emotion engine recognizes "confusion," the server provides this information to the instructor AI, which responds with, "You seem confused. Let me explain in more detail."

[0559] The server records the conversation content, progress, and emotional information in a database, which will be used in the next session.

[0560] Example of a prompt:

[0561] If an operator appears confused while tightening a bolt, recognize their confusion and provide a clear and detailed explanation.

[0562] This makes it possible to provide users and operators with adjusted dialogue and instructions that are tailored to their emotional state.

[0563] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0564] Program processing steps

[0565] Step 1:

[0566] The user launches the app on their smartphone and enters their user ID and password on the login screen. This generates the input data for user authentication.

[0567] Step 2:

[0568] The terminal encrypts the user authentication input data and sends it to the server. The server compares the received authentication information with the database, and if authentication is successful, generates a user-specific dashboard and sends it back to the terminal.

[0569] Step 3:

[0570] The user selects a topic they want to learn about (e.g., "Python Programming") within the dashboard, and the device sends this selection information to the server. This generates the input data for the topic selection.

[0571] Step 4:

[0572] The server analyzes the theme information and loads the appropriate instructor AI model. The instructor AI generates an initial message and sends it to the terminal.

[0573] Step 5:

[0574] The user inputs a question to the instructor AI, and the device sends the question to the server. The server passes the question to the instructor AI, which generates an answer and sends it back.

[0575] Step 6:

[0576] The server passes the user's questions and the instructor AI's answers to the emotion engine, which recognizes the user's emotional state in real time. This process generates emotion recognition data.

[0577] Step 7:

[0578] The emotion engine recognizes emotional information, which the server then provides to the instructor AI. The instructor AI adjusts the content of the dialogue. The server then sends the adjusted response generated by the AI ​​to the terminal.

[0579] Step 8:

[0580] Review the user's response and either ask the next question or continue the learning process.

[0581] Step 9:

[0582] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[0583] Step 10:

[0584] In the next session, the server will customize the instructor AI model based on past conversations and progress information to provide appropriate learning advice.

[0585] This makes it possible to provide adjusted dialogue and instructions that respond to various emotional states, such as when a user is confused or agitated.

[0586] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0587] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0588] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0589] [Second Embodiment]

[0590] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0591] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0592] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0593] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0594] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0595] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0596] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0597] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0598] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0599] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0600] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0601] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0602] This invention is a system for users to learn online via an internet connection. This system allows users to learn anytime, anywhere, while interacting with an AI instructor, and to manage their progress.

[0603] This system consists of the following main components: user terminals, servers, instructor AI models, and databases. The entire system also uses secure communication protocols (e.g., HTTPS) for data exchange.

[0604] The main functions and operation of the system

[0605] 1. User Authentication

[0606] The user enters their authentication information (user ID and password) on the login screen and presses the submit button.

[0607] The device sends these authentication credentials to the server.

[0608] The server compares the received authentication information with the database, and if authentication is successful, it generates a user-specific dashboard.

[0609] 2. Theme Selection

[0610] The user selects a topic they want to learn about from the dashboard (e.g., "Python Programming").

[0611] The device sends the selected theme information to the server.

[0612] The server loads the appropriate instructor AI model based on the selected theme.

[0613] 3. Interaction with the Instructor AI

[0614] The server starts the loaded instructor AI model and generates an initial message (e.g., "Let's get started with Python programming.").

[0615] The instructor AI will send this message to the user.

[0616] Users can send questions and instructions to the instructor AI while learning.

[0617] The device sends these messages to the server, which then passes them on to the instructor AI.

[0618] The instructor AI generates answers to the user's questions and sends them to the user.

[0619] 4. Recording progress

[0620] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database.

[0621] This information will be used in the next session.

[0622] Explanation with specific examples

[0623] 1. The user logs in.

[0624] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[0625] The terminal receives this and sends it to the server.

[0626] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific dashboard.

[0627] 2. The user selects a theme.

[0628] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[0629] The device sends this selection information to the server.

[0630] The server selects and starts the appropriate instructor AI model.

[0631] 3. Start the conversation with the instructor AI.

[0632] The instructor AI sends the user a message saying, "Let's start Python programming. First, we'll learn about basic data types."

[0633] The user asks, "How do I use variables?"

[0634] The device sends the question to the server, which then passes it on to the instructor AI.

[0635] The instructor AI responds with "In Python, variables are declared like this..." and generates a message that includes an example.

[0636] 4. Progress Management

[0637] The server records the conversation content and learning progress in a database.

[0638] In the next session, the server will provide appropriate learning content based on the progress information from the previous session.

[0639] This allows users to always progress in their learning based on their latest learning progress and receive appropriate feedback from the instructor AI in real time. In this way, the system of the present invention provides efficient and effective learning support to users.

[0640] The following describes the processing flow.

[0641] Step 1:

[0642] The user accesses the login screen and enters their authentication information (user ID and password). The user then clicks the "Login" button.

[0643] Step 2:

[0644] The terminal encodes the entered authentication information and sends it to the server using a secure communication protocol (HTTPS).

[0645] Step 3:

[0646] The server compares the received authentication information with the database. If the user ID and password match, authentication is considered successful.

[0647] Step 4:

[0648] The server generates a user-specific dashboard page along with an authentication success message and sends it to the device. If authentication fails, an error message is sent.

[0649] Step 5:

[0650] The user accesses the dashboard page and selects a topic they want to learn about from the provided list of topics (e.g., "Python Programming").

[0651] Step 6:

[0652] The device sends the selected theme information to the server.

[0653] Step 7:

[0654] The server receives the theme information and loads the corresponding instructor AI model. Once loading is complete, it generates an initial message.

[0655] Step 8:

[0656] The instructor AI generates an initial message, "Let's start Python programming," and sends it to the user.

[0657] Step 9:

[0658] The user enters questions or requests while learning, and the device sends them to the server.

[0659] Step 10:

[0660] The server passes the user's questions and requests to the instructor AI. The instructor AI then generates answers to those questions.

[0661] Step 11:

[0662] The instructor AI returns the generated answer to the server, which then sends that answer to the user.

[0663] Step 12:

[0664] The user receives answers from the instructor AI and continues learning. If there are any additional questions or requests, they can enter them again.

[0665] Step 13:

[0666] When a learning session ends, the server records the content of the user-instructor AI conversation and the learning progress in a database.

[0667] Step 14:

[0668] The server uses the recorded progress data to prepare the content for the next learning session. In the next session, new themes and problems will be provided based on the progress made in the previous session.

[0669] The above explains the program's processing in detail, step by step. This system allows users to effectively learn by interacting with the instructor AI anytime, anywhere.

[0670] (Example 1)

[0671] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0672] Traditional online learning systems lacked sufficient interactive learning support based on themes selected by learners. Furthermore, managing learning progress and preparing future learning plans required manual intervention, highlighting the need for more efficient learning support. Additionally, there was a lack of systems that could provide appropriate real-time answers to learners' questions. Therefore, the development of a system that provides more effective and interactive learning support was essential.

[0673] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0674] In this invention, the server includes means for the user to input authentication information and transmit it to the server via a computer network; means for the server to compare the received authentication information with a database and generate a user-specific interface if authentication is successful; means for the user to select a subject they wish to learn and transmit that selection information to the server; means for the server to load an educational support AI based on the selected subject and initiate a dialogue with the user in natural language; means for the educational support AI to generate answers to the user's questions in real time and transmit them to the user; means for the server to record the content of the dialogue with the educational support AI in a database and save the data for use in the next session; and means for the server to prepare the content of the next learning session based on the previous learning progress and for the educational support AI to provide appropriate learning advice as needed. As a result, the user can always receive appropriate learning support based on their latest learning progress.

[0675] A "user" refers to an individual or group that uses the system to learn.

[0676] "Authentication information" refers to identification information used by users to access the system, such as user IDs and passwords.

[0677] A "computer network" refers to infrastructure used for data communication between computer systems, such as the internet and local area networks (LANs).

[0678] A "server" refers to a computer system that receives and processes requests from users.

[0679] A "database" refers to a system that efficiently stores large amounts of information and allows for searching and updating.

[0680] "Interface" refers to the screens and methods of operation that users use to interact with a system.

[0681] "Subject" refers to the specific learning content or theme that the user wants to learn about.

[0682] "Educational support AI" refers to programs and systems that use artificial intelligence technology to support users' learning.

[0683] "Natural language" refers to the language that humans use on a daily basis, which is converted into a format that computer systems can easily understand.

[0684] "Dialogue" refers to two-way communication between a user and a system where information is exchanged bi-directionally.

[0685] "Real-time" refers to the immediate response to user actions and questions.

[0686] "Generating answers" refers to the process of creating appropriate information or answers in response to a user's question.

[0687] "Saving data" refers to recording conversation content and learning progress for future use.

[0688] "Learning progress" refers to information indicating how far a user has progressed in their learning.

[0689] "Learning advice" refers to suggestions and instructions that help users streamline their learning and deepen their understanding.

[0690] This invention provides a system for users to learn online via an internet connection. This system allows users to learn and manage their progress anytime, anywhere, while interacting with an educational support AI. The system consists of the following main components: a user terminal, a server, an educational support AI model, and a database. Furthermore, the entire system can exchange data using a secure communication protocol (e.g., HTTPS).

[0691] Hardware and software to use

[0692] 1. User terminal

[0693] Internet-connected devices such as smartphones, tablets, and personal computers

[0694] Use an application or web browser

[0695] 2. Server

[0696] High-performance computer systems (e.g., Linux servers)

[0697] Server-side scripts (e.g., Python, Node.js)

[0698] Secure authentication libraries (e.g., JWT)

[0699] 3. Database

[0700] Relational databases such as MySQL, PostgreSQL, and MongoDB, or NoSQL databases.

[0701] 4. AI Models for Educational Support

[0702] Generative AI models such as GPT-3 and BERT

[0703] Machine learning libraries such as PyTorch and TensorFlow

[0704] The main functions and operation of the system

[0705] 1. User Authentication

[0706] The user enters their authentication information (user ID and password) on the login screen and presses the submit button.

[0707] The device sends these authentication credentials to the server via HTTPS.

[0708] The server compares the received authentication information with the database, and if authentication is successful, it generates a user-specific interface.

[0709] 2. Theme Selection

[0710] The user selects a topic they want to learn from the dashboard (e.g., "Python Programming").

[0711] The terminal sends the selected subject information to the server.

[0712] The server loads and launches the appropriate educational support AI model based on the selected subject.

[0713] 3. Interaction with educational support AI

[0714] The server launches the loaded educational support AI model and generates an initial message (e.g., "Let's start Python programming.").

[0715] The educational support AI will send this message to the user.

[0716] Users can send questions and instructions to the educational support AI while learning.

[0717] The device sends these messages to the server, which then passes them on to the educational support AI.

[0718] The educational support AI generates answers to the user's questions and sends them to the user.

[0719] 4. Recording progress

[0720] The server records the content of the conversation between the educational support AI and the user, as well as the learning progress, in a database.

[0721] This information will be used in the next session.

[0722] Specific example

[0723] 1. The user logs in.

[0724] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[0725] The terminal receives this and sends it to the server.

[0726] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific interface.

[0727] Example of a prompt:

[0728] "A user is attempting to log in using the ID "user123" and password "password". Please explain the procedure."

[0729] 2. The user selects a theme.

[0730] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[0731] The device sends this selection information to the server.

[0732] The server selects and launches an appropriate AI model for educational support.

[0733] Example of a prompt:

[0734] "The user selected 'Python Programming' from the dashboard. Please describe the next server-side actions to take."

[0735] 3. Start the conversation with the educational support AI.

[0736] The educational support AI sends the user a message saying, "Let's start Python programming. First, we'll learn about basic data types."

[0737] The user asks, "How do I use variables?"

[0738] The device sends the question to the server, which then passes it on to the educational support AI.

[0739] The educational support AI responds with "In Python, variables are declared as follows..." and generates a message that includes an example.

[0740] Example of a prompt:

[0741] "The user asked, 'How do I use variables?' Generate an appropriate response that the educational support AI should provide."

[0742] 4. Progress Management

[0743] The server records the conversation content and learning progress in a database.

[0744] In the next session, the server will provide appropriate learning content based on the progress information from the previous session.

[0745] Example of a prompt:

[0746] "Please explain the procedure for recording the user's interaction with the educational support AI and their learning progress in a database."

[0747] This allows users to effectively progress in their learning based on their latest progress and receive appropriate feedback from the educational support AI in real time. In this way, the system of the present invention provides efficient and effective learning support to users.

[0748] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0749] Program processing flow

[0750] Step 1: User Authentication

[0751] 1.1 The user enters their authentication information on the login screen.

[0752] The user enters user ID "user123" and password "password" on the app's login screen and presses the login button.

[0753] Input: User ID and password

[0754] Output: Request to send input information

[0755] 1.2 The device sends authentication information to the server.

[0756] The terminal sends the entered user ID and password to the server via HTTPS.

[0757] Input: Authentication information entered by the user

[0758] Output: Authentication information is sent to the server.

[0759] 1.3 The server performs authentication in the database.

[0760] The server compares the received user ID and password with the database, and if authentication is successful, it generates a user-specific interface.

[0761] Input: Authentication information received by the server

[0762] Data processing: Database matching

[0763] sql

[0764] SELECT FROM users WHERE userID='user123' AND password='password';

[0765] Output: Authentication results and user-specific interface

[0766] Step 2: Theme Selection

[0767] 2.1 The user selects the topic they want to learn about.

[0768] The user selects "Python Programming" from the list of themes displayed on the dashboard screen.

[0769] Input: Selected theme (Python programming)

[0770] Output: Theme selection submission request

[0771] 2.2 The device sends theme information to the server.

[0772] The terminal sends the selected subject information to the server.

[0773] Input: user selected theme information

[0774] Output: Theme information is sent to the server.

[0775] 2.3 The server loads the instructor AI model.

[0776] The server loads and launches the appropriate educational support AI model based on the selected theme.

[0777] Input: Received theme information

[0778] Data processing: Loading and initializing AI models

[0779] Python

[0780] AI_model = load_model("Python_Programming_Model")

[0781] Output: Activated educational support AI model

[0782] Step 3: Interacting with the Instructor AI

[0783] 3.1 The server starts the instructor AI model.

[0784] The server starts the loaded instructor AI model and generates the initial message "Let's get started with Python programming."

[0785] Input: Loaded AI model

[0786] Data processing: Initial message generation

[0787] Python

[0788] initial_message = AI_model.generate_initial_message()

[0789] Output: Initial message

[0790] 3.2 Instructor AI sends an initial message to the user

[0791] The instructor AI generates an initial message which is then sent to the user.

[0792] Input: Initial message

[0793] Output: Initial message sent to the user

[0794] 3.3 Users submit questions or instructions during learning

[0795] The user enters a question such as "Please tell me how to use variables" and presses the submit button.

[0796] Input: Questions or instructions

[0797] Output: Request to send questions or instructions

[0798] 3.4 The terminal sends the user's question to the server.

[0799] The terminal sends the user's question to the server.

[0800] Input: User's question

[0801] Output: Question sent to the server

[0802] 3.5 The server passes the question to the instructor AI.

[0803] The server passes the user's question received to the educational support AI.

[0804] Input: Question received from the user

[0805] Output: Questions given to the educational support AI

[0806] 3.6 Instructor AI generates answers to questions

[0807] The educational support AI generates an answer such as, "In Python, variables are declared as follows..." and creates a message that includes an example.

[0808] Input: User's question

[0809] Data processing: Answer generation

[0810] Python

[0811] response = AI_model.generate_response("Please tell me how to use the variables")

[0812] Output: Response message

[0813] 3.7 The instructor AI sends the answer to the user.

[0814] The instructor AI generates the answer and sends it to the user.

[0815] Input: Generated answer

[0816] Output: Response sent to the user

[0817] Step 4: Recording progress

[0818] 4.1 The server records the conversation content and learning progress.

[0819] The server records the content of the conversation between the educational support AI and the user, as well as the learning progress, in a database.

[0820] Input: Dialogue content and learning progress

[0821] Data processing: Saving to a database

[0822] sql

[0823] INSERT INTO progress (userID, content, timestamp) VALUES ('user123', 'Learn how to use variables', NOW());

[0824] Output: Recorded data

[0825] 4.2 Utilize the progress made in the previous session in the next session

[0826] The server retrieves the user's previous learning progress from the database upon their next login and displays the message, "Last time, you learned how to use variables. Do you want to continue?"

[0827] Input: Previous learning progress data

[0828] Data processing: Message generation based on learning progress

[0829] sql

[0830] SELECT FROM progress WHERE userID='user123' ORDER BY timestamp DESC LIMIT 1;

[0831] Output: Learning progress message sent to the user

[0832] The above describes the processing flow in the system of the present invention. Each step is explained in detail, including specific inputs and outputs, and the data processing and calculations based on them.

[0833] (Application Example 1)

[0834] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0835] Traditional online learning systems have presented challenges such as difficulty for users to track their progress in real time and in obtaining quick answers to questions about the learning content. Furthermore, it was difficult for users to manage their own learning progress and plan their next learning sessions. Additionally, the lack of widespread use of mobile devices such as smartphones has resulted in a lack of convenience.

[0836] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0837] In this invention, the server includes means for the user to input and send authentication information to the server; means for the server to verify the received authentication information and generate a user dashboard if authentication is successful; means for the user to select a topic they wish to learn and send that selection information to the server; means for the server to activate an instructor AI based on the selected topic and begin a dialogue with the user; means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session; and means for the user to interact with the instructor AI in real time using natural language with a smartphone and provide responses to questions using prompt sentences generated by the AI. This allows the user to grasp their learning progress in real time and obtain quick answers, improving the convenience of learning using a smartphone.

[0838] A "user" is someone who uses the system to learn online.

[0839] "Authentication information" refers to information used to identify an individual, such as a user ID and password, which a user enters when accessing a system.

[0840] A "server" is a computer system that receives authentication information from users, performs searches, and performs authentication.

[0841] The "user dashboard" is an interface generated for users who have successfully authenticated, displaying their learning progress, selected learning themes, and other relevant information.

[0842] A "theme" is information that indicates the content or field that the user wants to learn about.

[0843] "Instructor AI" is an artificial intelligence system that is activated based on the theme selected by the user, interacts with the user, and generates answers in response to questions.

[0844] "Natural language" refers to the language that humans use on a daily basis, which is used for communication between the user and the instructor AI.

[0845] A "prompt message" is a series of texts that the instructor AI generates and provides as a response to a user's question.

[0846] A "smartphone" is a portable, multi-functional telephone that users use as their personal device to access systems and communicate.

[0847] "Real-time" refers to a state of immediate response where a user asks a question to the instructor AI and receives an answer right away.

[0848] This invention is a system for users to learn online while interacting in real time with an instructor AI using a smartphone. The system consists of a user terminal, a server, an instructor AI model, and a database. This allows users to receive appropriate learning advice and quick responses.

[0849] System Configuration and Operation

[0850] Hardware and software usage

[0851] User terminal: A smartphone is used. This device is used by the user to enter authentication information, select a theme, and interact with the instructor AI.

[0852] Server: This is the central device that verifies authentication information, activates the instructor AI, records the conversation content, and prepares the content for the next learning session.

[0853] Instructor AI Model: This artificial intelligence is activated based on the user's learning theme, interacts with the user in natural language, and generates prompt sentences in response to questions.

[0854] Database: A storage device that records user authentication information, learning progress information, dialogue content, etc.

[0855] Specific operating procedures and functions

[0856] 1. User Authentication

[0857] The user launches the app on their smartphone and enters their authentication information (user ID and password) on the login screen.

[0858] The user terminal sends this authentication information to the server, and the server performs authentication by comparing the received authentication information with the database.

[0859] If authentication is successful, the server will generate a user dashboard.

[0860] 2. Theme Selection

[0861] The user selects the topic they want to learn from the dashboard. For example, they might select "Python Programming".

[0862] The user terminal sends the selected theme information to the server, and the server loads the corresponding instructor AI model.

[0863] 3. Interaction with the Instructor AI

[0864] The server starts the loaded instructor AI model and generates an initial message. For example, it might generate a message like, "Let's get started with Python programming. We'll begin by learning basic data types."

[0865] The user sends a question to the instructor AI during the learning process (e.g., "Please explain how to use variables").

[0866] The user's terminal sends this question to the server, which then passes it on to the instructor AI.

[0867] The instructor AI generates an answer to the question (e.g., "In Python, variables are declared as follows...") and sends that answer to the user's terminal.

[0868] 4. Recording progress and preparing for the next learning session.

[0869] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database.

[0870] For the next session, the server will prepare appropriate learning content based on the progress from the previous session, and the instructor AI will provide the next learning content.

[0871] Examples of prompt statements

[0872] Log in with user ID and password: 'user123', 'password'

[0873] Select a topic you want to learn: 'Python Programming'

[0874] Question: 'How do I use variables?'

[0875] Save progress

[0876] In this way, the present invention enables users to efficiently and effectively learn by interacting with an instructor AI in real time using their smartphone.

[0877] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0878] Step 1:

[0879] User Authentication

[0880] The user launches the app on their smartphone and enters their authentication information (user ID and password).

[0881] The device sends this authentication information to the server. The entered data is the user ID and password.

[0882] The server compares the received authentication information with the database. If authentication is successful, it generates a user-specific dashboard and sends it to the user's terminal. The user's profile information is output upon successful authentication. An error message is output if authentication fails.

[0883] Step 2:

[0884] Theme Selection

[0885] The user selects a learning topic (for example, "Python Programming") from the dashboard.

[0886] The terminal sends the selected information (theme name) to the server. The entered data is the selected theme name.

[0887] The server loads and launches the appropriate instructor AI model from the database based on the selected theme. It also sends an initial message to the user's terminal based on the selected theme (e.g., "Let's start Python programming.").

[0888] Step 3:

[0889] Dialogue with the Instructor AI

[0890] The user enters and submits a question during the learning process (e.g., "Please explain how to use variables").

[0891] The terminal sends the question to the server. The input data is the user's question.

[0892] The server passes the question to the instructor AI, which then generates an answer. The answer includes a generated prompt (e.g., "In Python, variables are declared as follows...").

[0893] The server sends the instructor AI's response to the user's terminal and displays it to the user.

[0894] Step 4:

[0895] Progress log

[0896] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database. The input data consists of the interaction content and learning progress information.

[0897] We will save this data to use in the next session and prepare appropriate learning content for the next session.

[0898] Step 5:

[0899] Preparation for the next lesson

[0900] The server automatically prepares the next learning content based on the learning progress information stored. The input data is past learning progress information.

[0901] Based on the learning content prepared by the server, the instructor AI generates scenarios and prompts to provide appropriate learning advice, and saves them for use in the next learning session.

[0902] In this way, users can engage in online learning more efficiently and effectively through the system.

[0903] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0904] This invention is a system for users to learn online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the learning experience. This system allows users to learn anytime, anywhere, while interacting with an instructor AI, and to manage their progress and emotions.

[0905] This system consists of the following main components: user terminals, servers, instructor AI models, emotion engines, and databases. The entire system uses secure communication protocols (e.g., HTTPS) to exchange data.

[0906] The main functions and operation of the system

[0907] 1. User Authentication

[0908] The user accesses the login screen and enters their authentication information (user ID and password). This information is then sent from the terminal to the server.

[0909] The server compares the received authentication information with the database and, if authentication is successful, generates a user-specific dashboard.

[0910] 2. Theme Selection

[0911] The user selects a topic they want to learn about on the dashboard (e.g., "Python Programming").

[0912] The device sends the selected theme information to the server.

[0913] The server receives the theme information and loads the corresponding instructor AI model.

[0914] 3. Interaction with the Instructor AI

[0915] The server starts the loaded instructor AI model and generates an initial message.

[0916] The instructor AI will send this message to the user.

[0917] Users can send questions and instructions to the instructor AI while learning.

[0918] The device sends these messages to the server, which then passes them on to the instructor AI.

[0919] The instructor AI generates answers to the user's questions and sends them to the user.

[0920] 4. Emotion recognition

[0921] The server passes user input data (e.g., text messages) to the emotion engine, which recognizes the user's emotions in real time.

[0922] The emotion engine recognizes the emotion information, which the server then provides to the instructor AI.

[0923] The instructor AI generates responses by considering emotional information and adjusts the dialogue according to the user's emotions.

[0924] 5. Record progress and emotions

[0925] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[0926] This information will be used in the next session.

[0927] Explanation with specific examples

[0928] 1. The user logs in.

[0929] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[0930] The terminal receives this and sends it to the server.

[0931] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific dashboard.

[0932] 2. The user selects a theme.

[0933] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[0934] The device sends this selection information to the server.

[0935] The server selects and starts the appropriate instructor AI model.

[0936] 3. Start the conversation with the instructor AI.

[0937] The instructor AI generates an initial message saying, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user.

[0938] The user asks, "How do I use variables?"

[0939] The device sends the question to the server, which then passes it on to the instructor AI.

[0940] The instructor AI responds with "In Python, variables are declared like this..." and generates a message that includes an example.

[0941] 4. Emotion Recognition and Adaptation

[0942] The server uses an emotion engine to recognize the user's emotions (e.g., confusion, relief) based on the content of the user's questions.

[0943] The emotion engine recognizes the emotions, which the server then provides to the instructor AI.

[0944] The instructor AI adjusts its response based on the user's emotions, saying something like, "You seem confused. Let me explain in more detail," and sends it to the user.

[0945] 5. Managing progress and emotions

[0946] The server records the conversation content, learning progress, and recognized emotion information in a database.

[0947] In the next session, dialogue will be provided that takes into account not only what was learned in the previous session, but also the recognized emotional information.

[0948] In this way, by monitoring the user's learning experience in real time and providing optimal feedback based on their emotions, the quality and efficiency of learning can be improved. With such a system, users can always receive appropriate learning support, making it easier to maintain motivation for self-study.

[0949] The following describes the processing flow.

[0950] Step 1:

[0951] The user accesses the login screen and enters their authentication information (user ID and password). The user then clicks the "Login" button.

[0952] Step 2:

[0953] The terminal encodes the entered authentication information and sends it to the server using a secure communication protocol (HTTPS).

[0954] Step 3:

[0955] The server compares the received authentication information with the database. If the user ID and password match, authentication is considered successful.

[0956] Step 4:

[0957] The server generates a user-specific dashboard page along with an authentication success message and sends it to the device. If authentication fails, an error message is sent.

[0958] Step 5:

[0959] The user accesses the dashboard page and selects a topic they want to learn about from the provided list of topics (e.g., "Python Programming").

[0960] Step 6:

[0961] The device sends the selected theme information to the server.

[0962] Step 7:

[0963] The server receives the theme information and loads the corresponding instructor AI model. Once loading is complete, it generates an initial message.

[0964] Step 8:

[0965] The instructor AI generates an initial message, "Let's start Python programming," and sends it to the user.

[0966] Step 9:

[0967] The user enters questions or requests while learning, and the device sends them to the server.

[0968] Step 10:

[0969] The server passes the user's questions and requests to the instructor AI. The instructor AI then generates answers to those questions.

[0970] Step 11:

[0971] The instructor AI returns the generated answer to the server, which then sends that answer to the user.

[0972] Step 12:

[0973] The server passes user input data to the emotion engine, which recognizes the user's emotions in real time.

[0974] Step 13:

[0975] The emotion engine recognizes emotions from the user's input data and provides that emotion information to the server.

[0976] Step 14:

[0977] The server passes the user's emotional information, obtained from the emotion engine, to the instructor AI. The instructor AI then adjusts its response based on that emotional information.

[0978] Step 15:

[0979] The instructor AI generates emotion-based responses, creating messages such as, "You seem confused. Let me explain in more detail," and sends them to the user.

[0980] Step 16:

[0981] Users receive emotionally sensitive responses and continue learning. If they have additional questions or requests, they can enter them and submit again.

[0982] Step 17:

[0983] When a learning session ends, the server records the user-instructor AI dialogue, learning progress, and recognized emotion information in a database.

[0984] Step 18:

[0985] The server uses recorded progress data and sentiment information to prepare the content for the next learning session. In the next session, new themes and problems will be presented, taking into account the progress and sentiment information from the previous session.

[0986] The above outlines the specific processing steps of the system based on the invention that combines an emotion engine. This system allows users to effectively learn by interacting with an instructor AI anytime, anywhere, and to receive emotion-sensitive feedback.

[0987] (Example 2)

[0988] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0989] Traditional online learning systems often struggle to provide immediate and appropriate responses to user questions, potentially leading to decreased learning efficiency. Furthermore, they fail to optimize individual learning experiences by providing uniform responses without considering user emotions. Additionally, a lack of effective methods for reflecting learning progress in subsequent sessions hinders the overall improvement of the learning experience.

[0990] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the user to input and transmit authentication information to the server, means for the server to verify the received authentication information and generate a user dashboard if authentication is successful, means for the user to select a theme they want to learn and transmit the selection information to the server, means for the server to activate an instructional AI based on the selected theme and start a dialogue with the user, means for the server to pass the user's input data to an analysis engine and recognize the user's emotions in real time, means for the server to provide the recognized emotion information to the instructor AI and generate a response, and means for the server to record the content of the dialogue with the instructional AI and save the data for use in the next session. This enables the generation of appropriate responses in real time that take into account the user's emotions and the effective preparation of the next learning content according to the learning progress.

[0991] "User authentication information" refers to information such as the user ID and password that a user uses to log in to the system.

[0992] A "server" is a computer system that receives requests from users on a network and processes them accordingly.

[0993] The "user dashboard" is an interface that users can access after logging in, where they can check their learning progress, select topics, and more.

[0994] A "theme" refers to a specific learning topic or subject that a user wants to study.

[0995] "Teaching AI" refers to a model that uses artificial intelligence to provide educational guidance to users, including natural language dialogue and learning support.

[0996] An "analysis engine" is a collection of software and hardware that analyzes user input data and identifies specific information (e.g., emotions).

[0997] "Emotional information" refers to information that indicates the user's emotional state, extracted from the user's input data by the analysis engine.

[0998] "Dialogue content" refers to the record of texts and messages exchanged between the user and the instructional AI.

[0999] A "data storage method" refers to a means by which a server records dialogue content and learning progress information and stores it for use in the next session.

[1000] Modes for carrying out the invention

[1001] This invention is a system for users to learn online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the learning experience. The main components of the system are a user terminal, a server, an instructional AI model, an analysis engine, and a database. The entire system exchanges data using a secure communication protocol (e.g., HTTPS).

[1002] System Overview

[1003] This system consists of the following main components:

[1004] 1. User terminal:

[1005] This refers to the device that the user accesses, such as a smartphone, tablet, or personal computer.

[1006] 2. Server:

[1007] This is a computer system that performs user authentication, data analysis, manages the AI ​​for instruction, and communicates with the database.

[1008] 3. Instructional AI Model:

[1009] It is an artificial intelligence that interacts with users, using natural language processing to generate responses to user questions.

[1010] 4. Analysis Engine:

[1011] This part analyzes user input data (e.g., text messages) in real time to recognize emotions and intentions.

[1012] 5. Database:

[1013] This is a storage system that stores user authentication information, learning progress, conversation content, and sentiment data.

[1014] System operation

[1015] This system operates using the following steps:

[1016] 1. User Authentication:

[1017] The user accesses the login screen and enters their user ID and password. The entered information is sent from the terminal to the server.

[1018] The server verifies the authentication information against the database, and if authentication is successful, it generates a user-specific dashboard.

[1019] 2. Theme Selection:

[1020] The user selects the topic they want to learn about on the dashboard.

[1021] The device sends the selected theme information to the server, and the server loads the corresponding instructional AI model.

[1022] 3. Interaction with the instructional AI:

[1023] The server activates the instructional AI model and generates an initial message. The instructional AI then sends this message to the user.

[1024] The user enters and sends questions or instructions. The device sends these messages to the server, which then passes them on to the instructional AI.

[1025] The instructional AI generates answers to the user's questions and sends them to the user.

[1026] 4. Emotion recognition:

[1027] The server passes user input data to an analysis engine, which recognizes the user's emotions in real time.

[1028] The emotion engine recognizes the emotion information, which the server then provides to the guidance AI. The guidance AI then generates a response, taking the emotion information into consideration.

[1029] 5. Recording progress and emotions:

[1030] The server records the content of the interaction between the teaching AI and the user, as well as learning progress and recognized emotion information, in a database.

[1031] This information will be used in the next session.

[1032] Hardware and software to be used

[1033] User terminal:

[1034] Smartphone, tablet, or computer (e.g., iPhone, iPad, Windows PC)

[1035] server:

[1036] Cloud services or on-premises computer systems (e.g., AWS, Microsoft Azure)

[1037] Instructional AI model:

[1038] Generative AI models that use natural language processing (e.g., GPT-3)

[1039] Analysis engine:

[1040] Software that analyzes user emotions (e.g., IBM Watson Tone Analyzer)

[1041] Database:

[1042] SQL database or NoSQL database (e.g., MySQL, MongoDB)

[1043] Specific example

[1044] The user launches the smartphone app, enters the ID "user123" and password "password" on the login screen, and presses the login button. The device receives this information and sends it to the server. The server verifies the authentication information against the database, and if authentication is successful, displays a user-specific dashboard. The user selects "Python Programming" on the dashboard screen, and the device sends this selection information to the server. The server selects an appropriate teaching AI model and starts it. The teaching AI generates an initial message, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user. The user enters a question, "Please teach me how to use variables," and presses the send button. The device sends the question to the server, which passes it to the teaching AI. The teaching AI responds, "In Python, variables are declared as follows...", and generates a concrete example. The server uses an emotion engine to recognize the user's emotions (e.g., confusion, relief) from the user's question. The server provides the emotions recognized by the emotion engine to the teaching AI, which adjusts its response based on the emotions and sends it to the user. The server records the dialogue content, learning progress, and recognized emotion information in the database and uses this information in the next session.

[1045] In this way, by monitoring the user's learning experience in real time and providing optimal feedback based on their emotions, the quality and efficiency of learning can be improved.

[1046] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1047] Step 1:

[1048] The user enters and submits their login information (user ID and password). The terminal receives this information and sends it to the server. The input is the user ID and password, and the output is the login request sent to the server.

[1049] Step 2:

[1050] The server compares the received authentication information with the database. Data processing here involves extracting user authentication information from the database using SQL queries and comparing it with the input information. The output is whether the authentication was successful or not.

[1051] Step 3:

[1052] If authentication is successful, the server generates a user-specific dashboard and sends the information to the device. The output is data for displaying the dashboard, which is then sent to the device. The device receives this data and displays the dashboard to the user.

[1053] Step 4:

[1054] The user selects a topic they want to learn about on the dashboard and submits the topic information. The device receives this information and sends it to the server. The input is the topic selected by the user, and the output is the topic information sent to the server.

[1055] Step 5:

[1056] The server loads the corresponding instructional AI model based on the received theme information. Here, data processing involves loading the AI ​​model corresponding to the theme. The output is the initialization and startup of the AI ​​model.

[1057] Step 6:

[1058] The instructional AI generates an initial message and sends it to the terminal via the server. The input is the loaded AI model and the prompt for generating the initial message, and the output is the initial message sent to the user. The terminal displays this message to the user.

[1059] Step 7:

[1060] The user inputs questions or instructions and sends that information. The terminal receives this message and sends it to the server. The input is the user's questions or instructions, and the output is the user input data sent to the server.

[1061] Step 8:

[1062] The server passes user input data to the guidance AI, which then generates an answer. The input consists of the user's questions and instructions, while the output is the generated answer message.

[1063] Step 9:

[1064] The server sends the generated response message to the terminal, which then displays it to the user. The input is the response message from the AI, and the output is the message displayed to the user.

[1065] Step 10:

[1066] The server passes user input data to the analysis engine, which recognizes emotions in real time. The data processing here involves natural language processing for emotion analysis. The input is user input data, and the output is the recognized emotion information.

[1067] Step 11:

[1068] The emotion engine recognizes emotional information, which the server then provides to the guidance AI. Based on this information, the AI ​​generates a response. The input is emotional information, and the output is a response message that takes that emotional information into account.

[1069] Step 12:

[1070] The server records the content of the interaction between the teaching AI and the user, as well as learning progress and recognized emotion information, in a database. The inputs are the interaction content, learning progress, and emotion information, while the output is the information stored in the database.

[1071] Step 13:

[1072] In the next session, the server will have the instructional AI generate a customized learning plan based on the recorded information. The input will be the content of the previous conversation and emotional information, and the output will be the customized learning plan. The user will then start learning again based on this information.

[1073] In this way, by analyzing user sentiment information and learning progress in real time and providing appropriate feedback, the learning experience is improved.

[1074] (Application Example 2)

[1075] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1076] Conventional instructor AI systems failed to recognize the user's emotional state and could only provide uniform dialogue, resulting in a decline in the quality of the learning experience. Similarly, in factory robots, the inability to provide feedback and instructions that considered the operator's emotions posed a challenge to work efficiency and safety.

[1077] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input and transmit authentication information to the server; means for the server to verify the received authentication information and generate a user dashboard if authentication is successful; means for the user to select a theme they wish to learn and transmit the selection information to the server; means for the server to activate an instructor AI based on the selected theme and begin a dialogue with the user; means for the user to recognize their emotions and for the instructor AI to adjust the content of the dialogue based on those emotions; means installed in a robot to recognize the operator's emotional state and provide appropriate feedback and instructions; and means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session. This makes it possible to provide adjusted dialogue and instructions according to the emotional state of the user and operator.

[1078] User authentication is the process by which a user enters authentication information, sends it to a server for verification, and then verifies it.

[1079] A "server" is a computer system that receives user authentication information and selection information and processes it based on that information.

[1080] A "user dashboard" is a screen that the server generates upon successful authentication and allows the user to access it.

[1081] An "instructor AI" is an artificial intelligence model that interacts with users and operators, providing answers to questions and instructions.

[1082] "Emotion recognition" is the process of analyzing user or operator input data (voice, text, facial expressions, etc.) to identify emotional states.

[1083] "Dialogue content adjustment" refers to the instructor AI appropriately modifying the content of the dialogue based on recognized emotions.

[1084] A "robot" is a mechanical device that performs tasks within a factory, interacting with an operator and providing work instructions.

[1085] "Feedback" refers to the responses and instructions that the instructor AI provides to the user or operator.

[1086] A "database" is a data storage system where a server records conversation content, progress, emotional information, and other data for use in subsequent sessions.

[1087] This invention is a system for users to learn and work online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the experience. The system consists of the following main components: a user terminal, a server, an instructor AI model, an emotion engine, and a database. The entire system exchanges data using a secure communication protocol (e.g., HTTPS).

[1088] The main functions and operation of the system

[1089] 1. User Authentication

[1090] The user accesses the login screen and enters their authentication information (user ID and password). This information is then sent from the terminal to the server.

[1091] The server compares the received authentication information with the database and, if authentication is successful, generates a user-specific dashboard.

[1092] 2. Theme Selection

[1093] Users select a topic they want to learn about on the dashboard (e.g., "Python Programming").

[1094] The device sends the selected theme information to the server.

[1095] The server receives the theme information and loads the corresponding instructor AI model.

[1096] 3. Interaction with the Instructor AI

[1097] The server starts the loaded instructor AI model and generates an initial message.

[1098] The instructor AI will send this message to the user.

[1099] Users can send questions and instructions to the instructor AI while learning.

[1100] The device sends these messages to the server, which then passes them on to the instructor AI.

[1101] The instructor AI generates answers to the user's questions and sends them to the user.

[1102] 4. Emotion recognition

[1103] The server passes user input data (e.g., text messages) to the emotion engine, which recognizes the user's emotions in real time.

[1104] The emotion engine recognizes the emotion information, which the server then provides to the instructor AI.

[1105] The instructor AI generates responses by considering emotional information and adjusts the dialogue according to the user's emotions.

[1106] 5. Record progress and emotions

[1107] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[1108] This information will be used in the next session.

[1109] Hardware and software to be used

[1110] Hardware:

[1111] Camera (e.g., Logitech C920): Captures the user's facial expressions.

[1112] Microphone: Captures the user's voice.

[1113] software:

[1114] OpenCV: Used for inputting and processing camera images.

[1115] Emotion Recognition Library (e.g., emotion_recognition): Analyzes the user's emotions.

[1116] AI Model: As an instructor AI, it generates responses based on the task content (e.g., the natural language processing model ChatGPT).

[1117] Database (e.g., MySQL): Manages login information, work details, and sentiment information.

[1118] Specific example

[1119] Specific example 1:

[1120] The user launches the smartphone app, enters the ID "user123" and password "password" on the login screen, and presses the login button. The device receives this information and sends it to the server. The server verifies it against the database, and if authentication is successful, displays a user-specific dashboard.

[1121] The user selects "Python Programming" in the dashboard and sends this selection information to the server. The server receives this information and activates the instructor AI.

[1122] The instructor AI generates an initial message saying, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user. When the user asks, "How do I use variables?", the terminal sends the question to the server, and the instructor AI generates an answer.

[1123] The server passes the user's question to the emotion engine, which recognizes the user's emotions. If the emotion engine recognizes "confusion," the server provides this information to the instructor AI, which responds with, "You seem confused. Let me explain in more detail."

[1124] The server records the conversation content, progress, and emotional information in a database, which will be used in the next session.

[1125] Example of a prompt:

[1126] If an operator appears confused while tightening a bolt, recognize their confusion and provide a clear and detailed explanation.

[1127] This makes it possible to provide users and operators with adjusted dialogue and instructions that are tailored to their emotional state.

[1128] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1129] Program processing steps

[1130] Step 1:

[1131] The user launches the app on their smartphone and enters their user ID and password on the login screen. This generates the input data for user authentication.

[1132] Step 2:

[1133] The terminal encrypts the user authentication input data and sends it to the server. The server compares the received authentication information with the database, and if authentication is successful, generates a user-specific dashboard and sends it back to the terminal.

[1134] Step 3:

[1135] The user selects a topic they want to learn about (e.g., "Python Programming") within the dashboard, and the device sends this selection information to the server. This generates the input data for the topic selection.

[1136] Step 4:

[1137] The server analyzes the theme information and loads the appropriate instructor AI model. The instructor AI generates an initial message and sends it to the terminal.

[1138] Step 5:

[1139] The user inputs a question to the instructor AI, and the device sends the question to the server. The server passes the question to the instructor AI, which generates an answer and sends it back.

[1140] Step 6:

[1141] The server passes the user's questions and the instructor AI's answers to the emotion engine, which recognizes the user's emotional state in real time. This process generates emotion recognition data.

[1142] Step 7:

[1143] The emotion engine recognizes the emotion information, which the server provides to the instructor AI, and the instructor AI adjusts the content of the dialogue. The server then sends the adjusted response generated by the AI ​​to the terminal.

[1144] Step 8:

[1145] Review the user's response and either ask the next question or continue the learning process.

[1146] Step 9:

[1147] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[1148] Step 10:

[1149] In the next session, the server will customize the instructor AI model based on past conversations and progress information to provide appropriate learning advice.

[1150] This makes it possible to provide adjusted dialogue and instructions that respond to various emotional states, such as when a user is confused or agitated.

[1151] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1152] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1153] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1154] [Third Embodiment]

[1155] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1156] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1157] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1158] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1159] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1160] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1161] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1162] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1163] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1164] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1165] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1166] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1167] This invention is a system for users to learn online via an internet connection. This system allows users to learn anytime, anywhere, while interacting with an AI instructor, and to manage their progress.

[1168] This system consists of the following main components: user terminals, servers, instructor AI models, and databases. The entire system also uses secure communication protocols (e.g., HTTPS) for data exchange.

[1169] The main functions and operation of the system

[1170] 1. User Authentication

[1171] The user enters their authentication information (user ID and password) on the login screen and presses the submit button.

[1172] The device sends these authentication credentials to the server.

[1173] The server compares the received authentication information with the database, and if authentication is successful, it generates a user-specific dashboard.

[1174] 2. Theme Selection

[1175] The user selects a topic they want to learn about from the dashboard (e.g., "Python Programming").

[1176] The device sends the selected theme information to the server.

[1177] The server loads the appropriate instructor AI model based on the selected theme.

[1178] 3. Interaction with the Instructor AI

[1179] The server starts the loaded instructor AI model and generates an initial message (e.g., "Let's get started with Python programming.").

[1180] The instructor AI will send this message to the user.

[1181] Users can send questions and instructions to the instructor AI while learning.

[1182] The device sends these messages to the server, which then passes them on to the instructor AI.

[1183] The instructor AI generates answers to the user's questions and sends them to the user.

[1184] 4. Recording progress

[1185] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database.

[1186] This information will be used in the next session.

[1187] Explanation with specific examples

[1188] 1. The user logs in.

[1189] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[1190] The terminal receives this and sends it to the server.

[1191] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific dashboard.

[1192] 2. The user selects a theme.

[1193] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[1194] The device sends this selection information to the server.

[1195] The server selects and starts the appropriate instructor AI model.

[1196] 3. Start the conversation with the instructor AI.

[1197] The instructor AI sends the user a message saying, "Let's start Python programming. First, we'll learn about basic data types."

[1198] The user asks, "How do I use variables?"

[1199] The device sends the question to the server, which then passes it on to the instructor AI.

[1200] The instructor AI responds with "In Python, variables are declared like this..." and generates a message that includes an example.

[1201] 4. Progress Management

[1202] The server records the conversation content and learning progress in a database.

[1203] In the next session, the server will provide appropriate learning content based on the progress information from the previous session.

[1204] This allows users to always progress in their learning based on their latest learning progress and receive appropriate feedback from the instructor AI in real time. In this way, the system of the present invention provides efficient and effective learning support to users.

[1205] The following describes the processing flow.

[1206] Step 1:

[1207] The user accesses the login screen and enters their authentication information (user ID and password). The user then clicks the "Login" button.

[1208] Step 2:

[1209] The terminal encodes the entered authentication information and sends it to the server using a secure communication protocol (HTTPS).

[1210] Step 3:

[1211] The server compares the received authentication information with the database. If the user ID and password match, authentication is considered successful.

[1212] Step 4:

[1213] The server generates a user-specific dashboard page along with an authentication success message and sends it to the device. If authentication fails, an error message is sent.

[1214] Step 5:

[1215] The user accesses the dashboard page and selects a topic they want to learn about from the provided list of topics (e.g., "Python Programming").

[1216] Step 6:

[1217] The device sends the selected theme information to the server.

[1218] Step 7:

[1219] The server receives the theme information and loads the corresponding instructor AI model. Once loading is complete, it generates an initial message.

[1220] Step 8:

[1221] The instructor AI generates an initial message, "Let's start Python programming," and sends it to the user.

[1222] Step 9:

[1223] The user enters questions or requests while learning, and the device sends them to the server.

[1224] Step 10:

[1225] The server passes the user's questions and requests to the instructor AI. The instructor AI then generates answers to those questions.

[1226] Step 11:

[1227] The instructor AI returns the generated answer to the server, which then sends that answer to the user.

[1228] Step 12:

[1229] The user receives answers from the instructor AI and continues learning. If there are any additional questions or requests, they can enter them again.

[1230] Step 13:

[1231] When a learning session ends, the server records the content of the user-instructor AI conversation and the learning progress in a database.

[1232] Step 14:

[1233] The server uses the recorded progress data to prepare the content for the next learning session. In the next session, new themes and problems will be provided based on the progress made in the previous session.

[1234] The above explains the program's processing in detail, step by step. This system allows users to effectively learn by interacting with the instructor AI anytime, anywhere.

[1235] (Example 1)

[1236] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1237] Traditional online learning systems lacked sufficient interactive learning support based on themes selected by learners. Furthermore, managing learning progress and preparing future learning plans required manual intervention, highlighting the need for more efficient learning support. Additionally, there was a lack of systems that could provide appropriate real-time answers to learners' questions. Therefore, the development of a system that provides more effective and interactive learning support was essential.

[1238] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1239] In this invention, the server includes means for the user to input authentication information and transmit it to the server via a computer network; means for the server to compare the received authentication information with a database and generate a user-specific interface if authentication is successful; means for the user to select a subject they wish to learn and transmit that selection information to the server; means for the server to load an educational support AI based on the selected subject and initiate a dialogue with the user in natural language; means for the educational support AI to generate answers to the user's questions in real time and transmit them to the user; means for the server to record the content of the dialogue with the educational support AI in a database and save the data for use in the next session; and means for the server to prepare the content of the next learning session based on the previous learning progress and for the educational support AI to provide appropriate learning advice as needed. As a result, the user can always receive appropriate learning support based on their latest learning progress.

[1240] A "user" refers to an individual or group that uses the system to learn.

[1241] "Authentication information" refers to identification information used by users to access the system, such as user IDs and passwords.

[1242] A "computer network" refers to infrastructure used for data communication between computer systems, such as the internet and local area networks (LANs).

[1243] A "server" refers to a computer system that receives and processes requests from users.

[1244] A "database" refers to a system that efficiently stores large amounts of information and allows for searching and updating.

[1245] "Interface" refers to the screens and methods of operation that users use to interact with a system.

[1246] "Subject" refers to the specific learning content or theme that the user wants to learn about.

[1247] "Educational support AI" refers to programs and systems that use artificial intelligence technology to support users' learning.

[1248] "Natural language" refers to the language that humans use on a daily basis, which is converted into a format that computer systems can easily understand.

[1249] "Dialogue" refers to two-way communication between a user and a system where information is exchanged bi-directionally.

[1250] "Real-time" refers to the immediate response to user actions and questions.

[1251] "Generating answers" refers to the process of creating appropriate information or answers in response to a user's question.

[1252] "Saving data" refers to recording conversation content and learning progress for future use.

[1253] "Learning progress" refers to information indicating how far a user has progressed in their learning.

[1254] "Learning advice" refers to suggestions and instructions that help users streamline their learning and deepen their understanding.

[1255] This invention provides a system for users to learn online via an internet connection. This system allows users to learn and manage their progress anytime, anywhere, while interacting with an educational support AI. The system consists of the following main components: a user terminal, a server, an educational support AI model, and a database. Furthermore, the entire system can exchange data using a secure communication protocol (e.g., HTTPS).

[1256] Hardware and software to use

[1257] 1. User terminal

[1258] Internet-connected devices such as smartphones, tablets, and personal computers

[1259] Use an application or web browser

[1260] 2. Server

[1261] High-performance computer systems (e.g., Linux servers)

[1262] Server-side scripts (e.g., Python, Node.js)

[1263] Secure authentication libraries (e.g., JWT)

[1264] 3. Database

[1265] Relational databases such as MySQL, PostgreSQL, and MongoDB, or NoSQL databases.

[1266] 4. AI Models for Educational Support

[1267] Generative AI models such as GPT-3 and BERT

[1268] Machine learning libraries such as PyTorch and TensorFlow

[1269] The main functions and operation of the system

[1270] 1. User Authentication

[1271] The user enters their authentication information (user ID and password) on the login screen and presses the submit button.

[1272] The device sends these authentication credentials to the server via HTTPS.

[1273] The server compares the received authentication information with the database, and if authentication is successful, it generates a user-specific interface.

[1274] 2. Theme Selection

[1275] The user selects a topic they want to learn from the dashboard (e.g., "Python Programming").

[1276] The terminal sends the selected subject information to the server.

[1277] The server loads and launches the appropriate educational support AI model based on the selected subject.

[1278] 3. Interaction with educational support AI

[1279] The server launches the loaded educational support AI model and generates an initial message (e.g., "Let's start Python programming.").

[1280] The educational support AI will send this message to the user.

[1281] Users can send questions and instructions to the educational support AI while learning.

[1282] The device sends these messages to the server, which then passes them on to the educational support AI.

[1283] The educational support AI generates answers to the user's questions and sends them to the user.

[1284] 4. Recording progress

[1285] The server records the content of the conversation between the educational support AI and the user, as well as the learning progress, in a database.

[1286] This information will be used in the next session.

[1287] Specific example

[1288] 1. The user logs in.

[1289] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[1290] The terminal receives this and sends it to the server.

[1291] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific interface.

[1292] Example of a prompt:

[1293] "A user is attempting to log in using the ID "user123" and password "password". Please explain the procedure."

[1294] 2. The user selects a theme.

[1295] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[1296] The device sends this selection information to the server.

[1297] The server selects and launches an appropriate AI model for educational support.

[1298] Example of a prompt:

[1299] "The user selected 'Python Programming' from the dashboard. Please describe the next server-side actions to take."

[1300] 3. Start the conversation with the educational support AI.

[1301] The educational support AI sends the user a message saying, "Let's start Python programming. First, we'll learn about basic data types."

[1302] The user asks, "How do I use variables?"

[1303] The device sends the question to the server, which then passes it on to the educational support AI.

[1304] The educational support AI responds with "In Python, variables are declared as follows..." and generates a message that includes an example.

[1305] Example of a prompt:

[1306] "The user asked, 'How do I use variables?' Generate an appropriate response that the educational support AI should provide."

[1307] 4. Progress Management

[1308] The server records the conversation content and learning progress in a database.

[1309] In the next session, the server will provide appropriate learning content based on the progress information from the previous session.

[1310] Example of a prompt:

[1311] "Please explain the procedure for recording the user's interaction with the educational support AI and their learning progress in a database."

[1312] This allows users to effectively progress in their learning based on their latest progress and receive appropriate feedback from the educational support AI in real time. In this way, the system of the present invention provides efficient and effective learning support to users.

[1313] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1314] Program processing flow

[1315] Step 1: User Authentication

[1316] 1.1 The user enters their authentication information on the login screen.

[1317] The user enters user ID "user123" and password "password" on the app's login screen and presses the login button.

[1318] Input: User ID and password

[1319] Output: Request to send input information

[1320] 1.2 The device sends authentication information to the server.

[1321] The terminal sends the entered user ID and password to the server via HTTPS.

[1322] Input: Authentication information entered by the user

[1323] Output: Authentication information is sent to the server.

[1324] 1.3 The server performs authentication in the database.

[1325] The server compares the received user ID and password with the database, and if authentication is successful, it generates a user-specific interface.

[1326] Input: Authentication information received by the server

[1327] Data processing: Database matching

[1328] sql

[1329] SELECT FROM users WHERE userID='user123' AND password='password';

[1330] Output: Authentication results and user-specific interface

[1331] Step 2: Theme Selection

[1332] 2.1 The user selects the topic they want to learn about.

[1333] The user selects "Python Programming" from the list of themes displayed on the dashboard screen.

[1334] Input: Selected theme (Python programming)

[1335] Output: Theme selection submission request

[1336] 2.2 The device sends theme information to the server.

[1337] The terminal sends the selected subject information to the server.

[1338] Input: user selected theme information

[1339] Output: Theme information is sent to the server.

[1340] 2.3 The server loads the instructor AI model.

[1341] The server loads and launches the appropriate educational support AI model based on the selected theme.

[1342] Input: Received theme information

[1343] Data processing: Loading and initializing AI models

[1344] Python

[1345] AI_model = load_model("Python_Programming_Model")

[1346] Output: Activated educational support AI model

[1347] Step 3: Interacting with the Instructor AI

[1348] 3.1 The server starts the instructor AI model.

[1349] The server starts the loaded instructor AI model and generates the initial message "Let's get started with Python programming."

[1350] Input: Loaded AI model

[1351] Data processing: Initial message generation

[1352] Python

[1353] initial_message = AI_model.generate_initial_message()

[1354] Output: Initial message

[1355] 3.2 Instructor AI sends an initial message to the user

[1356] The instructor AI generates an initial message which is then sent to the user.

[1357] Input: Initial message

[1358] Output: Initial message sent to the user

[1359] 3.3 Users submit questions or instructions during learning

[1360] The user enters a question such as "Please tell me how to use variables" and presses the submit button.

[1361] Input: Questions or instructions

[1362] Output: Request to send questions or instructions

[1363] 3.4 The terminal sends the user's question to the server.

[1364] The terminal sends the user's question to the server.

[1365] Input: User's question

[1366] Output: Question sent to the server

[1367] 3.5 The server passes the question to the instructor AI.

[1368] The server passes the user's question received to the educational support AI.

[1369] Input: Question received from the user

[1370] Output: Questions given to the educational support AI

[1371] 3.6 Instructor AI generates answers to questions

[1372] The educational support AI generates an answer such as, "In Python, variables are declared as follows..." and creates a message that includes an example.

[1373] Input: User's question

[1374] Data processing: Answer generation

[1375] Python

[1376] response = AI_model.generate_response("Please tell me how to use the variables")

[1377] Output: Response message

[1378] 3.7 The instructor AI sends the answer to the user.

[1379] The instructor AI generates the answer and sends it to the user.

[1380] Input: Generated answer

[1381] Output: Response sent to the user

[1382] Step 4: Recording progress

[1383] 4.1 The server records the conversation content and learning progress.

[1384] The server records the content of the conversation between the educational support AI and the user, as well as the learning progress, in a database.

[1385] Input: Dialogue content and learning progress

[1386] Data processing: Saving to a database

[1387] sql

[1388] INSERT INTO progress (userID, content, timestamp) VALUES ('user123', 'Learn how to use variables', NOW());

[1389] Output: Recorded data

[1390] 4.2 Utilize the progress made in the previous session in the next session

[1391] The server retrieves the user's previous learning progress from the database upon their next login and displays the message, "Last time, you learned how to use variables. Do you want to continue?"

[1392] Input: Previous learning progress data

[1393] Data processing: Message generation based on learning progress

[1394] sql

[1395] SELECT FROM progress WHERE userID='user123' ORDER BY timestamp DESC LIMIT 1;

[1396] Output: Learning progress message sent to the user

[1397] The above describes the processing flow in the system of the present invention. Each step is explained in detail, including specific inputs and outputs, and the data processing and calculations based on them.

[1398] (Application Example 1)

[1399] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1400] Traditional online learning systems have presented challenges such as difficulty for users to track their progress in real time and in obtaining quick answers to questions about the learning content. Furthermore, it was difficult for users to manage their own learning progress and plan their next learning sessions. Additionally, the lack of widespread use of mobile devices such as smartphones has resulted in a lack of convenience.

[1401] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1402] In this invention, the server includes means for the user to input and send authentication information to the server; means for the server to verify the received authentication information and generate a user dashboard if authentication is successful; means for the user to select a topic they wish to learn and send that selection information to the server; means for the server to activate an instructor AI based on the selected topic and begin a dialogue with the user; means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session; and means for the user to interact with the instructor AI in real time using natural language with a smartphone and provide responses to questions using prompt sentences generated by the AI. This allows the user to grasp their learning progress in real time and obtain quick answers, improving the convenience of learning using a smartphone.

[1403] A "user" is someone who uses the system to learn online.

[1404] "Authentication information" refers to information used to identify an individual, such as a user ID and password, which a user enters when accessing a system.

[1405] A "server" is a computer system that receives authentication information from users, performs searches, and performs authentication.

[1406] The "user dashboard" is an interface generated for users who have successfully authenticated, displaying their learning progress, selected learning themes, and other relevant information.

[1407] A "theme" is information that indicates the content or field that the user wants to learn about.

[1408] "Instructor AI" is an artificial intelligence system that is activated based on the theme selected by the user, interacts with the user, and generates answers in response to questions.

[1409] "Natural language" refers to the language that humans use on a daily basis, which is used for communication between the user and the instructor AI.

[1410] A "prompt message" is a series of texts that the instructor AI generates and provides as a response to a user's question.

[1411] A "smartphone" is a portable, multi-functional telephone that users use as their personal device to access systems and communicate.

[1412] "Real-time" refers to a state of immediate response where a user asks a question to the instructor AI and receives an answer right away.

[1413] This invention is a system for users to learn online while interacting in real time with an instructor AI using a smartphone. The system consists of a user terminal, a server, an instructor AI model, and a database. This allows users to receive appropriate learning advice and quick responses.

[1414] System Configuration and Operation

[1415] Hardware and software usage

[1416] User terminal: A smartphone is used. This device is used by the user to enter authentication information, select a theme, and interact with the instructor AI.

[1417] Server: This is the central device that verifies authentication information, activates the instructor AI, records the conversation content, and prepares the content for the next learning session.

[1418] Instructor AI Model: This artificial intelligence is activated based on the user's learning theme, interacts with the user in natural language, and generates prompt sentences in response to questions.

[1419] Database: A storage device that records user authentication information, learning progress information, dialogue content, etc.

[1420] Specific operating procedures and functions

[1421] 1. User Authentication

[1422] The user launches the app on their smartphone and enters their authentication information (user ID and password) on the login screen.

[1423] The user terminal sends this authentication information to the server, and the server performs authentication by comparing the received authentication information with the database.

[1424] If authentication is successful, the server will generate a user dashboard.

[1425] 2. Theme Selection

[1426] The user selects the topic they want to learn from the dashboard. For example, they might select "Python Programming".

[1427] The user terminal sends the selected theme information to the server, and the server loads the corresponding instructor AI model.

[1428] 3. Interaction with the Instructor AI

[1429] The server starts the loaded instructor AI model and generates an initial message. For example, it might generate a message like, "Let's get started with Python programming. We'll begin by learning basic data types."

[1430] The user sends a question to the instructor AI during the learning process (e.g., "Please explain how to use variables").

[1431] The user's terminal sends this question to the server, which then passes it on to the instructor AI.

[1432] The instructor AI generates an answer to the question (e.g., "In Python, variables are declared as follows...") and sends that answer to the user's terminal.

[1433] 4. Recording progress and preparing for the next learning session.

[1434] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database.

[1435] For the next session, the server will prepare appropriate learning content based on the progress from the previous session, and the instructor AI will provide the next learning content.

[1436] Examples of prompt statements

[1437] Log in with user ID and password: 'user123', 'password'

[1438] Select a topic you want to learn: 'Python Programming'

[1439] Question: 'How do I use variables?'

[1440] Save progress

[1441] In this way, the present invention enables users to efficiently and effectively learn by interacting with an instructor AI in real time using their smartphone.

[1442] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1443] Step 1:

[1444] User Authentication

[1445] The user launches the app on their smartphone and enters their authentication information (user ID and password).

[1446] The device sends this authentication information to the server. The entered data is the user ID and password.

[1447] The server compares the received authentication information with the database. If authentication is successful, it generates a user-specific dashboard and sends it to the user's terminal. The user's profile information is output upon successful authentication. An error message is output if authentication fails.

[1448] Step 2:

[1449] Theme Selection

[1450] The user selects a learning topic (for example, "Python Programming") from the dashboard.

[1451] The terminal sends the selected information (theme name) to the server. The entered data is the selected theme name.

[1452] The server loads and launches the appropriate instructor AI model from the database based on the selected theme. It also sends an initial message to the user's terminal based on the selected theme (e.g., "Let's start Python programming.").

[1453] Step 3:

[1454] Dialogue with the Instructor AI

[1455] The user enters and submits a question during the learning process (e.g., "Please explain how to use variables").

[1456] The terminal sends the question to the server. The input data is the user's question.

[1457] The server passes the question to the instructor AI, which then generates an answer. The answer includes a generated prompt (e.g., "In Python, variables are declared as follows...").

[1458] The server sends the instructor AI's response to the user's terminal and displays it to the user.

[1459] Step 4:

[1460] Progress log

[1461] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database. The input data consists of the interaction content and learning progress information.

[1462] We will save this data to use in the next session and prepare appropriate learning content for the next session.

[1463] Step 5:

[1464] Preparation for the next lesson

[1465] The server automatically prepares the next learning content based on the learning progress information stored. The input data is past learning progress information.

[1466] Based on the learning content prepared by the server, the instructor AI generates scenarios and prompts to provide appropriate learning advice, and saves them for use in the next learning session.

[1467] In this way, users can engage in online learning more efficiently and effectively through the system.

[1468] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1469] This invention is a system for users to learn online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the learning experience. This system allows users to learn anytime, anywhere, while interacting with an instructor AI, and to manage their progress and emotions.

[1470] This system consists of the following main components: user terminals, servers, instructor AI models, emotion engines, and databases. The entire system uses secure communication protocols (e.g., HTTPS) to exchange data.

[1471] The main functions and operation of the system

[1472] 1. User Authentication

[1473] The user accesses the login screen and enters their authentication information (user ID and password). This information is then sent from the terminal to the server.

[1474] The server compares the received authentication information with the database and, if authentication is successful, generates a user-specific dashboard.

[1475] 2. Theme Selection

[1476] The user selects a topic they want to learn about on the dashboard (e.g., "Python Programming").

[1477] The device sends the selected theme information to the server.

[1478] The server receives the theme information and loads the corresponding instructor AI model.

[1479] 3. Interaction with the Instructor AI

[1480] The server starts the loaded instructor AI model and generates an initial message.

[1481] The instructor AI will send this message to the user.

[1482] Users can send questions and instructions to the instructor AI while learning.

[1483] The device sends these messages to the server, which then passes them on to the instructor AI.

[1484] The instructor AI generates answers to the user's questions and sends them to the user.

[1485] 4. Emotion recognition

[1486] The server passes user input data (e.g., text messages) to the emotion engine, which recognizes the user's emotions in real time.

[1487] The emotion engine recognizes the emotion information, which the server then provides to the instructor AI.

[1488] The instructor AI generates responses by considering emotional information and adjusts the dialogue according to the user's emotions.

[1489] 5. Record progress and emotions

[1490] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[1491] This information will be used in the next session.

[1492] Explanation with specific examples

[1493] 1. The user logs in.

[1494] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[1495] The terminal receives this and sends it to the server.

[1496] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific dashboard.

[1497] 2. The user selects a theme.

[1498] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[1499] The device sends this selection information to the server.

[1500] The server selects and starts the appropriate instructor AI model.

[1501] 3. Start the conversation with the instructor AI.

[1502] The instructor AI generates an initial message saying, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user.

[1503] The user asks, "How do I use variables?"

[1504] The device sends the question to the server, which then passes it on to the instructor AI.

[1505] The instructor AI responds with "In Python, variables are declared like this..." and generates a message that includes an example.

[1506] 4. Emotion Recognition and Adaptation

[1507] The server uses an emotion engine to recognize the user's emotions (e.g., confusion, relief) based on the content of the user's questions.

[1508] The emotion engine recognizes the emotions, which the server then provides to the instructor AI.

[1509] The instructor AI adjusts its response based on the user's emotions, saying something like, "You seem confused. Let me explain in more detail," and sends it to the user.

[1510] 5. Managing progress and emotions

[1511] The server records the conversation content, learning progress, and recognized emotion information in a database.

[1512] In the next session, dialogue will be provided that takes into account not only what was learned in the previous session, but also the recognized emotional information.

[1513] In this way, by monitoring the user's learning experience in real time and providing optimal feedback based on their emotions, the quality and efficiency of learning can be improved. With such a system, users can always receive appropriate learning support, making it easier to maintain motivation for self-study.

[1514] The following describes the processing flow.

[1515] Step 1:

[1516] The user accesses the login screen and enters their authentication information (user ID and password). The user then clicks the "Login" button.

[1517] Step 2:

[1518] The terminal encodes the entered authentication information and sends it to the server using a secure communication protocol (HTTPS).

[1519] Step 3:

[1520] The server compares the received authentication information with the database. If the user ID and password match, authentication is considered successful.

[1521] Step 4:

[1522] The server generates a user-specific dashboard page along with an authentication success message and sends it to the device. If authentication fails, an error message is sent.

[1523] Step 5:

[1524] The user accesses the dashboard page and selects a topic they want to learn about from the provided list of topics (e.g., "Python Programming").

[1525] Step 6:

[1526] The device sends the selected theme information to the server.

[1527] Step 7:

[1528] The server receives the theme information and loads the corresponding instructor AI model. Once loading is complete, it generates an initial message.

[1529] Step 8:

[1530] The instructor AI generates an initial message, "Let's start Python programming," and sends it to the user.

[1531] Step 9:

[1532] The user enters questions or requests while learning, and the device sends them to the server.

[1533] Step 10:

[1534] The server passes the user's questions and requests to the instructor AI. The instructor AI then generates answers to those questions.

[1535] Step 11:

[1536] The instructor AI returns the generated answer to the server, which then sends that answer to the user.

[1537] Step 12:

[1538] The server passes user input data to the emotion engine, which recognizes the user's emotions in real time.

[1539] Step 13:

[1540] The emotion engine recognizes emotions from the user's input data and provides that emotion information to the server.

[1541] Step 14:

[1542] The server passes the user's emotional information, obtained from the emotion engine, to the instructor AI. The instructor AI then adjusts its response based on that emotional information.

[1543] Step 15:

[1544] The instructor AI generates emotion-based responses, creating messages such as, "You seem confused. Let me explain in more detail," and sends them to the user.

[1545] Step 16:

[1546] Users receive emotionally sensitive responses and continue learning. If they have additional questions or requests, they can enter them and submit again.

[1547] Step 17:

[1548] When a learning session ends, the server records the user-instructor AI dialogue, learning progress, and recognized emotion information in a database.

[1549] Step 18:

[1550] The server uses recorded progress data and sentiment information to prepare the content for the next learning session. In the next session, new themes and problems will be presented, taking into account the progress and sentiment information from the previous session.

[1551] The above outlines the specific processing steps of the system based on the invention that combines an emotion engine. This system allows users to effectively learn by interacting with an instructor AI anytime, anywhere, and to receive emotion-sensitive feedback.

[1552] (Example 2)

[1553] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1554] Traditional online learning systems often struggle to provide immediate and appropriate responses to user questions, potentially leading to decreased learning efficiency. Furthermore, they fail to optimize individual learning experiences by providing uniform responses without considering user emotions. Additionally, a lack of effective methods for reflecting learning progress in subsequent sessions hinders the overall improvement of the learning experience.

[1555] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the user to input and transmit authentication information to the server, means for the server to verify the received authentication information and generate a user dashboard if authentication is successful, means for the user to select a theme they want to learn and transmit the selection information to the server, means for the server to activate an instructional AI based on the selected theme and start a dialogue with the user, means for the server to pass the user's input data to an analysis engine and recognize the user's emotions in real time, means for the server to provide the recognized emotion information to the instructor AI and generate a response, and means for the server to record the content of the dialogue with the instructional AI and save the data for use in the next session. This enables the generation of appropriate responses in real time that take into account the user's emotions and the effective preparation of the next learning content according to the learning progress.

[1556] "User authentication information" refers to information such as the user ID and password that a user uses to log in to the system.

[1557] A "server" is a computer system that receives requests from users on a network and processes them accordingly.

[1558] The "user dashboard" is an interface that users can access after logging in, where they can check their learning progress, select topics, and more.

[1559] A "theme" refers to a specific learning topic or subject that a user wants to study.

[1560] "Teaching AI" refers to a model that uses artificial intelligence to provide educational guidance to users, including natural language dialogue and learning support.

[1561] An "analysis engine" is a collection of software and hardware that analyzes user input data and identifies specific information (e.g., emotions).

[1562] "Emotional information" refers to information that indicates the user's emotional state, extracted from the user's input data by the analysis engine.

[1563] "Dialogue content" refers to the record of texts and messages exchanged between the user and the instructional AI.

[1564] A "data storage method" refers to a means by which a server records dialogue content and learning progress information and stores it for use in the next session.

[1565] Modes for carrying out the invention

[1566] This invention is a system for users to learn online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the learning experience. The main components of the system are a user terminal, a server, an instructional AI model, an analysis engine, and a database. The entire system exchanges data using a secure communication protocol (e.g., HTTPS).

[1567] System Overview

[1568] This system consists of the following main components:

[1569] 1. User terminal:

[1570] This refers to the device that the user accesses, such as a smartphone, tablet, or personal computer.

[1571] 2. Server:

[1572] This is a computer system that performs user authentication, data analysis, manages the AI ​​for instruction, and communicates with the database.

[1573] 3. Instructional AI Model:

[1574] It is an artificial intelligence that interacts with users, using natural language processing to generate responses to user questions.

[1575] 4. Analysis Engine:

[1576] This part analyzes user input data (e.g., text messages) in real time to recognize emotions and intentions.

[1577] 5. Database:

[1578] This is a storage system that stores user authentication information, learning progress, conversation content, and sentiment data.

[1579] System operation

[1580] This system operates using the following steps:

[1581] 1. User Authentication:

[1582] The user accesses the login screen and enters their user ID and password. The entered information is sent from the terminal to the server.

[1583] The server verifies the authentication information against the database, and if authentication is successful, it generates a user-specific dashboard.

[1584] 2. Theme Selection:

[1585] The user selects the topic they want to learn about on the dashboard.

[1586] The device sends the selected theme information to the server, and the server loads the corresponding instructional AI model.

[1587] 3. Interaction with the instructional AI:

[1588] The server activates the instructional AI model and generates an initial message. The instructional AI then sends this message to the user.

[1589] The user enters and sends questions or instructions. The device sends these messages to the server, which then passes them on to the instructional AI.

[1590] The instructional AI generates answers to the user's questions and sends them to the user.

[1591] 4. Emotion recognition:

[1592] The server passes user input data to an analysis engine, which recognizes the user's emotions in real time.

[1593] The emotion engine recognizes the emotion information, which the server then provides to the guidance AI. The guidance AI then generates a response, taking the emotion information into consideration.

[1594] 5. Recording progress and emotions:

[1595] The server records the content of the interaction between the teaching AI and the user, as well as learning progress and recognized emotion information, in a database.

[1596] This information will be used in the next session.

[1597] Hardware and software to be used

[1598] User terminal:

[1599] Smartphone, tablet, or computer (e.g., iPhone, iPad, Windows PC)

[1600] server:

[1601] Cloud services or on-premises computer systems (e.g., AWS, Microsoft Azure)

[1602] Instructional AI model:

[1603] Generative AI models that use natural language processing (e.g., GPT-3)

[1604] Analysis engine:

[1605] Software that analyzes user emotions (e.g., IBM Watson Tone Analyzer)

[1606] Database:

[1607] SQL database or NoSQL database (e.g., MySQL, MongoDB)

[1608] Specific example

[1609] The user launches the smartphone app, enters the ID "user123" and password "password" on the login screen, and presses the login button. The device receives this information and sends it to the server. The server verifies the authentication information against the database, and if authentication is successful, displays a user-specific dashboard. The user selects "Python Programming" on the dashboard screen, and the device sends this selection information to the server. The server selects an appropriate teaching AI model and starts it. The teaching AI generates an initial message, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user. The user enters a question, "Please teach me how to use variables," and presses the send button. The device sends the question to the server, which passes it to the teaching AI. The teaching AI responds, "In Python, variables are declared as follows...", and generates a concrete example. The server uses an emotion engine to recognize the user's emotions (e.g., confusion, relief) from the user's question. The server provides the emotions recognized by the emotion engine to the teaching AI, which adjusts its response based on the emotions and sends it to the user. The server records the dialogue content, learning progress, and recognized emotion information in the database and uses this information in the next session.

[1610] In this way, by monitoring the user's learning experience in real time and providing optimal feedback based on their emotions, the quality and efficiency of learning can be improved.

[1611] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1612] Step 1:

[1613] The user enters and submits their login information (user ID and password). The terminal receives this information and sends it to the server. The input is the user ID and password, and the output is the login request sent to the server.

[1614] Step 2:

[1615] The server compares the received authentication information with the database. Data processing here involves extracting user authentication information from the database using SQL queries and comparing it with the input information. The output is whether the authentication was successful or not.

[1616] Step 3:

[1617] If authentication is successful, the server generates a user-specific dashboard and sends the information to the device. The output is data for displaying the dashboard, which is then sent to the device. The device receives this data and displays the dashboard to the user.

[1618] Step 4:

[1619] The user selects a topic they want to learn about on the dashboard and submits the topic information. The device receives this information and sends it to the server. The input is the topic selected by the user, and the output is the topic information sent to the server.

[1620] Step 5:

[1621] The server loads the corresponding instructional AI model based on the received theme information. Here, data processing involves loading the AI ​​model corresponding to the theme. The output is the initialization and startup of the AI ​​model.

[1622] Step 6:

[1623] The instructional AI generates an initial message and sends it to the terminal via the server. The input is the loaded AI model and the prompt for generating the initial message, and the output is the initial message sent to the user. The terminal displays this message to the user.

[1624] Step 7:

[1625] The user inputs questions or instructions and sends that information. The terminal receives this message and sends it to the server. The input is the user's questions or instructions, and the output is the user input data sent to the server.

[1626] Step 8:

[1627] The server passes user input data to the guidance AI, which then generates an answer. The input consists of the user's questions and instructions, while the output is the generated answer message.

[1628] Step 9:

[1629] The server sends the generated response message to the terminal, which then displays it to the user. The input is the response message from the AI, and the output is the message displayed to the user.

[1630] Step 10:

[1631] The server passes user input data to the analysis engine, which recognizes emotions in real time. The data processing here involves natural language processing for emotion analysis. The input is user input data, and the output is the recognized emotion information.

[1632] Step 11:

[1633] The emotion engine recognizes emotional information, which the server then provides to the guidance AI. Based on this information, the AI ​​generates a response. The input is emotional information, and the output is a response message that takes that emotional information into account.

[1634] Step 12:

[1635] The server records the content of the interaction between the teaching AI and the user, as well as learning progress and recognized emotion information, in a database. The inputs are the interaction content, learning progress, and emotion information, while the output is the information stored in the database.

[1636] Step 13:

[1637] In the next session, the server will have the instructional AI generate a customized learning plan based on the recorded information. The input will be the content of the previous conversation and emotional information, and the output will be the customized learning plan. The user will then start learning again based on this information.

[1638] In this way, by analyzing user sentiment information and learning progress in real time and providing appropriate feedback, the learning experience is improved.

[1639] (Application Example 2)

[1640] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1641] Conventional instructor AI systems failed to recognize the user's emotional state and could only provide uniform dialogue, resulting in a decline in the quality of the learning experience. Similarly, in factory robots, the inability to provide feedback and instructions that considered the operator's emotions posed a challenge to work efficiency and safety.

[1642] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input and transmit authentication information to the server; means for the server to verify the received authentication information and generate a user dashboard if authentication is successful; means for the user to select a theme they wish to learn and transmit the selection information to the server; means for the server to activate an instructor AI based on the selected theme and begin a dialogue with the user; means for the user to recognize their emotions and for the instructor AI to adjust the content of the dialogue based on those emotions; means installed in a robot to recognize the operator's emotional state and provide appropriate feedback and instructions; and means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session. This makes it possible to provide adjusted dialogue and instructions according to the emotional state of the user and operator.

[1643] User authentication is the process by which a user enters authentication information, sends it to a server for verification, and then verifies it.

[1644] A "server" is a computer system that receives user authentication information and selection information and processes it based on that information.

[1645] A "user dashboard" is a screen that the server generates upon successful authentication and allows the user to access it.

[1646] An "instructor AI" is an artificial intelligence model that interacts with users and operators, providing answers to questions and instructions.

[1647] "Emotion recognition" is the process of analyzing user or operator input data (voice, text, facial expressions, etc.) to identify emotional states.

[1648] "Dialogue content adjustment" refers to the instructor AI appropriately modifying the content of the dialogue based on recognized emotions.

[1649] A "robot" is a mechanical device that performs tasks within a factory, interacting with an operator and providing work instructions.

[1650] "Feedback" refers to the responses and instructions that the instructor AI provides to the user or operator.

[1651] A "database" is a data storage system where a server records conversation content, progress, emotional information, and other data for use in subsequent sessions.

[1652] This invention is a system for users to learn and work online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the experience. The system consists of the following main components: a user terminal, a server, an instructor AI model, an emotion engine, and a database. The entire system exchanges data using a secure communication protocol (e.g., HTTPS).

[1653] The main functions and operation of the system

[1654] 1. User Authentication

[1655] The user accesses the login screen and enters their authentication information (user ID and password). This information is then sent from the terminal to the server.

[1656] The server compares the received authentication information with the database and, if authentication is successful, generates a user-specific dashboard.

[1657] 2. Theme Selection

[1658] Users select a topic they want to learn about on the dashboard (e.g., "Python Programming").

[1659] The device sends the selected theme information to the server.

[1660] The server receives the theme information and loads the corresponding instructor AI model.

[1661] 3. Interaction with the Instructor AI

[1662] The server starts the loaded instructor AI model and generates an initial message.

[1663] The instructor AI will send this message to the user.

[1664] Users can send questions and instructions to the instructor AI while learning.

[1665] The device sends these messages to the server, which then passes them on to the instructor AI.

[1666] The instructor AI generates answers to the user's questions and sends them to the user.

[1667] 4. Emotion recognition

[1668] The server passes user input data (e.g., text messages) to the emotion engine, which recognizes the user's emotions in real time.

[1669] The emotion engine recognizes the emotion information, which the server then provides to the instructor AI.

[1670] The instructor AI generates responses by considering emotional information and adjusts the dialogue according to the user's emotions.

[1671] 5. Record progress and emotions

[1672] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[1673] This information will be used in the next session.

[1674] Hardware and software to be used

[1675] Hardware:

[1676] Camera (e.g., Logitech C920): Captures the user's facial expressions.

[1677] Microphone: Captures the user's voice.

[1678] software:

[1679] OpenCV: Used for inputting and processing camera images.

[1680] Emotion Recognition Library (e.g., emotion_recognition): Analyzes the user's emotions.

[1681] AI Model: As an instructor AI, it generates responses based on the task content (e.g., the natural language processing model ChatGPT).

[1682] Database (e.g., MySQL): Manages login information, work details, and sentiment information.

[1683] Specific example

[1684] Specific example 1:

[1685] The user launches the smartphone app, enters the ID "user123" and password "password" on the login screen, and presses the login button. The device receives this information and sends it to the server. The server verifies it against the database, and if authentication is successful, displays a user-specific dashboard.

[1686] The user selects "Python Programming" in the dashboard and sends this selection information to the server. The server receives this information and activates the instructor AI.

[1687] The instructor AI generates an initial message saying, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user. When the user asks, "How do I use variables?", the terminal sends the question to the server, and the instructor AI generates an answer.

[1688] The server passes the user's question to the emotion engine, which recognizes the user's emotions. If the emotion engine recognizes "confusion," the server provides this information to the instructor AI, which responds with, "You seem confused. Let me explain in more detail."

[1689] The server records the conversation content, progress, and emotional information in a database, which will be used in the next session.

[1690] Example of a prompt:

[1691] If an operator appears confused while tightening a bolt, recognize their confusion and provide a clear and detailed explanation.

[1692] This makes it possible to provide users and operators with adjusted dialogue and instructions that are tailored to their emotional state.

[1693] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1694] Program processing steps

[1695] Step 1:

[1696] The user launches the app on their smartphone and enters their user ID and password on the login screen. This generates the input data for user authentication.

[1697] Step 2:

[1698] The terminal encrypts the user authentication input data and sends it to the server. The server compares the received authentication information with the database, and if authentication is successful, generates a user-specific dashboard and sends it back to the terminal.

[1699] Step 3:

[1700] The user selects a topic they want to learn about (e.g., "Python Programming") within the dashboard, and the device sends this selection information to the server. This generates the input data for the topic selection.

[1701] Step 4:

[1702] The server analyzes the theme information and loads the appropriate instructor AI model. The instructor AI generates an initial message and sends it to the terminal.

[1703] Step 5:

[1704] The user inputs a question to the instructor AI, and the device sends the question to the server. The server passes the question to the instructor AI, which generates an answer and sends it back.

[1705] Step 6:

[1706] The server passes the user's questions and the instructor AI's answers to the emotion engine, which recognizes the user's emotional state in real time. This process generates emotion recognition data.

[1707] Step 7:

[1708] The emotion engine recognizes the emotion information, which the server provides to the instructor AI, and the instructor AI adjusts the content of the dialogue. The server then sends the adjusted response generated by the AI ​​to the terminal.

[1709] Step 8:

[1710] Review the user's response and either ask the next question or continue the learning process.

[1711] Step 9:

[1712] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[1713] Step 10:

[1714] In the next session, the server will customize the instructor AI model based on past conversations and progress information to provide appropriate learning advice.

[1715] This makes it possible to provide adjusted dialogue and instructions that respond to various emotional states, such as when a user is confused or agitated.

[1716] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1717] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1718] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1719] [Fourth Embodiment]

[1720] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1721] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1722] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1723] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1724] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1725] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1726] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1727] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1728] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1729] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1730] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1731] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1732] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1733] This invention is a system for users to learn online via an internet connection. This system allows users to learn anytime, anywhere, while interacting with an AI instructor, and to manage their progress.

[1734] This system consists of the following main components: user terminals, servers, instructor AI models, and databases. The entire system also uses secure communication protocols (e.g., HTTPS) for data exchange.

[1735] The main functions and operation of the system

[1736] 1. User Authentication

[1737] The user enters their authentication information (user ID and password) on the login screen and presses the submit button.

[1738] The device sends these authentication credentials to the server.

[1739] The server compares the received authentication information with the database, and if authentication is successful, it generates a user-specific dashboard.

[1740] 2. Theme Selection

[1741] The user selects a topic they want to learn about from the dashboard (e.g., "Python Programming").

[1742] The device sends the selected theme information to the server.

[1743] The server loads the appropriate instructor AI model based on the selected theme.

[1744] 3. Interaction with the Instructor AI

[1745] The server starts the loaded instructor AI model and generates an initial message (e.g., "Let's get started with Python programming.").

[1746] The instructor AI will send this message to the user.

[1747] Users can send questions and instructions to the instructor AI while learning.

[1748] The device sends these messages to the server, which then passes them on to the instructor AI.

[1749] The instructor AI generates answers to the user's questions and sends them to the user.

[1750] 4. Recording progress

[1751] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database.

[1752] This information will be used in the next session.

[1753] Explanation with specific examples

[1754] 1. The user logs in.

[1755] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[1756] The terminal receives this and sends it to the server.

[1757] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific dashboard.

[1758] 2. The user selects a theme.

[1759] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[1760] The device sends this selection information to the server.

[1761] The server selects and starts the appropriate instructor AI model.

[1762] 3. Start the conversation with the instructor AI.

[1763] The instructor AI sends the user a message saying, "Let's start Python programming. First, we'll learn about basic data types."

[1764] The user asks, "How do I use variables?"

[1765] The device sends the question to the server, which then passes it on to the instructor AI.

[1766] The instructor AI responds with "In Python, variables are declared like this..." and generates a message that includes an example.

[1767] 4. Progress Management

[1768] The server records the conversation content and learning progress in a database.

[1769] In the next session, the server will provide appropriate learning content based on the progress information from the previous session.

[1770] This allows users to always progress in their learning based on their latest learning progress and receive appropriate feedback from the instructor AI in real time. In this way, the system of the present invention provides efficient and effective learning support to users.

[1771] The following describes the processing flow.

[1772] Step 1:

[1773] The user accesses the login screen and enters their authentication information (user ID and password). The user then clicks the "Login" button.

[1774] Step 2:

[1775] The terminal encodes the entered authentication information and sends it to the server using a secure communication protocol (HTTPS).

[1776] Step 3:

[1777] The server compares the received authentication information with the database. If the user ID and password match, authentication is considered successful.

[1778] Step 4:

[1779] The server generates a user-specific dashboard page along with an authentication success message and sends it to the device. If authentication fails, an error message is sent.

[1780] Step 5:

[1781] The user accesses the dashboard page and selects a topic they want to learn about from the provided list of topics (e.g., "Python Programming").

[1782] Step 6:

[1783] The device sends the selected theme information to the server.

[1784] Step 7:

[1785] The server receives the theme information and loads the corresponding instructor AI model. Once loading is complete, it generates an initial message.

[1786] Step 8:

[1787] The instructor AI generates an initial message, "Let's start Python programming," and sends it to the user.

[1788] Step 9:

[1789] The user enters questions or requests while learning, and the device sends them to the server.

[1790] Step 10:

[1791] The server passes the user's questions and requests to the instructor AI. The instructor AI then generates answers to those questions.

[1792] Step 11:

[1793] The instructor AI returns the generated answer to the server, which then sends that answer to the user.

[1794] Step 12:

[1795] The user receives answers from the instructor AI and continues learning. If there are any additional questions or requests, they can enter them again.

[1796] Step 13:

[1797] When a learning session ends, the server records the content of the user-instructor AI conversation and the learning progress in a database.

[1798] Step 14:

[1799] The server uses the recorded progress data to prepare the content for the next learning session. In the next session, new themes and problems will be provided based on the progress made in the previous session.

[1800] The above explains the program's processing in detail, step by step. This system allows users to effectively learn by interacting with the instructor AI anytime, anywhere.

[1801] (Example 1)

[1802] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1803] Traditional online learning systems lacked sufficient interactive learning support based on themes selected by learners. Furthermore, managing learning progress and preparing future learning plans required manual intervention, highlighting the need for more efficient learning support. Additionally, there was a lack of systems that could provide appropriate real-time answers to learners' questions. Therefore, the development of a system that provides more effective and interactive learning support was essential.

[1804] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1805] In this invention, the server includes means for the user to input authentication information and transmit it to the server via a computer network; means for the server to compare the received authentication information with a database and generate a user-specific interface if authentication is successful; means for the user to select a subject they wish to learn and transmit that selection information to the server; means for the server to load an educational support AI based on the selected subject and initiate a dialogue with the user in natural language; means for the educational support AI to generate answers to the user's questions in real time and transmit them to the user; means for the server to record the content of the dialogue with the educational support AI in a database and save the data for use in the next session; and means for the server to prepare the content of the next learning session based on the previous learning progress and for the educational support AI to provide appropriate learning advice as needed. As a result, the user can always receive appropriate learning support based on their latest learning progress.

[1806] A "user" refers to an individual or group that uses the system to learn.

[1807] "Authentication information" refers to identification information used by users to access the system, such as user IDs and passwords.

[1808] A "computer network" refers to infrastructure used for data communication between computer systems, such as the internet and local area networks (LANs).

[1809] A "server" refers to a computer system that receives and processes requests from users.

[1810] A "database" refers to a system that efficiently stores large amounts of information and allows for searching and updating.

[1811] "Interface" refers to the screens and methods of operation that users use to interact with a system.

[1812] "Subject" refers to the specific learning content or theme that the user wants to learn about.

[1813] "Educational support AI" refers to programs and systems that use artificial intelligence technology to support users' learning.

[1814] "Natural language" refers to the language that humans use on a daily basis, which is converted into a format that computer systems can easily understand.

[1815] "Dialogue" refers to two-way communication between a user and a system where information is exchanged bi-directionally.

[1816] "Real-time" refers to the immediate response to user actions and questions.

[1817] "Generating answers" refers to the process of creating appropriate information or answers in response to a user's question.

[1818] "Saving data" refers to recording conversation content and learning progress for future use.

[1819] "Learning progress" refers to information indicating how far a user has progressed in their learning.

[1820] "Learning advice" refers to suggestions and instructions that help users streamline their learning and deepen their understanding.

[1821] This invention provides a system for users to learn online via an internet connection. This system allows users to learn and manage their progress anytime, anywhere, while interacting with an educational support AI. The system consists of the following main components: a user terminal, a server, an educational support AI model, and a database. Furthermore, the entire system can exchange data using a secure communication protocol (e.g., HTTPS).

[1822] Hardware and software to use

[1823] 1. User terminal

[1824] Internet-connected devices such as smartphones, tablets, and personal computers

[1825] Use an application or web browser

[1826] 2. Server

[1827] High-performance computer systems (e.g., Linux servers)

[1828] Server-side scripts (e.g., Python, Node.js)

[1829] Secure authentication libraries (e.g., JWT)

[1830] 3. Database

[1831] Relational databases such as MySQL, PostgreSQL, and MongoDB, or NoSQL databases.

[1832] 4. AI Models for Educational Support

[1833] Generative AI models such as GPT-3 and BERT

[1834] Machine learning libraries such as PyTorch and TensorFlow

[1835] The main functions and operation of the system

[1836] 1. User Authentication

[1837] The user enters their authentication information (user ID and password) on the login screen and presses the submit button.

[1838] The device sends these authentication credentials to the server via HTTPS.

[1839] The server compares the received authentication information with the database, and if authentication is successful, it generates a user-specific interface.

[1840] 2. Theme Selection

[1841] The user selects a topic they want to learn from the dashboard (e.g., "Python Programming").

[1842] The terminal sends the selected subject information to the server.

[1843] The server loads and launches the appropriate educational support AI model based on the selected subject.

[1844] 3. Interaction with educational support AI

[1845] The server launches the loaded educational support AI model and generates an initial message (e.g., "Let's start Python programming.").

[1846] The educational support AI will send this message to the user.

[1847] Users can send questions and instructions to the educational support AI while learning.

[1848] The device sends these messages to the server, which then passes them on to the educational support AI.

[1849] The educational support AI generates answers to the user's questions and sends them to the user.

[1850] 4. Recording progress

[1851] The server records the content of the conversation between the educational support AI and the user, as well as the learning progress, in a database.

[1852] This information will be used in the next session.

[1853] Specific example

[1854] 1. The user logs in.

[1855] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[1856] The terminal receives this and sends it to the server.

[1857] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific interface.

[1858] Example of a prompt:

[1859] "A user is attempting to log in using the ID "user123" and password "password". Please explain the procedure."

[1860] 2. The user selects a theme.

[1861] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[1862] The device sends this selection information to the server.

[1863] The server selects and launches an appropriate AI model for educational support.

[1864] Example of a prompt:

[1865] "The user selected 'Python Programming' from the dashboard. Please describe the next server-side actions to take."

[1866] 3. Start the conversation with the educational support AI.

[1867] The educational support AI sends the user a message saying, "Let's start Python programming. First, we'll learn about basic data types."

[1868] The user asks, "How do I use variables?"

[1869] The device sends the question to the server, which then passes it on to the educational support AI.

[1870] The educational support AI responds with "In Python, variables are declared as follows..." and generates a message that includes an example.

[1871] Example of a prompt:

[1872] "The user asked, 'How do I use variables?' Generate an appropriate response that the educational support AI should provide."

[1873] 4. Progress Management

[1874] The server records the conversation content and learning progress in a database.

[1875] In the next session, the server will provide appropriate learning content based on the progress information from the previous session.

[1876] Example of a prompt:

[1877] "Please explain the procedure for recording the user's interaction with the educational support AI and their learning progress in a database."

[1878] This allows users to effectively progress in their learning based on their latest progress and receive appropriate feedback from the educational support AI in real time. In this way, the system of the present invention provides efficient and effective learning support to users.

[1879] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1880] Program processing flow

[1881] Step 1: User Authentication

[1882] 1.1 The user enters their authentication information on the login screen.

[1883] The user enters user ID "user123" and password "password" on the app's login screen and presses the login button.

[1884] Input: User ID and password

[1885] Output: Request to send input information

[1886] 1.2 The device sends authentication information to the server.

[1887] The terminal sends the entered user ID and password to the server via HTTPS.

[1888] Input: Authentication information entered by the user

[1889] Output: Authentication information is sent to the server.

[1890] 1.3 The server performs authentication in the database.

[1891] The server compares the received user ID and password with the database, and if authentication is successful, it generates a user-specific interface.

[1892] Input: Authentication information received by the server

[1893] Data processing: Database matching

[1894] sql

[1895] SELECT FROM users WHERE userID='user123' AND password='password';

[1896] Output: Authentication results and user-specific interface

[1897] Step 2: Theme Selection

[1898] 2.1 The user selects the topic they want to learn about.

[1899] The user selects "Python Programming" from the list of themes displayed on the dashboard screen.

[1900] Input: Selected theme (Python programming)

[1901] Output: Theme selection submission request

[1902] 2.2 The device sends theme information to the server.

[1903] The terminal sends the selected subject information to the server.

[1904] Input: user selected theme information

[1905] Output: Theme information is sent to the server.

[1906] 2.3 The server loads the instructor AI model.

[1907] The server loads and launches the appropriate educational support AI model based on the selected theme.

[1908] Input: Received theme information

[1909] Data processing: Loading and initializing AI models

[1910] Python

[1911] AI_model = load_model("Python_Programming_Model")

[1912] Output: Activated educational support AI model

[1913] Step 3: Interacting with the Instructor AI

[1914] 3.1 The server starts the instructor AI model.

[1915] The server starts the loaded instructor AI model and generates the initial message "Let's get started with Python programming."

[1916] Input: Loaded AI model

[1917] Data processing: Initial message generation

[1918] Python

[1919] initial_message = AI_model.generate_initial_message()

[1920] Output: Initial message

[1921] 3.2 Instructor AI sends an initial message to the user

[1922] The instructor AI generates an initial message which is then sent to the user.

[1923] Input: Initial message

[1924] Output: Initial message sent to the user

[1925] 3.3 Users submit questions or instructions during learning

[1926] The user enters a question such as "Please tell me how to use variables" and presses the submit button.

[1927] Input: Questions or instructions

[1928] Output: Request to send questions or instructions

[1929] 3.4 The terminal sends the user's question to the server.

[1930] The terminal sends the user's question to the server.

[1931] Input: User's question

[1932] Output: Question sent to the server

[1933] 3.5 The server passes the question to the instructor AI.

[1934] The server passes the user's question received to the educational support AI.

[1935] Input: Question received from the user

[1936] Output: Questions given to the educational support AI

[1937] 3.6 Instructor AI generates answers to questions

[1938] The educational support AI generates an answer such as, "In Python, variables are declared as follows..." and creates a message that includes an example.

[1939] Input: User's question

[1940] Data processing: Answer generation

[1941] Python

[1942] response = AI_model.generate_response("Please tell me how to use the variables")

[1943] Output: Response message

[1944] 3.7 The instructor AI sends the answer to the user.

[1945] The instructor AI generates the answer and sends it to the user.

[1946] Input: Generated answer

[1947] Output: Response sent to the user

[1948] Step 4: Recording progress

[1949] 4.1 The server records the conversation content and learning progress.

[1950] The server records the content of the conversation between the educational support AI and the user, as well as the learning progress, in a database.

[1951] Input: Dialogue content and learning progress

[1952] Data processing: Saving to a database

[1953] sql

[1954] INSERT INTO progress (userID, content, timestamp) VALUES ('user123', 'Learn how to use variables', NOW());

[1955] Output: Recorded data

[1956] 4.2 Utilize the progress made in the previous session in the next session

[1957] The server retrieves the user's previous learning progress from the database upon their next login and displays the message, "Last time, you learned how to use variables. Do you want to continue?"

[1958] Input: Previous learning progress data

[1959] Data processing: Message generation based on learning progress

[1960] sql

[1961] SELECT FROM progress WHERE userID='user123' ORDER BY timestamp DESC LIMIT 1;

[1962] Output: Learning progress message sent to the user

[1963] The above describes the processing flow in the system of the present invention. Each step is explained in detail, including specific inputs and outputs, and the data processing and calculations based on them.

[1964] (Application Example 1)

[1965] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1966] Traditional online learning systems have presented challenges such as difficulty for users to track their progress in real time and in obtaining quick answers to questions about the learning content. Furthermore, it was difficult for users to manage their own learning progress and plan their next learning sessions. Additionally, the lack of widespread use of mobile devices such as smartphones has resulted in a lack of convenience.

[1967] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1968] In this invention, the server includes means for the user to input and send authentication information to the server; means for the server to verify the received authentication information and generate a user dashboard if authentication is successful; means for the user to select a topic they wish to learn and send that selection information to the server; means for the server to activate an instructor AI based on the selected topic and begin a dialogue with the user; means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session; and means for the user to interact with the instructor AI in real time using natural language with a smartphone and provide responses to questions using prompt sentences generated by the AI. This allows the user to grasp their learning progress in real time and obtain quick answers, improving the convenience of learning using a smartphone.

[1969] A "user" is someone who uses the system to learn online.

[1970] "Authentication information" refers to information used to identify an individual, such as a user ID and password, which a user enters when accessing a system.

[1971] A "server" is a computer system that receives authentication information from users, performs searches, and performs authentication.

[1972] The "user dashboard" is an interface generated for users who have successfully authenticated, displaying their learning progress, selected learning themes, and other relevant information.

[1973] A "theme" is information that indicates the content or field that the user wants to learn about.

[1974] "Instructor AI" is an artificial intelligence system that is activated based on the theme selected by the user, interacts with the user, and generates answers in response to questions.

[1975] "Natural language" refers to the language that humans use on a daily basis, which is used for communication between the user and the instructor AI.

[1976] A "prompt message" is a series of texts that the instructor AI generates and provides as a response to a user's question.

[1977] A "smartphone" is a portable, multi-functional telephone that users use as their personal device to access systems and communicate.

[1978] "Real-time" refers to a state of immediate response where a user asks a question to the instructor AI and receives an answer right away.

[1979] This invention is a system for users to learn online while interacting in real time with an instructor AI using a smartphone. The system consists of a user terminal, a server, an instructor AI model, and a database. This allows users to receive appropriate learning advice and quick responses.

[1980] System Configuration and Operation

[1981] Hardware and software usage

[1982] User terminal: A smartphone is used. This device is used by the user to enter authentication information, select a theme, and interact with the instructor AI.

[1983] Server: This is the central device that verifies authentication information, activates the instructor AI, records the conversation content, and prepares the content for the next learning session.

[1984] Instructor AI Model: This artificial intelligence is activated based on the user's learning theme, interacts with the user in natural language, and generates prompt sentences in response to questions.

[1985] Database: A storage device that records user authentication information, learning progress information, dialogue content, etc.

[1986] Specific operating procedures and functions

[1987] 1. User Authentication

[1988] The user launches the app on their smartphone and enters their authentication information (user ID and password) on the login screen.

[1989] The user terminal sends this authentication information to the server, and the server performs authentication by comparing the received authentication information with the database.

[1990] If authentication is successful, the server will generate a user dashboard.

[1991] 2. Theme Selection

[1992] The user selects the topic they want to learn from the dashboard. For example, they might select "Python Programming".

[1993] The user terminal sends the selected theme information to the server, and the server loads the corresponding instructor AI model.

[1994] 3. Interaction with the Instructor AI

[1995] The server starts the loaded instructor AI model and generates an initial message. For example, it might generate a message like, "Let's get started with Python programming. We'll begin by learning basic data types."

[1996] The user sends a question to the instructor AI during the learning process (e.g., "Please explain how to use variables").

[1997] The user's terminal sends this question to the server, which then passes it on to the instructor AI.

[1998] The instructor AI generates an answer to the question (e.g., "In Python, variables are declared as follows...") and sends that answer to the user's terminal.

[1999] 4. Recording progress and preparing for the next learning session.

[2000] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database.

[2001] For the next session, the server will prepare appropriate learning content based on the progress from the previous session, and the instructor AI will provide the next learning content.

[2002] Examples of prompt statements

[2003] Log in with user ID and password: 'user123', 'password'

[2004] Select a topic you want to learn: 'Python Programming'

[2005] Question: 'How do I use variables?'

[2006] Save progress

[2007] In this way, the present invention enables users to efficiently and effectively learn by interacting with an instructor AI in real time using their smartphone.

[2008] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2009] Step 1:

[2010] User Authentication

[2011] The user launches the app on their smartphone and enters their authentication information (user ID and password).

[2012] The device sends this authentication information to the server. The entered data is the user ID and password.

[2013] The server compares the received authentication information with the database. If authentication is successful, it generates a user-specific dashboard and sends it to the user's terminal. The user's profile information is output upon successful authentication. An error message is output if authentication fails.

[2014] Step 2:

[2015] Theme Selection

[2016] The user selects a learning topic (for example, "Python Programming") from the dashboard.

[2017] The terminal sends the selected information (theme name) to the server. The entered data is the selected theme name.

[2018] The server loads and launches the appropriate instructor AI model from the database based on the selected theme. It also sends an initial message to the user's terminal based on the selected theme (e.g., "Let's start Python programming.").

[2019] Step 3:

[2020] Dialogue with the Instructor AI

[2021] The user enters and submits a question during the learning process (e.g., "Please explain how to use variables").

[2022] The terminal sends the question to the server. The input data is the user's question.

[2023] The server passes the question to the instructor AI, which then generates an answer. The answer includes a generated prompt (e.g., "In Python, variables are declared as follows...").

[2024] The server sends the instructor AI's response to the user's terminal and displays it to the user.

[2025] Step 4:

[2026] Progress log

[2027] The server records the content of the interaction between the instructor AI and the user, as well as the learning progress, in a database. The input data consists of the interaction content and learning progress information.

[2028] We will save this data to use in the next session and prepare appropriate learning content for the next session.

[2029] Step 5:

[2030] Preparation for the next lesson

[2031] The server automatically prepares the next learning content based on the learning progress information stored. The input data is past learning progress information.

[2032] Based on the learning content prepared by the server, the instructor AI generates scenarios and prompts to provide appropriate learning advice, and saves them for use in the next learning session.

[2033] In this way, users can engage in online learning more efficiently and effectively through the system.

[2034] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2035] This invention is a system for users to learn online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the learning experience. This system allows users to learn anytime, anywhere, while interacting with an instructor AI, and to manage their progress and emotions.

[2036] This system consists of the following main components: user terminals, servers, instructor AI models, emotion engines, and databases. The entire system uses secure communication protocols (e.g., HTTPS) to exchange data.

[2037] The main functions and operation of the system

[2038] 1. User Authentication

[2039] The user accesses the login screen and enters their authentication information (user ID and password). This information is then sent from the terminal to the server.

[2040] The server compares the received authentication information with the database and, if authentication is successful, generates a user-specific dashboard.

[2041] 2. Theme Selection

[2042] The user selects a topic they want to learn about on the dashboard (e.g., "Python Programming").

[2043] The device sends the selected theme information to the server.

[2044] The server receives the theme information and loads the corresponding instructor AI model.

[2045] 3. Interaction with the Instructor AI

[2046] The server starts the loaded instructor AI model and generates an initial message.

[2047] The instructor AI will send this message to the user.

[2048] Users can send questions and instructions to the instructor AI while learning.

[2049] The device sends these messages to the server, which then passes them on to the instructor AI.

[2050] The instructor AI generates answers to the user's questions and sends them to the user.

[2051] 4. Emotion recognition

[2052] The server passes user input data (e.g., text messages) to the emotion engine, which recognizes the user's emotions in real time.

[2053] The emotion engine recognizes the emotion information, which the server then provides to the instructor AI.

[2054] The instructor AI generates responses by considering emotional information and adjusts the dialogue according to the user's emotions.

[2055] 5. Record progress and emotions

[2056] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[2057] This information will be used in the next session.

[2058] Explanation with specific examples

[2059] 1. The user logs in.

[2060] The user launches the app on their smartphone, enters the ID "user123" and password "password" on the login screen, and presses the login button.

[2061] The terminal receives this and sends it to the server.

[2062] The server verifies the authentication information against the database, and if authentication is successful, it displays a user-specific dashboard.

[2063] 2. The user selects a theme.

[2064] Select "Python Programming" from the list of themes displayed on the dashboard screen.

[2065] The device sends this selection information to the server.

[2066] The server selects and starts the appropriate instructor AI model.

[2067] 3. Start the conversation with the instructor AI.

[2068] The instructor AI generates an initial message saying, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user.

[2069] The user asks, "How do I use variables?"

[2070] The device sends the question to the server, which then passes it on to the instructor AI.

[2071] The instructor AI responds with "In Python, variables are declared like this..." and generates a message that includes an example.

[2072] 4. Emotion Recognition and Adaptation

[2073] The server uses an emotion engine to recognize the user's emotions (e.g., confusion, relief) based on the content of the user's questions.

[2074] The emotion engine recognizes the emotions, which the server then provides to the instructor AI.

[2075] The instructor AI adjusts its response based on the user's emotions, saying something like, "You seem confused. Let me explain in more detail," and sends it to the user.

[2076] 5. Managing progress and emotions

[2077] The server records the conversation content, learning progress, and recognized emotion information in a database.

[2078] In the next session, dialogue will be provided that takes into account not only what was learned in the previous session, but also the recognized emotional information.

[2079] In this way, by monitoring the user's learning experience in real time and providing optimal feedback based on their emotions, the quality and efficiency of learning can be improved. With such a system, users can always receive appropriate learning support, making it easier to maintain motivation for self-study.

[2080] The following describes the processing flow.

[2081] Step 1:

[2082] The user accesses the login screen and enters their authentication information (user ID and password). The user then clicks the "Login" button.

[2083] Step 2:

[2084] The terminal encodes the entered authentication information and sends it to the server using a secure communication protocol (HTTPS).

[2085] Step 3:

[2086] The server compares the received authentication information with the database. If the user ID and password match, authentication is considered successful.

[2087] Step 4:

[2088] The server generates a user-specific dashboard page along with an authentication success message and sends it to the device. If authentication fails, an error message is sent.

[2089] Step 5:

[2090] The user accesses the dashboard page and selects a topic they want to learn about from the provided list of topics (e.g., "Python Programming").

[2091] Step 6:

[2092] The device sends the selected theme information to the server.

[2093] Step 7:

[2094] The server receives the theme information and loads the corresponding instructor AI model. Once loading is complete, it generates an initial message.

[2095] Step 8:

[2096] The instructor AI generates an initial message, "Let's start Python programming," and sends it to the user.

[2097] Step 9:

[2098] The user enters questions or requests while learning, and the device sends them to the server.

[2099] Step 10:

[2100] The server passes the user's questions and requests to the instructor AI. The instructor AI then generates answers to those questions.

[2101] Step 11:

[2102] The instructor AI returns the generated answer to the server, which then sends that answer to the user.

[2103] Step 12:

[2104] The server passes user input data to the emotion engine, which recognizes the user's emotions in real time.

[2105] Step 13:

[2106] The emotion engine recognizes emotions from the user's input data and provides that emotion information to the server.

[2107] Step 14:

[2108] The server passes the user's emotional information, obtained from the emotion engine, to the instructor AI. The instructor AI then adjusts its response based on that emotional information.

[2109] Step 15:

[2110] The instructor AI generates emotion-based responses, creating messages such as, "You seem confused. Let me explain in more detail," and sends them to the user.

[2111] Step 16:

[2112] Users receive emotionally sensitive responses and continue learning. If they have additional questions or requests, they can enter them and submit again.

[2113] Step 17:

[2114] When a learning session ends, the server records the user-instructor AI dialogue, learning progress, and recognized emotion information in a database.

[2115] Step 18:

[2116] The server uses recorded progress data and sentiment information to prepare the content for the next learning session. In the next session, new themes and problems will be presented, taking into account the progress and sentiment information from the previous session.

[2117] The above outlines the specific processing steps of the system based on the invention that combines an emotion engine. This system allows users to effectively learn by interacting with an instructor AI anytime, anywhere, and to receive emotion-sensitive feedback.

[2118] (Example 2)

[2119] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2120] Traditional online learning systems often struggle to provide immediate and appropriate responses to user questions, potentially leading to decreased learning efficiency. Furthermore, they fail to optimize individual learning experiences by providing uniform responses without considering user emotions. Additionally, a lack of effective methods for reflecting learning progress in subsequent sessions hinders the overall improvement of the learning experience.

[2121] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for the user to input and transmit authentication information to the server, means for the server to verify the received authentication information and generate a user dashboard if authentication is successful, means for the user to select a theme they want to learn and transmit the selection information to the server, means for the server to activate an instructional AI based on the selected theme and start a dialogue with the user, means for the server to pass the user's input data to an analysis engine and recognize the user's emotions in real time, means for the server to provide the recognized emotion information to the instructor AI and generate a response, and means for the server to record the content of the dialogue with the instructional AI and save the data for use in the next session. This enables the generation of appropriate responses in real time that take into account the user's emotions and the effective preparation of the next learning content according to the learning progress.

[2122] "User authentication information" refers to information such as the user ID and password that a user uses to log in to the system.

[2123] A "server" is a computer system that receives requests from users on a network and processes them accordingly.

[2124] The "user dashboard" is an interface that users can access after logging in, where they can check their learning progress, select topics, and more.

[2125] A "theme" refers to a specific learning topic or subject that a user wants to study.

[2126] "Teaching AI" refers to a model that uses artificial intelligence to provide educational guidance to users, including natural language dialogue and learning support.

[2127] An "analysis engine" is a collection of software and hardware that analyzes user input data and identifies specific information (e.g., emotions).

[2128] "Emotional information" refers to information that indicates the user's emotional state, extracted from the user's input data by the analysis engine.

[2129] "Dialogue content" refers to the record of texts and messages exchanged between the user and the instructional AI.

[2130] A "data storage method" refers to a means by which a server records dialogue content and learning progress information and stores it for use in the next session.

[2131] Modes for carrying out the invention

[2132] This invention is a system for users to learn online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the learning experience. The main components of the system are a user terminal, a server, an instructional AI model, an analysis engine, and a database. The entire system exchanges data using a secure communication protocol (e.g., HTTPS).

[2133] System Overview

[2134] This system consists of the following main components:

[2135] 1. User terminal:

[2136] This refers to the device that the user accesses, such as a smartphone, tablet, or personal computer.

[2137] 2. Server:

[2138] This is a computer system that performs user authentication, data analysis, manages the AI ​​for instruction, and communicates with the database.

[2139] 3. Instructional AI Model:

[2140] It is an artificial intelligence that interacts with users, using natural language processing to generate responses to user questions.

[2141] 4. Analysis Engine:

[2142] This part analyzes user input data (e.g., text messages) in real time to recognize emotions and intentions.

[2143] 5. Database:

[2144] This is a storage system that stores user authentication information, learning progress, conversation content, and sentiment data.

[2145] System operation

[2146] This system operates using the following steps:

[2147] 1. User Authentication:

[2148] The user accesses the login screen and enters their user ID and password. The entered information is sent from the terminal to the server.

[2149] The server verifies the authentication information against the database, and if authentication is successful, it generates a user-specific dashboard.

[2150] 2. Theme Selection:

[2151] The user selects the topic they want to learn about on the dashboard.

[2152] The device sends the selected theme information to the server, and the server loads the corresponding instructional AI model.

[2153] 3. Interaction with the instructional AI:

[2154] The server activates the instructional AI model and generates an initial message. The instructional AI then sends this message to the user.

[2155] The user enters and sends questions or instructions. The device sends these messages to the server, which then passes them on to the instructional AI.

[2156] The instructional AI generates answers to the user's questions and sends them to the user.

[2157] 4. Emotion recognition:

[2158] The server passes user input data to an analysis engine, which recognizes the user's emotions in real time.

[2159] The emotion engine recognizes the emotion information, which the server then provides to the guidance AI. The guidance AI then generates a response, taking the emotion information into consideration.

[2160] 5. Recording progress and emotions:

[2161] The server records the content of the interaction between the teaching AI and the user, as well as learning progress and recognized emotion information, in a database.

[2162] This information will be used in the next session.

[2163] Hardware and software to be used

[2164] User terminal:

[2165] Smartphone, tablet, or computer (e.g., iPhone, iPad, Windows PC)

[2166] server:

[2167] Cloud services or on-premises computer systems (e.g., AWS, Microsoft Azure)

[2168] Instructional AI model:

[2169] Generative AI models that use natural language processing (e.g., GPT-3)

[2170] Analysis engine:

[2171] Software that analyzes user emotions (e.g., IBM Watson Tone Analyzer)

[2172] Database:

[2173] SQL database or NoSQL database (e.g., MySQL, MongoDB)

[2174] Specific example

[2175] The user launches the smartphone app, enters the ID "user123" and password "password" on the login screen, and presses the login button. The device receives this information and sends it to the server. The server verifies the authentication information against the database, and if authentication is successful, displays a user-specific dashboard. The user selects "Python Programming" on the dashboard screen, and the device sends this selection information to the server. The server selects an appropriate teaching AI model and starts it. The teaching AI generates an initial message, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user. The user enters a question, "Please teach me how to use variables," and presses the send button. The device sends the question to the server, which passes it to the teaching AI. The teaching AI responds, "In Python, variables are declared as follows...", and generates a concrete example. The server uses an emotion engine to recognize the user's emotions (e.g., confusion, relief) from the user's question. The server provides the emotions recognized by the emotion engine to the teaching AI, which adjusts its response based on the emotions and sends it to the user. The server records the dialogue content, learning progress, and recognized emotion information in the database and uses this information in the next session.

[2176] In this way, by monitoring the user's learning experience in real time and providing optimal feedback based on their emotions, the quality and efficiency of learning can be improved.

[2177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2178] Step 1:

[2179] The user enters and submits their login information (user ID and password). The terminal receives this information and sends it to the server. The input is the user ID and password, and the output is the login request sent to the server.

[2180] Step 2:

[2181] The server compares the received authentication information with the database. Data processing here involves extracting user authentication information from the database using SQL queries and comparing it with the input information. The output is whether the authentication was successful or not.

[2182] Step 3:

[2183] If authentication is successful, the server generates a user-specific dashboard and sends the information to the device. The output is data for displaying the dashboard, which is then sent to the device. The device receives this data and displays the dashboard to the user.

[2184] Step 4:

[2185] The user selects a topic they want to learn about on the dashboard and submits the topic information. The device receives this information and sends it to the server. The input is the topic selected by the user, and the output is the topic information sent to the server.

[2186] Step 5:

[2187] The server loads the corresponding instructional AI model based on the received theme information. Here, data processing involves loading the AI ​​model corresponding to the theme. The output is the initialization and startup of the AI ​​model.

[2188] Step 6:

[2189] The instructional AI generates an initial message and sends it to the terminal via the server. The input is the loaded AI model and the prompt for generating the initial message, and the output is the initial message sent to the user. The terminal displays this message to the user.

[2190] Step 7:

[2191] The user inputs questions or instructions and sends that information. The terminal receives this message and sends it to the server. The input is the user's questions or instructions, and the output is the user input data sent to the server.

[2192] Step 8:

[2193] The server passes user input data to the guidance AI, which then generates an answer. The input consists of the user's questions and instructions, while the output is the generated answer message.

[2194] Step 9:

[2195] The server sends the generated response message to the terminal, which then displays it to the user. The input is the response message from the AI, and the output is the message displayed to the user.

[2196] Step 10:

[2197] The server passes user input data to the analysis engine, which recognizes emotions in real time. The data processing here involves natural language processing for emotion analysis. The input is user input data, and the output is the recognized emotion information.

[2198] Step 11:

[2199] The emotion engine recognizes emotional information, which the server then provides to the guidance AI. Based on this information, the AI ​​generates a response. The input is emotional information, and the output is a response message that takes that emotional information into account.

[2200] Step 12:

[2201] The server records the content of the interaction between the teaching AI and the user, as well as learning progress and recognized emotion information, in a database. The inputs are the interaction content, learning progress, and emotion information, while the output is the information stored in the database.

[2202] Step 13:

[2203] In the next session, the server will have the instructional AI generate a customized learning plan based on the recorded information. The input will be the content of the previous conversation and emotional information, and the output will be the customized learning plan. The user will then start learning again based on this information.

[2204] In this way, by analyzing user sentiment information and learning progress in real time and providing appropriate feedback, the learning experience is improved.

[2205] (Application Example 2)

[2206] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2207] Conventional instructor AI systems failed to recognize the user's emotional state and could only provide uniform dialogue, resulting in a decline in the quality of the learning experience. Similarly, in factory robots, the inability to provide feedback and instructions that considered the operator's emotions posed a challenge to work efficiency and safety.

[2208] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input and transmit authentication information to the server; means for the server to verify the received authentication information and generate a user dashboard if authentication is successful; means for the user to select a theme they wish to learn and transmit the selection information to the server; means for the server to activate an instructor AI based on the selected theme and begin a dialogue with the user; means for the user to recognize their emotions and for the instructor AI to adjust the content of the dialogue based on those emotions; means installed in a robot to recognize the operator's emotional state and provide appropriate feedback and instructions; and means for the server to record the content of the dialogue with the instructor AI and save the data for use in the next session. This makes it possible to provide adjusted dialogue and instructions according to the emotional state of the user and operator.

[2209] User authentication is the process by which a user enters authentication information, sends it to a server for verification, and then verifies it.

[2210] A "server" is a computer system that receives user authentication information and selection information and processes it based on that information.

[2211] A "user dashboard" is a screen that the server generates upon successful authentication and allows the user to access it.

[2212] An "instructor AI" is an artificial intelligence model that interacts with users and operators, providing answers to questions and instructions.

[2213] "Emotion recognition" is the process of analyzing user or operator input data (voice, text, facial expressions, etc.) to identify emotional states.

[2214] "Dialogue content adjustment" refers to the instructor AI appropriately modifying the content of the dialogue based on recognized emotions.

[2215] A "robot" is a mechanical device that performs tasks within a factory, interacting with an operator and providing work instructions.

[2216] "Feedback" refers to the responses and instructions that the instructor AI provides to the user or operator.

[2217] A "database" is a data storage system where a server records conversation content, progress, emotional information, and other data for use in subsequent sessions.

[2218] This invention is a system for users to learn and work online via an internet connection, and also aims to recognize the user's emotions and use that information to improve the experience. The system consists of the following main components: a user terminal, a server, an instructor AI model, an emotion engine, and a database. The entire system exchanges data using a secure communication protocol (e.g., HTTPS).

[2219] The main functions and operation of the system

[2220] 1. User Authentication

[2221] The user accesses the login screen and enters their authentication information (user ID and password). This information is then sent from the terminal to the server.

[2222] The server compares the received authentication information with the database and, if authentication is successful, generates a user-specific dashboard.

[2223] 2. Theme Selection

[2224] Users select a topic they want to learn about on the dashboard (e.g., "Python Programming").

[2225] The device sends the selected theme information to the server.

[2226] The server receives the theme information and loads the corresponding instructor AI model.

[2227] 3. Interaction with the Instructor AI

[2228] The server starts the loaded instructor AI model and generates an initial message.

[2229] The instructor AI will send this message to the user.

[2230] Users can send questions and instructions to the instructor AI while learning.

[2231] The device sends these messages to the server, which then passes them on to the instructor AI.

[2232] The instructor AI generates answers to the user's questions and sends them to the user.

[2233] 4. Emotion recognition

[2234] The server passes user input data (e.g., text messages) to the emotion engine, which recognizes the user's emotions in real time.

[2235] The emotion engine recognizes the emotion information, which the server then provides to the instructor AI.

[2236] The instructor AI generates responses by considering emotional information and adjusts the dialogue according to the user's emotions.

[2237] 5. Record progress and emotions

[2238] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[2239] This information will be used in the next session.

[2240] Hardware and software to be used

[2241] Hardware:

[2242] Camera (e.g., Logitech C920): Captures the user's facial expressions.

[2243] Microphone: Captures the user's voice.

[2244] software:

[2245] OpenCV: Used for inputting and processing camera images.

[2246] Emotion Recognition Library (e.g., emotion_recognition): Analyzes the user's emotions.

[2247] AI Model: As an instructor AI, it generates responses based on the task content (e.g., the natural language processing model ChatGPT).

[2248] Database (e.g., MySQL): Manages login information, work details, and sentiment information.

[2249] Specific example

[2250] Specific example 1:

[2251] The user launches the smartphone app, enters the ID "user123" and password "password" on the login screen, and presses the login button. The device receives this information and sends it to the server. The server verifies it against the database, and if authentication is successful, displays a user-specific dashboard.

[2252] The user selects "Python Programming" in the dashboard and sends this selection information to the server. The server receives this information and activates the instructor AI.

[2253] The instructor AI generates an initial message saying, "Let's start Python programming. First, we'll learn about basic data types," and sends it to the user. When the user asks, "How do I use variables?", the terminal sends the question to the server, and the instructor AI generates an answer.

[2254] The server passes the user's question to the emotion engine, which recognizes the user's emotions. If the emotion engine recognizes "confusion," the server provides this information to the instructor AI, which responds with, "You seem confused. Let me explain in more detail."

[2255] The server records the conversation content, progress, and emotional information in a database, which will be used in the next session.

[2256] Example of a prompt:

[2257] If an operator appears confused while tightening a bolt, recognize their confusion and provide a clear and detailed explanation.

[2258] This makes it possible to provide users and operators with adjusted dialogue and instructions that are tailored to their emotional state.

[2259] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2260] Program processing steps

[2261] Step 1:

[2262] The user launches the app on their smartphone and enters their user ID and password on the login screen. This generates the input data for user authentication.

[2263] Step 2:

[2264] The terminal encrypts the user authentication input data and sends it to the server. The server compares the received authentication information with the database, and if authentication is successful, generates a user-specific dashboard and sends it back to the terminal.

[2265] Step 3:

[2266] The user selects a topic they want to learn about (e.g., "Python Programming") within the dashboard, and the device sends this selection information to the server. This generates the input data for the topic selection.

[2267] Step 4:

[2268] The server analyzes the theme information and loads the appropriate instructor AI model. The instructor AI generates an initial message and sends it to the terminal.

[2269] Step 5:

[2270] The user inputs a question to the instructor AI, and the device sends the question to the server. The server passes the question to the instructor AI, which generates an answer and sends it back.

[2271] Step 6:

[2272] The server passes the user's questions and the instructor AI's answers to the emotion engine, which recognizes the user's emotional state in real time. This process generates emotion recognition data.

[2273] Step 7:

[2274] The emotion engine recognizes the emotion information, which the server provides to the instructor AI, and the instructor AI adjusts the content of the dialogue. The server then sends the adjusted response generated by the AI ​​to the terminal.

[2275] Step 8:

[2276] Review the user's response and either ask the next question or continue the learning process.

[2277] Step 9:

[2278] The server records the content of the conversation between the instructor AI and the user, as well as learning progress and recognized emotion information, in a database.

[2279] Step 10:

[2280] In the next session, the server will customize the instructor AI model based on past conversations and progress information to provide appropriate learning advice.

[2281] This makes it possible to provide adjusted dialogue and instructions that respond to various emotional states, such as when a user is confused or agitated.

[2282] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2283] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2284] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2285] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2286] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2287] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2288] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2289] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2290] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2291] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2292] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2293] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2294] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2295] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2296] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2297] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2298] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2299] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[2300] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[2301] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure,...

Claims

1. A means for the user to enter authentication information and send it to the server, A means for the server to verify the received authentication information and generate a user dashboard if authentication is successful, A means for a user to select a topic they want to learn about and send that selection information to the server, A means by which the server activates the instructor AI based on the selected theme and initiates interaction with the user, The server records the content of the conversation with the instructor AI and stores the data for use in the next session. A system that includes this.

2. The system according to claim 1, wherein a user and an instructor AI interact in natural language and generate answers to questions in real time.

3. The system according to claim 1, wherein the server prepares the next learning content based on the user's progress, and the instructor AI provides appropriate learning advice as needed.

Images