system

Abstract

We provide the system. [Solution] Initialization method, User authentication method, Methods for selecting a learning program, Means of providing learning content, Progress tracking methods, Feedback processing means, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Many modern companies are facing a shortage of human resources to effectively provide specific knowledge such as qualification learning and in-company manners in employee education and training. As a result, it is difficult for many new employees and existing employees to obtain appropriate learning opportunities, leading to a problem of decreased efficiency and productivity across the company. In addition, existing educational systems have difficulty balancing specific knowledge and general knowledge, and there is also a problem that there is no mechanism for appropriately tracking the learning progress of users and effectively utilizing feedback.

Means for Solving the Problems

[0005] To address these challenges, the present invention provides an educational system including initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means. The initialization means loads generated data and manual mode data, the user authentication means authenticates the user's credentials and starts a session. The learning program selection means provides a learning program including both general data and manual mode data, and the learning content provision means delivers appropriate learning content to the user. The progress tracking means records the user's learning progress and sends it to the server. The feedback processing means stores the feedback received from the user in a database and performs analysis. This enhances the effectiveness of education, allows for continuous tracking of learning progress, and enables the use of feedback to improve the educational program.

[0006] An "initialization means" refers to a device or program that has the function of loading and preparing generated data and manual mode data when the system starts up.

[0007] A "user authentication method" refers to a device or program that verifies a user's credentials and authenticates their legitimacy, thereby granting access to the system.

[0008] A "learning program selection means" refers to a device or program that allows a user to select their desired program from a list of available learning programs.

[0009] "Learning content provision means" refers to a device or program that provides appropriate learning content to a user based on a selected learning program.

[0010] A "progress tracking device" refers to a device or program that records a user's learning progress and sends it to a server.

[0011] A "feedback processing means" refers to a device or program that stores user feedback in a database and performs analysis on it.

[0012] "Generated data" refers to data composed of machine learning models and existing general knowledge data.

[0013] "Manual mode data" refers to custom data and programs that have been manually added by the training staff.

[0014] A "server" is a central computer that manages the data for the entire system and performs various processes.

[0015] A "terminal" is a device that a user directly operates to access a system. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10]Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means.

[0038] First, when the system starts up, the server loads generated data and manual mode data using initialization mechanisms. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[0039] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[0040] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[0041] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct the learning content to be provided to the user. For example, specific knowledge such as that needed for certification exams or new employee training is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[0042] During learning, the device uses progress tracking to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[0043] After completing the learning program, users use a feedback processing mechanism to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from the terminal to the server, which stores the received feedback in a database and performs analysis. This analysis is then used to improve future programs.

[0044] Specific example

[0045] For example, if a new employee chooses the "New Employee Training Program":

[0046] 1. The server loads generated data and manual mode data when the system starts up.

[0047] 2. The terminal displays a screen for the user to log in to the system. The user enters their ID and password to authenticate.

[0048] 3. The server starts a session if authentication is successful and displays a list of available learning programs to the user.

[0049] 4. The user selects the "New Employee Training Program".

[0050] 5. The terminal sends the selected program information to the server.

[0051] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[0052] 7. The device displays learning content to the user. For example, the section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[0053] 8. The device records the user's learning progress and sends it to the server.

[0054] 9. The server saves progress data to a database, allowing users to resume their learning progress the next time they log in.

[0055] 10. After completing the learning process, the user submits feedback and sends it to the server.

[0056] 11. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[0057] In this way, the educational system of the present invention provides users with effective and balanced education, and enables continuous improvement through tracking of learning progress and feedback.

[0058] The following describes the processing flow.

[0059] Step 1:

[0060] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This includes machine learning models, existing general knowledge data, and custom data and programs manually added by educators.

[0061] Step 2:

[0062] The terminal displays a login screen to the user. The user enters their ID and password and submits their credentials.

[0063] Step 3:

[0064] The server uses user authentication methods to verify the user's credentials and authenticate their legitimacy. If authentication is successful, it starts a session and sends an authentication token to the device.

[0065] Step 4:

[0066] After logging in, users can browse a list of learning programs and select a program that matches their learning goals. This is made possible through a learning program selection tool.

[0067] Step 5:

[0068] The terminal sends the program information selected by the user to the server.

[0069] Step 6:

[0070] The server integrates the generated data and manual mode data based on the received program ID to construct the learning content.

[0071] Step 7:

[0072] The server sends the configured learning content to the terminal. For example, in a new employee training program, company etiquette is provided from manual mode data, and business etiquette is provided from generated data.

[0073] Step 8:

[0074] The device displays the received learning content to the user. The user then proceeds with their learning while referring to it.

[0075] Step 9:

[0076] The device uses a progress tracking system to record the user's learning progress in real time. The learning progress is periodically sent to the server.

[0077] Step 10:

[0078] The server saves the received progress data to a database. This allows the user to continue their learning progress the next time they log in.

[0079] Step 11:

[0080] After completing the learning process, users use a feedback processing mechanism to enter their evaluation and comments on the learning content into a feedback form. Once the feedback is complete, it is sent to the server.

[0081] Step 12:

[0082] The server stores the received feedback in a database for later analysis, which can then be used to improve the educational program.

[0083] In this way, it becomes possible to provide users with effective learning content, track their progress, and utilize their feedback throughout each processing step of the system.

[0084] (Example 1)

[0085] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0086] Traditional education systems have a fixed approach to providing learning content, making it difficult to deliver balanced learning programs that appropriately integrate general and custom data. Furthermore, they lacked mechanisms to track user learning progress in real time and improve the system through continuous feedback. This resulted in insufficient educational effectiveness and decreased user motivation.

[0087] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0088] In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, means for integrating generated data and custom data after learning program selection to constitute learning content, and means for sending progress data to the server periodically or at the end of an operation. This makes it possible to provide individually customized learning content to users, track progress in real time, and continuously improve the educational program by utilizing feedback.

[0089] An "initialization method" is a means of loading generated data and custom data when the system starts up, and bringing the system into an operational state.

[0090] "User authentication means" refers to a method for verifying the credentials a user uses to access a system and determining their legitimacy.

[0091] The "learning program selection means" is a means of displaying a list of learning programs available to the user and sending the selected program information to the server.

[0092] A "means for providing learning content" refers to a means of integrating generated data and custom data based on a selected learning program to construct learning content and provide it to the user.

[0093] A "progress tracking method" is a means of recording the user's learning progress and sending it to the server at regular intervals or upon completion of an operation.

[0094] A "feedback processing method" is a means by which users input evaluations and comments on the learning content and send them to the server.

[0095] "Generated data" refers to data that is automatically generated by a system, such as machine learning models and general knowledge data.

[0096] "Custom data" refers to customized data that has been manually added by the training staff.

[0097] A "learning program" is a plan that includes a set of learning content and activities designed to teach specific knowledge or skills.

[0098] A "server" is a central computer system that handles data processing and management for the entire system.

[0099] A "terminal" is a device used by a user to access the system and utilize learning content.

[0100] A "session" refers to a specific period of time during which a user operates within a system, from the time they log in until they log out.

[0101] A "database" is a structured collection of data used to efficiently store, manage, and retrieve data within a system.

[0102] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means. This makes it possible to provide individually customized learning content to users, track progress in real time, and continuously improve the educational program by utilizing feedback. Embodiments of the present invention are described in detail below.

[0103] First, when the system starts up, the server loads generated data and custom data using initialization methods. Specifically, generated data includes machine learning models and existing general knowledge data, which are retrieved from a database or external storage. Custom data, on the other hand, includes customized data manually added by educators.

[0104] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password. Using user authentication, the server verifies the credentials and determines their legitimacy. If authentication is successful, a session is started, and the user can begin using the system.

[0105] After authentication, the server generates a list of available learning programs and sends it to the terminal. The user then uses the learning program selection tool to choose their desired program. The selected program information is then sent from the terminal to the server.

[0106] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and custom data to construct the learning content. Specifically, particular knowledge such as that required for certification exams or new employee training is provided from custom data, while general knowledge such as business etiquette is provided from generated data. This results in well-balanced learning content.

[0107] As the learning process progresses, the device uses progress tracking to record the user's learning progress. Progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume learning based on their progress the next time they log in.

[0108] After completing the learning program, users use a feedback processing mechanism to input evaluations and comments about the learning content. This feedback is sent from the terminal to the server, which stores it in a database and performs analysis. The results of this analysis are used to improve future programs.

[0109] Specific example

[0110] For example, if a new employee selects the "New Employee Training Program," the following process takes place: The server loads generated data and custom data when the system starts up. The terminal displays a login screen, and the user authenticates with their ID and password. If authentication is successful, the server generates a list of available learning programs and displays it to the user. The user selects the "New Employee Training Program," and the terminal sends this information to the server. The server integrates the generated data and custom data to create learning content and sends it to the terminal. The terminal displays the learning content, records progress, and sends it to the server. Finally, the user submits feedback, which the server saves to a database and analyzes.

[0111] Example of a prompt

[0112] "Please describe the system's process for integrating the content of a new employee training program using both generated and custom data, and providing it to users while tracking their progress."

[0113] As described above, the educational system of the present invention provides users with efficient and balanced education, and enables continuous improvement through tracking of learning progress and feedback.

[0114] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0115] Step 1:

[0116] The server loads generated data and custom data using initialization methods when the system starts up.

[0117] Input: Generated data from databases and external storage, and custom data.

[0118] Processing: Access databases and external storage and load the necessary data into memory.

[0119] Output: Generated data and custom data stored in memory

[0120] Specific operation: The server uses Python or SQL scripts to connect to the database and load generated data (e.g., machine learning models) and custom data (materials added by educators).

[0121] Step 2:

[0122] The device displays a login screen and prompts the user to enter their ID and password.

[0123] Input: User ID and password

[0124] Process: Render the login screen and accept input.

[0125] Output: Authentication information (ID and password) sent to the server.

[0126] Specific operation: The terminal displays a login form in a web browser or dedicated application, accepts user input, and has a submit button to send authentication information to the server.

[0127] Step 3:

[0128] The server verifies credentials using user authentication methods and determines their legitimacy.

[0129] Input: Authentication information (ID and password) sent to the server

[0130] Process: Verify authentication information against user information in the database.

[0131] Output: Status indicating authentication success or failure

[0132] Specific operation: The server compares the ID and password with the internal user database and generates a session ID if authentication is successful.

[0133] Step 4:

[0134] The server generates a list of available learning programs after successful authentication and sends it to the terminal.

[0135] Input: Authentication success status and learning program information from the internal database.

[0136] Process: Extract learning program information from the database and generate a list.

[0137] Output: List of learning programs sent to the terminal

[0138] Specific operation: The server extracts a list of learning programs from the database using an SQL query and sends it to the terminal in JSON format.

[0139] Step 5:

[0140] The user selects their desired learning program.

[0141] Input: List of learning programs displayed on the terminal

[0142] Process: The user selects a learning program and sends that selection information to the server.

[0143] Output: Selected learning program information sent to the server

[0144] Specific operation: The user clicks or taps the desired program on the device screen and presses the send button to send the selected program information.

[0145] Step 6:

[0146] The server integrates generated data and custom data based on the selected program information to construct the learning content.

[0147] Input: User-selected learning program information and generated and custom data loaded during initialization.

[0148] Process: Generate learning content using generated data and custom data.

[0149] Output: Integrated learning content sent to the device

[0150] Specific operation: The server matches generated data and custom data related to the selected learning program and converts the consistent learning content into a single HTML document or application display format.

[0151] Step 7:

[0152] The terminal displays learning content sent from the server to the user.

[0153] Input: Learning content sent from the server

[0154] Processing: Render and display the learning content.

[0155] Output: Learning content displayed to the user

[0156] Specific operation: The device uses HTML, CSS, JavaScript (registered trademark), etc., to display learning content in a user-friendly format.

[0157] Step 8:

[0158] The device records the user's learning progress and sends it to the server at regular intervals or after the operation is completed.

[0159] Input: User's learning progress (completed sections, answer results, etc.)

[0160] Process: Log the progress and send it to the server.

[0161] Output: Training progress data sent to the server

[0162] Specific operation: The terminal records user actions using JavaScript or other scripts and periodically sends this data to the server in JSON format.

[0163] Step 9:

[0164] The server saves progress data to a database, allowing users to resume their learning the next time they log in.

[0165] Input: Learning progress data sent from the device

[0166] Process: Store progress data in the database and associate it with the user session.

[0167] Output: Progress data stored in the database

[0168] Specific operation: The server receives progress data and stores it in the database using SQL. It also associates the user's session ID with the progress data and stores that information.

[0169] Step 10:

[0170] After completing the learning process, users enter their evaluation and comments in a feedback form.

[0171] Input: Ratings and comments entered in the feedback form

[0172] Process: Enter your feedback.

[0173] Output: Feedback data sent to the server

[0174] Specific action: The user enters text into the feedback form on the device and presses the submit button.

[0175] Step 11:

[0176] The device sends the feedback content to the server.

[0177] Input: Feedback data

[0178] Processing: Send feedback data to the server.

[0179] Output: Feedback data sent to the server

[0180] Specific operation: The terminal sends feedback data to the server using an HTTP request.

[0181] Step 12:

[0182] The server stores the feedback in a database and performs analysis.

[0183] Input: Feedback data sent from the device

[0184] Processing: Store feedback in the database and analyze it.

[0185] Output: Feedback and analysis results stored in the database

[0186] Specific operation: The server receives feedback data and stores it in a database using SQL. Then, natural language processing tools and machine learning models are used to analyze the feedback and use it to improve the educational program.

[0187] The above describes the program processing flow of the system, including its specific actions.

[0188] (Application Example 1)

[0189] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0190] Traditional education systems have made efficient learning and progress management difficult, and in the industrial sector in particular, there has been a lack of effective means for learning new machinery and operating procedures. Furthermore, the lack of integrated systems for learning robot operating procedures, tracking progress, and processing feedback made efficient operation difficult. Therefore, there is a need to improve the efficiency of learning and practice using robots.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0192] In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, means for providing learning content to the robot, means for tracking the robot's learning progress, and means for processing feedback from the robot. This enables the robot to effectively learn new machines and work procedures, to grasp its progress in detail, and to provide appropriate feedback.

[0193] An "initialization method" is a means for loading generated data and specific data when the system starts up.

[0194] "User authentication means" refers to the method of authentication using an ID and password when a user accesses a system.

[0195] "Means for selecting a learning program" refers to the means by which learners can select available learning programs.

[0196] "Means of providing learning content" refers to means of constructing learning content based on a learning program and providing it to learners.

[0197] A "progress tracking system" is a means of recording a learner's learning progress and sending it to a server.

[0198] A "feedback processing method" is a means of collecting evaluations and comments after learning and using them to improve the system.

[0199] "Means for providing learning content to robots" refers to means for effectively providing learning content to robots and enabling them to learn operating procedures and business processes.

[0200] "Means for tracking the learning progress of a robot" refers to means for tracking the progress of a robot as it learns and recording it on a server.

[0201] "Means for processing robot feedback" refers to methods for collecting feedback that a robot provides after learning and using that feedback to improve the system.

[0202] This invention provides an educational system for effectively teaching factory robots new operating procedures and work processes. Specific embodiments of the system are described below.

[0203] Explanation of program generation and processing

[0204] This system consists of the following hardware and software.

[0205] Hardware:

[0206] Robot control unit (e.g., NVIDIA Jetson)

[0207] Server (e.g., AWS® EC2 instance)

[0208] Communication module (e.g., Wi-Fi / Bluetooth)

[0209] software:

[0210] Machine learning frameworks (e.g., TENSORFLOW®)

[0211] Database management systems (e.g., MySQL (registered trademark))

[0212] Robot Operating System (ROS)

[0213] Application frameworks (e.g., Django)

[0214] Initialization means

[0215] The server loads generated data and specific data using an initialization mechanism upon startup. The generated data includes machine learning models, while the specific data includes manually added custom data.

[0216] User authentication methods

[0217] When a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy. If authentication is successful, a session is started.

[0218] Learning program selection method

[0219] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user selects their desired program, and this information is sent from the terminal to the server.

[0220] Means of providing learning content

[0221] Based on the selected program, the server integrates generated data and specific data to construct learning content. For example, the operating procedures for a specific machine are provided from specific data, while general operating concepts are provided from generated data.

[0222] Progress tracking methods

[0223] During learning, the device records the user's learning progress using a progress tracking mechanism. Additionally, progress data is sent to the server at regular intervals or when the user finishes an operation, and the server stores it in a database.

[0224] Feedback processing means

[0225] After completing the learning program, users enter their evaluations and comments into a feedback form. This information is sent from the device to the server, which stores the feedback in a database and performs analysis. The results of this analysis will be used to improve future programs.

[0226] Specific example

[0227] For example, when a robot learns the operating procedures for a new polishing machine, the server loads generated data for the operating procedures and manual operation procedure data. The robot logs in and sends authentication information to the server. If authentication is successful, the robot is presented with a "polishing machine operation program" and selects it. The server provides learning content by integrating the general polishing process from the generated data and the specific machine operation procedures from the manual data. The robot records its progress as it performs the operation and sends it to the server. Once the operation is complete, it inputs feedback and sends it to the server. Based on this, the next operating procedures are improved.

[0228] Example of a prompt

[0229] "I would like to learn how to operate the new polishing machine. First, please teach me the basic safety procedures. Then, please explain the overall process."

[0230] As described above, the educational system of the present invention provides robots with effective and balanced education, and enables continuous improvement through tracking and feedback on learning progress.

[0231] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0232] Step 1:

[0233] The server loads generated data and specific data using initialization methods when the system starts up. The generated data includes machine learning models, and the specific data includes custom data added manually. This allows the server to prepare the dataset necessary for training. Specifically, the server loads the machine learning models from disk into memory, reads the custom data, and saves it to the database.

[0234] Input: Generated data on disk and specific data

[0235] Data processing / calculation: File reading, loading into memory, and performing necessary transformations.

[0236] Output: Prepared dataset

[0237] Step 2:

[0238] When a user accesses the system, the terminal displays a login screen. The user enters their ID and password, and the terminal sends this authentication information to the server. The server verifies the credentials using user authentication methods and authenticates their legitimacy. If authentication is successful, a session is started. Specifically, the server compares the user information stored in the database with the entered credentials, and if they match, a session is started.

[0239] Input: User ID, Password

[0240] Data processing / data calculations: Database queries, credential matching.

[0241] Output: Authentication result (success / failure)

[0242] Step 3:

[0243] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program, and this information is sent from the terminal to the server. The server receives the selected program information and begins preparing the corresponding learning content. Specifically, the server uses the selected program ID as a key to retrieve the corresponding learning content data from the database.

[0244] Input: User-selected program ID

[0245] Data processing / data calculation: Database queries, retrieval of learning content

[0246] Output: Training content dataset

[0247] Step 4:

[0248] Using a learning content delivery method, the server integrates generated data and specific data. The server provides operating procedures for specific machines from the specific data and general operating concepts from the generated data. This integrates content aligned with the learning program and transmits it to the terminal. Specifically, the server integrates generated data and specific data to create content that includes advanced operating procedures and general operating principles.

[0249] Input: Generated data, specific data

[0250] Data processing / data calculations: data integration, content generation

[0251] Output: Integrated learning content

[0252] Step 5:

[0253] During learning, the device records the user's learning progress using progress tracking mechanisms. It also sends progress data to the server at regular intervals or when the user finishes an operation, and the server stores this data in a database. Specifically, the device collects operation logs and progress information in real time and periodically transfers them to the server.

[0254] Input: User operation log, progress information

[0255] Data processing / data calculation: data recording, periodic transfer

[0256] Output: Progress data

[0257] Step 6:

[0258] After completing the learning program, users enter their evaluations and comments into a feedback form. This information is sent from the device to the server, which stores the feedback in a database and performs analysis. Specifically, the server analyzes the feedback data to extract statistical evaluations and areas for improvement.

[0259] Input: User ratings, comments

[0260] Data processing / data calculation: Storage and analysis of feedback data.

[0261] Output: analysis results, statistical information

[0262] Example of a prompt

[0263] "I would like to learn how to operate the new polishing machine. First, please teach me the basic safety procedures. Then, please explain the overall process."

[0264] The above describes the specific processing steps of the system. In each step, appropriate data processing and calculations are performed based on the input data to obtain output, thereby enabling the operation of the entire system.

[0265] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0266] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, and an emotion engine.

[0267] First, when the system starts up, the server loads generated data and manual mode data using initialization mechanisms. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[0268] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[0269] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[0270] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct the learning content. For example, specific knowledge such as that required for certification exams or new employee training is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[0271] Furthermore, during learning, the device uses an emotion engine to recognize emotions from the user's facial expressions and voice. The recognized emotion information is sent to the server in real time.

[0272] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it can suggest a break or reduce the intensity of the content. It can also provide additional explanations and support if the user is feeling anxious.

[0273] During learning, the device uses progress tracking to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[0274] After the learning program ends, the user uses the feedback processing means to input evaluations and comments on the learning content into the feedback form. The feedback content is transmitted from the terminal to the server, and the server saves the received feedback in the database and performs analysis. This can be used to improve the educational program.

[0275] Specific example

[0276] For example, when a new employee selects the "New Employee Training Program":

[0277] 1. The server loads the generated data and the manual mode data when the system starts up.

[0278] 2. The terminal displays a screen for the user to log in to the system. The user inputs an ID and password for authentication.

[0279] 3. When the authentication is successful, the server starts a session and displays a list of available learning programs for the user.

[0280] 4. The user selects the "New Employee Training Program".

[0281] 5. The terminal transmits the selected program information to the server.

[0282] 6. The server integrates the generated data and the manual mode data to compose the learning content of the "New Employee Training Program" and transmits it to the terminal.

[0283] 7. The terminal displays the learning content to the user. For example, the part of the company manners is provided from the manual mode data, and the part of general business manners is provided from the generated data.

[0284] 8. The terminal analyzes the emotion in real time from the user's expression and voice and transmits it to the server.

[0285] 9. The server dynamically adjusts the learning content based on the emotion data. If the user is tired, it proposes a break or changes the learning content to a simplified one.

[0286] 10. The terminal records the user's learning progress and sends it to the server.

[0287] 11. The server saves the progress data in the database and enables the continuation of the learning content at the next login.

[0288] 12. The user submits feedback after the learning is completed and sends it to the server.

[0289] 13. The server saves the feedback in the database, conducts analysis, and uses it to improve the educational program.

[0290] In this way, the educational system of the present invention provides effective and balanced education to the user, realizes dynamic learning corresponding to the user's emotions through the emotion engine, and can also track the learning progress and continuously improve through feedback.

[0291] The following describes the processing flow.

[0292] Step 1:

[0293] The server uses the initialization means to load the generated data and the manual mode data when the system is started. The generated data includes a machine learning model and existing general knowledge data, and the manual mode data includes custom data and programs manually added by the education staff.

[0294] Step 2:

[0295] The terminal displays the login screen to the user. The user enters the ID and password and submits the qualification information.

[0296] Step 3:

[0297] The server uses user authentication means to confirm the user's qualification information and authenticate its legitimacy. If the authentication is successful, a session is started and an authentication token is sent to the terminal.

[0298] Step 4:

[0299] After the user logs in, the user browses the list of learning programs and selects a program that matches their learning goals. This is made possible by the learning program selection means.

[0300] Step 5:

[0301] The terminal sends the program information selected by the user to the server.

[0302] Step 6:

[0303] Based on the received program ID, the server integrates the generated data and the manual mode data to constitute the learning content.

[0304] Step 7:

[0305] The server sends the constituted learning content to the terminal. For example, in the new employee training program, the in-company manners are provided from the manual mode data and the business manners are provided from the generated data.

[0306] Step 8:

[0307] The terminal displays the received learning content to the user. The user proceeds with learning while referring to it.

[0308] Step 9:

[0309] During learning, the terminal uses the emotion engine to recognize the emotion from the user's expression and voice. The recognized emotion information is sent to the server in real time.

[0310] Step 10:

[0311] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it suggests a break or reduces the intensity of the content. It also provides additional explanations and support if the user is feeling anxious.

[0312] Step 11:

[0313] The device records the user's learning progress using a progress tracking mechanism. Progress data is sent to the server at regular intervals or when the user finishes their operation.

[0314] Step 12:

[0315] The server saves the received progress data to a database. This allows the user to continue their learning progress the next time they log in.

[0316] Step 13:

[0317] After completing the learning program, the user uses a feedback processing mechanism to enter their evaluation and comments on the learning content into a feedback form. The feedback content is then sent from the terminal to the server.

[0318] Step 14:

[0319] The server stores the received feedback in a database and performs analysis. This helps to improve the educational program.

[0320] Specific example

[0321] If a new employee chooses the "New Employee Training Program":

[0322] 1. The server loads generated data and manual mode data when the system starts up.

[0323] 2. The terminal displays a screen for the user to log in to the system, and the user enters their ID and password.

[0324] 3. If authentication is successful, the server starts a session and displays a list of learning programs.

[0325] 4. The user selects the "New Employee Training Program".

[0326] 5. The terminal sends the selected program information to the server.

[0327] 6. The server integrates the generated data and the manual mode data to create the learning content for the "New Employee Training Program".

[0328] 7. The server sends the configured learning content to the device.

[0329] 8. The device displays learning content to the user, and the user proceeds with the learning.

[0330] 9. During learning, the device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[0331] 10. The server dynamically adjusts learning content based on sentiment data. If the user is tired, it will suggest a break or simplify the learning content.

[0332] 11. The device records the user's learning progress and sends it to the server.

[0333] 12. The server saves the progress data to the database.

[0334] 13. After completing the learning process, the user enters feedback and sends it to the server.

[0335] 14. The server stores the feedback in a database and analyzes it to help improve the educational program.

[0336] In this way, the educational system of the present invention provides users with effective and balanced education, and enables dynamic learning that responds to the user's emotions through an emotion engine. Furthermore, continuous improvement is possible through tracking of learning progress and feedback.

[0337] (Example 2)

[0338] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0339] Traditional education systems often fail to consider the emotional state of users when providing learning content, leading to decreased learning efficiency and learner satisfaction. Furthermore, managing learning progress and processing feedback are frequently done manually, which is time-consuming and labor-intensive, making it difficult to provide efficient learning.

[0340] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, emotion recognition means, and means for dynamically adjusting learning content based on emotion information. This enables flexible provision of learning content according to the learner's emotional state, as well as efficient progress management and feedback processing.

[0341] An "initialization method" is a means of loading the data necessary for system startup and setting the system to its initial state.

[0342] "User authentication means" are methods used to verify the legitimacy of a user's access to a system.

[0343] A "learning program selection method" is a means that allows users to select their desired program from among the available learning programs.

[0344] "Means of providing learning content" refers to means of providing learning content suitable for the user based on the selected learning program.

[0345] A "progress tracking method" is a means of recording the user's learning progress and sending it to the server as needed.

[0346] A "feedback processing method" is a means of collecting user feedback, sending it to a server, and performing analysis on it.

[0347] An "emotion recognition method" is a means of analyzing a user's facial expressions and voice to recognize their emotional state in real time.

[0348] "Means for dynamically adjusting learning content based on emotional information" refers to means of changing the content and progression of learning content in a timely manner according to the user's emotional state.

[0349] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, emotion recognition means, and means for dynamically adjusting learning content based on emotion information.

[0350] First, when the system starts up, the server loads generated data and manual mode data using initialization methods. This generated data includes machine learning models and general knowledge data, while the manual mode data includes custom data and programs manually added by the instructor. The server uses Apache® server software and Python programs to load this data from disk into memory.

[0351] Next, the terminal displays a login screen when the user accesses the system. The user enters their ID and password to authenticate. Based on the user authentication method, the server uses the Django framework to verify the credentials and authenticate their legitimacy. If authentication is successful, a session is started and the user is redirected to the dashboard screen.

[0352] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program from the list. This information is sent to the server via a JavaScript event handler.

[0353] Based on the selected program, the server integrates generated data and manual mode data to create learning content. Specifically, it extracts specific data from an SQL database using a Python script and supplements the content as needed using a machine learning model. The server then sends the constructed learning content to the device via an API.

[0354] The device uses JavaScript to display learning content to the user in real time. Furthermore, it utilizes emotion recognition technology, analyzing the user's facial expressions and voice data in real time using OpenCV and Federated Learning techniques. This emotion data is sent to the server in JSON format.

[0355] The server analyzes the received emotional information and dynamically adjusts the content and progression of the learning materials according to the user's learning state. For example, if the server determines that the user is tired, it will suggest a break or simplify the content. This is done using conditional branching in a Python program.

[0356] Learning progress is temporarily saved to local storage by the device using JavaScript and sent to the server periodically or after the operation is completed. Upon receiving the progress data, the server saves it to a PostgreSQL database and makes it available the next time the user logs in.

[0357] After completing the learning program, users enter their evaluations and comments into a feedback form. This input data is sent from the terminal to the server, which stores the feedback in a database and performs statistical processing of the feedback using a Python script.

[0358] Specific example

[0359] For example, the procedure for a new employee who selects the "New Employee Training Program" is as follows:

[0360] 1. The server loads generated data and manual mode data when the system starts up.

[0361] 2. The terminal displays the user's login screen, and the user enters their ID and password to authenticate.

[0362] 3. If authentication is successful, the server will start a session and display a list of available learning programs to the user.

[0363] 4. The user selects the "New Employee Training Program".

[0364] 5. The terminal sends the selected program information to the server.

[0365] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[0366] 7. The device displays learning content to the user. The section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[0367] 8. The device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[0368] 9. The server dynamically adjusts learning content based on emotional data. For example, if the user is tired, it suggests a break and simplifies the learning content.

[0369] 10. The device records the user's learning progress and sends it to the server.

[0370] 11. The server saves progress data to a database, allowing users to continue their learning progress the next time they log in.

[0371] 12. After completing the learning process, the user submits feedback and sends it to the server.

[0372] 13. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[0373] Examples of inputs to generative AI models

[0374] "Generate learning content suitable for a new employee training program."

[0375] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0376] Program processing steps

[0377] Step 1:

[0378] The server loads generated data and manual mode data using initialization methods during system startup. At this time, the server begins operating the Apache server software and uses a Python program to load generated data (machine learning models and general knowledge data) and manual mode data (custom data and programs) from disk into memory. The input is the command used during system startup, and the output is the data loaded into memory.

[0379] Step 2:

[0380] The terminal displays a login screen when a user accesses the system. Here, the user enters their ID and password. The entered authentication information is sent to the server using the HTTPS protocol. The input is the user ID and password, and the output is the transmission of the authentication information. Specifically, JavaScript on the terminal collects the authentication information and creates an HTTP request to send it to the server.

[0381] Step 3:

[0382] The server verifies the legitimacy of the credentials sent using user authentication methods. This is done using the Django framework and a database. Specifically, the server compares the received credentials with the database, and if authentication is successful, it starts a session and instructs the user to redirect to the dashboard screen. The input is the credentials, and the output is the authentication status and session information.

[0383] Step 4:

[0384] After successful login, the terminal displays a list of available learning programs to the user using a learning program selection mechanism. A JavaScript-based user interface is applied here. The input is a list of learning programs sent from the server, and the output is the display of the program list to the user.

[0385] Step 5:

[0386] The user selects their desired program from a displayed list of learning programs. The selection information is sent to the server via a JavaScript event handler. The input is the user's program selection, and the output is the transmission of the selection information to the server.

[0387] Step 6:

[0388] The server integrates generated data and manual mode data based on the selected program to generate learning content. It extracts specific data from an SQL database using a Python script and uses a machine learning model to supplement the content as needed. The input is the selected program information, and the output is the generated learning content. The generated content is sent to the terminal via an API.

[0389] Step 7:

[0390] The device uses JavaScript to display learning content to the user. At the same time, it utilizes emotion recognition to analyze the user's facial expressions and voice data in real time using OpenCV and Federated Learning technologies. The analyzed emotion data is sent to the server in JSON format. Inputs are the learning content from the server and the user's facial expressions and voice data, while outputs are the content displayed to the user and the transmitted emotion data.

[0391] Step 8:

[0392] The server analyzes emotional information and dynamically adjusts the learning content. Using a Python script, if the server determines that the user is tired, it suggests a break or changes the content to a simplified version. The input is the user's emotional information, and the output is the adjusted learning content.

[0393] Step 9:

[0394] The device records progress during learning and sends it to the server periodically or after the operation is completed. JavaScript is used to temporarily store progress data in local storage and to create an HTTP request when sending it. The input is the user's learning progress, and the output is the transmission of progress data to the server.

[0395] Step 10:

[0396] The server saves the received progress data to a database so that the game can resume from the same state the next time the user logs in. A Python script parses the progress data and stores it in the database. The input is the progress data, and the output is the progress information stored in the database.

[0397] Step 11:

[0398] After completing the learning program, users enter their evaluations and comments into a feedback form and send it from their device to the server. The input is feedback information, and the output is the transmission of feedback data to the server.

[0399] Step 12:

[0400] The server stores the received feedback in a database and performs statistical processing on the feedback using a Python script. The input is the feedback information, and the output is the feedback content stored in the database and the results of its analysis.

[0401] (Application Example 2)

[0402] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0403] In today's workplace, effective and real-time learning and training are essential. However, traditional education systems often lack the ability to dynamically adjust to learners' emotions and circumstances, leading to decreased learning effectiveness. Furthermore, in on-site training, such as in physical stores, learning must progress during actual work, and insufficient progress management and feedback functions in such situations pose a challenge.

[0404] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes an initialization means, a user authentication means, a learning program selection means, a learning content provision means, a progress tracking means, a feedback processing means, an emotion analysis means, and a learning content dynamic adjustment means. This enables effective and personalized learning by analyzing the learner's emotional information in real time and dynamically adjusting the learning content. Furthermore, continuous improvement of the training program is achieved through progress management and the collection and analysis of feedback.

[0405] "Initialization means" refers to the means of loading generated data and manual mode data when the system starts up.

[0406] "User authentication means" refers to the method used to verify the credentials and authenticate the legitimacy of a user when they access a system using their ID and password.

[0407] "Learning program selection means" refers to a means that allows the user to select their desired learning program, and is a means of displaying a list of available programs.

[0408] A "means for providing learning content" refers to a means of integrating generated data and manual mode data to construct learning content and provide it to the user.

[0409] A "progress tracking method" is a means of recording a user's learning progress and sending it to a server.

[0410] A "feedback processing method" is a means of collecting evaluations and comments from users after the learning process is complete, and sending and storing them on a server.

[0411] "Emotion analysis methods" are means of analyzing emotional information in real time from a user's facial expressions and voice.

[0412] A "dynamic learning content adjustment method" is a means of dynamically adjusting learning content based on information obtained from emotion analysis methods.

[0413] The present invention is a system that effectively supports the training of new staff in physical stores. This system includes an initialization means, a user authentication means, a learning program selection means, a learning content provision means, a progress tracking means, a feedback processing means, an emotion analysis means, and a learning content dynamic adjustment means.

[0414] First, the server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This generated data includes machine learning models and existing general knowledge data. On the other hand, the manual mode data includes custom data and programs that are manually added by the instructor.

[0415] Next, when a user accesses the system, a device such as smart glasses or a smartphone displays a login screen. The user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[0416] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[0417] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct learning content. For example, specific knowledge such as product display methods, customer service etiquette, and store cleaning procedures is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[0418] Furthermore, during learning, devices such as smart glasses and smartphones use emotion analysis tools to recognize emotions from the user's facial expressions and voice. The recognized emotion information is transmitted to the server in real time. For example, EmotionEngine analyzes the user's facial expressions and determines emotions such as fatigue or stress.

[0419] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it can suggest a break or reduce the intensity of the content. It can also provide additional explanations and support if the user is feeling anxious.

[0420] During learning, devices such as smart glasses and smartphones use progress tracking mechanisms to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[0421] After completing the learning program, users use a feedback processing system to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from devices such as smart glasses or smartphones to a server, which stores the received feedback in a database and analyzes it. This allows for the improvement of the educational program.

[0422] As a concrete example, consider a scenario where a new staff member starts a "new employee training program" using smart glasses. First, the server loads the necessary data using an initialization mechanism, and the staff member logs into the system through a user authentication mechanism. Next, the staff member selects the "new employee training program" using a learning program selection mechanism, and the specific training content is displayed on the smart glasses by a learning content provision mechanism. During the training, the staff member's emotions are analyzed by an emotion analysis mechanism, and the content is dynamically adjusted as needed. Learning progress is recorded by a progress tracking mechanism, and finally, training evaluations are collected by a feedback processing mechanism. This enables effective and personalized training.

[0423] Examples of prompts to input into a generative AI model:

[0424] Please generate the learning content for the new employee training program. Include the following items: product display methods, customer service etiquette, and store cleaning procedures. Dynamic adjustments should also be made based on the learners' emotions.

[0425] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0426] Step 1:

[0427] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This prepares the server with the basic data necessary to run the learning program. The input is the generated data and manual mode data pre-stored in the system, and the output is the initialized data profile. Specifically, the server reads the necessary files from the database and loads them into memory.

[0428] Step 2:

[0429] The terminal provides the user with a login screen, and the user authenticates by entering their ID and password. Using the user authentication method, the server verifies the credentials and authenticates their legitimacy. The input is the ID and password entered by the user, and the output is the authentication success or failure status. Specifically, the server compares the entered information with the database and starts a session if they match.

[0430] Step 3:

[0431] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program, and the selected program information is sent from the terminal to the server. The input is the learning program selected by the user, and the output is specific information about the selected program. Specifically, the server receives the user's selection and sends an appropriate response back to the terminal.

[0432] Step 4:

[0433] The server uses a learning content delivery mechanism to integrate generated data and manual mode data based on the selected program, thereby generating learning content. This makes it possible to provide users with well-balanced learning content. The input is the selected program information, generated data, and manual mode data, and the output is the generated learning content. Specifically, the server analyzes the data, extracts relevant information, and integrates it.

[0434] Step 5:

[0435] During learning, devices such as smart glasses use emotion analysis techniques to recognize emotions from the user's facial expressions and voice, and transmit this information to a server in real time. The input is the user's facial expressions and voice data, and the output is the emotion analysis result. Specifically, data is collected using the device's camera and microphone, and EmotionEngine analyzes the data.

[0436] Step 6:

[0437] The server dynamically adjusts the learning content based on the emotional information it receives. For example, if it detects that the user is tired, it may suggest a break or change to lighter content. The input is the result of the emotional analysis, and the output is the adjusted learning content. Specifically, the server changes the current content and provides feedback according to the user's state.

[0438] Step 7:

[0439] During learning, devices such as smart glasses use progress tracking mechanisms to record the user's learning progress and send this data to the server at regular intervals or when the user finishes their activity. The input is the user's learning activity data, and the output is the progress record. Specifically, the device monitors the completion status of learning steps in real time and reports the progress to the server.

[0440] Step 8:

[0441] After completing the learning program, the user uses a feedback processing mechanism to enter evaluations and comments on the learning content into a feedback form and sends them from the terminal to the server. The input is the user's feedback comments, and the output is the collected feedback data. Specifically, the terminal receives input from the user and forwards it to the server.

[0442] Step 9:

[0443] The server stores the received feedback in a database and performs analysis. This helps improve the educational program. The input is user feedback data, and the output is the analysis results and improvement items. Specifically, the server analyzes the collected feedback and generates new program improvement proposals.

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0445] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0446] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0447] [Second Embodiment]

[0448] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0449] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0450] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0451] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0452] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0454] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0455] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0456] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0457] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0458] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0459] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0460] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means.

[0461] First, when the system starts up, the server loads generated data and manual mode data using initialization mechanisms. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[0462] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[0463] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[0464] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct the learning content to be provided to the user. For example, specific knowledge such as that needed for certification exams or new employee training is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[0465] During learning, the device uses progress tracking to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[0466] After completing the learning program, users use a feedback processing mechanism to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from the terminal to the server, which stores the received feedback in a database and performs analysis. This analysis is then used to improve future programs.

[0467] Specific example

[0468] For example, if a new employee chooses the "New Employee Training Program":

[0469] 1. The server loads generated data and manual mode data when the system starts up.

[0470] 2. The terminal displays a screen for the user to log in to the system. The user enters their ID and password to authenticate.

[0471] 3. The server starts a session if authentication is successful and displays a list of available learning programs to the user.

[0472] 4. The user selects the "New Employee Training Program".

[0473] 5. The terminal sends the selected program information to the server.

[0474] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[0475] 7. The device displays learning content to the user. For example, the section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[0476] 8. The device records the user's learning progress and sends it to the server.

[0477] 9. The server saves progress data to a database, allowing users to resume their learning progress the next time they log in.

[0478] 10. After completing the learning process, the user submits feedback and sends it to the server.

[0479] 11. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[0480] In this way, the educational system of the present invention provides users with effective and balanced education, and enables continuous improvement through tracking of learning progress and feedback.

[0481] The following describes the processing flow.

[0482] Step 1:

[0483] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This includes machine learning models, existing general knowledge data, and custom data and programs manually added by educators.

[0484] Step 2:

[0485] The terminal displays a login screen to the user. The user enters their ID and password and submits their credentials.

[0486] Step 3:

[0487] The server uses user authentication methods to verify the user's credentials and authenticate their legitimacy. If authentication is successful, it starts a session and sends an authentication token to the device.

[0488] Step 4:

[0489] After logging in, users can browse a list of learning programs and select a program that matches their learning goals. This is made possible through a learning program selection tool.

[0490] Step 5:

[0491] The terminal sends the program information selected by the user to the server.

[0492] Step 6:

[0493] The server integrates the generated data and manual mode data based on the received program ID to construct the learning content.

[0494] Step 7:

[0495] The server sends the configured learning content to the terminal. For example, in a new employee training program, company etiquette is provided from manual mode data, and business etiquette is provided from generated data.

[0496] Step 8:

[0497] The device displays the received learning content to the user. The user then proceeds with their learning while referring to it.

[0498] Step 9:

[0499] The device uses a progress tracking system to record the user's learning progress in real time. The learning progress is periodically sent to the server.

[0500] Step 10:

[0501] The server saves the received progress data to a database. This allows the user to continue their learning progress the next time they log in.

[0502] Step 11:

[0503] After completing the learning process, users use a feedback processing mechanism to enter their evaluation and comments on the learning content into a feedback form. Once the feedback is complete, it is sent to the server.

[0504] Step 12:

[0505] The server stores the received feedback in a database for later analysis, which can then be used to improve the educational program.

[0506] In this way, it becomes possible to provide users with effective learning content, track their progress, and utilize their feedback throughout each processing step of the system.

[0507] (Example 1)

[0508] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0509] Traditional education systems have a fixed approach to providing learning content, making it difficult to deliver balanced learning programs that appropriately integrate general and custom data. Furthermore, they lacked mechanisms to track user learning progress in real time and improve the system through continuous feedback. This resulted in insufficient educational effectiveness and decreased user motivation.

[0510] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0511] In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, means for integrating generated data and custom data after learning program selection to constitute learning content, and means for sending progress data to the server periodically or at the end of an operation. This makes it possible to provide individually customized learning content to users, track progress in real time, and continuously improve the educational program by utilizing feedback.

[0512] An "initialization method" is a means of loading generated data and custom data when the system starts up, and bringing the system into an operational state.

[0513] "User authentication means" refers to a method for verifying the credentials a user uses to access a system and determining their legitimacy.

[0514] The "learning program selection means" is a means of displaying a list of learning programs available to the user and sending the selected program information to the server.

[0515] A "means for providing learning content" refers to a means of integrating generated data and custom data based on a selected learning program to construct learning content and provide it to the user.

[0516] A "progress tracking method" is a means of recording the user's learning progress and sending it to the server at regular intervals or upon completion of an operation.

[0517] A "feedback processing method" is a means by which users input evaluations and comments on the learning content and send them to the server.

[0518] "Generated data" refers to data that is automatically generated by a system, such as machine learning models and general knowledge data.

[0519] "Custom data" refers to customized data that has been manually added by the training staff.

[0520] A "learning program" is a plan that includes a set of learning content and activities designed to teach specific knowledge or skills.

[0521] A "server" is a central computer system that handles data processing and management for the entire system.

[0522] A "terminal" is a device used by a user to access the system and utilize learning content.

[0523] A "session" refers to a specific period of time during which a user operates within a system, from the time they log in until they log out.

[0524] A "database" is a structured collection of data used to efficiently store, manage, and retrieve data within a system.

[0525] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means. This makes it possible to provide individually customized learning content to users, track progress in real time, and continuously improve the educational program by utilizing feedback. Embodiments of the present invention are described in detail below.

[0526] First, when the system starts up, the server loads generated data and custom data using initialization methods. Specifically, generated data includes machine learning models and existing general knowledge data, which are retrieved from a database or external storage. Custom data, on the other hand, includes customized data manually added by educators.

[0527] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password. Using user authentication, the server verifies the credentials and determines their legitimacy. If authentication is successful, a session is started, and the user can begin using the system.

[0528] After authentication, the server generates a list of available learning programs and sends it to the terminal. The user then uses the learning program selection tool to choose their desired program. The selected program information is then sent from the terminal to the server.

[0529] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and custom data to construct the learning content. Specifically, particular knowledge such as that required for certification exams or new employee training is provided from custom data, while general knowledge such as business etiquette is provided from generated data. This results in well-balanced learning content.

[0530] As the learning process progresses, the device uses progress tracking to record the user's learning progress. Progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume learning based on their progress the next time they log in.

[0531] After completing the learning program, users use a feedback processing mechanism to input evaluations and comments about the learning content. This feedback is sent from the terminal to the server, which stores it in a database and performs analysis. The results of this analysis are used to improve future programs.

[0532] Specific example

[0533] For example, if a new employee selects the "New Employee Training Program," the following process takes place: The server loads generated data and custom data when the system starts up. The terminal displays a login screen, and the user authenticates with their ID and password. If authentication is successful, the server generates a list of available learning programs and displays it to the user. The user selects the "New Employee Training Program," and the terminal sends this information to the server. The server integrates the generated data and custom data to create learning content and sends it to the terminal. The terminal displays the learning content, records progress, and sends it to the server. Finally, the user submits feedback, which the server saves to a database and analyzes.

[0534] Example of a prompt

[0535] "Please describe the system's process for integrating the content of a new employee training program using both generated and custom data, and providing it to users while tracking their progress."

[0536] As described above, the educational system of the present invention provides users with efficient and balanced education, and enables continuous improvement through tracking of learning progress and feedback.

[0537] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0538] Step 1:

[0539] The server loads generated data and custom data using initialization methods when the system starts up.

[0540] Input: Generated data from databases and external storage, and custom data.

[0541] Processing: Access databases and external storage and load the necessary data into memory.

[0542] Output: Generated data and custom data stored in memory

[0543] Specific operation: The server uses Python or SQL scripts to connect to the database and load generated data (e.g., machine learning models) and custom data (materials added by educators).

[0544] Step 2:

[0545] The device displays a login screen and prompts the user to enter their ID and password.

[0546] Input: User ID and password

[0547] Process: Render the login screen and accept input.

[0548] Output: Authentication information (ID and password) sent to the server.

[0549] Specific operation: The terminal displays a login form in a web browser or dedicated application, accepts user input, and has a submit button to send authentication information to the server.

[0550] Step 3:

[0551] The server verifies credentials using user authentication methods and determines their legitimacy.

[0552] Input: Authentication information (ID and password) sent to the server

[0553] Process: Verify authentication information against user information in the database.

[0554] Output: Status indicating authentication success or failure

[0555] Specific operation: The server compares the ID and password with the internal user database and generates a session ID if authentication is successful.

[0556] Step 4:

[0557] The server generates a list of available learning programs after successful authentication and sends it to the terminal.

[0558] Input: Authentication success status and learning program information from the internal database.

[0559] Process: Extract learning program information from the database and generate a list.

[0560] Output: List of learning programs sent to the terminal

[0561] Specific operation: The server extracts a list of learning programs from the database using an SQL query and sends it to the terminal in JSON format.

[0562] Step 5:

[0563] The user selects their desired learning program.

[0564] Input: List of learning programs displayed on the terminal

[0565] Process: The user selects a learning program and sends that selection information to the server.

[0566] Output: Selected learning program information sent to the server

[0567] Specific operation: The user clicks or taps the desired program on the device screen and presses the send button to send the selected program information.

[0568] Step 6:

[0569] The server integrates generated data and custom data based on the selected program information to construct the learning content.

[0570] Input: User-selected learning program information and generated and custom data loaded during initialization.

[0571] Process: Generate learning content using generated data and custom data.

[0572] Output: Integrated learning content sent to the device

[0573] Specific operation: The server matches generated data and custom data related to the selected learning program and converts the consistent learning content into a single HTML document or application display format.

[0574] Step 7:

[0575] The terminal displays learning content sent from the server to the user.

[0576] Input: Learning content sent from the server

[0577] Processing: Render and display the learning content.

[0578] Output: Learning content displayed to the user

[0579] Specific operation: The device uses HTML, CSS, JavaScript, etc., to display learning content in a user-friendly format.

[0580] Step 8:

[0581] The device records the user's learning progress and sends it to the server at regular intervals or after the operation is completed.

[0582] Input: User's learning progress (completed sections, answer results, etc.)

[0583] Process: Log the progress and send it to the server.

[0584] Output: Training progress data sent to the server

[0585] Specific operation: The terminal records user actions using JavaScript or other scripts and periodically sends this data to the server in JSON format.

[0586] Step 9:

[0587] The server saves progress data to a database, allowing users to resume their learning the next time they log in.

[0588] Input: Learning progress data sent from the device

[0589] Process: Store progress data in the database and associate it with the user session.

[0590] Output: Progress data stored in the database

[0591] Specific operation: The server receives progress data and stores it in the database using SQL. It also associates the user's session ID with the progress data and stores that information.

[0592] Step 10:

[0593] After completing the learning process, users enter their evaluation and comments in a feedback form.

[0594] Input: Ratings and comments entered in the feedback form

[0595] Process: Enter your feedback.

[0596] Output: Feedback data sent to the server

[0597] Specific action: The user enters text into the feedback form on the device and presses the submit button.

[0598] Step 11:

[0599] The device sends the feedback content to the server.

[0600] Input: Feedback data

[0601] Processing: Send feedback data to the server.

[0602] Output: Feedback data sent to the server

[0603] Specific operation: The terminal sends feedback data to the server using an HTTP request.

[0604] Step 12:

[0605] The server stores the feedback in a database and performs analysis.

[0606] Input: Feedback data sent from the device

[0607] Processing: Store feedback in the database and analyze it.

[0608] Output: Feedback and analysis results stored in the database

[0609] Specific operation: The server receives feedback data and stores it in a database using SQL. Then, natural language processing tools and machine learning models are used to analyze the feedback and use it to improve the educational program.

[0610] The above describes the program processing flow of the system, including its specific actions.

[0611] (Application Example 1)

[0612] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0613] Traditional education systems have made efficient learning and progress management difficult, and in the industrial sector in particular, there has been a lack of effective means for learning new machinery and operating procedures. Furthermore, the lack of integrated systems for learning robot operating procedures, tracking progress, and processing feedback made efficient operation difficult. Therefore, there is a need to improve the efficiency of learning and practice using robots.

[0614] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0615] In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, means for providing learning content to the robot, means for tracking the robot's learning progress, and means for processing feedback from the robot. This enables the robot to effectively learn new machines and work procedures, to grasp its progress in detail, and to provide appropriate feedback.

[0616] An "initialization method" is a means for loading generated data and specific data when the system starts up.

[0617] "User authentication means" refers to the method of authentication using an ID and password when a user accesses a system.

[0618] "Means for selecting a learning program" refers to the means by which learners can select available learning programs.

[0619] "Means of providing learning content" refers to means of constructing learning content based on a learning program and providing it to learners.

[0620] A "progress tracking system" is a means of recording a learner's learning progress and sending it to a server.

[0621] A "feedback processing method" is a means of collecting evaluations and comments after learning and using them to improve the system.

[0622] "Means for providing learning content to robots" refers to means for effectively providing learning content to robots and enabling them to learn operating procedures and business processes.

[0623] "Means for tracking the learning progress of a robot" refers to means for tracking the progress of a robot as it learns and recording it on a server.

[0624] "Means for processing robot feedback" refers to methods for collecting feedback that a robot provides after learning and using that feedback to improve the system.

[0625] This invention provides an educational system for effectively teaching factory robots new operating procedures and work processes. Specific embodiments of the system are described below.

[0626] Explanation of program generation and processing

[0627] This system consists of the following hardware and software.

[0628] Hardware:

[0629] Robot control unit (e.g., NVIDIA Jetson)

[0630] Server (e.g., AWS EC2 instance)

[0631] Communication module (e.g., Wi-Fi / Bluetooth)

[0632] software:

[0633] Machine learning frameworks (e.g., TensorFlow)

[0634] Database management system (e.g., MySQL)

[0635] Robot Operating System (ROS)

[0636] Application frameworks (e.g., Django)

[0637] Initialization means

[0638] The server loads generated data and specific data using an initialization mechanism upon startup. The generated data includes machine learning models, while the specific data includes manually added custom data.

[0639] User authentication methods

[0640] When a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy. If authentication is successful, a session is started.

[0641] Learning program selection method

[0642] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user selects their desired program, and this information is sent from the terminal to the server.

[0643] Means of providing learning content

[0644] Based on the selected program, the server integrates generated data and specific data to construct learning content. For example, the operating procedures for a specific machine are provided from specific data, while general operating concepts are provided from generated data.

[0645] Progress tracking methods

[0646] During learning, the device records the user's learning progress using a progress tracking mechanism. Additionally, progress data is sent to the server at regular intervals or when the user finishes an operation, and the server stores it in a database.

[0647] Feedback processing means

[0648] After completing the learning program, users enter their evaluations and comments into a feedback form. This information is sent from the device to the server, which stores the feedback in a database and performs analysis. The results of this analysis will be used to improve future programs.

[0649] Specific example

[0650] For example, when a robot learns the operating procedures for a new polishing machine, the server loads generated data for the operating procedures and manual operation procedure data. The robot logs in and sends authentication information to the server. If authentication is successful, the robot is presented with a "polishing machine operation program" and selects it. The server provides learning content by integrating the general polishing process from the generated data and the specific machine operation procedures from the manual data. The robot records its progress as it performs the operation and sends it to the server. Once the operation is complete, it inputs feedback and sends it to the server. Based on this, the next operating procedures are improved.

[0651] Example of a prompt

[0652] "I would like to learn how to operate the new polishing machine. First, please teach me the basic safety procedures. Then, please explain the overall process."

[0653] As described above, the educational system of the present invention provides robots with effective and balanced education, and enables continuous improvement through tracking and feedback on learning progress.

[0654] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0655] Step 1:

[0656] The server loads generated data and specific data using initialization methods when the system starts up. The generated data includes machine learning models, and the specific data includes custom data added manually. This allows the server to prepare the dataset necessary for training. Specifically, the server loads the machine learning models from disk into memory, reads the custom data, and saves it to the database.

[0657] Input: Generated data on disk and specific data

[0658] Data processing / calculation: File reading, loading into memory, and performing necessary transformations.

[0659] Output: Prepared dataset

[0660] Step 2:

[0661] When a user accesses the system, the terminal displays a login screen. The user enters their ID and password, and the terminal sends this authentication information to the server. The server verifies the credentials using user authentication methods and authenticates their legitimacy. If authentication is successful, a session is started. Specifically, the server compares the user information stored in the database with the entered credentials, and if they match, a session is started.

[0662] Input: User ID, Password

[0663] Data processing / data calculations: Database queries, credential matching.

[0664] Output: Authentication result (success / failure)

[0665] Step 3:

[0666] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program, and this information is sent from the terminal to the server. The server receives the selected program information and begins preparing the corresponding learning content. Specifically, the server uses the selected program ID as a key to retrieve the corresponding learning content data from the database.

[0667] Input: User-selected program ID

[0668] Data processing / data calculation: Database queries, retrieval of learning content

[0669] Output: Training content dataset

[0670] Step 4:

[0671] Using a learning content delivery method, the server integrates generated data and specific data. The server provides operating procedures for specific machines from the specific data and general operating concepts from the generated data. This integrates content aligned with the learning program and transmits it to the terminal. Specifically, the server integrates generated data and specific data to create content that includes advanced operating procedures and general operating principles.

[0672] Input: Generated data, specific data

[0673] Data processing / data calculations: data integration, content generation

[0674] Output: Integrated learning content

[0675] Step 5:

[0676] During learning, the device records the user's learning progress using progress tracking mechanisms. It also sends progress data to the server at regular intervals or when the user finishes an operation, and the server stores this data in a database. Specifically, the device collects operation logs and progress information in real time and periodically transfers them to the server.

[0677] Input: User operation log, progress information

[0678] Data processing / data calculation: data recording, periodic transfer

[0679] Output: Progress data

[0680] Step 6:

[0681] After completing the learning program, users enter their evaluations and comments into a feedback form. This information is sent from the device to the server, which stores the feedback in a database and performs analysis. Specifically, the server analyzes the feedback data to extract statistical evaluations and areas for improvement.

[0682] Input: User ratings, comments

[0683] Data processing / data calculation: Storage and analysis of feedback data.

[0684] Output: analysis results, statistical information

[0685] Example of a prompt

[0686] "I would like to learn how to operate the new polishing machine. First, please teach me the basic safety procedures. Then, please explain the overall process."

[0687] The above describes the specific processing steps of the system. In each step, appropriate data processing and calculations are performed based on the input data to obtain output, thereby enabling the operation of the entire system.

[0688] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0689] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, and an emotion engine.

[0690] First, when the system starts up, the server loads generated data and manual mode data using initialization mechanisms. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[0691] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[0692] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[0693] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct the learning content. For example, specific knowledge such as that required for certification exams or new employee training is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[0694] Furthermore, during learning, the device uses an emotion engine to recognize emotions from the user's facial expressions and voice. The recognized emotion information is sent to the server in real time.

[0695] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it can suggest a break or reduce the intensity of the content. It can also provide additional explanations and support if the user is feeling anxious.

[0696] During learning, the device uses progress tracking to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[0697] After completing the learning program, users use a feedback processing system to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from the terminal to the server, which stores the received feedback in a database and performs analysis. This allows for the improvement of the educational program.

[0698] Specific example

[0699] For example, if a new employee chooses the "New Employee Training Program":

[0700] 1. The server loads generated data and manual mode data when the system starts up.

[0701] 2. The terminal displays a screen for the user to log in to the system. The user enters their ID and password to authenticate.

[0702] 3. The server starts a session if authentication is successful and displays a list of available learning programs to the user.

[0703] 4. The user selects the "New Employee Training Program".

[0704] 5. The terminal sends the selected program information to the server.

[0705] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[0706] 7. The device displays learning content to the user. For example, the section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[0707] 8. The device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[0708] 9. The server dynamically adjusts learning content based on sentiment data. If the user is tired, it will suggest a break or change the learning content to a simplified version.

[0709] 10. The device records the user's learning progress and sends it to the server.

[0710] 11. The server saves progress data to a database, allowing users to continue their learning progress the next time they log in.

[0711] 12. After completing the learning process, the user submits feedback and sends it to the server.

[0712] 13. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[0713] In this way, the educational system of the present invention provides users with effective and balanced education, and enables dynamic learning that responds to the user's emotions through an emotion engine. Furthermore, continuous improvement is possible through tracking of learning progress and feedback.

[0714] The following describes the processing flow.

[0715] Step 1:

[0716] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[0717] Step 2:

[0718] The terminal displays a login screen to the user. The user enters their ID and password and submits their credentials.

[0719] Step 3:

[0720] The server uses user authentication methods to verify the user's credentials and authenticate their legitimacy. If authentication is successful, it starts a session and sends an authentication token to the device.

[0721] Step 4:

[0722] After logging in, users can browse a list of learning programs and select one that matches their learning goals. This is made possible by the learning program selection method.

[0723] Step 5:

[0724] The terminal sends the program information selected by the user to the server.

[0725] Step 6:

[0726] Based on the received program ID, the server integrates the generated data and manual mode data to construct the learning content.

[0727] Step 7:

[0728] The server sends the configured learning content to the terminal. For example, in a new employee training program, company etiquette is provided from manual mode data, and business etiquette is provided from generated data.

[0729] Step 8:

[0730] The device displays the received learning content to the user. The user then proceeds with their learning while referring to it.

[0731] Step 9:

[0732] During training, the device uses an emotion engine to recognize emotions from the user's facial expressions and voice. The recognized emotion information is sent to the server in real time.

[0733] Step 10:

[0734] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it suggests a break or reduces the intensity of the content. It also provides additional explanations and support if the user is feeling anxious.

[0735] Step 11:

[0736] The device records the user's learning progress using a progress tracking mechanism. Progress data is sent to the server at regular intervals or when the user finishes their operation.

[0737] Step 12:

[0738] The server saves the received progress data to a database. This allows the user to continue their learning progress the next time they log in.

[0739] Step 13:

[0740] After completing the learning program, the user uses a feedback processing mechanism to enter their evaluation and comments on the learning content into a feedback form. The feedback content is then sent from the terminal to the server.

[0741] Step 14:

[0742] The server stores the received feedback in a database and performs analysis. This helps to improve the educational program.

[0743] Specific example

[0744] If a new employee chooses the "New Employee Training Program":

[0745] 1. The server loads generated data and manual mode data when the system starts up.

[0746] 2. The terminal displays a screen for the user to log in to the system, and the user enters their ID and password.

[0747] 3. If authentication is successful, the server starts a session and displays a list of learning programs.

[0748] 4. The user selects the "New Employee Training Program".

[0749] 5. The terminal sends the selected program information to the server.

[0750] 6. The server integrates the generated data and the manual mode data to create the learning content for the "New Employee Training Program".

[0751] 7. The server sends the configured learning content to the device.

[0752] 8. The device displays learning content to the user, and the user proceeds with the learning.

[0753] 9. During learning, the device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[0754] 10. The server dynamically adjusts learning content based on sentiment data. If the user is tired, it will suggest a break or simplify the learning content.

[0755] 11. The device records the user's learning progress and sends it to the server.

[0756] 12. The server saves the progress data to the database.

[0757] 13. After completing the learning process, the user enters feedback and sends it to the server.

[0758] 14. The server stores the feedback in a database and analyzes it to help improve the educational program.

[0759] In this way, the educational system of the present invention provides users with effective and balanced education, and enables dynamic learning that responds to the user's emotions through an emotion engine. Furthermore, continuous improvement is possible through tracking of learning progress and feedback.

[0760] (Example 2)

[0761] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0762] Traditional education systems often fail to consider the emotional state of users when providing learning content, leading to decreased learning efficiency and learner satisfaction. Furthermore, managing learning progress and processing feedback are frequently done manually, which is time-consuming and labor-intensive, making it difficult to provide efficient learning.

[0763] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, emotion recognition means, and means for dynamically adjusting learning content based on emotion information. This enables flexible provision of learning content according to the learner's emotional state, as well as efficient progress management and feedback processing.

[0764] An "initialization method" is a means of loading the data necessary for system startup and setting the system to its initial state.

[0765] "User authentication means" are methods used to verify the legitimacy of a user's access to a system.

[0766] A "learning program selection method" is a means that allows users to select their desired program from among the available learning programs.

[0767] "Means of providing learning content" refers to means of providing learning content suitable for the user based on the selected learning program.

[0768] A "progress tracking method" is a means of recording the user's learning progress and sending it to the server as needed.

[0769] A "feedback processing method" is a means of collecting user feedback, sending it to a server, and performing analysis on it.

[0770] An "emotion recognition method" is a means of analyzing a user's facial expressions and voice to recognize their emotional state in real time.

[0771] "Means for dynamically adjusting learning content based on emotional information" refers to means of changing the content and progression of learning content in a timely manner according to the user's emotional state.

[0772] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, emotion recognition means, and means for dynamically adjusting learning content based on emotion information.

[0773] First, upon system startup, the server loads generated data and manual mode data using initialization mechanisms. This generated data includes machine learning models and general knowledge data, while the manual mode data includes custom data and programs manually added by instructors. The server uses Apache server software and Python programs to load this data from disk into memory.

[0774] Next, the terminal displays a login screen when the user accesses the system. The user enters their ID and password to authenticate. Based on the user authentication method, the server uses the Django framework to verify the credentials and authenticate their legitimacy. If authentication is successful, a session is started and the user is redirected to the dashboard screen.

[0775] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program from the list. This information is sent to the server via a JavaScript event handler.

[0776] Based on the selected program, the server integrates generated data and manual mode data to create learning content. Specifically, it extracts specific data from an SQL database using a Python script and supplements the content as needed using a machine learning model. The server then sends the constructed learning content to the device via an API.

[0777] The device uses JavaScript to display learning content to the user in real time. Furthermore, it utilizes emotion recognition technology, analyzing the user's facial expressions and voice data in real time using OpenCV and Federated Learning techniques. This emotion data is sent to the server in JSON format.

[0778] The server analyzes the received emotional information and dynamically adjusts the content and progression of the learning materials according to the user's learning state. For example, if the server determines that the user is tired, it will suggest a break or simplify the content. This is done using conditional branching in a Python program.

[0779] Learning progress is temporarily saved to local storage by the device using JavaScript and sent to the server periodically or after the operation is completed. Upon receiving the progress data, the server saves it to a PostgreSQL database and makes it available the next time the user logs in.

[0780] After completing the learning program, users enter their evaluations and comments into a feedback form. This input data is sent from the terminal to the server, which stores the feedback in a database and performs statistical processing of the feedback using a Python script.

[0781] Specific example

[0782] For example, the procedure for a new employee who selects the "New Employee Training Program" is as follows:

[0783] 1. The server loads generated data and manual mode data when the system starts up.

[0784] 2. The terminal displays the user's login screen, and the user enters their ID and password to authenticate.

[0785] 3. If authentication is successful, the server will start a session and display a list of available learning programs to the user.

[0786] 4. The user selects the "New Employee Training Program".

[0787] 5. The terminal sends the selected program information to the server.

[0788] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[0789] 7. The device displays learning content to the user. The section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[0790] 8. The device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[0791] 9. The server dynamically adjusts learning content based on emotional data. For example, if the user is tired, it suggests a break and simplifies the learning content.

[0792] 10. The device records the user's learning progress and sends it to the server.

[0793] 11. The server saves progress data to a database, allowing users to continue their learning progress the next time they log in.

[0794] 12. After completing the learning process, the user submits feedback and sends it to the server.

[0795] 13. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[0796] Examples of inputs to generative AI models

[0797] "Generate learning content suitable for a new employee training program."

[0798] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0799] Program processing steps

[0800] Step 1:

[0801] The server loads generated data and manual mode data using initialization methods during system startup. At this time, the server begins operating the Apache server software and uses a Python program to load generated data (machine learning models and general knowledge data) and manual mode data (custom data and programs) from disk into memory. The input is the command used during system startup, and the output is the data loaded into memory.

[0802] Step 2:

[0803] The terminal displays a login screen when a user accesses the system. Here, the user enters their ID and password. The entered authentication information is sent to the server using the HTTPS protocol. The input is the user ID and password, and the output is the transmission of the authentication information. Specifically, JavaScript on the terminal collects the authentication information and creates an HTTP request to send it to the server.

[0804] Step 3:

[0805] The server verifies the legitimacy of the credentials sent using user authentication methods. This is done using the Django framework and a database. Specifically, the server compares the received credentials with the database, and if authentication is successful, it starts a session and instructs the user to redirect to the dashboard screen. The input is the credentials, and the output is the authentication status and session information.

[0806] Step 4:

[0807] After successful login, the terminal displays a list of available learning programs to the user using a learning program selection mechanism. A JavaScript-based user interface is applied here. The input is a list of learning programs sent from the server, and the output is the display of the program list to the user.

[0808] Step 5:

[0809] The user selects their desired program from a displayed list of learning programs. The selection information is sent to the server via a JavaScript event handler. The input is the user's program selection, and the output is the transmission of the selection information to the server.

[0810] Step 6:

[0811] The server integrates generated data and manual mode data based on the selected program to generate learning content. It extracts specific data from an SQL database using a Python script and uses a machine learning model to supplement the content as needed. The input is the selected program information, and the output is the generated learning content. The generated content is sent to the terminal via an API.

[0812] Step 7:

[0813] The device uses JavaScript to display learning content to the user. At the same time, it utilizes emotion recognition to analyze the user's facial expressions and voice data in real time using OpenCV and Federated Learning technologies. The analyzed emotion data is sent to the server in JSON format. Inputs are the learning content from the server and the user's facial expressions and voice data, while outputs are the content displayed to the user and the transmitted emotion data.

[0814] Step 8:

[0815] The server analyzes emotional information and dynamically adjusts the learning content. Using a Python script, if the server determines that the user is tired, it suggests a break or changes the content to a simplified version. The input is the user's emotional information, and the output is the adjusted learning content.

[0816] Step 9:

[0817] The device records progress during learning and sends it to the server periodically or after the operation is completed. JavaScript is used to temporarily store progress data in local storage and to create an HTTP request when sending it. The input is the user's learning progress, and the output is the transmission of progress data to the server.

[0818] Step 10:

[0819] The server saves the received progress data to a database so that the game can resume from the same state the next time the user logs in. A Python script parses the progress data and stores it in the database. The input is the progress data, and the output is the progress information stored in the database.

[0820] Step 11:

[0821] After completing the learning program, users enter their evaluations and comments into a feedback form and send it from their device to the server. The input is feedback information, and the output is the transmission of feedback data to the server.

[0822] Step 12:

[0823] The server stores the received feedback in a database and performs statistical processing on the feedback using a Python script. The input is the feedback information, and the output is the feedback content stored in the database and the results of its analysis.

[0824] (Application Example 2)

[0825] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0826] In today's workplace, effective and real-time learning and training are essential. However, traditional education systems often lack the ability to dynamically adjust to learners' emotions and circumstances, leading to decreased learning effectiveness. Furthermore, in on-site training, such as in physical stores, learning must progress during actual work, and insufficient progress management and feedback functions in such situations pose a challenge.

[0827] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes an initialization means, a user authentication means, a learning program selection means, a learning content provision means, a progress tracking means, a feedback processing means, an emotion analysis means, and a learning content dynamic adjustment means. This enables effective and personalized learning by analyzing the learner's emotional information in real time and dynamically adjusting the learning content. Furthermore, continuous improvement of the training program is achieved through progress management and the collection and analysis of feedback.

[0828] "Initialization means" refers to the means of loading generated data and manual mode data when the system starts up.

[0829] "User authentication means" refers to the method used to verify the credentials and authenticate the legitimacy of a user when they access a system using their ID and password.

[0830] "Learning program selection means" refers to a means that allows the user to select their desired learning program, and is a means of displaying a list of available programs.

[0831] A "means for providing learning content" refers to a means of integrating generated data and manual mode data to construct learning content and provide it to the user.

[0832] A "progress tracking method" is a means of recording a user's learning progress and sending it to a server.

[0833] A "feedback processing method" is a means of collecting evaluations and comments from users after the learning process is complete, and sending and storing them on a server.

[0834] "Emotion analysis methods" are means of analyzing emotional information in real time from a user's facial expressions and voice.

[0835] A "dynamic learning content adjustment method" is a means of dynamically adjusting learning content based on information obtained from emotion analysis methods.

[0836] The present invention is a system that effectively supports the training of new staff in physical stores. This system includes an initialization means, a user authentication means, a learning program selection means, a learning content provision means, a progress tracking means, a feedback processing means, an emotion analysis means, and a learning content dynamic adjustment means.

[0837] First, the server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This generated data includes machine learning models and existing general knowledge data. On the other hand, the manual mode data includes custom data and programs that are manually added by the instructor.

[0838] Next, when a user accesses the system, a device such as smart glasses or a smartphone displays a login screen. The user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[0839] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[0840] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct learning content. For example, specific knowledge such as product display methods, customer service etiquette, and store cleaning procedures is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[0841] Furthermore, during learning, devices such as smart glasses and smartphones use emotion analysis tools to recognize emotions from the user's facial expressions and voice. The recognized emotion information is transmitted to the server in real time. For example, EmotionEngine analyzes the user's facial expressions and determines emotions such as fatigue or stress.

[0842] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it can suggest a break or reduce the intensity of the content. It can also provide additional explanations and support if the user is feeling anxious.

[0843] During learning, devices such as smart glasses and smartphones use progress tracking mechanisms to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[0844] After completing the learning program, users use a feedback processing system to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from devices such as smart glasses or smartphones to a server, which stores the received feedback in a database and analyzes it. This allows for the improvement of the educational program.

[0845] As a concrete example, consider a scenario where a new staff member starts a "new employee training program" using smart glasses. First, the server loads the necessary data using an initialization mechanism, and the staff member logs into the system through a user authentication mechanism. Next, the staff member selects the "new employee training program" using a learning program selection mechanism, and the specific training content is displayed on the smart glasses by a learning content provision mechanism. During the training, the staff member's emotions are analyzed by an emotion analysis mechanism, and the content is dynamically adjusted as needed. Learning progress is recorded by a progress tracking mechanism, and finally, training evaluations are collected by a feedback processing mechanism. This enables effective and personalized training.

[0846] Examples of prompts to input into a generative AI model:

[0847] Please generate the learning content for the new employee training program. Include the following items: product display methods, customer service etiquette, and store cleaning procedures. Dynamic adjustments should also be made based on the learners' emotions.

[0848] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0849] Step 1:

[0850] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This prepares the server with the basic data necessary to run the learning program. The input is the generated data and manual mode data pre-stored in the system, and the output is the initialized data profile. Specifically, the server reads the necessary files from the database and loads them into memory.

[0851] Step 2:

[0852] The terminal provides the user with a login screen, and the user authenticates by entering their ID and password. Using the user authentication method, the server verifies the credentials and authenticates their legitimacy. The input is the ID and password entered by the user, and the output is the authentication success or failure status. Specifically, the server compares the entered information with the database and starts a session if they match.

[0853] Step 3:

[0854] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program, and the selected program information is sent from the terminal to the server. The input is the learning program selected by the user, and the output is specific information about the selected program. Specifically, the server receives the user's selection and sends an appropriate response back to the terminal.

[0855] Step 4:

[0856] The server uses a learning content delivery mechanism to integrate generated data and manual mode data based on the selected program, thereby generating learning content. This makes it possible to provide users with well-balanced learning content. The input is the selected program information, generated data, and manual mode data, and the output is the generated learning content. Specifically, the server analyzes the data, extracts relevant information, and integrates it.

[0857] Step 5:

[0858] During learning, devices such as smart glasses use emotion analysis techniques to recognize emotions from the user's facial expressions and voice, and transmit this information to a server in real time. The input is the user's facial expressions and voice data, and the output is the emotion analysis result. Specifically, data is collected using the device's camera and microphone, and EmotionEngine analyzes the data.

[0859] Step 6:

[0860] The server dynamically adjusts the learning content based on the emotional information it receives. For example, if it detects that the user is tired, it may suggest a break or change to lighter content. The input is the result of the emotional analysis, and the output is the adjusted learning content. Specifically, the server changes the current content and provides feedback according to the user's state.

[0861] Step 7:

[0862] During learning, devices such as smart glasses use progress tracking mechanisms to record the user's learning progress and send this data to the server at regular intervals or when the user finishes their activity. The input is the user's learning activity data, and the output is the progress record. Specifically, the device monitors the completion status of learning steps in real time and reports the progress to the server.

[0863] Step 8:

[0864] After completing the learning program, the user uses a feedback processing mechanism to enter evaluations and comments on the learning content into a feedback form and sends them from the terminal to the server. The input is the user's feedback comments, and the output is the collected feedback data. Specifically, the terminal receives input from the user and forwards it to the server.

[0865] Step 9:

[0866] The server stores the received feedback in a database and performs analysis. This helps improve the educational program. The input is user feedback data, and the output is the analysis results and improvement items. Specifically, the server analyzes the collected feedback and generates new program improvement proposals.

[0867] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0868] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0869] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0870] [Third Embodiment]

[0871] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0872] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0873] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0874] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0875] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0876] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0877] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0878] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0879] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0880] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0881] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0882] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0883] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means.

[0884] First, when the system starts up, the server loads generated data and manual mode data using initialization mechanisms. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[0885] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[0886] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[0887] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct the learning content to be provided to the user. For example, specific knowledge such as that needed for certification exams or new employee training is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[0888] During learning, the device uses progress tracking to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[0889] After completing the learning program, users use a feedback processing mechanism to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from the terminal to the server, which stores the received feedback in a database and performs analysis. This analysis is then used to improve future programs.

[0890] Specific example

[0891] For example, if a new employee chooses the "New Employee Training Program":

[0892] 1. The server loads generated data and manual mode data when the system starts up.

[0893] 2. The terminal displays a screen for the user to log in to the system. The user enters their ID and password to authenticate.

[0894] 3. The server starts a session if authentication is successful and displays a list of available learning programs to the user.

[0895] 4. The user selects the "New Employee Training Program".

[0896] 5. The terminal sends the selected program information to the server.

[0897] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[0898] 7. The device displays learning content to the user. For example, the section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[0899] 8. The device records the user's learning progress and sends it to the server.

[0900] 9. The server saves progress data to a database, allowing users to resume their learning progress the next time they log in.

[0901] 10. After completing the learning process, the user submits feedback and sends it to the server.

[0902] 11. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[0903] In this way, the educational system of the present invention provides users with effective and balanced education, and enables continuous improvement through tracking of learning progress and feedback.

[0904] The following describes the processing flow.

[0905] Step 1:

[0906] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This includes machine learning models, existing general knowledge data, and custom data and programs manually added by educators.

[0907] Step 2:

[0908] The terminal displays a login screen to the user. The user enters their ID and password and submits their credentials.

[0909] Step 3:

[0910] The server uses user authentication methods to verify the user's credentials and authenticate their legitimacy. If authentication is successful, it starts a session and sends an authentication token to the device.

[0911] Step 4:

[0912] After logging in, users can browse a list of learning programs and select a program that matches their learning goals. This is made possible through a learning program selection tool.

[0913] Step 5:

[0914] The terminal sends the program information selected by the user to the server.

[0915] Step 6:

[0916] The server integrates the generated data and manual mode data based on the received program ID to construct the learning content.

[0917] Step 7:

[0918] The server sends the configured learning content to the terminal. For example, in a new employee training program, company etiquette is provided from manual mode data, and business etiquette is provided from generated data.

[0919] Step 8:

[0920] The device displays the received learning content to the user. The user then proceeds with their learning while referring to it.

[0921] Step 9:

[0922] The device uses a progress tracking system to record the user's learning progress in real time. The learning progress is periodically sent to the server.

[0923] Step 10:

[0924] The server saves the received progress data to a database. This allows the user to continue their learning progress the next time they log in.

[0925] Step 11:

[0926] After completing the learning process, users use a feedback processing mechanism to enter their evaluation and comments on the learning content into a feedback form. Once the feedback is complete, it is sent to the server.

[0927] Step 12:

[0928] The server stores the received feedback in a database for later analysis, which can then be used to improve the educational program.

[0929] In this way, it becomes possible to provide users with effective learning content, track their progress, and utilize their feedback throughout each processing step of the system.

[0930] (Example 1)

[0931] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0932] Traditional education systems have a fixed approach to providing learning content, making it difficult to deliver balanced learning programs that appropriately integrate general and custom data. Furthermore, they lacked mechanisms to track user learning progress in real time and improve the system through continuous feedback. This resulted in insufficient educational effectiveness and decreased user motivation.

[0933] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0934] In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, means for integrating generated data and custom data after learning program selection to constitute learning content, and means for sending progress data to the server periodically or at the end of an operation. This makes it possible to provide individually customized learning content to users, track progress in real time, and continuously improve the educational program by utilizing feedback.

[0935] An "initialization method" is a means of loading generated data and custom data when the system starts up, and bringing the system into an operational state.

[0936] "User authentication means" refers to a method for verifying the credentials a user uses to access a system and determining their legitimacy.

[0937] The "learning program selection means" is a means of displaying a list of learning programs available to the user and sending the selected program information to the server.

[0938] A "means for providing learning content" refers to a means of integrating generated data and custom data based on a selected learning program to construct learning content and provide it to the user.

[0939] A "progress tracking method" is a means of recording the user's learning progress and sending it to the server at regular intervals or upon completion of an operation.

[0940] A "feedback processing method" is a means by which users input evaluations and comments on the learning content and send them to the server.

[0941] "Generated data" refers to data that is automatically generated by a system, such as machine learning models and general knowledge data.

[0942] "Custom data" refers to customized data that has been manually added by the training staff.

[0943] A "learning program" is a plan that includes a set of learning content and activities designed to teach specific knowledge or skills.

[0944] A "server" is a central computer system that handles data processing and management for the entire system.

[0945] A "terminal" is a device used by a user to access the system and utilize learning content.

[0946] A "session" refers to a specific period of time during which a user operates within a system, from the time they log in until they log out.

[0947] A "database" is a structured collection of data used to efficiently store, manage, and retrieve data within a system.

[0948] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means. This makes it possible to provide individually customized learning content to users, track progress in real time, and continuously improve the educational program by utilizing feedback. Embodiments of the present invention are described in detail below.

[0949] First, when the system starts up, the server loads generated data and custom data using initialization methods. Specifically, generated data includes machine learning models and existing general knowledge data, which are retrieved from a database or external storage. Custom data, on the other hand, includes customized data manually added by educators.

[0950] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password. Using user authentication, the server verifies the credentials and determines their legitimacy. If authentication is successful, a session is started, and the user can begin using the system.

[0951] After authentication, the server generates a list of available learning programs and sends it to the terminal. The user then uses the learning program selection tool to choose their desired program. The selected program information is then sent from the terminal to the server.

[0952] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and custom data to construct the learning content. Specifically, particular knowledge such as that required for certification exams or new employee training is provided from custom data, while general knowledge such as business etiquette is provided from generated data. This results in well-balanced learning content.

[0953] As the learning process progresses, the device uses progress tracking to record the user's learning progress. Progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume learning based on their progress the next time they log in.

[0954] After completing the learning program, users use a feedback processing mechanism to input evaluations and comments about the learning content. This feedback is sent from the terminal to the server, which stores it in a database and performs analysis. The results of this analysis are used to improve future programs.

[0955] Specific example

[0956] For example, if a new employee selects the "New Employee Training Program," the following process takes place: The server loads generated data and custom data when the system starts up. The terminal displays a login screen, and the user authenticates with their ID and password. If authentication is successful, the server generates a list of available learning programs and displays it to the user. The user selects the "New Employee Training Program," and the terminal sends this information to the server. The server integrates the generated data and custom data to create learning content and sends it to the terminal. The terminal displays the learning content, records progress, and sends it to the server. Finally, the user submits feedback, which the server saves to a database and analyzes.

[0957] Example of a prompt

[0958] "Please describe the system's process for integrating the content of a new employee training program using both generated and custom data, and providing it to users while tracking their progress."

[0959] As described above, the educational system of the present invention provides users with efficient and balanced education, and enables continuous improvement through tracking of learning progress and feedback.

[0960] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0961] Step 1:

[0962] The server loads generated data and custom data using initialization methods when the system starts up.

[0963] Input: Generated data from databases and external storage, and custom data.

[0964] Processing: Access databases and external storage and load the necessary data into memory.

[0965] Output: Generated data and custom data stored in memory

[0966] Specific operation: The server uses Python or SQL scripts to connect to the database and load generated data (e.g., machine learning models) and custom data (materials added by educators).

[0967] Step 2:

[0968] The device displays a login screen and prompts the user to enter their ID and password.

[0969] Input: User ID and password

[0970] Process: Render the login screen and accept input.

[0971] Output: Authentication information (ID and password) sent to the server.

[0972] Specific operation: The terminal displays a login form in a web browser or dedicated application, accepts user input, and has a submit button to send authentication information to the server.

[0973] Step 3:

[0974] The server verifies credentials using user authentication methods and determines their legitimacy.

[0975] Input: Authentication information (ID and password) sent to the server

[0976] Process: Verify authentication information against user information in the database.

[0977] Output: Status indicating authentication success or failure

[0978] Specific operation: The server compares the ID and password with the internal user database and generates a session ID if authentication is successful.

[0979] Step 4:

[0980] The server generates a list of available learning programs after successful authentication and sends it to the terminal.

[0981] Input: Authentication success status and learning program information from the internal database.

[0982] Process: Extract learning program information from the database and generate a list.

[0983] Output: List of learning programs sent to the terminal

[0984] Specific operation: The server extracts a list of learning programs from the database using an SQL query and sends it to the terminal in JSON format.

[0985] Step 5:

[0986] The user selects their desired learning program.

[0987] Input: List of learning programs displayed on the terminal

[0988] Process: The user selects a learning program and sends that selection information to the server.

[0989] Output: Selected learning program information sent to the server

[0990] Specific operation: The user clicks or taps the desired program on the device screen and presses the send button to send the selected program information.

[0991] Step 6:

[0992] The server integrates generated data and custom data based on the selected program information to construct the learning content.

[0993] Input: User-selected learning program information and generated and custom data loaded during initialization.

[0994] Process: Generate learning content using generated data and custom data.

[0995] Output: Integrated learning content sent to the device

[0996] Specific operation: The server matches generated data and custom data related to the selected learning program and converts the consistent learning content into a single HTML document or application display format.

[0997] Step 7:

[0998] The terminal displays learning content sent from the server to the user.

[0999] Input: Learning content sent from the server

[1000] Processing: Render and display the learning content.

[1001] Output: Learning content displayed to the user

[1002] Specific operation: The device uses HTML, CSS, JavaScript, etc., to display learning content in a user-friendly format.

[1003] Step 8:

[1004] The device records the user's learning progress and sends it to the server at regular intervals or after the operation is completed.

[1005] Input: User's learning progress (completed sections, answer results, etc.)

[1006] Process: Log the progress and send it to the server.

[1007] Output: Training progress data sent to the server

[1008] Specific operation: The terminal records user actions using JavaScript or other scripts and periodically sends this data to the server in JSON format.

[1009] Step 9:

[1010] The server saves progress data to a database, allowing users to resume their learning the next time they log in.

[1011] Input: Learning progress data sent from the device

[1012] Process: Store progress data in the database and associate it with the user session.

[1013] Output: Progress data stored in the database

[1014] Specific operation: The server receives progress data and stores it in the database using SQL. It also associates the user's session ID with the progress data and stores that information.

[1015] Step 10:

[1016] After completing the learning process, users enter their evaluation and comments in a feedback form.

[1017] Input: Ratings and comments entered in the feedback form

[1018] Process: Enter your feedback.

[1019] Output: Feedback data sent to the server

[1020] Specific action: The user enters text into the feedback form on the device and presses the submit button.

[1021] Step 11:

[1022] The device sends the feedback content to the server.

[1023] Input: Feedback data

[1024] Processing: Send feedback data to the server.

[1025] Output: Feedback data sent to the server

[1026] Specific operation: The terminal sends feedback data to the server using an HTTP request.

[1027] Step 12:

[1028] The server stores the feedback in a database and performs analysis.

[1029] Input: Feedback data sent from the device

[1030] Processing: Store feedback in the database and analyze it.

[1031] Output: Feedback and analysis results stored in the database

[1032] Specific operation: The server receives feedback data and stores it in a database using SQL. Then, natural language processing tools and machine learning models are used to analyze the feedback and use it to improve the educational program.

[1033] The above describes the program processing flow of the system, including its specific actions.

[1034] (Application Example 1)

[1035] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1036] Traditional education systems have made efficient learning and progress management difficult, and in the industrial sector in particular, there has been a lack of effective means for learning new machinery and operating procedures. Furthermore, the lack of integrated systems for learning robot operating procedures, tracking progress, and processing feedback made efficient operation difficult. Therefore, there is a need to improve the efficiency of learning and practice using robots.

[1037] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1038] In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, means for providing learning content to the robot, means for tracking the robot's learning progress, and means for processing feedback from the robot. This enables the robot to effectively learn new machines and work procedures, to grasp its progress in detail, and to provide appropriate feedback.

[1039] An "initialization method" is a means for loading generated data and specific data when the system starts up.

[1040] "User authentication means" refers to the method of authentication using an ID and password when a user accesses a system.

[1041] "Means for selecting a learning program" refers to the means by which learners can select available learning programs.

[1042] "Means of providing learning content" refers to means of constructing learning content based on a learning program and providing it to learners.

[1043] A "progress tracking system" is a means of recording a learner's learning progress and sending it to a server.

[1044] A "feedback processing method" is a means of collecting evaluations and comments after learning and using them to improve the system.

[1045] "Means for providing learning content to robots" refers to means for effectively providing learning content to robots and enabling them to learn operating procedures and business processes.

[1046] "Means for tracking the learning progress of a robot" refers to means for tracking the progress of a robot as it learns and recording it on a server.

[1047] "Means for processing robot feedback" refers to methods for collecting feedback that a robot provides after learning and using that feedback to improve the system.

[1048] This invention provides an educational system for effectively teaching factory robots new operating procedures and work processes. Specific embodiments of the system are described below.

[1049] Explanation of program generation and processing

[1050] This system consists of the following hardware and software.

[1051] Hardware:

[1052] Robot control unit (e.g., NVIDIA Jetson)

[1053] Server (e.g., AWS EC2 instance)

[1054] Communication module (e.g., Wi-Fi / Bluetooth)

[1055] software:

[1056] Machine learning frameworks (e.g., TensorFlow)

[1057] Database management system (e.g., MySQL)

[1058] Robot Operating System (ROS)

[1059] Application frameworks (e.g., Django)

[1060] Initialization means

[1061] The server loads generated data and specific data using an initialization mechanism upon startup. The generated data includes machine learning models, while the specific data includes manually added custom data.

[1062] User authentication methods

[1063] When a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy. If authentication is successful, a session is started.

[1064] Learning program selection method

[1065] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user selects their desired program, and this information is sent from the terminal to the server.

[1066] Means of providing learning content

[1067] Based on the selected program, the server integrates generated data and specific data to construct learning content. For example, the operating procedures for a specific machine are provided from specific data, while general operating concepts are provided from generated data.

[1068] Progress tracking methods

[1069] During learning, the device records the user's learning progress using a progress tracking mechanism. Additionally, progress data is sent to the server at regular intervals or when the user finishes an operation, and the server stores it in a database.

[1070] Feedback processing means

[1071] After completing the learning program, users enter their evaluations and comments into a feedback form. This information is sent from the device to the server, which stores the feedback in a database and performs analysis. The results of this analysis will be used to improve future programs.

[1072] Specific example

[1073] For example, when a robot learns the operating procedures for a new polishing machine, the server loads generated data for the operating procedures and manual operation procedure data. The robot logs in and sends authentication information to the server. If authentication is successful, the robot is presented with a "polishing machine operation program" and selects it. The server provides learning content by integrating the general polishing process from the generated data and the specific machine operation procedures from the manual data. The robot records its progress as it performs the operation and sends it to the server. Once the operation is complete, it inputs feedback and sends it to the server. Based on this, the next operating procedures are improved.

[1074] Example of a prompt

[1075] "I would like to learn how to operate the new polishing machine. First, please teach me the basic safety procedures. Then, please explain the overall process."

[1076] As described above, the educational system of the present invention provides robots with effective and balanced education, and enables continuous improvement through tracking and feedback on learning progress.

[1077] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1078] Step 1:

[1079] The server loads generated data and specific data using initialization methods when the system starts up. The generated data includes machine learning models, and the specific data includes custom data added manually. This allows the server to prepare the dataset necessary for training. Specifically, the server loads the machine learning models from disk into memory, reads the custom data, and saves it to the database.

[1080] Input: Generated data on disk and specific data

[1081] Data processing / calculation: File reading, loading into memory, and performing necessary transformations.

[1082] Output: Prepared dataset

[1083] Step 2:

[1084] When a user accesses the system, the terminal displays a login screen. The user enters their ID and password, and the terminal sends this authentication information to the server. The server verifies the credentials using user authentication methods and authenticates their legitimacy. If authentication is successful, a session is started. Specifically, the server compares the user information stored in the database with the entered credentials, and if they match, a session is started.

[1085] Input: User ID, Password

[1086] Data processing / data calculations: Database queries, credential matching.

[1087] Output: Authentication result (success / failure)

[1088] Step 3:

[1089] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program, and this information is sent from the terminal to the server. The server receives the selected program information and begins preparing the corresponding learning content. Specifically, the server uses the selected program ID as a key to retrieve the corresponding learning content data from the database.

[1090] Input: User-selected program ID

[1091] Data processing / data calculation: Database queries, retrieval of learning content

[1092] Output: Training content dataset

[1093] Step 4:

[1094] Using a learning content delivery method, the server integrates generated data and specific data. The server provides operating procedures for specific machines from the specific data and general operating concepts from the generated data. This integrates content aligned with the learning program and transmits it to the terminal. Specifically, the server integrates generated data and specific data to create content that includes advanced operating procedures and general operating principles.

[1095] Input: Generated data, specific data

[1096] Data processing / data calculations: data integration, content generation

[1097] Output: Integrated learning content

[1098] Step 5:

[1099] During learning, the device records the user's learning progress using progress tracking mechanisms. It also sends progress data to the server at regular intervals or when the user finishes an operation, and the server stores this data in a database. Specifically, the device collects operation logs and progress information in real time and periodically transfers them to the server.

[1100] Input: User operation log, progress information

[1101] Data processing / data calculation: data recording, periodic transfer

[1102] Output: Progress data

[1103] Step 6:

[1104] After completing the learning program, users enter their evaluations and comments into a feedback form. This information is sent from the device to the server, which stores the feedback in a database and performs analysis. Specifically, the server analyzes the feedback data to extract statistical evaluations and areas for improvement.

[1105] Input: User ratings, comments

[1106] Data processing / data calculation: Storage and analysis of feedback data.

[1107] Output: analysis results, statistical information

[1108] Example of a prompt

[1109] "I would like to learn how to operate the new polishing machine. First, please teach me the basic safety procedures. Then, please explain the overall process."

[1110] The above describes the specific processing steps of the system. In each step, appropriate data processing and calculations are performed based on the input data to obtain output, thereby enabling the operation of the entire system.

[1111] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1112] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, and an emotion engine.

[1113] First, when the system starts up, the server loads generated data and manual mode data using initialization mechanisms. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[1114] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[1115] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[1116] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct the learning content. For example, specific knowledge such as that required for certification exams or new employee training is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[1117] Furthermore, during learning, the device uses an emotion engine to recognize emotions from the user's facial expressions and voice. The recognized emotion information is sent to the server in real time.

[1118] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it can suggest a break or reduce the intensity of the content. It can also provide additional explanations and support if the user is feeling anxious.

[1119] During learning, the device uses progress tracking to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[1120] After completing the learning program, users use a feedback processing system to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from the terminal to the server, which stores the received feedback in a database and performs analysis. This allows for the improvement of the educational program.

[1121] Specific example

[1122] For example, if a new employee chooses the "New Employee Training Program":

[1123] 1. The server loads generated data and manual mode data when the system starts up.

[1124] 2. The terminal displays a screen for the user to log in to the system. The user enters their ID and password to authenticate.

[1125] 3. The server starts a session if authentication is successful and displays a list of available learning programs to the user.

[1126] 4. The user selects the "New Employee Training Program".

[1127] 5. The terminal sends the selected program information to the server.

[1128] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[1129] 7. The device displays learning content to the user. For example, the section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[1130] 8. The device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[1131] 9. The server dynamically adjusts learning content based on sentiment data. If the user is tired, it will suggest a break or change the learning content to a simplified version.

[1132] 10. The device records the user's learning progress and sends it to the server.

[1133] 11. The server saves progress data to a database, allowing users to continue their learning progress the next time they log in.

[1134] 12. After completing the learning process, the user submits feedback and sends it to the server.

[1135] 13. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[1136] In this way, the educational system of the present invention provides users with effective and balanced education, and enables dynamic learning that responds to the user's emotions through an emotion engine. Furthermore, continuous improvement is possible through tracking of learning progress and feedback.

[1137] The following describes the processing flow.

[1138] Step 1:

[1139] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[1140] Step 2:

[1141] The terminal displays a login screen to the user. The user enters their ID and password and submits their credentials.

[1142] Step 3:

[1143] The server uses user authentication methods to verify the user's credentials and authenticate their legitimacy. If authentication is successful, it starts a session and sends an authentication token to the device.

[1144] Step 4:

[1145] After logging in, users can browse a list of learning programs and select one that matches their learning goals. This is made possible by the learning program selection method.

[1146] Step 5:

[1147] The terminal sends the program information selected by the user to the server.

[1148] Step 6:

[1149] Based on the received program ID, the server integrates the generated data and manual mode data to construct the learning content.

[1150] Step 7:

[1151] The server sends the configured learning content to the terminal. For example, in a new employee training program, company etiquette is provided from manual mode data, and business etiquette is provided from generated data.

[1152] Step 8:

[1153] The device displays the received learning content to the user. The user then proceeds with their learning while referring to it.

[1154] Step 9:

[1155] During training, the device uses an emotion engine to recognize emotions from the user's facial expressions and voice. The recognized emotion information is sent to the server in real time.

[1156] Step 10:

[1157] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it suggests a break or reduces the intensity of the content. It also provides additional explanations and support if the user is feeling anxious.

[1158] Step 11:

[1159] The device records the user's learning progress using a progress tracking mechanism. Progress data is sent to the server at regular intervals or when the user finishes their operation.

[1160] Step 12:

[1161] The server saves the received progress data to a database. This allows the user to continue their learning progress the next time they log in.

[1162] Step 13:

[1163] After completing the learning program, the user uses a feedback processing mechanism to enter their evaluation and comments on the learning content into a feedback form. The feedback content is then sent from the terminal to the server.

[1164] Step 14:

[1165] The server stores the received feedback in a database and performs analysis. This helps to improve the educational program.

[1166] Specific example

[1167] If a new employee chooses the "New Employee Training Program":

[1168] 1. The server loads generated data and manual mode data when the system starts up.

[1169] 2. The terminal displays a screen for the user to log in to the system, and the user enters their ID and password.

[1170] 3. If authentication is successful, the server starts a session and displays a list of learning programs.

[1171] 4. The user selects the "New Employee Training Program".

[1172] 5. The terminal sends the selected program information to the server.

[1173] 6. The server integrates the generated data and the manual mode data to create the learning content for the "New Employee Training Program".

[1174] 7. The server sends the configured learning content to the device.

[1175] 8. The device displays learning content to the user, and the user proceeds with the learning.

[1176] 9. During learning, the device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[1177] 10. The server dynamically adjusts learning content based on sentiment data. If the user is tired, it will suggest a break or simplify the learning content.

[1178] 11. The device records the user's learning progress and sends it to the server.

[1179] 12. The server saves the progress data to the database.

[1180] 13. After completing the learning process, the user enters feedback and sends it to the server.

[1181] 14. The server stores the feedback in a database and analyzes it to help improve the educational program.

[1182] In this way, the educational system of the present invention provides users with effective and balanced education, and enables dynamic learning that responds to the user's emotions through an emotion engine. Furthermore, continuous improvement is possible through tracking of learning progress and feedback.

[1183] (Example 2)

[1184] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1185] Traditional education systems often fail to consider the emotional state of users when providing learning content, leading to decreased learning efficiency and learner satisfaction. Furthermore, managing learning progress and processing feedback are frequently done manually, which is time-consuming and labor-intensive, making it difficult to provide efficient learning.

[1186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, emotion recognition means, and means for dynamically adjusting learning content based on emotion information. This enables flexible provision of learning content according to the learner's emotional state, as well as efficient progress management and feedback processing.

[1187] An "initialization method" is a means of loading the data necessary for system startup and setting the system to its initial state.

[1188] "User authentication means" are methods used to verify the legitimacy of a user's access to a system.

[1189] A "learning program selection method" is a means that allows users to select their desired program from among the available learning programs.

[1190] "Means of providing learning content" refers to means of providing learning content suitable for the user based on the selected learning program.

[1191] A "progress tracking method" is a means of recording the user's learning progress and sending it to the server as needed.

[1192] A "feedback processing method" is a means of collecting user feedback, sending it to a server, and performing analysis on it.

[1193] An "emotion recognition method" is a means of analyzing a user's facial expressions and voice to recognize their emotional state in real time.

[1194] "Means for dynamically adjusting learning content based on emotional information" refers to means of changing the content and progression of learning content in a timely manner according to the user's emotional state.

[1195] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, emotion recognition means, and means for dynamically adjusting learning content based on emotion information.

[1196] First, upon system startup, the server loads generated data and manual mode data using initialization mechanisms. This generated data includes machine learning models and general knowledge data, while the manual mode data includes custom data and programs manually added by instructors. The server uses Apache server software and Python programs to load this data from disk into memory.

[1197] Next, the terminal displays a login screen when the user accesses the system. The user enters their ID and password to authenticate. Based on the user authentication method, the server uses the Django framework to verify the credentials and authenticate their legitimacy. If authentication is successful, a session is started and the user is redirected to the dashboard screen.

[1198] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program from the list. This information is sent to the server via a JavaScript event handler.

[1199] Based on the selected program, the server integrates generated data and manual mode data to create learning content. Specifically, it extracts specific data from an SQL database using a Python script and supplements the content as needed using a machine learning model. The server then sends the constructed learning content to the device via an API.

[1200] The device uses JavaScript to display learning content to the user in real time. Furthermore, it utilizes emotion recognition technology, analyzing the user's facial expressions and voice data in real time using OpenCV and Federated Learning techniques. This emotion data is sent to the server in JSON format.

[1201] The server analyzes the received emotional information and dynamically adjusts the content and progression of the learning materials according to the user's learning state. For example, if the server determines that the user is tired, it will suggest a break or simplify the content. This is done using conditional branching in a Python program.

[1202] Learning progress is temporarily saved to local storage by the device using JavaScript and sent to the server periodically or after the operation is completed. Upon receiving the progress data, the server saves it to a PostgreSQL database and makes it available the next time the user logs in.

[1203] After completing the learning program, users enter their evaluations and comments into a feedback form. This input data is sent from the terminal to the server, which stores the feedback in a database and performs statistical processing of the feedback using a Python script.

[1204] Specific example

[1205] For example, the procedure for a new employee who selects the "New Employee Training Program" is as follows:

[1206] 1. The server loads generated data and manual mode data when the system starts up.

[1207] 2. The terminal displays the user's login screen, and the user enters their ID and password to authenticate.

[1208] 3. If authentication is successful, the server will start a session and display a list of available learning programs to the user.

[1209] 4. The user selects the "New Employee Training Program".

[1210] 5. The terminal sends the selected program information to the server.

[1211] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[1212] 7. The device displays learning content to the user. The section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[1213] 8. The device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[1214] 9. The server dynamically adjusts learning content based on emotional data. For example, if the user is tired, it suggests a break and simplifies the learning content.

[1215] 10. The device records the user's learning progress and sends it to the server.

[1216] 11. The server saves progress data to a database, allowing users to continue their learning progress the next time they log in.

[1217] 12. After completing the learning process, the user submits feedback and sends it to the server.

[1218] 13. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[1219] Examples of inputs to generative AI models

[1220] "Generate learning content suitable for a new employee training program."

[1221] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1222] Program processing steps

[1223] Step 1:

[1224] The server loads generated data and manual mode data using initialization methods during system startup. At this time, the server begins operating the Apache server software and uses a Python program to load generated data (machine learning models and general knowledge data) and manual mode data (custom data and programs) from disk into memory. The input is the command used during system startup, and the output is the data loaded into memory.

[1225] Step 2:

[1226] The terminal displays a login screen when a user accesses the system. Here, the user enters their ID and password. The entered authentication information is sent to the server using the HTTPS protocol. The input is the user ID and password, and the output is the transmission of the authentication information. Specifically, JavaScript on the terminal collects the authentication information and creates an HTTP request to send it to the server.

[1227] Step 3:

[1228] The server verifies the legitimacy of the credentials sent using user authentication methods. This is done using the Django framework and a database. Specifically, the server compares the received credentials with the database, and if authentication is successful, it starts a session and instructs the user to redirect to the dashboard screen. The input is the credentials, and the output is the authentication status and session information.

[1229] Step 4:

[1230] After successful login, the terminal displays a list of available learning programs to the user using a learning program selection mechanism. A JavaScript-based user interface is applied here. The input is a list of learning programs sent from the server, and the output is the display of the program list to the user.

[1231] Step 5:

[1232] The user selects their desired program from a displayed list of learning programs. The selection information is sent to the server via a JavaScript event handler. The input is the user's program selection, and the output is the transmission of the selection information to the server.

[1233] Step 6:

[1234] The server integrates generated data and manual mode data based on the selected program to generate learning content. It extracts specific data from an SQL database using a Python script and uses a machine learning model to supplement the content as needed. The input is the selected program information, and the output is the generated learning content. The generated content is sent to the terminal via an API.

[1235] Step 7:

[1236] The device uses JavaScript to display learning content to the user. At the same time, it utilizes emotion recognition to analyze the user's facial expressions and voice data in real time using OpenCV and Federated Learning technologies. The analyzed emotion data is sent to the server in JSON format. Inputs are the learning content from the server and the user's facial expressions and voice data, while outputs are the content displayed to the user and the transmitted emotion data.

[1237] Step 8:

[1238] The server analyzes emotional information and dynamically adjusts the learning content. Using a Python script, if the server determines that the user is tired, it suggests a break or changes the content to a simplified version. The input is the user's emotional information, and the output is the adjusted learning content.

[1239] Step 9:

[1240] The device records progress during learning and sends it to the server periodically or after the operation is completed. JavaScript is used to temporarily store progress data in local storage and to create an HTTP request when sending it. The input is the user's learning progress, and the output is the transmission of progress data to the server.

[1241] Step 10:

[1242] The server saves the received progress data to a database so that the game can resume from the same state the next time the user logs in. A Python script parses the progress data and stores it in the database. The input is the progress data, and the output is the progress information stored in the database.

[1243] Step 11:

[1244] After completing the learning program, users enter their evaluations and comments into a feedback form and send it from their device to the server. The input is feedback information, and the output is the transmission of feedback data to the server.

[1245] Step 12:

[1246] The server stores the received feedback in a database and performs statistical processing on the feedback using a Python script. The input is the feedback information, and the output is the feedback content stored in the database and the results of its analysis.

[1247] (Application Example 2)

[1248] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1249] In today's workplace, effective and real-time learning and training are essential. However, traditional education systems often lack the ability to dynamically adjust to learners' emotions and circumstances, leading to decreased learning effectiveness. Furthermore, in on-site training, such as in physical stores, learning must progress during actual work, and insufficient progress management and feedback functions in such situations pose a challenge.

[1250] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes an initialization means, a user authentication means, a learning program selection means, a learning content provision means, a progress tracking means, a feedback processing means, an emotion analysis means, and a learning content dynamic adjustment means. This enables effective and personalized learning by analyzing the learner's emotional information in real time and dynamically adjusting the learning content. Furthermore, continuous improvement of the training program is achieved through progress management and the collection and analysis of feedback.

[1251] "Initialization means" refers to the means of loading generated data and manual mode data when the system starts up.

[1252] "User authentication means" refers to the method used to verify the credentials and authenticate the legitimacy of a user when they access a system using their ID and password.

[1253] "Learning program selection means" refers to a means that allows the user to select their desired learning program, and is a means of displaying a list of available programs.

[1254] A "means for providing learning content" refers to a means of integrating generated data and manual mode data to construct learning content and provide it to the user.

[1255] A "progress tracking method" is a means of recording a user's learning progress and sending it to a server.

[1256] A "feedback processing method" is a means of collecting evaluations and comments from users after the learning process is complete, and sending and storing them on a server.

[1257] "Emotion analysis methods" are means of analyzing emotional information in real time from a user's facial expressions and voice.

[1258] A "dynamic learning content adjustment method" is a means of dynamically adjusting learning content based on information obtained from emotion analysis methods.

[1259] The present invention is a system that effectively supports the training of new staff in physical stores. This system includes an initialization means, a user authentication means, a learning program selection means, a learning content provision means, a progress tracking means, a feedback processing means, an emotion analysis means, and a learning content dynamic adjustment means.

[1260] First, the server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This generated data includes machine learning models and existing general knowledge data. On the other hand, the manual mode data includes custom data and programs that are manually added by the instructor.

[1261] Next, when a user accesses the system, a device such as smart glasses or a smartphone displays a login screen. The user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[1262] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[1263] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct learning content. For example, specific knowledge such as product display methods, customer service etiquette, and store cleaning procedures is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[1264] Furthermore, during learning, devices such as smart glasses and smartphones use emotion analysis tools to recognize emotions from the user's facial expressions and voice. The recognized emotion information is transmitted to the server in real time. For example, EmotionEngine analyzes the user's facial expressions and determines emotions such as fatigue or stress.

[1265] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it can suggest a break or reduce the intensity of the content. It can also provide additional explanations and support if the user is feeling anxious.

[1266] During learning, devices such as smart glasses and smartphones use progress tracking mechanisms to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[1267] After completing the learning program, users use a feedback processing system to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from devices such as smart glasses or smartphones to a server, which stores the received feedback in a database and analyzes it. This allows for the improvement of the educational program.

[1268] As a concrete example, consider a scenario where a new staff member starts a "new employee training program" using smart glasses. First, the server loads the necessary data using an initialization mechanism, and the staff member logs into the system through a user authentication mechanism. Next, the staff member selects the "new employee training program" using a learning program selection mechanism, and the specific training content is displayed on the smart glasses by a learning content provision mechanism. During the training, the staff member's emotions are analyzed by an emotion analysis mechanism, and the content is dynamically adjusted as needed. Learning progress is recorded by a progress tracking mechanism, and finally, training evaluations are collected by a feedback processing mechanism. This enables effective and personalized training.

[1269] Examples of prompts to input into a generative AI model:

[1270] Please generate the learning content for the new employee training program. Include the following items: product display methods, customer service etiquette, and store cleaning procedures. Dynamic adjustments should also be made based on the learners' emotions.

[1271] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1272] Step 1:

[1273] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This prepares the server with the basic data necessary to run the learning program. The input is the generated data and manual mode data pre-stored in the system, and the output is the initialized data profile. Specifically, the server reads the necessary files from the database and loads them into memory.

[1274] Step 2:

[1275] The terminal provides the user with a login screen, and the user authenticates by entering their ID and password. Using the user authentication method, the server verifies the credentials and authenticates their legitimacy. The input is the ID and password entered by the user, and the output is the authentication success or failure status. Specifically, the server compares the entered information with the database and starts a session if they match.

[1276] Step 3:

[1277] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program, and the selected program information is sent from the terminal to the server. The input is the learning program selected by the user, and the output is specific information about the selected program. Specifically, the server receives the user's selection and sends an appropriate response back to the terminal.

[1278] Step 4:

[1279] The server uses a learning content delivery mechanism to integrate generated data and manual mode data based on the selected program, thereby generating learning content. This makes it possible to provide users with well-balanced learning content. The input is the selected program information, generated data, and manual mode data, and the output is the generated learning content. Specifically, the server analyzes the data, extracts relevant information, and integrates it.

[1280] Step 5:

[1281] During learning, devices such as smart glasses use emotion analysis techniques to recognize emotions from the user's facial expressions and voice, and transmit this information to a server in real time. The input is the user's facial expressions and voice data, and the output is the emotion analysis result. Specifically, data is collected using the device's camera and microphone, and EmotionEngine analyzes the data.

[1282] Step 6:

[1283] The server dynamically adjusts the learning content based on the emotional information it receives. For example, if it detects that the user is tired, it may suggest a break or change to lighter content. The input is the result of the emotional analysis, and the output is the adjusted learning content. Specifically, the server changes the current content and provides feedback according to the user's state.

[1284] Step 7:

[1285] During learning, devices such as smart glasses use progress tracking mechanisms to record the user's learning progress and send this data to the server at regular intervals or when the user finishes their activity. The input is the user's learning activity data, and the output is the progress record. Specifically, the device monitors the completion status of learning steps in real time and reports the progress to the server.

[1286] Step 8:

[1287] After completing the learning program, the user uses a feedback processing mechanism to enter evaluations and comments on the learning content into a feedback form and sends them from the terminal to the server. The input is the user's feedback comments, and the output is the collected feedback data. Specifically, the terminal receives input from the user and forwards it to the server.

[1288] Step 9:

[1289] The server stores the received feedback in a database and performs analysis. This helps improve the educational program. The input is user feedback data, and the output is the analysis results and improvement items. Specifically, the server analyzes the collected feedback and generates new program improvement proposals.

[1290] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1291] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1292] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1293] [Fourth Embodiment]

[1294] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1295] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1296] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1297] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1298] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1299] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1300] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1301] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1302] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1303] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1304] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1305] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1306] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1307] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means.

[1308] First, when the system starts up, the server loads generated data and manual mode data using initialization mechanisms. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[1309] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[1310] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[1311] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct the learning content to be provided to the user. For example, specific knowledge such as that needed for certification exams or new employee training is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[1312] During learning, the device uses progress tracking to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[1313] After completing the learning program, users use a feedback processing mechanism to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from the terminal to the server, which stores the received feedback in a database and performs analysis. This analysis is then used to improve future programs.

[1314] Specific example

[1315] For example, if a new employee chooses the "New Employee Training Program":

[1316] 1. The server loads generated data and manual mode data when the system starts up.

[1317] 2. The terminal displays a screen for the user to log in to the system. The user enters their ID and password to authenticate.

[1318] 3. The server starts a session if authentication is successful and displays a list of available learning programs to the user.

[1319] 4. The user selects the "New Employee Training Program".

[1320] 5. The terminal sends the selected program information to the server.

[1321] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[1322] 7. The device displays learning content to the user. For example, the section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[1323] 8. The device records the user's learning progress and sends it to the server.

[1324] 9. The server saves progress data to a database, allowing users to resume their learning progress the next time they log in.

[1325] 10. After completing the learning process, the user submits feedback and sends it to the server.

[1326] 11. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[1327] In this way, the educational system of the present invention provides users with effective and balanced education, and enables continuous improvement through tracking of learning progress and feedback.

[1328] The following describes the processing flow.

[1329] Step 1:

[1330] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This includes machine learning models, existing general knowledge data, and custom data and programs manually added by educators.

[1331] Step 2:

[1332] The terminal displays a login screen to the user. The user enters their ID and password and submits their credentials.

[1333] Step 3:

[1334] The server uses user authentication methods to verify the user's credentials and authenticate their legitimacy. If authentication is successful, it starts a session and sends an authentication token to the device.

[1335] Step 4:

[1336] After logging in, users can browse a list of learning programs and select a program that matches their learning goals. This is made possible through a learning program selection tool.

[1337] Step 5:

[1338] The terminal sends the program information selected by the user to the server.

[1339] Step 6:

[1340] The server integrates the generated data and manual mode data based on the received program ID to construct the learning content.

[1341] Step 7:

[1342] The server sends the configured learning content to the terminal. For example, in a new employee training program, company etiquette is provided from manual mode data, and business etiquette is provided from generated data.

[1343] Step 8:

[1344] The device displays the received learning content to the user. The user then proceeds with their learning while referring to it.

[1345] Step 9:

[1346] The device uses a progress tracking system to record the user's learning progress in real time. The learning progress is periodically sent to the server.

[1347] Step 10:

[1348] The server saves the received progress data to a database. This allows the user to continue their learning progress the next time they log in.

[1349] Step 11:

[1350] After completing the learning process, users use a feedback processing mechanism to enter their evaluation and comments on the learning content into a feedback form. Once the feedback is complete, it is sent to the server.

[1351] Step 12:

[1352] The server stores the received feedback in a database for later analysis, which can then be used to improve the educational program.

[1353] In this way, it becomes possible to provide users with effective learning content, track their progress, and utilize their feedback throughout each processing step of the system.

[1354] (Example 1)

[1355] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1356] Traditional education systems have a fixed approach to providing learning content, making it difficult to deliver balanced learning programs that appropriately integrate general and custom data. Furthermore, they lacked mechanisms to track user learning progress in real time and improve the system through continuous feedback. This resulted in insufficient educational effectiveness and decreased user motivation.

[1357] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1358] In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, means for integrating generated data and custom data after learning program selection to constitute learning content, and means for sending progress data to the server periodically or at the end of an operation. This makes it possible to provide individually customized learning content to users, track progress in real time, and continuously improve the educational program by utilizing feedback.

[1359] An "initialization method" is a means of loading generated data and custom data when the system starts up, and bringing the system into an operational state.

[1360] "User authentication means" refers to a method for verifying the credentials a user uses to access a system and determining their legitimacy.

[1361] The "learning program selection means" is a means of displaying a list of learning programs available to the user and sending the selected program information to the server.

[1362] A "means for providing learning content" refers to a means of integrating generated data and custom data based on a selected learning program to construct learning content and provide it to the user.

[1363] A "progress tracking method" is a means of recording the user's learning progress and sending it to the server at regular intervals or upon completion of an operation.

[1364] A "feedback processing method" is a means by which users input evaluations and comments on the learning content and send them to the server.

[1365] "Generated data" refers to data that is automatically generated by a system, such as machine learning models and general knowledge data.

[1366] "Custom data" refers to customized data that has been manually added by the training staff.

[1367] A "learning program" is a plan that includes a set of learning content and activities designed to teach specific knowledge or skills.

[1368] A "server" is a central computer system that handles data processing and management for the entire system.

[1369] A "terminal" is a device used by a user to access the system and utilize learning content.

[1370] A "session" refers to a specific period of time during which a user operates within a system, from the time they log in until they log out.

[1371] A "database" is a structured collection of data used to efficiently store, manage, and retrieve data within a system.

[1372] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, and feedback processing means. This makes it possible to provide individually customized learning content to users, track progress in real time, and continuously improve the educational program by utilizing feedback. Embodiments of the present invention are described in detail below.

[1373] First, when the system starts up, the server loads generated data and custom data using initialization methods. Specifically, generated data includes machine learning models and existing general knowledge data, which are retrieved from a database or external storage. Custom data, on the other hand, includes customized data manually added by educators.

[1374] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password. Using user authentication, the server verifies the credentials and determines their legitimacy. If authentication is successful, a session is started, and the user can begin using the system.

[1375] After authentication, the server generates a list of available learning programs and sends it to the terminal. The user then uses the learning program selection tool to choose their desired program. The selected program information is then sent from the terminal to the server.

[1376] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and custom data to construct the learning content. Specifically, particular knowledge such as that required for certification exams or new employee training is provided from custom data, while general knowledge such as business etiquette is provided from generated data. This results in well-balanced learning content.

[1377] As the learning process progresses, the device uses progress tracking to record the user's learning progress. Progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume learning based on their progress the next time they log in.

[1378] After completing the learning program, users use a feedback processing mechanism to input evaluations and comments about the learning content. This feedback is sent from the terminal to the server, which stores it in a database and performs analysis. The results of this analysis are used to improve future programs.

[1379] Specific example

[1380] For example, if a new employee selects the "New Employee Training Program," the following process takes place: The server loads generated data and custom data when the system starts up. The terminal displays a login screen, and the user authenticates with their ID and password. If authentication is successful, the server generates a list of available learning programs and displays it to the user. The user selects the "New Employee Training Program," and the terminal sends this information to the server. The server integrates the generated data and custom data to create learning content and sends it to the terminal. The terminal displays the learning content, records progress, and sends it to the server. Finally, the user submits feedback, which the server saves to a database and analyzes.

[1381] Example of a prompt

[1382] "Please describe the system's process for integrating the content of a new employee training program using both generated and custom data, and providing it to users while tracking their progress."

[1383] As described above, the educational system of the present invention provides users with efficient and balanced education, and enables continuous improvement through tracking of learning progress and feedback.

[1384] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1385] Step 1:

[1386] The server loads generated data and custom data using initialization methods when the system starts up.

[1387] Input: Generated data from databases and external storage, and custom data.

[1388] Processing: Access databases and external storage and load the necessary data into memory.

[1389] Output: Generated data and custom data stored in memory

[1390] Specific operation: The server uses Python or SQL scripts to connect to the database and load generated data (e.g., machine learning models) and custom data (materials added by educators).

[1391] Step 2:

[1392] The device displays a login screen and prompts the user to enter their ID and password.

[1393] Input: User ID and password

[1394] Process: Render the login screen and accept input.

[1395] Output: Authentication information (ID and password) sent to the server.

[1396] Specific operation: The terminal displays a login form in a web browser or dedicated application, accepts user input, and has a submit button to send authentication information to the server.

[1397] Step 3:

[1398] The server verifies credentials using user authentication methods and determines their legitimacy.

[1399] Input: Authentication information (ID and password) sent to the server

[1400] Process: Verify authentication information against user information in the database.

[1401] Output: Status indicating authentication success or failure

[1402] Specific operation: The server compares the ID and password with the internal user database and generates a session ID if authentication is successful.

[1403] Step 4:

[1404] The server generates a list of available learning programs after successful authentication and sends it to the terminal.

[1405] Input: Authentication success status and learning program information from the internal database.

[1406] Process: Extract learning program information from the database and generate a list.

[1407] Output: List of learning programs sent to the terminal

[1408] Specific operation: The server extracts a list of learning programs from the database using an SQL query and sends it to the terminal in JSON format.

[1409] Step 5:

[1410] The user selects their desired learning program.

[1411] Input: List of learning programs displayed on the terminal

[1412] Process: The user selects a learning program and sends that selection information to the server.

[1413] Output: Selected learning program information sent to the server

[1414] Specific operation: The user clicks or taps the desired program on the device screen and presses the send button to send the selected program information.

[1415] Step 6:

[1416] The server integrates generated data and custom data based on the selected program information to construct the learning content.

[1417] Input: User-selected learning program information and generated and custom data loaded during initialization.

[1418] Process: Generate learning content using generated data and custom data.

[1419] Output: Integrated learning content sent to the device

[1420] Specific operation: The server matches generated data and custom data related to the selected learning program and converts the consistent learning content into a single HTML document or application display format.

[1421] Step 7:

[1422] The terminal displays learning content sent from the server to the user.

[1423] Input: Learning content sent from the server

[1424] Processing: Render and display the learning content.

[1425] Output: Learning content displayed to the user

[1426] Specific operation: The device uses HTML, CSS, JavaScript, etc., to display learning content in a user-friendly format.

[1427] Step 8:

[1428] The device records the user's learning progress and sends it to the server at regular intervals or after the operation is completed.

[1429] Input: User's learning progress (completed sections, answer results, etc.)

[1430] Process: Log the progress and send it to the server.

[1431] Output: Training progress data sent to the server

[1432] Specific operation: The terminal records user actions using JavaScript or other scripts and periodically sends this data to the server in JSON format.

[1433] Step 9:

[1434] The server saves progress data to a database, allowing users to resume their learning the next time they log in.

[1435] Input: Learning progress data sent from the device

[1436] Process: Store progress data in the database and associate it with the user session.

[1437] Output: Progress data stored in the database

[1438] Specific operation: The server receives progress data and stores it in the database using SQL. It also associates the user's session ID with the progress data and stores that information.

[1439] Step 10:

[1440] After completing the learning process, users enter their evaluation and comments in a feedback form.

[1441] Input: Ratings and comments entered in the feedback form

[1442] Process: Enter your feedback.

[1443] Output: Feedback data sent to the server

[1444] Specific action: The user enters text into the feedback form on the device and presses the submit button.

[1445] Step 11:

[1446] The device sends the feedback content to the server.

[1447] Input: Feedback data

[1448] Processing: Send feedback data to the server.

[1449] Output: Feedback data sent to the server

[1450] Specific operation: The terminal sends feedback data to the server using an HTTP request.

[1451] Step 12:

[1452] The server stores the feedback in a database and performs analysis.

[1453] Input: Feedback data sent from the device

[1454] Processing: Store feedback in the database and analyze it.

[1455] Output: Feedback and analysis results stored in the database

[1456] Specific operation: The server receives feedback data and stores it in a database using SQL. Then, natural language processing tools and machine learning models are used to analyze the feedback and use it to improve the educational program.

[1457] The above describes the program processing flow of the system, including its specific actions.

[1458] (Application Example 1)

[1459] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1460] Traditional education systems have made efficient learning and progress management difficult, and in the industrial sector in particular, there has been a lack of effective means for learning new machinery and operating procedures. Furthermore, the lack of integrated systems for learning robot operating procedures, tracking progress, and processing feedback made efficient operation difficult. Therefore, there is a need to improve the efficiency of learning and practice using robots.

[1461] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1462] In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, means for providing learning content to the robot, means for tracking the robot's learning progress, and means for processing feedback from the robot. This enables the robot to effectively learn new machines and work procedures, to grasp its progress in detail, and to provide appropriate feedback.

[1463] An "initialization method" is a means for loading generated data and specific data when the system starts up.

[1464] "User authentication means" refers to the method of authentication using an ID and password when a user accesses a system.

[1465] "Means for selecting a learning program" refers to the means by which learners can select available learning programs.

[1466] "Means of providing learning content" refers to means of constructing learning content based on a learning program and providing it to learners.

[1467] A "progress tracking system" is a means of recording a learner's learning progress and sending it to a server.

[1468] A "feedback processing method" is a means of collecting evaluations and comments after learning and using them to improve the system.

[1469] "Means for providing learning content to robots" refers to means for effectively providing learning content to robots and enabling them to learn operating procedures and business processes.

[1470] "Means for tracking the learning progress of a robot" refers to means for tracking the progress of a robot as it learns and recording it on a server.

[1471] "Means for processing robot feedback" refers to methods for collecting feedback that a robot provides after learning and using that feedback to improve the system.

[1472] This invention provides an educational system for effectively teaching factory robots new operating procedures and work processes. Specific embodiments of the system are described below.

[1473] Explanation of program generation and processing

[1474] This system consists of the following hardware and software.

[1475] Hardware:

[1476] Robot control unit (e.g., NVIDIA Jetson)

[1477] Server (e.g., AWS EC2 instance)

[1478] Communication module (e.g., Wi-Fi / Bluetooth)

[1479] software:

[1480] Machine learning frameworks (e.g., TensorFlow)

[1481] Database management system (e.g., MySQL)

[1482] Robot Operating System (ROS)

[1483] Application frameworks (e.g., Django)

[1484] Initialization means

[1485] The server loads generated data and specific data using an initialization mechanism upon startup. The generated data includes machine learning models, while the specific data includes manually added custom data.

[1486] User authentication methods

[1487] When a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy. If authentication is successful, a session is started.

[1488] Learning program selection method

[1489] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user selects their desired program, and this information is sent from the terminal to the server.

[1490] Means of providing learning content

[1491] Based on the selected program, the server integrates generated data and specific data to construct learning content. For example, the operating procedures for a specific machine are provided from specific data, while general operating concepts are provided from generated data.

[1492] Progress tracking methods

[1493] During learning, the device records the user's learning progress using a progress tracking mechanism. Additionally, progress data is sent to the server at regular intervals or when the user finishes an operation, and the server stores it in a database.

[1494] Feedback processing means

[1495] After completing the learning program, users enter their evaluations and comments into a feedback form. This information is sent from the device to the server, which stores the feedback in a database and performs analysis. The results of this analysis will be used to improve future programs.

[1496] Specific example

[1497] For example, when a robot learns the operating procedures for a new polishing machine, the server loads generated data for the operating procedures and manual operation procedure data. The robot logs in and sends authentication information to the server. If authentication is successful, the robot is presented with a "polishing machine operation program" and selects it. The server provides learning content by integrating the general polishing process from the generated data and the specific machine operation procedures from the manual data. The robot records its progress as it performs the operation and sends it to the server. Once the operation is complete, it inputs feedback and sends it to the server. Based on this, the next operating procedures are improved.

[1498] Example of a prompt

[1499] "I would like to learn how to operate the new polishing machine. First, please teach me the basic safety procedures. Then, please explain the overall process."

[1500] As described above, the educational system of the present invention provides robots with effective and balanced education, and enables continuous improvement through tracking and feedback on learning progress.

[1501] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1502] Step 1:

[1503] The server loads generated data and specific data using initialization methods when the system starts up. The generated data includes machine learning models, and the specific data includes custom data added manually. This allows the server to prepare the dataset necessary for training. Specifically, the server loads the machine learning models from disk into memory, reads the custom data, and saves it to the database.

[1504] Input: Generated data on disk and specific data

[1505] Data processing / calculation: File reading, loading into memory, and performing necessary transformations.

[1506] Output: Prepared dataset

[1507] Step 2:

[1508] When a user accesses the system, the terminal displays a login screen. The user enters their ID and password, and the terminal sends this authentication information to the server. The server verifies the credentials using user authentication methods and authenticates their legitimacy. If authentication is successful, a session is started. Specifically, the server compares the user information stored in the database with the entered credentials, and if they match, a session is started.

[1509] Input: User ID, Password

[1510] Data processing / data calculations: Database queries, credential matching.

[1511] Output: Authentication result (success / failure)

[1512] Step 3:

[1513] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program, and this information is sent from the terminal to the server. The server receives the selected program information and begins preparing the corresponding learning content. Specifically, the server uses the selected program ID as a key to retrieve the corresponding learning content data from the database.

[1514] Input: User-selected program ID

[1515] Data processing / data calculation: Database queries, retrieval of learning content

[1516] Output: Training content dataset

[1517] Step 4:

[1518] Using a learning content delivery method, the server integrates generated data and specific data. The server provides operating procedures for specific machines from the specific data and general operating concepts from the generated data. This integrates content aligned with the learning program and transmits it to the terminal. Specifically, the server integrates generated data and specific data to create content that includes advanced operating procedures and general operating principles.

[1519] Input: Generated data, specific data

[1520] Data processing / data calculations: data integration, content generation

[1521] Output: Integrated learning content

[1522] Step 5:

[1523] During learning, the device records the user's learning progress using progress tracking mechanisms. It also sends progress data to the server at regular intervals or when the user finishes an operation, and the server stores this data in a database. Specifically, the device collects operation logs and progress information in real time and periodically transfers them to the server.

[1524] Input: User operation log, progress information

[1525] Data processing / data calculation: data recording, periodic transfer

[1526] Output: Progress data

[1527] Step 6:

[1528] After completing the learning program, users enter their evaluations and comments into a feedback form. This information is sent from the device to the server, which stores the feedback in a database and performs analysis. Specifically, the server analyzes the feedback data to extract statistical evaluations and areas for improvement.

[1529] Input: User ratings, comments

[1530] Data processing / data calculation: Storage and analysis of feedback data.

[1531] Output: analysis results, statistical information

[1532] Example of a prompt

[1533] "I would like to learn how to operate the new polishing machine. First, please teach me the basic safety procedures. Then, please explain the overall process."

[1534] The above describes the specific processing steps of the system. In each step, appropriate data processing and calculations are performed based on the input data to obtain output, thereby enabling the operation of the entire system.

[1535] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1536] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, and an emotion engine.

[1537] First, when the system starts up, the server loads generated data and manual mode data using initialization mechanisms. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[1538] Next, when a user accesses the system, the terminal displays a login screen, and the user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[1539] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[1540] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct the learning content. For example, specific knowledge such as that required for certification exams or new employee training is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[1541] Furthermore, during learning, the device uses an emotion engine to recognize emotions from the user's facial expressions and voice. The recognized emotion information is sent to the server in real time.

[1542] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it can suggest a break or reduce the intensity of the content. It can also provide additional explanations and support if the user is feeling anxious.

[1543] During learning, the device uses progress tracking to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[1544] After completing the learning program, users use a feedback processing system to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from the terminal to the server, which stores the received feedback in a database and performs analysis. This allows for the improvement of the educational program.

[1545] Specific example

[1546] For example, if a new employee chooses the "New Employee Training Program":

[1547] 1. The server loads generated data and manual mode data when the system starts up.

[1548] 2. The terminal displays a screen for the user to log in to the system. The user enters their ID and password to authenticate.

[1549] 3. The server starts a session if authentication is successful and displays a list of available learning programs to the user.

[1550] 4. The user selects the "New Employee Training Program".

[1551] 5. The terminal sends the selected program information to the server.

[1552] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[1553] 7. The device displays learning content to the user. For example, the section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[1554] 8. The device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[1555] 9. The server dynamically adjusts learning content based on sentiment data. If the user is tired, it will suggest a break or change the learning content to a simplified version.

[1556] 10. The device records the user's learning progress and sends it to the server.

[1557] 11. The server saves progress data to a database, allowing users to continue their learning progress the next time they log in.

[1558] 12. After completing the learning process, the user submits feedback and sends it to the server.

[1559] 13. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[1560] In this way, the educational system of the present invention provides users with effective and balanced education, and enables dynamic learning that responds to the user's emotions through an emotion engine. Furthermore, continuous improvement is possible through tracking of learning progress and feedback.

[1561] The following describes the processing flow.

[1562] Step 1:

[1563] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. Generated data includes machine learning models and existing general knowledge data, while manual mode data includes custom data and programs manually added by the instructor.

[1564] Step 2:

[1565] The terminal displays a login screen to the user. The user enters their ID and password and submits their credentials.

[1566] Step 3:

[1567] The server uses user authentication methods to verify the user's credentials and authenticate their legitimacy. If authentication is successful, it starts a session and sends an authentication token to the device.

[1568] Step 4:

[1569] After logging in, users can browse a list of learning programs and select one that matches their learning goals. This is made possible by the learning program selection method.

[1570] Step 5:

[1571] The terminal sends the program information selected by the user to the server.

[1572] Step 6:

[1573] Based on the received program ID, the server integrates the generated data and manual mode data to construct the learning content.

[1574] Step 7:

[1575] The server sends the configured learning content to the terminal. For example, in a new employee training program, company etiquette is provided from manual mode data, and business etiquette is provided from generated data.

[1576] Step 8:

[1577] The device displays the received learning content to the user. The user then proceeds with their learning while referring to it.

[1578] Step 9:

[1579] During training, the device uses an emotion engine to recognize emotions from the user's facial expressions and voice. The recognized emotion information is sent to the server in real time.

[1580] Step 10:

[1581] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it suggests a break or reduces the intensity of the content. It also provides additional explanations and support if the user is feeling anxious.

[1582] Step 11:

[1583] The device records the user's learning progress using a progress tracking mechanism. Progress data is sent to the server at regular intervals or when the user finishes their operation.

[1584] Step 12:

[1585] The server saves the received progress data to a database. This allows the user to continue their learning progress the next time they log in.

[1586] Step 13:

[1587] After completing the learning program, the user uses a feedback processing mechanism to enter their evaluation and comments on the learning content into a feedback form. The feedback content is then sent from the terminal to the server.

[1588] Step 14:

[1589] The server stores the received feedback in a database and performs analysis. This helps to improve the educational program.

[1590] Specific example

[1591] If a new employee chooses the "New Employee Training Program":

[1592] 1. The server loads generated data and manual mode data when the system starts up.

[1593] 2. The terminal displays a screen for the user to log in to the system, and the user enters their ID and password.

[1594] 3. If authentication is successful, the server starts a session and displays a list of learning programs.

[1595] 4. The user selects the "New Employee Training Program".

[1596] 5. The terminal sends the selected program information to the server.

[1597] 6. The server integrates the generated data and the manual mode data to create the learning content for the "New Employee Training Program".

[1598] 7. The server sends the configured learning content to the device.

[1599] 8. The device displays learning content to the user, and the user proceeds with the learning.

[1600] 9. During learning, the device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[1601] 10. The server dynamically adjusts learning content based on sentiment data. If the user is tired, it will suggest a break or simplify the learning content.

[1602] 11. The device records the user's learning progress and sends it to the server.

[1603] 12. The server saves the progress data to the database.

[1604] 13. After completing the learning process, the user enters feedback and sends it to the server.

[1605] 14. The server stores the feedback in a database and analyzes it to help improve the educational program.

[1606] In this way, the educational system of the present invention provides users with effective and balanced education, and enables dynamic learning that responds to the user's emotions through an emotion engine. Furthermore, continuous improvement is possible through tracking of learning progress and feedback.

[1607] (Example 2)

[1608] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1609] Traditional education systems often fail to consider the emotional state of users when providing learning content, leading to decreased learning efficiency and learner satisfaction. Furthermore, managing learning progress and processing feedback are frequently done manually, which is time-consuming and labor-intensive, making it difficult to provide efficient learning.

[1610] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, emotion recognition means, and means for dynamically adjusting learning content based on emotion information. This enables flexible provision of learning content according to the learner's emotional state, as well as efficient progress management and feedback processing.

[1611] An "initialization method" is a means of loading the data necessary for system startup and setting the system to its initial state.

[1612] "User authentication means" are methods used to verify the legitimacy of a user's access to a system.

[1613] A "learning program selection method" is a means that allows users to select their desired program from among the available learning programs.

[1614] "Means of providing learning content" refers to means of providing learning content suitable for the user based on the selected learning program.

[1615] A "progress tracking method" is a means of recording the user's learning progress and sending it to the server as needed.

[1616] A "feedback processing method" is a means of collecting user feedback, sending it to a server, and performing analysis on it.

[1617] An "emotion recognition method" is a means of analyzing a user's facial expressions and voice to recognize their emotional state in real time.

[1618] "Means for dynamically adjusting learning content based on emotional information" refers to means of changing the content and progression of learning content in a timely manner according to the user's emotional state.

[1619] The educational system of the present invention includes initialization means, user authentication means, learning program selection means, learning content provision means, progress tracking means, feedback processing means, emotion recognition means, and means for dynamically adjusting learning content based on emotion information.

[1620] First, upon system startup, the server loads generated data and manual mode data using initialization mechanisms. This generated data includes machine learning models and general knowledge data, while the manual mode data includes custom data and programs manually added by instructors. The server uses Apache server software and Python programs to load this data from disk into memory.

[1621] Next, the terminal displays a login screen when the user accesses the system. The user enters their ID and password to authenticate. Based on the user authentication method, the server uses the Django framework to verify the credentials and authenticate their legitimacy. If authentication is successful, a session is started and the user is redirected to the dashboard screen.

[1622] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program from the list. This information is sent to the server via a JavaScript event handler.

[1623] Based on the selected program, the server integrates generated data and manual mode data to create learning content. Specifically, it extracts specific data from an SQL database using a Python script and supplements the content as needed using a machine learning model. The server then sends the constructed learning content to the device via an API.

[1624] The device uses JavaScript to display learning content to the user in real time. Furthermore, it utilizes emotion recognition technology, analyzing the user's facial expressions and voice data in real time using OpenCV and Federated Learning techniques. This emotion data is sent to the server in JSON format.

[1625] The server analyzes the received emotional information and dynamically adjusts the content and progression of the learning materials according to the user's learning state. For example, if the server determines that the user is tired, it will suggest a break or simplify the content. This is done using conditional branching in a Python program.

[1626] Learning progress is temporarily saved to local storage by the device using JavaScript and sent to the server periodically or after the operation is completed. Upon receiving the progress data, the server saves it to a PostgreSQL database and makes it available the next time the user logs in.

[1627] After completing the learning program, users enter their evaluations and comments into a feedback form. This input data is sent from the terminal to the server, which stores the feedback in a database and performs statistical processing of the feedback using a Python script.

[1628] Specific example

[1629] For example, the procedure for a new employee who selects the "New Employee Training Program" is as follows:

[1630] 1. The server loads generated data and manual mode data when the system starts up.

[1631] 2. The terminal displays the user's login screen, and the user enters their ID and password to authenticate.

[1632] 3. If authentication is successful, the server will start a session and display a list of available learning programs to the user.

[1633] 4. The user selects the "New Employee Training Program".

[1634] 5. The terminal sends the selected program information to the server.

[1635] 6. The server integrates the generated data and manual mode data to create the learning content for the "New Employee Training Program" and sends it to the terminal.

[1636] 7. The device displays learning content to the user. The section on company etiquette is provided from manual mode data, while the section on general business etiquette is provided from generated data.

[1637] 8. The device analyzes the user's emotions in real time from their facial expressions and voice, and sends the information to the server.

[1638] 9. The server dynamically adjusts learning content based on emotional data. For example, if the user is tired, it suggests a break and simplifies the learning content.

[1639] 10. The device records the user's learning progress and sends it to the server.

[1640] 11. The server saves progress data to a database, allowing users to continue their learning progress the next time they log in.

[1641] 12. After completing the learning process, the user submits feedback and sends it to the server.

[1642] 13. The server stores the feedback in a database, analyzes it, and uses it to improve the educational program.

[1643] Examples of inputs to generative AI models

[1644] "Generate learning content suitable for a new employee training program."

[1645] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1646] Program processing steps

[1647] Step 1:

[1648] The server loads generated data and manual mode data using initialization methods during system startup. At this time, the server begins operating the Apache server software and uses a Python program to load generated data (machine learning models and general knowledge data) and manual mode data (custom data and programs) from disk into memory. The input is the command used during system startup, and the output is the data loaded into memory.

[1649] Step 2:

[1650] The terminal displays a login screen when a user accesses the system. Here, the user enters their ID and password. The entered authentication information is sent to the server using the HTTPS protocol. The input is the user ID and password, and the output is the transmission of the authentication information. Specifically, JavaScript on the terminal collects the authentication information and creates an HTTP request to send it to the server.

[1651] Step 3:

[1652] The server verifies the legitimacy of the credentials sent using user authentication methods. This is done using the Django framework and a database. Specifically, the server compares the received credentials with the database, and if authentication is successful, it starts a session and instructs the user to redirect to the dashboard screen. The input is the credentials, and the output is the authentication status and session information.

[1653] Step 4:

[1654] After successful login, the terminal displays a list of available learning programs to the user using a learning program selection mechanism. A JavaScript-based user interface is applied here. The input is a list of learning programs sent from the server, and the output is the display of the program list to the user.

[1655] Step 5:

[1656] The user selects their desired program from a displayed list of learning programs. The selection information is sent to the server via a JavaScript event handler. The input is the user's program selection, and the output is the transmission of the selection information to the server.

[1657] Step 6:

[1658] The server integrates generated data and manual mode data based on the selected program to generate learning content. It extracts specific data from an SQL database using a Python script and uses a machine learning model to supplement the content as needed. The input is the selected program information, and the output is the generated learning content. The generated content is sent to the terminal via an API.

[1659] Step 7:

[1660] The device uses JavaScript to display learning content to the user. At the same time, it utilizes emotion recognition to analyze the user's facial expressions and voice data in real time using OpenCV and Federated Learning technologies. The analyzed emotion data is sent to the server in JSON format. Inputs are the learning content from the server and the user's facial expressions and voice data, while outputs are the content displayed to the user and the transmitted emotion data.

[1661] Step 8:

[1662] The server analyzes emotional information and dynamically adjusts the learning content. Using a Python script, if the server determines that the user is tired, it suggests a break or changes the content to a simplified version. The input is the user's emotional information, and the output is the adjusted learning content.

[1663] Step 9:

[1664] The device records progress during learning and sends it to the server periodically or after the operation is completed. JavaScript is used to temporarily store progress data in local storage and to create an HTTP request when sending it. The input is the user's learning progress, and the output is the transmission of progress data to the server.

[1665] Step 10:

[1666] The server saves the received progress data to a database so that the game can resume from the same state the next time the user logs in. A Python script parses the progress data and stores it in the database. The input is the progress data, and the output is the progress information stored in the database.

[1667] Step 11:

[1668] After completing the learning program, users enter their evaluations and comments into a feedback form and send it from their device to the server. The input is feedback information, and the output is the transmission of feedback data to the server.

[1669] Step 12:

[1670] The server stores the received feedback in a database and performs statistical processing on the feedback using a Python script. The input is the feedback information, and the output is the feedback content stored in the database and the results of its analysis.

[1671] (Application Example 2)

[1672] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1673] In today's workplace, effective and real-time learning and training are essential. However, traditional education systems often lack the ability to dynamically adjust to learners' emotions and circumstances, leading to decreased learning effectiveness. Furthermore, in on-site training, such as in physical stores, learning must progress during actual work, and insufficient progress management and feedback functions in such situations pose a challenge.

[1674] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes an initialization means, a user authentication means, a learning program selection means, a learning content provision means, a progress tracking means, a feedback processing means, an emotion analysis means, and a learning content dynamic adjustment means. This enables effective and personalized learning by analyzing the learner's emotional information in real time and dynamically adjusting the learning content. Furthermore, continuous improvement of the training program is achieved through progress management and the collection and analysis of feedback.

[1675] "Initialization means" refers to the means of loading generated data and manual mode data when the system starts up.

[1676] "User authentication means" refers to the method used to verify the credentials and authenticate the legitimacy of a user when they access a system using their ID and password.

[1677] "Learning program selection means" refers to a means that allows the user to select their desired learning program, and is a means of displaying a list of available programs.

[1678] A "means for providing learning content" refers to a means of integrating generated data and manual mode data to construct learning content and provide it to the user.

[1679] A "progress tracking method" is a means of recording a user's learning progress and sending it to a server.

[1680] A "feedback processing method" is a means of collecting evaluations and comments from users after the learning process is complete, and sending and storing them on a server.

[1681] "Emotion analysis methods" are means of analyzing emotional information in real time from a user's facial expressions and voice.

[1682] A "dynamic learning content adjustment method" is a means of dynamically adjusting learning content based on information obtained from emotion analysis methods.

[1683] The present invention is a system that effectively supports the training of new staff in physical stores. This system includes an initialization means, a user authentication means, a learning program selection means, a learning content provision means, a progress tracking means, a feedback processing means, an emotion analysis means, and a learning content dynamic adjustment means.

[1684] First, the server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This generated data includes machine learning models and existing general knowledge data. On the other hand, the manual mode data includes custom data and programs that are manually added by the instructor.

[1685] Next, when a user accesses the system, a device such as smart glasses or a smartphone displays a login screen. The user enters their ID and password to authenticate. The server verifies the credentials and authenticates their legitimacy using the user authentication method. If authentication is successful, a session is started, and the user can begin using the system.

[1686] After the user logs in, a list of available learning programs is displayed using the learning program selection tool. The user then selects their desired program. The selected program information is sent from the terminal to the server.

[1687] Based on the selected program, the server uses a learning content delivery mechanism to integrate generated data and manual mode data to construct learning content. For example, specific knowledge such as product display methods, customer service etiquette, and store cleaning procedures is provided from manual mode data, while general business etiquette is provided from generated data. This results in well-balanced learning content.

[1688] Furthermore, during learning, devices such as smart glasses and smartphones use emotion analysis tools to recognize emotions from the user's facial expressions and voice. The recognized emotion information is transmitted to the server in real time. For example, EmotionEngine analyzes the user's facial expressions and determines emotions such as fatigue or stress.

[1689] The server dynamically adjusts learning content based on the emotional information it receives. For example, if it detects that the user is tired, it can suggest a break or reduce the intensity of the content. It can also provide additional explanations and support if the user is feeling anxious.

[1690] During learning, devices such as smart glasses and smartphones use progress tracking mechanisms to record the user's learning progress. Additionally, progress data is sent to the server at regular intervals or when the user finishes their activity. The server stores the received progress data in a database, allowing the user to resume their learning the next time they log in.

[1691] After completing the learning program, users use a feedback processing system to enter their evaluations and comments on the learning content into a feedback form. The feedback is sent from devices such as smart glasses or smartphones to a server, which stores the received feedback in a database and analyzes it. This allows for the improvement of the educational program.

[1692] As a concrete example, consider a scenario where a new staff member starts a "new employee training program" using smart glasses. First, the server loads the necessary data using an initialization mechanism, and the staff member logs into the system through a user authentication mechanism. Next, the staff member selects the "new employee training program" using a learning program selection mechanism, and the specific training content is displayed on the smart glasses by a learning content provision mechanism. During the training, the staff member's emotions are analyzed by an emotion analysis mechanism, and the content is dynamically adjusted as needed. Learning progress is recorded by a progress tracking mechanism, and finally, training evaluations are collected by a feedback processing mechanism. This enables effective and personalized training.

[1693] Examples of prompts to input into a generative AI model:

[1694] Please generate the learning content for the new employee training program. Include the following items: product display methods, customer service etiquette, and store cleaning procedures. Dynamic adjustments should also be made based on the learners' emotions.

[1695] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1696] Step 1:

[1697] The server uses an initialization mechanism to load generated data and manual mode data when the system starts up. This prepares the server with the basic data necessary to run the learning program. The input is the generated data and manual mode data pre-stored in the system, and the output is the initialized data profile. Specifically, the server reads the necessary files from the database and loads them into memory.

[1698] Step 2:

[1699] The terminal provides the user with a login screen, and the user authenticates by entering their ID and password. Using the user authentication method, the server verifies the credentials and authenticates their legitimacy. The input is the ID and password entered by the user, and the output is the authentication success or failure status. Specifically, the server compares the entered information with the database and starts a session if they match.

[1700] Step 3:

[1701] After the user logs in, a list of available learning programs is displayed on the terminal using the learning program selection tool. The user selects their desired program, and the selected program information is sent from the terminal to the server. The input is the learning program selected by the user, and the output is specific information about the selected program. Specifically, the server receives the user's selection and sends an appropriate response back to the terminal.

[1702] Step 4:

[1703] The server uses a learning content delivery mechanism to integrate generated data and manual mode data based on the selected program, thereby generating learning content. This makes it possible to provide users with well-balanced learning content. The input is the selected program information, generated data, and manual mode data, and the output is the generated learning content. Specifically, the server analyzes the data, extracts relevant information, and integrates it.

[1704] Step 5:

[1705] During learning, devices such as smart glasses use emotion analysis techniques to recognize emotions from the user's facial expressions and voice, and transmit this information to a server in real time. The input is the user's facial expressions and voice data, and the output is the emotion analysis result. Specifically, data is collected using the device's camera and microphone, and EmotionEngine analyzes the data.

[1706] Step 6:

[1707] The server dynamically adjusts the learning content based on the emotional information it receives. For example, if it detects that the user is tired, it may suggest a break or change to lighter content. The input is the result of the emotional analysis, and the output is the adjusted learning content. Specifically, the server changes the current content and provides feedback according to the user's state.

[1708] Step 7:

[1709] During learning, devices such as smart glasses use progress tracking mechanisms to record the user's learning progress and send this data to the server at regular intervals or when the user finishes their activity. The input is the user's learning activity data, and the output is the progress record. Specifically, the device monitors the completion status of learning steps in real time and reports the progress to the server.

[1710] Step 8:

[1711] After completing the learning program, the user uses a feedback processing mechanism to enter evaluations and comments on the learning content into a feedback form and sends them from the terminal to the server. The input is the user's feedback comments, and the output is the collected feedback data. Specifically, the terminal receives input from the user and forwards it to the server.

[1712] Step 9:

[1713] The server stores the received feedback in a database and performs analysis. This helps improve the educational program. The input is user feedback data, and the output is the analysis results and improvement items. Specifically, the server analyzes the collected feedback and generates new program improvement proposals.

[1714] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1715] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1716] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1717] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1718] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is ...

Claims

[Claim 1] An initialization means for loading generated data and manual mode data, A user authentication method that authenticates user credentials and initiates a session, A learning program selection means for selecting a desired program from a list of learning programs that include both general data and manual mode data, A learning content provision means that provides appropriate learning content to the user based on the selected learning program, A progress tracking means that records the user's learning progress and sends it to the server, A feedback processing means that stores user feedback in a database and performs analysis, A system that includes this.

Images