System

Abstract

A system is provided.SOLUTION: A means for providing a user interface for a user to perform account registration and input personal information and a purpose of learning, a means for managing a course and learning progress selected by the user via a dedicated terminal, a means for recording and analyzing the learning progress and result of the user in real time, a means for providing feedback to the user based on the analysis result, a means for posting a question by the user via a mobile application and analyzing the question content to provide an answer, and an event management means for the user to perform participation registration in an offline event.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The current education system faces the problem of a lack of learning skills and educational opportunities in STEAM education, particularly for elementary, junior high, and high school students. Another issue is that educators themselves lack the skill sets for new educational technologies and methodologies, making it difficult to provide efficient instruction. Effective learning requires interactive and personalized feedback, which the current education system does not adequately provide. It is also important to introduce systems that allow elementary, junior high, and high school students to learn in a fun way, while also providing practical learning opportunities through communication with other students and experts. Additionally, a system is needed to monitor learning progress in real time and provide appropriate feedback. [Means for solving the problem]

[0005] The present invention solves these problems by providing a system including: a means for providing a user interface for users to register an account and input personal information and learning objectives; a means for managing the user's selected courses and learning progress via a dedicated terminal; a means for recording and analyzing the user's learning progress and results in real time; a means for providing feedback to the user based on the analysis results; a means for users to post questions via a mobile app, analyze the questions, and provide answers; and an event management means for users to register for offline events. By including a means for customizing the user's learning progress using generative artificial intelligence and generating automatic feedback, and a means for optimizing the learning environment by providing educational institutions with high-speed internet and the latest learning equipment, the system achieves even more effective educational support.

[0006] "User" refers to students and educators who use this system to study.

[0007] A "dedicated terminal" is a hardware device specially designed for the educational support services provided by this system.

[0008] "User interface" refers to the screens and input devices through which a user interacts with a system and inputs or displays information.

[0009] "Study Progress" refers to the progress and achievements of a User in a course of study selected by the User.

[0010] "Real-time" refers to immediate response to user operations and data input, and data processing and feedback.

[0011] "Analysis" refers to the process of evaluating recorded data and progress and extracting meaningful information.

[0012] "Feedback" refers to information provided to a user regarding their learning activities for evaluation and improvement.

[0013] A "mobile app" is application software that runs on a mobile device and allows a user to access and interact with a system.

[0014] A "question" refers to an inquiry that a user poses to the system regarding a question or uncertainty that arises during the course of study.

[0015] "Escalation" refers to the process by which a basic inquiry is handed over to a specialist or higher-level support person.

[0016] The "event management means" is a function that allows users to register for and manage information about offline learning events and workshops.

[0017] "Generative AI" refers to technology that automatically generates optimal learning progress management and feedback based on a user's learning data.

[0018] "High-speed Internet" refers to an Internet connection environment in which data can be sent and received very quickly.

[0019] "Learning equipment" is a general term for hardware and software necessary for education.

[0020] The term "system" refers to an environment that is configured by combining the above elements and provides comprehensive support for users' learning. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives, a function for managing the user's selected courses and learning progress via a dedicated terminal, a function for recording and analyzing the user's learning progress and results in real time, a function for providing feedback based on the analysis results, a function for posting questions via the mobile app, analyzing them, and providing answers, and an event management function for participating in offline events. However, this does not involve writing program code.

[0043] The overall system flow is as follows:

[0044] User Registration

[0045] First, the user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal then sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[0046] Course Selection and Learning Management

[0047] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[0048] Learning progression and feedback

[0049] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[0050] Communication and Support

[0051] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[0052] Participating in offline events

[0053] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[0054] Specific examples

[0055] For example, if a junior high school student user selects the "Science Experiment" course and proceeds through the program, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. The user can continue learning by using this as a reference when reviewing the course or moving on to the next module.

[0056] This system allows users to learn STEAM education interactively and effectively, and provides customized feedback based on their progress and level of understanding.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] The user accesses the dedicated terminal, opens the account creation screen, enters required information such as name, school name, grade, and email address, and presses the "Register" button.

[0060] Step 2:

[0061] The terminal sends the input information to the server.

[0062] Step 3:

[0063] The server stores the received user information in a database and generates authentication information such as a user ID and password.

[0064] Step 4:

[0065] The server sends the generated authentication information to the terminal.

[0066] Step 5:

[0067] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[0068] Step 6:

[0069] To log in to the online platform, the user enters their authentication information and presses the login button.

[0070] Step 7:

[0071] The device sends the login information to the server.

[0072] Step 8:

[0073] The server checks the authentication information, and if authentication is successful, it sends the dashboard screen data (learning progress, recommended courses, etc.) to the device.

[0074] Step 9:

[0075] The terminal displays the dashboard screen, and the user opens the "Course Selection" menu.

[0076] Step 10:

[0077] The terminal sends a request for a course list to the server.

[0078] Step 11:

[0079] The server sends a list of available STEAM courses to the device.

[0080] Step 12:

[0081] The user selects the desired course from the course list.

[0082] Step 13:

[0083] The terminal transmits the selected course information to the server.

[0084] Step 14:

[0085] The server generates a learning plan based on the selected course and transmits it to the terminal.

[0086] Step 15:

[0087] The device displays the learning plan, and the user clicks the "Start" button to display the first learning module.

[0088] Step 16:

[0089] Users watch learning content (e.g., video tutorials) and complete experiments and quizzes.

[0090] Step 17:

[0091] The device records the user's learning data (experiment results, quiz answers, etc.) in real time and sends it to the server.

[0092] Step 18:

[0093] The server analyzes the data it receives and evaluates the user's learning progress and level of understanding.

[0094] Step 19:

[0095] The server generates feedback based on the evaluation results and sends it to the terminal.

[0096] Step 20:

[0097] The device displays feedback to the user and prompts them to relearn or proceed to the next module as needed.

[0098] Step 21:

[0099] A user uses the mobile app to post a question via the LINE chat function.

[0100] Step 22:

[0101] The terminal sends the user's question to the server.

[0102] Step 23:

[0103] The server analyzes the question, generates an appropriate answer, and sends it to the terminal.

[0104] Step 24:

[0105] The terminal displays the answer received from the server to the user.

[0106] Step 25:

[0107] If the user is not satisfied with the answer, they can post the question again and the server will escalate the question to an expert or employee.

[0108] Step 26:

[0109] The server periodically generates and sends offline event notifications to the terminal.

[0110] Step 27:

[0111] Users receive event notifications and view detailed information on mobile apps and online platforms.

[0112] Step 28:

[0113] A user registers to participate in an offline event, enters information in the application form, and presses the "Register" button.

[0114] Step 29:

[0115] The terminal transmits the participation registration information to the server.

[0116] Step 30:

[0117] The server stores the registration information in a database and generates a confirmation message.

[0118] Step 31:

[0119] The server sends a participation confirmation message to the terminal and notifies the user.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] Current STEAM education support systems struggle to effectively manage users' learning progress and comprehension, and provide individually customized feedback. They also lack systems that allow users to ask questions in real time and receive immediate answers. Furthermore, they lack a means to seamlessly link online and offline learning environments. This makes it difficult for users to progress efficiently.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes means for recording and analyzing a user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, and means for the user to post questions via a mobile app, analyze the questions, and provide answers. This makes it possible to instantly check a user's learning progress and suggest areas for improvement, thereby providing individually customized learning support. Furthermore, the real-time exchange of questions and answers via the mobile app allows users to quickly resolve their doubts and enable efficient learning. Furthermore, seamless integration of online and offline events can increase users' motivation to learn and encourage continuous learning.

[0125] "User" refers to a person who uses the system to carry out learning activities.

[0126] "Dedicated Terminal" refers to a specific electronic device used to access the system.

[0127] "Mobile app" refers to application software that runs on smartphones and tablets.

[0128] "Server" refers to a central computer system that stores, processes, and serves data.

[0129] "User interface" refers to the screen and operating means that allow a user to directly interact with a system.

[0130] "Learning curriculum" refers to various learning contents and courses selected by the user.

[0131] "Study progress" refers to the progress of a user in a learning curriculum selected by the user.

[0132] "Learning Content" refers to information and materials such as videos, texts, and quizzes provided to support learning.

[0133] "Feedback" refers to information that provides an evaluation of a user's learning activities and suggestions for improvement.

[0134] "Analysis" refers to the process of processing collected data to derive useful information or results.

[0135] "Non-online events" refer to learning or education-related events that take place in a physical location.

[0136] A "generative AI model" refers to an artificial intelligence algorithm that generates data-based feedback to track a user's learning progress and provide adaptive feedback.

[0137] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system provides a user interface for users to register an account and enter personal information and learning objectives. Specific hardware includes dedicated terminals, mobile terminals, and servers. Software includes database management software (e.g., MySQL or PostgreSQL), real-time data analysis software (e.g., Apache Kafka), front-end frameworks (e.g., React or Angular), and mobile app development frameworks (e.g., Flutter or React Native).

[0138] System configuration

[0139] 1. User Registration and Authentication

[0140] A user accesses a dedicated terminal and creates an account by entering profile information. The terminal sends the user's input information to the server, which stores the received information in a database. The server generates authentication information and sends it to the terminal. The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[0141] 2. Course Selection and Learning Management

[0142] The user logs in to a dedicated terminal using authentication information, and the server verifies the authentication information. If authentication is successful, the server sends the user's dedicated dashboard data to the terminal. The user selects the desired learning curriculum from the dashboard, and the terminal sends the selection information to the server. The server generates an optimal learning plan based on the user's selection and sends it to the terminal. The terminal displays the generated learning plan, and the user begins learning.

[0143] 3. Learning and Feedback

[0144] Users progress through learning content (e.g., video tutorials, textbooks, quizzes) provided on dedicated devices. The devices record the user's learning progress and quiz answers in real time and send them to the server. The server analyzes the received data and evaluates the user's level of understanding and progress. Feedback generated based on the evaluation is sent to the device, which then displays it to the user.

[0145] 4. Real-time Q&A

[0146] Users use a mobile app to post questions. The questions are sent from the device to a server, which analyzes them. An appropriate answer is generated and sent to the device. The user can review the answer and ask the question again if necessary. Complex questions are answered by experts or employees.

[0147] 5. Participating in offline events

[0148] The server periodically generates notifications for non-online events and sends them to the device. The user receives the notifications and views the details. When registering for an event, the user enters the required information and the device sends the registration information to the server. The server stores the information in a database and sends a participation confirmation message to the device.

[0149] Specific examples

[0150] For example, if a junior high school student user selects the "Science Experiment" course and uses the system, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. The user can continue learning by using this as a reference when reviewing the course or moving on to the next module.

[0151] Prompt Sentence Examples

[0152] "Please tell me the specific steps to provide lesson plans and video tutorials for a science lab course for middle school students to learn basic scientific principles."

[0153] The present invention allows users to learn STEAM education interactively and effectively, and receive customized feedback based on their progress and level of understanding, maximizing learning efficiency.

[0154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0155] Step 1: User registers for an account

[0156] The user accesses the dedicated terminal and opens the account creation screen.

[0157] Input: The user enters required information such as name, school name, grade, and email address.

[0158] The terminal sends the input information to the server.

[0159] Output: User information is saved in the database on the server side. Authentication information (user ID, password) is generated and sent to the terminal.

[0160] Specific operation: The terminal sends the input information as an HTTP POST request, and the server executes an SQL INSERT statement against the database.

[0161] Step 2: User enters credentials and logs in

[0162] The user enters authentication information (user ID, password) on the dedicated terminal and presses the login button.

[0163] Input: The user enters the user ID and password.

[0164] The device sends the login information to the server.

[0165] Output: The server verifies the authentication information and, if successful, sends the dashboard data to the device.

[0166] Specific operation: The terminal sends an HTTP POST request, and the server searches for user information in the database and compares the password.

[0167] Step 3: User makes course selection

[0168] After logging in, the user selects the "Course Selection" menu from the dashboard and chooses the desired learning curriculum.

[0169] Input: The user selects the desired learning curriculum.

[0170] The terminal transmits the course selection information to the server.

[0171] Output: The server generates the optimal learning plan for the user and sends it to the device.

[0172] How it works: The device sends an HTTP POST request to the server containing the selected course information, and the server uses an algorithm to generate a learning plan.

[0173] Step 4: Users progress through the learning content

[0174] The user starts the provided learning content on a dedicated device.

[0175] Input: User watches video tutorials, performs experiments and quizzes.

[0176] The device records the user's progress and quiz answers and sends them to the server.

[0177] Output: The server receives the data in real time and generates feedback based on the analysis results.

[0178] Specific operation: The device records user operation data in real time and sends it to the server via HTTP POST request. The server then runs an analysis algorithm to evaluate the data.

[0179] Step 5: Provide feedback based on the analysis results

[0180] The server analyzes the user's learning data and evaluates their level of understanding and progress.

[0181] Input: User learning progress data and quiz answer data.

[0182] The server generates the analysis results and sends feedback to the device.

[0183] Output: The device displays feedback to the user.

[0184] Specific operation: The server analyzes the data in the database, generates a feedback message based on the evaluation criteria, and sends the generated feedback to the terminal as an HTTP response.

[0185] Step 6: Users post questions and get answers

[0186] A user posts a question using a mobile app.

[0187] Input: The user enters the question and clicks the post button.

[0188] The device sends the question to the server.

[0189] Output: The server analyzes the question, generates an appropriate answer, and sends it to the device.

[0190] Specific operation: The device sends an HTTP POST request containing a question to the server, and the server analyzes the question using a natural language processing algorithm, generates an answer, and sends the answer to the device as an HTTP response.

[0191] Step 7: User registers for a non-online event

[0192] The server periodically generates and sends notifications of non-online events to the terminal.

[0193] Input: Event details and registration information.

[0194] Users receive notifications, view details, and register to attend.

[0195] Output: The terminal sends the participation registration information to the server, which stores the information in a database. The server generates a participation confirmation message and sends it to the terminal.

[0196] Specific operation: The server periodically generates an event notification and sends it to the terminal as an HTTP response. The user registers for participation, and the terminal sends it to the server as an HTTP POST request. The server executes an SQL INSERT statement in the database, generates a participation confirmation message, and sends it to the terminal.

[0197] (Application example 1)

[0198] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0199] Conventional STEAM education systems have limited functionality for tracking learners' progress and providing feedback, making real-time analysis and customized instruction difficult. They also lack the ability to adequately manage interactive virtual learning experiences and offline events. Furthermore, the adoption of new learning methods utilizing wearable devices has been slow, creating a need for ways to maximize the effectiveness of education.

[0200] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0201] In this invention, the server includes: means for providing a user interface for users to register an account and input personal information and learning objectives; means for managing the user's selected course and learning progress via a dedicated terminal; means for recording and analyzing the user's learning progress and results in real time; means for providing feedback to the user based on the analysis results; means for users to post questions via a mobile app or a wearable device (smart glasses or a head-mounted display), analyze the questions, and provide answers; means for conducting experiments in a virtual space, simulating the experiment results, and providing feedback in real time; and event management means for displaying regular offline event notifications and detailed information and registering participants. This enables an interactive learning experience using a variety of devices, realizes real-time analysis of learning progress, and provides customized feedback, thereby improving the quality of education and maximizing learning effectiveness.

[0202] "A user interface for users to register an account and enter personal information and learning objectives" is a means of providing an operation screen for users to create their own account in an educational system and enter personal information and learning objectives.

[0203] "Means for managing a user's selected course and learning progress via a dedicated terminal" refers to a means in an educational system that provides a user with the ability to track and manage their selected learning course and learning progress using a dedicated device.

[0204] "Means for recording and analyzing a user's learning progress and results in real time" refers to a means for providing a function in an educational system to collect a user's learning progress and results as data in real time and analyze them using statistical or machine learning methods.

[0205] "Means for providing feedback to the user based on the analysis results" refers to means for providing the user with a function for suggesting areas for improvement and next learning steps based on the learning data analyzed in the educational system.

[0206] "A means for users to post questions via a mobile app or wearable device, and for the content of the question to be analyzed and an answer to be provided" refers to a means in an educational system that allows users to send questions using a smartphone app or a wearable device such as smart glasses, and provides a function that analyzes the content of the question using a natural language processing model or the like and provides an appropriate answer.

[0207] "Means for conducting experiments in a virtual space, simulating the experimental results, and providing feedback in real time" refers to a means for realizing the function in an educational system in which a user conducts an experiment in a virtual reality or mixed reality space, analyzes the results using computer simulation, and provides feedback in real time.

[0208] "Event management means for displaying regular offline event notifications and detailed information, and registering for participation" refers to a means for providing users in an educational system with regular notifications of offline event information, allowing them to view detailed information, and providing the functionality for registering for participation.

[0209] "Means for analyzing data in real time using cloud computing and providing optimal feedback using generative artificial intelligence" refers to a means in an educational system that uses cloud services via the internet to process and analyze collected data in real time and generate and provide optimal feedback to users using machine learning models.

[0210] "Means for providing an interactive learning experience using mixed reality technology to create a virtual space" refers to a means for using mixed reality (MR) technology in an educational system to create a learning environment that combines reality and virtual space, and to provide users with an interactive and immersive learning experience.

[0211] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students.

[0212] Overall structure

[0213] 1. User Registration:

[0214] The server provides a user interface for users to enter personal information such as name, school name, grade, and email address, as well as their learning goals. When a user registers an account from a dedicated terminal, the terminal sends the information to the server, which stores it in a database. After registration is complete, the server generates a user ID and password and sends them to the terminal.

[0215] 2. Course Selection and Learning Management:

[0216] The server manages the courses selected by the user via a dedicated terminal and their learning progress. When the user logs in to the online platform and selects a course, the server generates an optimal learning plan and sends the data to the terminal. The terminal displays the learning plan, and the user starts the learning module.

[0217] 3. Learning progression and feedback:

[0218] Users progress through learning content on their devices. The devices record the user's learning progress and results (e.g., scientific experiment results and quiz answers) in real time and send them to a cloud server. The server analyzes the data, generates feedback based on the analysis results, and sends it to the device. Users can also conduct experiments in virtual space, simulate the results, and receive feedback in real time.

[0219] 4. Real-time communication and support:

[0220] When a user posts a question using a mobile app or a wearable device (smart glasses or a head-mounted display), the device sends the question to a server, which then analyzes the question using a generative AI model (e.g., an NLP model), generates an appropriate answer, and sends it to the device.

[0221] 5. Offline event notification and participation management:

[0222] The server periodically generates notifications for offline events and sends them to the terminal. When a user receives the notification, views detailed information, and registers for participation, the terminal sends the participation registration information to the server, which stores the information in a database. A participation confirmation message is sent to the terminal to notify the user.

[0223] Hardware and Software Use

[0224] Hardware:

[0225] Dedicated devices (PC, smartphone, smart glasses, head-mounted display)

[0226] Server (cloud-based, e.g. AWS, Google Cloud)

[0227] software:

[0228] Frontend: React Native, Flutter, Unity engine

[0229] Backend: Express.js, Firebase Functions

[0230] Database: MongoDB, Firestore database

[0231] Analysis: TensorFlow, PyTorch, AWS Lambda functions

[0232] Communication: LINE API, Twilio Chat API

[0233] Specific examples

[0234] Examples of how the virtual laboratory can be used:

[0235] 1. Scenario: A middle school student user selects the "Virtual Lab" course.

[0236] 2. Procedure: Users simulate scientific experiments in a virtual space built with the Unity engine. The results are analyzed in real time in the Azure cloud, and feedback is generated using Azure ML.

[0237] 3. Questions: When a user asks a question about something they don't understand, the question is sent using the LINE chat function, and an answer analyzed by the BERT model is provided in real time.

[0238] 4. Offline Events: Users receive offline event notifications, view details, and register to attend.

[0239] Example prompts to input to a generative AI model:

[0240] Prompt: We are developing a STEAM education application for elementary, middle, and high school students. The app allows users to conduct experiments in a virtual space, analyzes the results in real time, and provides feedback. Additionally, the app uses an NLP model to generate answers when users post questions. It also periodically notifies users of offline events. Please explain in detail the use case for this app and the technologies required.

[0241] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0242] Step 1:

[0243] User Registration:

[0244] The user accesses the account registration screen from a dedicated terminal and enters personal information such as name, school name, grade, and email address. The terminal sends this input information to the server. The server stores the received information in a database, generates a user ID and password, and sends them to the terminal. The terminal then displays a notification to the user that registration is complete. The input is the user's personal information, and the output is the user information stored in the database and the generated user ID and password.

[0245] Step 2:

[0246] Course Selection and Learning Management:

[0247] The user enters authentication information to log in to the online platform from a dedicated terminal and presses the login button. The terminal sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. The user opens the "Course Selection" menu from the dashboard and selects the desired course. The terminal sends the course selection information to the server, and the server generates an optimal learning plan for the user and sends it to the terminal. The input is the user's authentication information and course selection information, and the output is the dashboard screen data and learning plan.

[0248] Step 3:

[0249] Learning progression and feedback:

[0250] The user progresses through the learning content on the device. The device records the user's learning progress and results (e.g., scientific experiment results or quiz answers) in real time and sends them to a cloud server. The server analyzes the data in real time (e.g., using a machine learning model), generates feedback based on the analysis results, and sends it to the device. The device displays the feedback to the user. The input is the user's learning progress and results, and the output is feedback based on the analysis results.

[0251] Step 4:

[0252] Real-time communication and support:

[0253] A user posts a question using a mobile app or a wearable device (smart glasses or a head-mounted display). The device sends the question to the server, which analyzes the question using a generative AI model (e.g., an NLP model). An appropriate answer is generated and sent to the device. The device displays the answer to the user. The input is the user's question, and the output is the analyzed answer.

[0254] Step 5:

[0255] Virtual experiments and simulations:

[0256] Users conduct scientific experiments in a virtual space using dedicated terminals or wearable devices. The experimental results are recorded in real time on the terminal and sent to a cloud server. The server simulates the data and generates analytical results. Feedback is generated in real time and sent to the terminal. The terminal displays the feedback to the user. The input is the results of the virtual experiment, and the output is feedback based on the simulation and analytical results.

[0257] Step 6:

[0258] Offline event notification and participation management:

[0259] The server periodically generates offline event notifications and sends them to the terminal. When a user receives the notification, views detailed information, and registers to participate, the terminal sends the registration information to the server, which stores the information in a database. A participation confirmation message is sent to the terminal, notifying the user. The input is the offline event notification and the user's registration information, and the output is the registration information stored in the database and the participation confirmation message.

[0260] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0261] This invention provides a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives; a function for managing the user's selected courses and learning progress via a dedicated terminal; a function for recording and analyzing the user's learning progress and results in real time; a function for providing feedback based on the analysis results; a function for posting questions via the mobile app, analyzing them, and providing answers; and an event management function for participating in offline events. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system provides a more personalized learning experience and more effective educational support.

[0262] The overall system flow is as follows:

[0263] User Registration

[0264] First, the user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal then sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[0265] Course Selection and Learning Management

[0266] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[0267] Learning progression and feedback

[0268] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[0269] Communication and Support

[0270] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[0271] Participating in offline events

[0272] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[0273] Customizing the learning experience with an emotional engine

[0274] While the user is studying, the emotion engine collects emotional data in real time. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their studies, it presents more difficult tasks. This ensures that the user's learning experience is always optimal.

[0275] Specific examples

[0276] For example, if a junior high school student user selects a "science experiment" course and proceeds with the program, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then actually performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. Furthermore, the emotion engine detects stress from the user's facial expressions and sends instructions to the server to ease some of the learning as necessary. As a result, the user can proceed with their learning while reducing stress.

[0277] This system allows users to learn STEAM education interactively and effectively, providing an optimal learning environment based on their emotions and customized feedback according to their progress and level of understanding.

[0278] The processing flow will be explained below.

[0279] Step 1:

[0280] The user accesses the dedicated terminal, opens the account creation screen, enters required information such as name, school name, grade, and email address, and presses the "Register" button.

[0281] Step 2:

[0282] The terminal sends the input information to the server.

[0283] Step 3:

[0284] The server stores the received user information in a database and generates authentication information such as a user ID and password.

[0285] Step 4:

[0286] The server sends the generated authentication information to the terminal.

[0287] Step 5:

[0288] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[0289] Step 6:

[0290] To log in to the online platform, the user enters their authentication information and presses the login button.

[0291] Step 7:

[0292] The device sends the login information to the server.

[0293] Step 8:

[0294] The server checks the authentication information, and if authentication is successful, it sends the dashboard screen data (learning progress, recommended courses, etc.) to the device.

[0295] Step 9:

[0296] The terminal displays the dashboard screen, and the user opens the "Course Selection" menu.

[0297] Step 10:

[0298] The terminal sends a request for a course list to the server.

[0299] Step 11:

[0300] The server sends a list of available STEAM courses to the device.

[0301] Step 12:

[0302] The user selects the desired course from the course list.

[0303] Step 13:

[0304] The terminal transmits the selected course information to the server.

[0305] Step 14:

[0306] The server generates a learning plan based on the selected course and transmits it to the terminal.

[0307] Step 15:

[0308] The device displays the learning plan, and the user clicks the "Start" button to display the first learning module.

[0309] Step 16:

[0310] Users watch learning content (e.g., video tutorials) and complete experiments and quizzes.

[0311] Step 17:

[0312] The device records the user's learning data (experiment results, quiz answers, etc.) in real time and sends it to the server.

[0313] Step 18:

[0314] The server analyzes the data it receives and evaluates the user's learning progress and level of understanding.

[0315] Step 19:

[0316] The server generates feedback based on the evaluation results and sends it to the terminal.

[0317] Step 20:

[0318] The device displays feedback to the user and prompts them to relearn or proceed to the next module as needed.

[0319] Step 21:

[0320] A user uses the mobile app to post a question via the LINE chat function.

[0321] Step 22:

[0322] The terminal sends the user's question to the server.

[0323] Step 23:

[0324] The server analyzes the question, generates an appropriate answer, and sends it to the terminal.

[0325] Step 24:

[0326] The terminal displays the answer received from the server to the user.

[0327] Step 25:

[0328] If the user is not satisfied with the answer, they can post the question again and the server will escalate the question to an expert or employee.

[0329] Step 26:

[0330] The server periodically generates and sends offline event notifications to the terminal.

[0331] Step 27:

[0332] Users receive event notifications and view detailed information on mobile apps and online platforms.

[0333] Step 28:

[0334] A user registers to participate in an offline event, enters information in the application form, and presses the "Register" button.

[0335] Step 29:

[0336] The terminal transmits the participation registration information to the server.

[0337] Step 30:

[0338] The server stores the registration information in a database and generates a confirmation message.

[0339] Step 31:

[0340] The server sends a participation confirmation message to the terminal and notifies the user.

[0341] Form using emotion engine

[0342] Step 32:

[0343] During learning, the device collects facial and voice data in real time through the device's camera and microphone.

[0344] Step 33:

[0345] The device transmits the collected emotion data to a server.

[0346] Step 34:

[0347] The server uses an emotion engine to analyze the user's emotion data and evaluate the user's stress level and concentration.

[0348] Step 35:

[0349] The server adjusts the difficulty and progress speed of the learning content based on the results of emotion analysis.

[0350] Step 36:

[0351] The server transmits the adjusted learning content to the terminal.

[0352] Step 37:

[0353] The terminal displays the adjusted learning content to the user, and the user continues learning.

[0354] As a concrete example, when a junior high school student user is working through a "science experiment" course, if the user is concentrating during the experiment, the server will present more difficult experimental tasks, and conversely, if the user is feeling stressed, the server will present easier auxiliary tasks and provide relaxation techniques, allowing the user to always enjoy an optimal learning experience.

[0355] Example 2

[0356] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] Conventional educational systems have difficulty providing appropriate feedback in real time based on each user's learning progress and level of understanding, resulting in a decline in user learning efficiency.In addition, they are unable to grasp the stress and concentration levels felt by users in real time and adjust the learning environment accordingly, making it impossible to optimize each individual learning experience.

[0358] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0359] In this invention, the server includes means for providing a user interface for users to register an account and input personal information and learning objectives, means for managing the user's selected course and learning progress via a dedicated terminal, means for recording and analyzing the user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, means for users to post questions via a mobile app and provide answers by analyzing the questions, event management means for users to register for offline events, and means for collecting emotional data while the user is learning and adjusting the learning experience. This enables users to receive appropriate feedback in real time according to their individual progress and level of understanding, and to have their learning environment adjusted based on the emotional data.

[0360] "User" refers to a general user who uses the system to access learning content and manage their progress.

[0361] "Account Registration" refers to the process by which a User registers with the System by providing personal information and authentication information required to access the System.

[0362] "Personal information" refers to information that can be used to identify an individual, such as the user's name, school name, grade, and email address.

[0363] "Learning objectives" refer to the educational goals or needs that users want to achieve by using the system.

[0364] "User interface" refers to the screens and means by which a user logs into a system or enters information.

[0365] "Dedicated Terminal" refers to a device used to access the system and view learning content.

[0366] "Course" refers to a program of study or curriculum aligned with a specific educational objective.

[0367] "Study progress" refers to the progress of a user's studies in a course selected by the user.

[0368] "Results" refers to the achievements and evaluations that users obtain through learning and quizzes.

[0369] "Real-time" refers to immediate responses to user operations and actions.

[0370] "Analysis" refers to the process of creating evaluations and feedback based on collected data.

[0371] "Feedback" refers to evaluations and advice provided based on a user's learning progress and results.

[0372] "Mobile app" refers to software that runs on mobile devices such as smartphones and tablets.

[0373] "Posting a question" refers to the act of a user submitting a question that arises during study to the system.

[0374] "Event management" refers to the process of registering for and providing information about offline events.

[0375] "Emotional data" refers to information about emotions collected from the user's facial expressions, tone of voice, etc.

[0376] "Learning experience" refers to the entire process and environment in which a user learns using a system.

[0377] "Generative artificial intelligence" refers to technology that uses natural language processing and machine learning to analyze data and automatically generate feedback and answers.

[0378] "Educational organization" refers to an organization that carries out educational activities, such as a school or educational institution.

[0379] "High-speed internet" refers to an internet connection that can send and receive large amounts of data quickly.

[0380] "Modern learning equipment" refers to advanced educational devices that utilize modern technology.

[0381] System Overview

[0382] This invention provides a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives. It also includes functions for managing the user's selected courses and learning progress via a dedicated device, recording and analyzing the user's learning progress and results in real time, providing feedback based on the analysis results, posting questions via the mobile app, analyzing them, and providing answers, and event management for participating in offline events. In addition, by combining it with an emotion engine that recognizes the user's emotions, the learning experience is more personalized, providing effective educational support.

[0383] User Registration

[0384] The user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal sends the entered information to the server, which stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[0385] Course Selection and Learning Management

[0386] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[0387] Learning progression and feedback

[0388] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[0389] Communication and Support

[0390] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[0391] Participating in offline events

[0392] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[0393] Customizing the learning experience with an emotional engine

[0394] While the user is studying, the emotion engine collects emotional data in real time. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their studies, it presents more difficult tasks. This ensures that the user's learning experience is always optimal.

[0395] Examples of prompt statements

[0396] 1. "Select a science experiment course and generate feedback after completing the video tutorial."

[0397] 2. "What are some examples of relaxation content for users when they are feeling stressed?"

[0398] 3. "Generate notification messages for offline events."

[0399] This system allows users to learn STEAM education interactively and effectively, and provides an optimal learning environment based on their emotions and customized feedback according to their progress and level of understanding.

[0400] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0401] Step 1:

[0402] The user accesses the dedicated terminal and opens the account creation screen.

[0403] (Input): None

[0404] (Output): Account creation screen

[0405] (Specific operation): The user opens the browser on the dedicated device, enters the system URL, and displays the account creation screen.

[0406] Step 2:

[0407] The user enters their name, school name, grade, and email address and presses the "Register" button.

[0408] (Input): User's name, school name, grade, email address

[0409] (Output): Personal information entered

[0410] (Specific operation): The user enters the required information into the form on the account creation screen and clicks the "Register" button.

[0411] Step 3:

[0412] The terminal sends the input information to the server.

[0413] (Input): User's personal information

[0414] (Output): User information sent to the server

[0415] (Specific operation): The device sends the data entered in the form to the server via an HTTPS POST request. The server receives the data.

[0416] Step 4:

[0417] The server stores the input information in a database.

[0418] (Input): User's personal information

[0419] (Output): User information stored in the database

[0420] (Specific operation): The server saves the received user information in a database (e.g., MySQL or PostgreSQL) using an INSERT statement.

[0421] Step 5:

[0422] The server generates authentication information such as a user ID and password and sends it to the terminal.

[0423] (Input): User's personal information

[0424] (Output): Generated authentication information (user ID, password)

[0425] (Specific operation): The server generates a unique user ID from the user information, sets a password in a secure manner, and sends the generated authentication information to the terminal as an HTTPS response.

[0426] Step 6:

[0427] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[0428] (Input): Generated credentials

[0429] (Output): Registration completion notification and displayed authentication information

[0430] (Specific operation): The terminal displays the received authentication information on the screen in a format that is easy for the user to understand, and notifies the user that "registration is complete."

[0431] Step 7:

[0432] A user enters their credentials to log into an online platform.

[0433] (Input): User ID, password

[0434] (Output): Login information

[0435] (Specific operation): The user enters the user ID and password on the login screen and clicks the "Login" button.

[0436] Step 8:

[0437] The device sends the login information to the server.

[0438] (Input): User ID, password

[0439] (Output): Login information sent to the server

[0440] (Specific operation): The device sends the entered login information to the server via an HTTPS POST request.

[0441] Step 9:

[0442] The server verifies the authentication information.

[0443] (Input): Login information

[0444] (Output): Authentication result (success or failure)

[0445] (Specific operation): The server checks the received login information against the database to verify its validity. If successful, it returns an authentication success message to the terminal, and if unsuccessful, it returns an error message.

[0446] Step 10:

[0447] If the authentication is successful, the server sends the dashboard data to the terminal.

[0448] (Input): Authentication success result

[0449] (Output): Dashboard data

[0450] (Specific operation): After successful authentication, the server generates the necessary information for the user's dashboard screen (recommended courses, learning progress, etc.) and sends it to the device as an HTTPS response.

[0451] Step 11:

[0452] The device displays the dashboard.

[0453] (Input): Dashboard data

[0454] (Output): Dashboard displayed

[0455] (Specific operation): The dashboard data received by the device is displayed on the screen.

[0456] Step 12:

[0457] The user opens the "Course Selection" menu and selects the desired course.

[0458] (Input): User selection

[0459] (Output): Selected course information

[0460] (Specific operation): The user clicks on the "Course Selection" menu on the dashboard screen and selects the desired course from the displayed list of courses.

[0461] Step 13:

[0462] The terminal transmits the course selection information to the server.

[0463] (Input): Selected course information

[0464] (Output): Course selection information sent to the server

[0465] (Specific operation): The device sends the selected course information to the server via an HTTPS POST request.

[0466] Step 14:

[0467] The server generates the optimal learning plan for the user and sends it to the device.

[0468] (Input): Selected course information, user information

[0469] (Output): Generated lesson plan

[0470] (Specific operation): The server uses an AI model to generate an optimal learning plan based on the selected course information and the user's learning history, and sends it to the device as an HTTPS response.

[0471] Step 15:

[0472] The device displays the learning plan and the user begins the learning module.

[0473] (Input): Generated lesson plan

[0474] (Output): Displayed learning plan, start of learning module

[0475] (Specific operation): The device displays the received learning plan on the screen, and the user selects and starts a learning module.

[0476] Step 16:

[0477] Users progress through the learning content on dedicated devices.

[0478] (Input): Learning plan content

[0479] (Output): Learning progress data

[0480] (Specific actions): The user watches a video tutorial and proceeds through experiments and quizzes according to the learning content.

[0481] Step 17:

[0482] The device records experimental results and quiz answers in real time and sends them to the server.

[0483] (Input): Experiment results, quiz answers

[0484] (Output): Submitted experiment results, quiz answers

[0485] (Specific operation): The device records the user's input and selections in real time and sends them to the server via an HTTPS POST request.

[0486] Step 18:

[0487] The server analyzes the received data and evaluates the user's level of understanding and progress.

[0488] (Input): Experiment results, quiz answers

[0489] (Output): Analysis results (level of understanding, progress evaluation)

[0490] (Specific operation): The server analyzes the data received using an AI model to evaluate the user's learning comprehension and progress.

[0491] Step 19:

[0492] The server generates feedback based on the evaluation results and sends it to the terminal.

[0493] (Input): Understanding level, progress evaluation

[0494] (Output): Generated feedback

[0495] (Specific operation): Based on the evaluation results, the server generates a specific feedback message and sends it to the terminal as an HTTPS response.

[0496] Step 20:

[0497] The device displays feedback to the user, who can use it to improve and take next steps as they progress through their learning.

[0498] (Input): Feedback message

[0499] (Output): The displayed feedback message

[0500] (Specific operation): The device displays the received feedback message on the screen and notifies the user of the next learning step or areas for improvement.

[0501] Step 21:

[0502] A user posts a question using a mobile app.

[0503] (Input): Question

[0504] (Output): Question sent

[0505] (Specific action): A user types a question using the chat function of a mobile app and presses the "Send" button.

[0506] Step 22:

[0507] The device sends the question to the server.

[0508] (Input): Question

[0509] (Output): The question sent to the server

[0510] (Specific operation): The device sends the entered question to the server via an HTTPS POST request.

[0511] Step 23:

[0512] The server analyzes the question, generates an appropriate answer, and sends it to the device.

[0513] (Input): Question

[0514] (Output): Generated answer

[0515] (Specific operation): The server analyzes the received question using natural language processing (NLP), generates an appropriate answer using an AI model, and sends it to the device.

[0516] Step 24:

[0517] The device displays the answer for the user to review, and if they are not satisfied, they can post the question again.

[0518] (Input): Generated answer

[0519] (Output): Displayed answer, re-ask if necessary

[0520] (Specific operation): The device displays the received answer on the screen for the user to confirm. If the user is not satisfied, they can re-enter the question and submit it.

[0521] Step 25:

[0522] The server periodically generates and sends offline event notifications to the terminal.

[0523] (Input): None

[0524] (Output): Generated notification

[0525] (Specific operation): The server generates detailed information about offline events based on a pre-set schedule and sends notifications to the device.

[0526] Step 26:

[0527] The user receives a notification and registers for the event.

[0528] (Input): Notification content

[0529] (Output): Participant registration information

[0530] (Specific actions): The user checks the notification sent to them, and if they wish to participate, they enter the necessary information into the dedicated form and press the "Register" button.

[0531] Step 27:

[0532] The terminal transmits the participation registration information to the server.

[0533] (Input): Registration information

[0534] (Output): Registration information sent to the server

[0535] (Specific operation): The device sends the entered participation registration information to the server via an HTTPS POST request.

[0536] Step 28:

[0537] The server generates a participation confirmation message and sends it to the terminal to notify the user.

[0538] (Input): Registration information

[0539] (Output): The generated confirmation message

[0540] (Specific operation): The server stores the received registration information in a database, generates a participation confirmation message and sends it to the terminal, which then displays the confirmation message to the user.

[0541] Step 29:

[0542] While the user is learning, the emotion engine collects emotion data in real time.

[0543] (Input): User's facial expression, tone of voice

[0544] (Output): Real-time emotion data

[0545] (Specific operation): Using the camera and microphone of the dedicated device, the user's facial expressions and tone of voice are sensed in real time and collected as emotional data.

[0546] Step 30:

[0547] The server analyzes the emotional data and adjusts the learning content according to the user's emotional state.

[0548] (Input): Emotion data

[0549] (Output): Tailored learning content

[0550] (Specific operation): Based on the emotional data received by the server, the system evaluates the user's stress level and concentration level, and adjusts the difficulty of the learning content. It may also add relaxation content.

[0551] Step 31:

[0552] The terminal displays the adjusted content to the user.

[0553] (Input): Tailored learning content

[0554] (Output): The adjusted content displayed

[0555] (Specific operation): The device displays tailored learning and relaxation content on the screen and provides it to the user.

[0556] Through the above processing steps, the system can provide users with interactive and effective STEAM education, providing appropriate feedback and adjusting the learning environment according to individual progress and level of understanding.

[0557] (Application example 2)

[0558] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0559] Traditional STEAM education platforms and mobile apps focus on managing users' learning progress and providing feedback, but lack the emotional analysis capabilities to personalize the learning experience. As a result, they often provide uniform content without considering users' stress levels or concentration levels, making effective learning difficult. They also struggle to efficiently respond to questions in real time or manage offline events. A comprehensive system that can resolve these issues and provide a more personalized learning experience is needed.

[0560] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0561] In this invention, the server includes means for recording and analyzing the user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, and emotion analysis means for collecting the user's emotion data and analyzing their stress level and concentration level to optimize the learning experience. This makes it possible to analyze the user's learning progress in real time and provide optimal content based on the emotion data.

[0562] 1. "User interface" refers to the screens and operating means through which users register an account and enter personal information and learning objectives.

[0563] 2. "Dedicated Device" means a hardware device or software application used to manage the user's selected course and learning progress.

[0564] 3. "Means for recording and analyzing learning progress and results in real time" refers to algorithms or computer programs for instantly storing and analyzing a user's learning progress and results in a database.

[0565] 4. "Means for providing feedback" refers to a method or system for returning advice or instructions regarding the user's learning based on the analysis results.

[0566] 5. "Means for posting questions, analyzing the content of the questions, and providing answers" refers to a system that allows users to submit questions via a mobile app, analyzes the questions using natural language processing technology, and generates appropriate answers.

[0567] 6. "Event Management Means" means a method or software that allows a user to register to participate in an offline event and manage detailed information about that event.

[0568] 7. "Emotion analysis means" refers to technologies and systems that collect users' emotional data, analyze their stress levels and concentration levels, and optimize learning content.

[0569] This invention is a system that realizes a comprehensive online platform and mobile application that supports STEAM education for elementary, middle, and high school students. The entire system is composed as follows.

[0570] User Registration

[0571] The user accesses the dedicated terminal and first opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and then presses the "Register" button. The terminal sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal displays this authentication information on the screen and notifies the user that registration is complete.

[0572] Course Selection and Learning Management

[0573] The user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal then sends the course selection information to the server, which generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[0574] Learning progression and feedback

[0575] Users progress through learning content on a dedicated device. For example, they watch video tutorials and conduct scientific experiments. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. Based on the evaluation results, the server generates feedback and sends it to the device. The device displays the feedback to the user, who can use it as a reference for areas for improvement and next steps to continue their learning.

[0576] Question and Answer Support

[0577] When a user posts a question using the mobile app, the application sends the question to a server. The server uses a generative AI model to analyze the question and generate an appropriate answer. This answer is then sent to the device, where it is displayed to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to an expert.

[0578] Participating in offline events

[0579] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which stores it in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[0580] Sentiment analysis and customised learning experiences

[0581] During learning, the server uses emotion analysis to collect the user's emotional data. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice data. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their learning, it presents more difficult tasks.

[0582] Hardware and software used

[0583] The system uses the following elements:

[0584] React Native is used for the user interface of the dedicated terminal.

[0585] The backend development uses the Django framework and PostgreSQL.

[0586] TensorFlow is used for sentiment analysis.

[0587] Generative AI models such as ChatGPT API are used to analyze questions and generate answers.

[0588] Examples of concrete examples and prompts

[0589] For example, if a junior high school student user selects the "Science Experiment" course, the server generates a learning plan including a video tutorial and experiment procedures. The user watches the video and records the experiment results on their device. The server receives and analyzes the results in real time and generates appropriate feedback. At the same time, an emotion analysis means detects the user's stress level from facial expression data and adjusts the learning content as needed.

[0590] Example prompt sentence:

[0591] "Analyze the stress level of the user from facial expression data."

[0592] "Please analyze the level of concentration from user voice data."

[0593] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0594] Step 1: User Registration

[0595] The user opens the account creation screen using a dedicated terminal. They enter required information such as name, school name, grade, and email address, and press the "Register" button. The input data is sent from the terminal to the server. The server receives it, stores it in a database, and generates authentication information such as a user ID and password. The generated authentication information is sent from the server to the terminal, and the terminal notifies the user.

[0596] Input data: Name, school name, grade, email address

[0597] Output data: User ID, password

[0598] Step 2: Log in and view the dashboard

[0599] The user enters authentication information on the dedicated terminal and presses the login button. The terminal sends the login information to the server, which then verifies the authentication information. If authentication is successful, the server sends data about the user's learning course and progress to the terminal as dashboard screen data. The terminal displays this data.

[0600] Input data: User ID, password

[0601] Output data: Dashboard screen data

[0602] Step 3: Select your courses and create your study plan

[0603] The user opens the "Course Selection" menu from the dashboard and selects the desired course. The device sends the course selection information to the server. The server generates an optimal learning plan for the user and sends the learning plan data to the device. The device displays the learning plan, and the user starts the learning module.

[0604] Input data: Course selection information

[0605] Output data: Learning plan data

[0606] Step 4: Learning progression and data recording

[0607] Users progress through learning content on a dedicated device. When watching video tutorials or conducting scientific experiments, the device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress.

[0608] Input data: Experiment results, quiz answers

[0609] Output data: Comprehension assessment, progress data

[0610] Step 5: Generate and display feedback

[0611] The server generates feedback based on the understanding assessment and progress data. The generated feedback is sent to the device, which displays it to the user. The user can then refer to the feedback to identify areas for improvement and next steps.

[0612] Input data: Comprehension assessment, progress data

[0613] Output Data: Feedback

[0614] Step 6: Sentiment Analysis

[0615] While learning, a dedicated device records the user's facial expressions and voice in real time to collect emotional data. This data is sent from the device to a server, which uses an emotion analysis model to evaluate the user's stress level and concentration. Based on the evaluation results, the server adjusts the difficulty of the learning content.

[0616] Input data: facial expression data, voice data

[0617] Output data: stress level, concentration evaluation

[0618] Step 7: Questions and Answers

[0619] When a user posts a question using a mobile app, the device sends the question to the server, which uses a generative AI model to analyze the question and generate an appropriate answer. The generated answer is then sent to the device, which displays it to the user.

[0620] Input data: Question content

[0621] Output data: Answer

[0622] Step 8: Offline event management

[0623] The server periodically generates notifications for offline events and sends them to the device. The user receives the notification, enters the necessary information in the application form, and presses the "Register" button. The device sends the participation registration information to the server, which stores it in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[0624] Input data: Event details, application information

[0625] Output data: Participation confirmation message

[0626] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0627] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0628] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0629] [Second embodiment]

[0630] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0631] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0632] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0633] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0634] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0635] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0636] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0637] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0638] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0639] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0640] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0641] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0642] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives, a function for managing the user's selected courses and learning progress via a dedicated terminal, a function for recording and analyzing the user's learning progress and results in real time, a function for providing feedback based on the analysis results, a function for posting questions via the mobile app, analyzing them, and providing answers, and an event management function for participating in offline events. However, this does not involve writing program code.

[0643] The overall system flow is as follows:

[0644] User Registration

[0645] First, the user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal then sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[0646] Course Selection and Learning Management

[0647] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[0648] Learning progression and feedback

[0649] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[0650] Communication and Support

[0651] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[0652] Participating in offline events

[0653] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[0654] Specific examples

[0655] For example, if a junior high school student user selects the "Science Experiment" course and proceeds through the program, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. The user can continue learning by using this as a reference when reviewing the course or moving on to the next module.

[0656] This system allows users to learn STEAM education interactively and effectively, and provides customized feedback based on their progress and level of understanding.

[0657] The processing flow will be explained below.

[0658] Step 1:

[0659] The user accesses the dedicated terminal, opens the account creation screen, enters required information such as name, school name, grade, and email address, and presses the "Register" button.

[0660] Step 2:

[0661] The terminal sends the input information to the server.

[0662] Step 3:

[0663] The server stores the received user information in a database and generates authentication information such as a user ID and password.

[0664] Step 4:

[0665] The server sends the generated authentication information to the terminal.

[0666] Step 5:

[0667] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[0668] Step 6:

[0669] To log in to the online platform, the user enters their authentication information and presses the login button.

[0670] Step 7:

[0671] The device sends the login information to the server.

[0672] Step 8:

[0673] The server checks the authentication information, and if authentication is successful, it sends the dashboard screen data (learning progress, recommended courses, etc.) to the device.

[0674] Step 9:

[0675] The terminal displays the dashboard screen, and the user opens the "Course Selection" menu.

[0676] Step 10:

[0677] The terminal sends a request for a course list to the server.

[0678] Step 11:

[0679] The server sends a list of available STEAM courses to the device.

[0680] Step 12:

[0681] The user selects the desired course from the course list.

[0682] Step 13:

[0683] The terminal transmits the selected course information to the server.

[0684] Step 14:

[0685] The server generates a learning plan based on the selected course and transmits it to the terminal.

[0686] Step 15:

[0687] The device displays the learning plan, and the user clicks the "Start" button to display the first learning module.

[0688] Step 16:

[0689] Users watch learning content (e.g., video tutorials) and complete experiments and quizzes.

[0690] Step 17:

[0691] The device records the user's learning data (experiment results, quiz answers, etc.) in real time and sends it to the server.

[0692] Step 18:

[0693] The server analyzes the data it receives and evaluates the user's learning progress and level of understanding.

[0694] Step 19:

[0695] The server generates feedback based on the evaluation results and sends it to the terminal.

[0696] Step 20:

[0697] The device displays feedback to the user and prompts them to relearn or proceed to the next module as needed.

[0698] Step 21:

[0699] A user uses the mobile app to post a question via the LINE chat function.

[0700] Step 22:

[0701] The terminal sends the user's question to the server.

[0702] Step 23:

[0703] The server analyzes the question, generates an appropriate answer, and sends it to the terminal.

[0704] Step 24:

[0705] The terminal displays the answer received from the server to the user.

[0706] Step 25:

[0707] If the user is not satisfied with the answer, they can post the question again and the server will escalate the question to an expert or employee.

[0708] Step 26:

[0709] The server periodically generates and sends offline event notifications to the terminal.

[0710] Step 27:

[0711] Users receive event notifications and view detailed information on mobile apps and online platforms.

[0712] Step 28:

[0713] A user registers to participate in an offline event, enters information in the application form, and presses the "Register" button.

[0714] Step 29:

[0715] The terminal transmits the participation registration information to the server.

[0716] Step 30:

[0717] The server stores the registration information in a database and generates a confirmation message.

[0718] Step 31:

[0719] The server sends a participation confirmation message to the terminal and notifies the user.

[0720] Example 1

[0721] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0722] Current STEAM education support systems struggle to effectively manage users' learning progress and comprehension, and provide individually customized feedback. They also lack systems that allow users to ask questions in real time and receive immediate answers. Furthermore, they lack a means to seamlessly link online and offline learning environments. This makes it difficult for users to progress efficiently.

[0723] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0724] In this invention, the server includes means for recording and analyzing a user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, and means for the user to post questions via a mobile app, analyze the questions, and provide answers. This makes it possible to instantly check a user's learning progress and suggest areas for improvement, thereby providing individually customized learning support. Furthermore, the real-time exchange of questions and answers via the mobile app allows users to quickly resolve their doubts and enable efficient learning. Furthermore, seamless integration of online and offline events can increase users' motivation to learn and encourage continuous learning.

[0725] "User" refers to a person who uses the system to carry out learning activities.

[0726] "Dedicated Terminal" refers to a specific electronic device used to access the system.

[0727] "Mobile app" refers to application software that runs on smartphones and tablets.

[0728] "Server" refers to a central computer system that stores, processes, and serves data.

[0729] "User interface" refers to the screen and operating means that allow a user to directly interact with a system.

[0730] "Learning curriculum" refers to various learning contents and courses selected by the user.

[0731] "Study progress" refers to the progress of a user in a learning curriculum selected by the user.

[0732] "Learning Content" refers to information and materials such as videos, texts, and quizzes provided to support learning.

[0733] "Feedback" refers to information that provides an evaluation of a user's learning activities and suggestions for improvement.

[0734] "Analysis" refers to the process of processing collected data to derive useful information or results.

[0735] "Non-online events" refer to learning or education-related events that take place in a physical location.

[0736] A "generative AI model" refers to an artificial intelligence algorithm that generates data-based feedback to track a user's learning progress and provide adaptive feedback.

[0737] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system provides a user interface for users to register an account and enter personal information and learning objectives. Specific hardware includes dedicated terminals, mobile terminals, and servers. Software includes database management software (e.g., MySQL or PostgreSQL), real-time data analysis software (e.g., Apache Kafka), front-end frameworks (e.g., React or Angular), and mobile app development frameworks (e.g., Flutter or React Native).

[0738] System configuration

[0739] 1. User Registration and Authentication

[0740] A user accesses a dedicated terminal and creates an account by entering profile information. The terminal sends the user's input information to the server, which stores the received information in a database. The server generates authentication information and sends it to the terminal. The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[0741] 2. Course Selection and Learning Management

[0742] The user logs in to a dedicated terminal using authentication information, and the server verifies the authentication information. If authentication is successful, the server sends the user's dedicated dashboard data to the terminal. The user selects the desired learning curriculum from the dashboard, and the terminal sends the selection information to the server. The server generates an optimal learning plan based on the user's selection and sends it to the terminal. The terminal displays the generated learning plan, and the user begins learning.

[0743] 3. Learning and Feedback

[0744] Users progress through learning content (e.g., video tutorials, textbooks, quizzes) provided on dedicated devices. The devices record the user's learning progress and quiz answers in real time and send them to the server. The server analyzes the received data and evaluates the user's level of understanding and progress. Feedback generated based on the evaluation is sent to the device, which then displays it to the user.

[0745] 4. Real-time Q&A

[0746] Users use a mobile app to post questions. The questions are sent from the device to a server, which analyzes them. An appropriate answer is generated and sent to the device. The user can review the answer and ask the question again if necessary. Complex questions are answered by experts or employees.

[0747] 5. Participating in offline events

[0748] The server periodically generates notifications for non-online events and sends them to the device. The user receives the notifications and views the details. When registering for an event, the user enters the required information and the device sends the registration information to the server. The server stores the information in a database and sends a participation confirmation message to the device.

[0749] Specific examples

[0750] For example, if a junior high school student user selects the "Science Experiment" course and uses the system, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. The user can continue learning by using this as a reference when reviewing the course or moving on to the next module.

[0751] Prompt Sentence Examples

[0752] "Please tell me the specific steps to provide lesson plans and video tutorials for a science lab course for middle school students to learn basic scientific principles."

[0753] The present invention allows users to learn STEAM education interactively and effectively, and receive customized feedback based on their progress and level of understanding, maximizing learning efficiency.

[0754] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0755] Step 1: User registers for an account

[0756] The user accesses the dedicated terminal and opens the account creation screen.

[0757] Input: The user enters required information such as name, school name, grade, and email address.

[0758] The terminal sends the input information to the server.

[0759] Output: User information is saved in the database on the server side. Authentication information (user ID, password) is generated and sent to the terminal.

[0760] Specific operation: The terminal sends the input information as an HTTP POST request, and the server executes an SQL INSERT statement against the database.

[0761] Step 2: User enters credentials and logs in

[0762] The user enters authentication information (user ID, password) on the dedicated terminal and presses the login button.

[0763] Input: The user enters the user ID and password.

[0764] The device sends the login information to the server.

[0765] Output: The server verifies the authentication information and, if successful, sends the dashboard data to the device.

[0766] Specific operation: The terminal sends an HTTP POST request, and the server searches for user information in the database and compares the password.

[0767] Step 3: User makes course selection

[0768] After logging in, the user selects the "Course Selection" menu from the dashboard and chooses the desired learning curriculum.

[0769] Input: The user selects the desired learning curriculum.

[0770] The terminal transmits the course selection information to the server.

[0771] Output: The server generates the optimal learning plan for the user and sends it to the device.

[0772] How it works: The device sends an HTTP POST request to the server containing the selected course information, and the server uses an algorithm to generate a learning plan.

[0773] Step 4: Users progress through the learning content

[0774] The user starts the provided learning content on a dedicated device.

[0775] Input: User watches video tutorials, performs experiments and quizzes.

[0776] The device records the user's progress and quiz answers and sends them to the server.

[0777] Output: The server receives the data in real time and generates feedback based on the analysis results.

[0778] Specific operation: The device records user operation data in real time and sends it to the server via HTTP POST request. The server then runs an analysis algorithm to evaluate the data.

[0779] Step 5: Provide feedback based on the analysis results

[0780] The server analyzes the user's learning data and evaluates their level of understanding and progress.

[0781] Input: User learning progress data and quiz answer data.

[0782] The server generates the analysis results and sends feedback to the device.

[0783] Output: The device displays feedback to the user.

[0784] Specific operation: The server analyzes the data in the database, generates a feedback message based on the evaluation criteria, and sends the generated feedback to the terminal as an HTTP response.

[0785] Step 6: Users post questions and get answers

[0786] A user posts a question using a mobile app.

[0787] Input: The user enters the question and clicks the post button.

[0788] The device sends the question to the server.

[0789] Output: The server analyzes the question, generates an appropriate answer, and sends it to the device.

[0790] Specific operation: The device sends an HTTP POST request containing a question to the server, and the server analyzes the question using a natural language processing algorithm, generates an answer, and sends the answer to the device as an HTTP response.

[0791] Step 7: User registers for a non-online event

[0792] The server periodically generates and sends notifications of non-online events to the terminal.

[0793] Input: Event details and registration information.

[0794] Users receive notifications, view details, and register to attend.

[0795] Output: The terminal sends the participation registration information to the server, which stores the information in a database. The server generates a participation confirmation message and sends it to the terminal.

[0796] Specific operation: The server periodically generates an event notification and sends it to the terminal as an HTTP response. The user registers for participation, and the terminal sends it to the server as an HTTP POST request. The server executes an SQL INSERT statement in the database, generates a participation confirmation message, and sends it to the terminal.

[0797] (Application example 1)

[0798] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0799] Conventional STEAM education systems have limited functionality for tracking learners' progress and providing feedback, making real-time analysis and customized instruction difficult. They also lack the ability to adequately manage interactive virtual learning experiences and offline events. Furthermore, the adoption of new learning methods utilizing wearable devices has been slow, creating a need for ways to maximize the effectiveness of education.

[0800] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0801] In this invention, the server includes: means for providing a user interface for users to register an account and input personal information and learning objectives; means for managing the user's selected course and learning progress via a dedicated terminal; means for recording and analyzing the user's learning progress and results in real time; means for providing feedback to the user based on the analysis results; means for users to post questions via a mobile app or a wearable device (smart glasses or a head-mounted display), analyze the questions, and provide answers; means for conducting experiments in a virtual space, simulating the experiment results, and providing feedback in real time; and event management means for displaying regular offline event notifications and detailed information and registering participants. This enables an interactive learning experience using a variety of devices, realizes real-time analysis of learning progress, and provides customized feedback, thereby improving the quality of education and maximizing learning effectiveness.

[0802] "A user interface for users to register an account and enter personal information and learning objectives" is a means of providing an operation screen for users to create their own account in an educational system and enter personal information and learning objectives.

[0803] "Means for managing a user's selected course and learning progress via a dedicated terminal" refers to a means in an educational system that provides a user with the ability to track and manage their selected learning course and learning progress using a dedicated device.

[0804] "Means for recording and analyzing a user's learning progress and results in real time" refers to a means for providing a function in an educational system to collect a user's learning progress and results as data in real time and analyze them using statistical or machine learning methods.

[0805] "Means for providing feedback to the user based on the analysis results" refers to means for providing the user with a function for suggesting areas for improvement and next learning steps based on the learning data analyzed in the educational system.

[0806] "A means for users to post questions via a mobile app or wearable device, and for the content of the question to be analyzed and an answer to be provided" refers to a means in an educational system that allows users to send questions using a smartphone app or a wearable device such as smart glasses, and provides a function that analyzes the content of the question using a natural language processing model or the like and provides an appropriate answer.

[0807] "Means for conducting experiments in a virtual space, simulating the experimental results, and providing feedback in real time" refers to a means for realizing the function in an educational system in which a user conducts an experiment in a virtual reality or mixed reality space, analyzes the results using computer simulation, and provides feedback in real time.

[0808] "Event management means for displaying regular offline event notifications and detailed information, and registering for participation" refers to a means for providing users in an educational system with regular notifications of offline event information, allowing them to view detailed information, and providing the functionality for registering for participation.

[0809] "Means for analyzing data in real time using cloud computing and providing optimal feedback using generative artificial intelligence" refers to a means in an educational system that uses cloud services via the internet to process and analyze collected data in real time and generate and provide optimal feedback to users using machine learning models.

[0810] "Means for providing an interactive learning experience using mixed reality technology to create a virtual space" refers to a means for using mixed reality (MR) technology in an educational system to create a learning environment that combines reality and virtual space, and to provide users with an interactive and immersive learning experience.

[0811] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students.

[0812] Overall structure

[0813] 1. User Registration:

[0814] The server provides a user interface for users to enter personal information such as name, school name, grade, and email address, as well as their learning goals. When a user registers an account from a dedicated terminal, the terminal sends the information to the server, which stores it in a database. After registration is complete, the server generates a user ID and password and sends them to the terminal.

[0815] 2. Course Selection and Learning Management:

[0816] The server manages the courses selected by the user via a dedicated terminal and their learning progress. When the user logs in to the online platform and selects a course, the server generates an optimal learning plan and sends the data to the terminal. The terminal displays the learning plan, and the user starts the learning module.

[0817] 3. Learning progression and feedback:

[0818] Users progress through learning content on their devices. The devices record the user's learning progress and results (e.g., scientific experiment results and quiz answers) in real time and send them to a cloud server. The server analyzes the data, generates feedback based on the analysis results, and sends it to the device. Users can also conduct experiments in virtual space, simulate the results, and receive feedback in real time.

[0819] 4. Real-time communication and support:

[0820] When a user posts a question using a mobile app or a wearable device (smart glasses or a head-mounted display), the device sends the question to a server, which then analyzes the question using a generative AI model (e.g., an NLP model), generates an appropriate answer, and sends it to the device.

[0821] 5. Offline event notification and participation management:

[0822] The server periodically generates notifications for offline events and sends them to the terminal. When a user receives the notification, views detailed information, and registers for participation, the terminal sends the participation registration information to the server, which stores the information in a database. A participation confirmation message is sent to the terminal to notify the user.

[0823] Hardware and Software Use

[0824] Hardware:

[0825] Dedicated devices (PC, smartphone, smart glasses, head-mounted display)

[0826] Server (cloud-based, e.g. AWS, Google Cloud)

[0827] software:

[0828] Frontend: React Native, Flutter, Unity engine

[0829] Backend: Express.js, Firebase Functions

[0830] Database: MongoDB, Firestore database

[0831] Analysis: TensorFlow, PyTorch, AWS Lambda functions

[0832] Communication: LINE API, Twilio Chat API

[0833] Specific examples

[0834] Examples of how the virtual laboratory can be used:

[0835] 1. Scenario: A middle school student user selects the "Virtual Lab" course.

[0836] 2. Procedure: Users simulate scientific experiments in a virtual space built with the Unity engine. The results are analyzed in real time in the Azure cloud, and feedback is generated using Azure ML.

[0837] 3. Questions: When a user asks a question about something they don't understand, the question is sent using the LINE chat function, and an answer analyzed by the BERT model is provided in real time.

[0838] 4. Offline Events: Users receive offline event notifications, view details, and register to attend.

[0839] Example prompts to input to a generative AI model:

[0840] Prompt: We are developing a STEAM education application for elementary, middle, and high school students. The app allows users to conduct experiments in a virtual space, analyzes the results in real time, and provides feedback. Additionally, the app uses an NLP model to generate answers when users post questions. It also periodically notifies users of offline events. Please explain in detail the use case for this app and the technologies required.

[0841] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0842] Step 1:

[0843] User Registration:

[0844] The user accesses the account registration screen from a dedicated terminal and enters personal information such as name, school name, grade, and email address. The terminal sends this input information to the server. The server stores the received information in a database, generates a user ID and password, and sends them to the terminal. The terminal then displays a notification to the user that registration is complete. The input is the user's personal information, and the output is the user information stored in the database and the generated user ID and password.

[0845] Step 2:

[0846] Course Selection and Learning Management:

[0847] The user enters authentication information to log in to the online platform from a dedicated terminal and presses the login button. The terminal sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. The user opens the "Course Selection" menu from the dashboard and selects the desired course. The terminal sends the course selection information to the server, and the server generates an optimal learning plan for the user and sends it to the terminal. The input is the user's authentication information and course selection information, and the output is the dashboard screen data and learning plan.

[0848] Step 3:

[0849] Learning progression and feedback:

[0850] The user progresses through the learning content on the device. The device records the user's learning progress and results (e.g., scientific experiment results or quiz answers) in real time and sends them to a cloud server. The server analyzes the data in real time (e.g., using a machine learning model), generates feedback based on the analysis results, and sends it to the device. The device displays the feedback to the user. The input is the user's learning progress and results, and the output is feedback based on the analysis results.

[0851] Step 4:

[0852] Real-time communication and support:

[0853] A user posts a question using a mobile app or a wearable device (smart glasses or a head-mounted display). The device sends the question to the server, which analyzes the question using a generative AI model (e.g., an NLP model). An appropriate answer is generated and sent to the device. The device displays the answer to the user. The input is the user's question, and the output is the analyzed answer.

[0854] Step 5:

[0855] Virtual experiments and simulations:

[0856] Users conduct scientific experiments in a virtual space using dedicated terminals or wearable devices. The experimental results are recorded in real time on the terminal and sent to a cloud server. The server simulates the data and generates analytical results. Feedback is generated in real time and sent to the terminal. The terminal displays the feedback to the user. The input is the results of the virtual experiment, and the output is feedback based on the simulation and analytical results.

[0857] Step 6:

[0858] Offline event notification and participation management:

[0859] The server periodically generates offline event notifications and sends them to the terminal. When a user receives the notification, views detailed information, and registers to participate, the terminal sends the registration information to the server, which stores the information in a database. A participation confirmation message is sent to the terminal, notifying the user. The input is the offline event notification and the user's registration information, and the output is the registration information stored in the database and the participation confirmation message.

[0860] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0861] This invention provides a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives; a function for managing the user's selected courses and learning progress via a dedicated terminal; a function for recording and analyzing the user's learning progress and results in real time; a function for providing feedback based on the analysis results; a function for posting questions via the mobile app, analyzing them, and providing answers; and an event management function for participating in offline events. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system provides a more personalized learning experience and more effective educational support.

[0862] The overall system flow is as follows:

[0863] User Registration

[0864] First, the user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal then sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[0865] Course Selection and Learning Management

[0866] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[0867] Learning progression and feedback

[0868] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[0869] Communication and Support

[0870] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[0871] Participating in offline events

[0872] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[0873] Customizing the learning experience with an emotional engine

[0874] While the user is studying, the emotion engine collects emotional data in real time. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their studies, it presents more difficult tasks. This ensures that the user's learning experience is always optimal.

[0875] Specific examples

[0876] For example, if a junior high school student user selects a "science experiment" course and proceeds with the program, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then actually performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. Furthermore, the emotion engine detects stress from the user's facial expressions and sends instructions to the server to ease some of the learning as necessary. As a result, the user can proceed with their learning while reducing stress.

[0877] This system allows users to learn STEAM education interactively and effectively, providing an optimal learning environment based on their emotions and customized feedback according to their progress and level of understanding.

[0878] The processing flow will be explained below.

[0879] Step 1:

[0880] The user accesses the dedicated terminal, opens the account creation screen, enters required information such as name, school name, grade, and email address, and presses the "Register" button.

[0881] Step 2:

[0882] The terminal sends the input information to the server.

[0883] Step 3:

[0884] The server stores the received user information in a database and generates authentication information such as a user ID and password.

[0885] Step 4:

[0886] The server sends the generated authentication information to the terminal.

[0887] Step 5:

[0888] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[0889] Step 6:

[0890] To log in to the online platform, the user enters their authentication information and presses the login button.

[0891] Step 7:

[0892] The device sends the login information to the server.

[0893] Step 8:

[0894] The server checks the authentication information, and if authentication is successful, it sends the dashboard screen data (learning progress, recommended courses, etc.) to the device.

[0895] Step 9:

[0896] The terminal displays the dashboard screen, and the user opens the "Course Selection" menu.

[0897] Step 10:

[0898] The terminal sends a request for a course list to the server.

[0899] Step 11:

[0900] The server sends a list of available STEAM courses to the device.

[0901] Step 12:

[0902] The user selects the desired course from the course list.

[0903] Step 13:

[0904] The terminal transmits the selected course information to the server.

[0905] Step 14:

[0906] The server generates a learning plan based on the selected course and transmits it to the terminal.

[0907] Step 15:

[0908] The device displays the learning plan, and the user clicks the "Start" button to display the first learning module.

[0909] Step 16:

[0910] Users watch learning content (e.g., video tutorials) and complete experiments and quizzes.

[0911] Step 17:

[0912] The device records the user's learning data (experiment results, quiz answers, etc.) in real time and sends it to the server.

[0913] Step 18:

[0914] The server analyzes the data it receives and evaluates the user's learning progress and level of understanding.

[0915] Step 19:

[0916] The server generates feedback based on the evaluation results and sends it to the terminal.

[0917] Step 20:

[0918] The device displays feedback to the user and prompts them to relearn or proceed to the next module as needed.

[0919] Step 21:

[0920] A user uses the mobile app to post a question via the LINE chat function.

[0921] Step 22:

[0922] The terminal sends the user's question to the server.

[0923] Step 23:

[0924] The server analyzes the question, generates an appropriate answer, and sends it to the terminal.

[0925] Step 24:

[0926] The terminal displays the answer received from the server to the user.

[0927] Step 25:

[0928] If the user is not satisfied with the answer, they can post the question again and the server will escalate the question to an expert or employee.

[0929] Step 26:

[0930] The server periodically generates and sends offline event notifications to the terminal.

[0931] Step 27:

[0932] Users receive event notifications and view detailed information on mobile apps and online platforms.

[0933] Step 28:

[0934] A user registers to participate in an offline event, enters information in the application form, and presses the "Register" button.

[0935] Step 29:

[0936] The terminal transmits the participation registration information to the server.

[0937] Step 30:

[0938] The server stores the registration information in a database and generates a confirmation message.

[0939] Step 31:

[0940] The server sends a participation confirmation message to the terminal and notifies the user.

[0941] Form using emotion engine

[0942] Step 32:

[0943] During learning, the device collects facial and voice data in real time through the device's camera and microphone.

[0944] Step 33:

[0945] The device transmits the collected emotion data to a server.

[0946] Step 34:

[0947] The server uses an emotion engine to analyze the user's emotion data and evaluate the user's stress level and concentration.

[0948] Step 35:

[0949] The server adjusts the difficulty and progress speed of the learning content based on the results of emotion analysis.

[0950] Step 36:

[0951] The server transmits the adjusted learning content to the terminal.

[0952] Step 37:

[0953] The terminal displays the adjusted learning content to the user, and the user continues learning.

[0954] As a concrete example, when a junior high school student user is working through a "science experiment" course, if the user is concentrating during the experiment, the server will present more difficult experimental tasks, and conversely, if the user is feeling stressed, the server will present easier auxiliary tasks and provide relaxation techniques, allowing the user to always enjoy an optimal learning experience.

[0955] Example 2

[0956] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0957] Conventional educational systems have difficulty providing appropriate feedback in real time based on each user's learning progress and level of understanding, resulting in a decline in user learning efficiency.In addition, they are unable to grasp the stress and concentration levels felt by users in real time and adjust the learning environment accordingly, making it impossible to optimize each individual learning experience.

[0958] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0959] In this invention, the server includes means for providing a user interface for users to register an account and input personal information and learning objectives, means for managing the user's selected course and learning progress via a dedicated terminal, means for recording and analyzing the user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, means for users to post questions via a mobile app and provide answers by analyzing the questions, event management means for users to register for offline events, and means for collecting emotional data while the user is learning and adjusting the learning experience. This enables users to receive appropriate feedback in real time according to their individual progress and level of understanding, and to have their learning environment adjusted based on the emotional data.

[0960] "User" refers to a general user who uses the system to access learning content and manage their progress.

[0961] "Account Registration" refers to the process by which a User registers with the System by providing personal information and authentication information required to access the System.

[0962] "Personal information" refers to information that can be used to identify an individual, such as the user's name, school name, grade, and email address.

[0963] "Learning objectives" refer to the educational goals or needs that users want to achieve by using the system.

[0964] "User interface" refers to the screens and means by which a user logs into a system or enters information.

[0965] "Dedicated Terminal" refers to a device used to access the system and view learning content.

[0966] "Course" refers to a program of study or curriculum aligned with a specific educational objective.

[0967] "Study progress" refers to the progress of a user's studies in a course selected by the user.

[0968] "Results" refers to the achievements and evaluations that users obtain through learning and quizzes.

[0969] "Real-time" refers to immediate responses to user operations and actions.

[0970] "Analysis" refers to the process of creating evaluations and feedback based on collected data.

[0971] "Feedback" refers to evaluations and advice provided based on a user's learning progress and results.

[0972] "Mobile app" refers to software that runs on mobile devices such as smartphones and tablets.

[0973] "Posting a question" refers to the act of a user submitting a question that arises during study to the system.

[0974] "Event management" refers to the process of registering for and providing information about offline events.

[0975] "Emotional data" refers to information about emotions collected from the user's facial expressions, tone of voice, etc.

[0976] "Learning experience" refers to the entire process and environment in which a user learns using a system.

[0977] "Generative artificial intelligence" refers to technology that uses natural language processing and machine learning to analyze data and automatically generate feedback and answers.

[0978] "Educational organization" refers to an organization that carries out educational activities, such as a school or educational institution.

[0979] "High-speed internet" refers to an internet connection that can send and receive large amounts of data quickly.

[0980] "Modern learning equipment" refers to advanced educational devices that utilize modern technology.

[0981] System Overview

[0982] This invention provides a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives. It also includes functions for managing the user's selected courses and learning progress via a dedicated device, recording and analyzing the user's learning progress and results in real time, providing feedback based on the analysis results, posting questions via the mobile app, analyzing them, and providing answers, and event management for participating in offline events. In addition, by combining it with an emotion engine that recognizes the user's emotions, the learning experience is more personalized, providing effective educational support.

[0983] User Registration

[0984] The user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal sends the entered information to the server, which stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[0985] Course Selection and Learning Management

[0986] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[0987] Learning progression and feedback

[0988] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[0989] Communication and Support

[0990] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[0991] Participating in offline events

[0992] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[0993] Customizing the learning experience with an emotional engine

[0994] While the user is studying, the emotion engine collects emotional data in real time. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their studies, it presents more difficult tasks. This ensures that the user's learning experience is always optimal.

[0995] Examples of prompt statements

[0996] 1. "Select a science experiment course and generate feedback after completing the video tutorial."

[0997] 2. "What are some examples of relaxation content for users when they are feeling stressed?"

[0998] 3. "Generate notification messages for offline events."

[0999] This system allows users to learn STEAM education interactively and effectively, and provides an optimal learning environment based on their emotions and customized feedback according to their progress and level of understanding.

[1000] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1001] Step 1:

[1002] The user accesses the dedicated terminal and opens the account creation screen.

[1003] (Input): None

[1004] (Output): Account creation screen

[1005] (Specific operation): The user opens the browser on the dedicated device, enters the system URL, and displays the account creation screen.

[1006] Step 2:

[1007] The user enters their name, school name, grade, and email address and presses the "Register" button.

[1008] (Input): User's name, school name, grade, email address

[1009] (Output): Personal information entered

[1010] (Specific operation): The user enters the required information into the form on the account creation screen and clicks the "Register" button.

[1011] Step 3:

[1012] The terminal sends the input information to the server.

[1013] (Input): User's personal information

[1014] (Output): User information sent to the server

[1015] (Specific operation): The device sends the data entered in the form to the server via an HTTPS POST request. The server receives the data.

[1016] Step 4:

[1017] The server stores the input information in a database.

[1018] (Input): User's personal information

[1019] (Output): User information stored in the database

[1020] (Specific operation): The server saves the received user information in a database (e.g., MySQL or PostgreSQL) using an INSERT statement.

[1021] Step 5:

[1022] The server generates authentication information such as a user ID and password and sends it to the terminal.

[1023] (Input): User's personal information

[1024] (Output): Generated authentication information (user ID, password)

[1025] (Specific operation): The server generates a unique user ID from the user information, sets a password in a secure manner, and sends the generated authentication information to the terminal as an HTTPS response.

[1026] Step 6:

[1027] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[1028] (Input): Generated credentials

[1029] (Output): Registration completion notification and displayed authentication information

[1030] (Specific operation): The terminal displays the received authentication information on the screen in a format that is easy for the user to understand, and notifies the user that "registration is complete."

[1031] Step 7:

[1032] A user enters their credentials to log into an online platform.

[1033] (Input): User ID, password

[1034] (Output): Login information

[1035] (Specific operation): The user enters the user ID and password on the login screen and clicks the "Login" button.

[1036] Step 8:

[1037] The device sends the login information to the server.

[1038] (Input): User ID, password

[1039] (Output): Login information sent to the server

[1040] (Specific operation): The device sends the entered login information to the server via an HTTPS POST request.

[1041] Step 9:

[1042] The server verifies the authentication information.

[1043] (Input): Login information

[1044] (Output): Authentication result (success or failure)

[1045] (Specific operation): The server checks the received login information against the database to verify its validity. If successful, it returns an authentication success message to the terminal, and if unsuccessful, it returns an error message.

[1046] Step 10:

[1047] If the authentication is successful, the server sends the dashboard data to the terminal.

[1048] (Input): Authentication success result

[1049] (Output): Dashboard data

[1050] (Specific operation): After successful authentication, the server generates the necessary information for the user's dashboard screen (recommended courses, learning progress, etc.) and sends it to the device as an HTTPS response.

[1051] Step 11:

[1052] The device displays the dashboard.

[1053] (Input): Dashboard data

[1054] (Output): Dashboard displayed

[1055] (Specific operation): The dashboard data received by the device is displayed on the screen.

[1056] Step 12:

[1057] The user opens the "Course Selection" menu and selects the desired course.

[1058] (Input): User selection

[1059] (Output): Selected course information

[1060] (Specific operation): The user clicks on the "Course Selection" menu on the dashboard screen and selects the desired course from the displayed list of courses.

[1061] Step 13:

[1062] The terminal transmits the course selection information to the server.

[1063] (Input): Selected course information

[1064] (Output): Course selection information sent to the server

[1065] (Specific operation): The device sends the selected course information to the server via an HTTPS POST request.

[1066] Step 14:

[1067] The server generates the optimal learning plan for the user and sends it to the device.

[1068] (Input): Selected course information, user information

[1069] (Output): Generated lesson plan

[1070] (Specific operation): The server uses an AI model to generate an optimal learning plan based on the selected course information and the user's learning history, and sends it to the device as an HTTPS response.

[1071] Step 15:

[1072] The device displays the learning plan and the user begins the learning module.

[1073] (Input): Generated lesson plan

[1074] (Output): Displayed learning plan, start of learning module

[1075] (Specific operation): The device displays the received learning plan on the screen, and the user selects and starts a learning module.

[1076] Step 16:

[1077] Users progress through the learning content on dedicated devices.

[1078] (Input): Learning plan content

[1079] (Output): Learning progress data

[1080] (Specific actions): The user watches a video tutorial and proceeds through experiments and quizzes according to the learning content.

[1081] Step 17:

[1082] The device records experimental results and quiz answers in real time and sends them to the server.

[1083] (Input): Experiment results, quiz answers

[1084] (Output): Submitted experiment results, quiz answers

[1085] (Specific operation): The device records the user's input and selections in real time and sends them to the server via an HTTPS POST request.

[1086] Step 18:

[1087] The server analyzes the received data and evaluates the user's level of understanding and progress.

[1088] (Input): Experiment results, quiz answers

[1089] (Output): Analysis results (level of understanding, progress evaluation)

[1090] (Specific operation): The server analyzes the data received using an AI model to evaluate the user's learning comprehension and progress.

[1091] Step 19:

[1092] The server generates feedback based on the evaluation results and sends it to the terminal.

[1093] (Input): Understanding level, progress evaluation

[1094] (Output): Generated feedback

[1095] (Specific operation): Based on the evaluation results, the server generates a specific feedback message and sends it to the terminal as an HTTPS response.

[1096] Step 20:

[1097] The device displays feedback to the user, who can use it to improve and take next steps as they progress through their learning.

[1098] (Input): Feedback message

[1099] (Output): The displayed feedback message

[1100] (Specific operation): The device displays the received feedback message on the screen and notifies the user of the next learning step or areas for improvement.

[1101] Step 21:

[1102] A user posts a question using a mobile app.

[1103] (Input): Question

[1104] (Output): Question sent

[1105] (Specific action): A user types a question using the chat function of a mobile app and presses the "Send" button.

[1106] Step 22:

[1107] The device sends the question to the server.

[1108] (Input): Question

[1109] (Output): The question sent to the server

[1110] (Specific operation): The device sends the entered question to the server via an HTTPS POST request.

[1111] Step 23:

[1112] The server analyzes the question, generates an appropriate answer, and sends it to the device.

[1113] (Input): Question

[1114] (Output): Generated answer

[1115] (Specific operation): The server analyzes the received question using natural language processing (NLP), generates an appropriate answer using an AI model, and sends it to the device.

[1116] Step 24:

[1117] The device displays the answer for the user to review, and if they are not satisfied, they can post the question again.

[1118] (Input): Generated answer

[1119] (Output): Displayed answer, re-ask if necessary

[1120] (Specific operation): The device displays the received answer on the screen for the user to confirm. If the user is not satisfied, they can re-enter the question and submit it.

[1121] Step 25:

[1122] The server periodically generates and sends offline event notifications to the terminal.

[1123] (Input): None

[1124] (Output): Generated notification

[1125] (Specific operation): The server generates detailed information about offline events based on a pre-set schedule and sends notifications to the device.

[1126] Step 26:

[1127] The user receives a notification and registers for the event.

[1128] (Input): Notification content

[1129] (Output): Participant registration information

[1130] (Specific actions): The user checks the notification sent to them, and if they wish to participate, they enter the necessary information into the dedicated form and press the "Register" button.

[1131] Step 27:

[1132] The terminal transmits the participation registration information to the server.

[1133] (Input): Registration information

[1134] (Output): Registration information sent to the server

[1135] (Specific operation): The device sends the entered participation registration information to the server via an HTTPS POST request.

[1136] Step 28:

[1137] The server generates a participation confirmation message and sends it to the terminal to notify the user.

[1138] (Input): Registration information

[1139] (Output): The generated confirmation message

[1140] (Specific operation): The server stores the received registration information in a database, generates a participation confirmation message and sends it to the terminal, which then displays the confirmation message to the user.

[1141] Step 29:

[1142] While the user is learning, the emotion engine collects emotion data in real time.

[1143] (Input): User's facial expression, tone of voice

[1144] (Output): Real-time emotion data

[1145] (Specific operation): Using the camera and microphone of the dedicated device, the user's facial expressions and tone of voice are sensed in real time and collected as emotional data.

[1146] Step 30:

[1147] The server analyzes the emotional data and adjusts the learning content according to the user's emotional state.

[1148] (Input): Emotion data

[1149] (Output): Tailored learning content

[1150] (Specific operation): Based on the emotional data received by the server, the system evaluates the user's stress level and concentration level, and adjusts the difficulty of the learning content. It may also add relaxation content.

[1151] Step 31:

[1152] The terminal displays the adjusted content to the user.

[1153] (Input): Tailored learning content

[1154] (Output): The adjusted content displayed

[1155] (Specific operation): The device displays tailored learning and relaxation content on the screen and provides it to the user.

[1156] Through the above processing steps, the system can provide users with interactive and effective STEAM education, providing appropriate feedback and adjusting the learning environment according to individual progress and level of understanding.

[1157] (Application example 2)

[1158] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1159] Traditional STEAM education platforms and mobile apps focus on managing users' learning progress and providing feedback, but lack the emotional analysis capabilities to personalize the learning experience. As a result, they often provide uniform content without considering users' stress levels or concentration levels, making effective learning difficult. They also struggle to efficiently respond to questions in real time or manage offline events. A comprehensive system that can resolve these issues and provide a more personalized learning experience is needed.

[1160] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1161] In this invention, the server includes means for recording and analyzing the user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, and emotion analysis means for collecting the user's emotion data and analyzing their stress level and concentration level to optimize the learning experience. This makes it possible to analyze the user's learning progress in real time and provide optimal content based on the emotion data.

[1162] 1. "User interface" refers to the screens and operating means through which users register an account and enter personal information and learning objectives.

[1163] 2. "Dedicated Device" means a hardware device or software application used to manage the user's selected course and learning progress.

[1164] 3. "Means for recording and analyzing learning progress and results in real time" refers to algorithms or computer programs for instantly storing and analyzing a user's learning progress and results in a database.

[1165] 4. "Means for providing feedback" refers to a method or system for returning advice or instructions regarding the user's learning based on the analysis results.

[1166] 5. "Means for posting questions, analyzing the content of the questions, and providing answers" refers to a system that allows users to submit questions via a mobile app, analyzes the questions using natural language processing technology, and generates appropriate answers.

[1167] 6. "Event Management Means" means a method or software that allows a user to register to participate in an offline event and manage detailed information about that event.

[1168] 7. "Emotion analysis means" refers to technologies and systems that collect users' emotional data, analyze their stress levels and concentration levels, and optimize learning content.

[1169] This invention is a system that realizes a comprehensive online platform and mobile application that supports STEAM education for elementary, middle, and high school students. The entire system is composed as follows.

[1170] User Registration

[1171] The user accesses the dedicated terminal and first opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and then presses the "Register" button. The terminal sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal displays this authentication information on the screen and notifies the user that registration is complete.

[1172] Course Selection and Learning Management

[1173] The user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal then sends the course selection information to the server, which generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[1174] Learning progression and feedback

[1175] Users progress through learning content on a dedicated device. For example, they watch video tutorials and conduct scientific experiments. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. Based on the evaluation results, the server generates feedback and sends it to the device. The device displays the feedback to the user, who can use it as a reference for areas for improvement and next steps to continue their learning.

[1176] Question and Answer Support

[1177] When a user posts a question using the mobile app, the application sends the question to a server. The server uses a generative AI model to analyze the question and generate an appropriate answer. This answer is then sent to the device, where it is displayed to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to an expert.

[1178] Participating in offline events

[1179] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which stores it in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[1180] Sentiment analysis and customised learning experiences

[1181] During learning, the server uses emotion analysis to collect the user's emotional data. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice data. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their learning, it presents more difficult tasks.

[1182] Hardware and software used

[1183] The system uses the following elements:

[1184] React Native is used for the user interface of the dedicated terminal.

[1185] The backend development uses the Django framework and PostgreSQL.

[1186] TensorFlow is used for sentiment analysis.

[1187] Generative AI models such as ChatGPT API are used to analyze questions and generate answers.

[1188] Examples of concrete examples and prompts

[1189] For example, if a junior high school student user selects the "Science Experiment" course, the server generates a learning plan including a video tutorial and experiment procedures. The user watches the video and records the experiment results on their device. The server receives and analyzes the results in real time and generates appropriate feedback. At the same time, an emotion analysis means detects the user's stress level from facial expression data and adjusts the learning content as needed.

[1190] Example prompt sentence:

[1191] "Analyze the stress level of the user from facial expression data."

[1192] "Please analyze the level of concentration from user voice data."

[1193] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1194] Step 1: User Registration

[1195] The user opens the account creation screen using a dedicated terminal. They enter required information such as name, school name, grade, and email address, and press the "Register" button. The input data is sent from the terminal to the server. The server receives it, stores it in a database, and generates authentication information such as a user ID and password. The generated authentication information is sent from the server to the terminal, and the terminal notifies the user.

[1196] Input data: Name, school name, grade, email address

[1197] Output data: User ID, password

[1198] Step 2: Log in and view the dashboard

[1199] The user enters authentication information on the dedicated terminal and presses the login button. The terminal sends the login information to the server, which then verifies the authentication information. If authentication is successful, the server sends data about the user's learning course and progress to the terminal as dashboard screen data. The terminal displays this data.

[1200] Input data: User ID, password

[1201] Output data: Dashboard screen data

[1202] Step 3: Select your courses and create your study plan

[1203] The user opens the "Course Selection" menu from the dashboard and selects the desired course. The device sends the course selection information to the server. The server generates an optimal learning plan for the user and sends the learning plan data to the device. The device displays the learning plan, and the user starts the learning module.

[1204] Input data: Course selection information

[1205] Output data: Learning plan data

[1206] Step 4: Learning progression and data recording

[1207] Users progress through learning content on a dedicated device. When watching video tutorials or conducting scientific experiments, the device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress.

[1208] Input data: Experiment results, quiz answers

[1209] Output data: Comprehension assessment, progress data

[1210] Step 5: Generate and display feedback

[1211] The server generates feedback based on the understanding assessment and progress data. The generated feedback is sent to the device, which displays it to the user. The user can then refer to the feedback to identify areas for improvement and next steps.

[1212] Input data: Comprehension assessment, progress data

[1213] Output Data: Feedback

[1214] Step 6: Sentiment Analysis

[1215] While learning, a dedicated device records the user's facial expressions and voice in real time to collect emotional data. This data is sent from the device to a server, which uses an emotion analysis model to evaluate the user's stress level and concentration. Based on the evaluation results, the server adjusts the difficulty of the learning content.

[1216] Input data: facial expression data, voice data

[1217] Output data: stress level, concentration evaluation

[1218] Step 7: Questions and Answers

[1219] When a user posts a question using a mobile app, the device sends the question to the server, which uses a generative AI model to analyze the question and generate an appropriate answer. The generated answer is then sent to the device, which displays it to the user.

[1220] Input data: Question content

[1221] Output data: Answer

[1222] Step 8: Offline event management

[1223] The server periodically generates notifications for offline events and sends them to the device. The user receives the notification, enters the necessary information in the application form, and presses the "Register" button. The device sends the participation registration information to the server, which stores it in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[1224] Input data: Event details, application information

[1225] Output data: Participation confirmation message

[1226] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1227] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1228] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1229] [Third embodiment]

[1230] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1231] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1232] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1233] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1234] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1235] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1236] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1237] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1238] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1239] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1240] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1241] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1242] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives, a function for managing the user's selected courses and learning progress via a dedicated terminal, a function for recording and analyzing the user's learning progress and results in real time, a function for providing feedback based on the analysis results, a function for posting questions via the mobile app, analyzing them, and providing answers, and an event management function for participating in offline events. However, this does not involve writing program code.

[1243] The overall system flow is as follows:

[1244] User Registration

[1245] First, the user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal then sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[1246] Course Selection and Learning Management

[1247] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[1248] Learning progression and feedback

[1249] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[1250] Communication and Support

[1251] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[1252] Participating in offline events

[1253] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[1254] Specific examples

[1255] For example, if a junior high school student user selects the "Science Experiment" course and proceeds through the program, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. The user can continue learning by using this as a reference when reviewing the course or moving on to the next module.

[1256] This system allows users to learn STEAM education interactively and effectively, and provides customized feedback based on their progress and level of understanding.

[1257] The processing flow will be explained below.

[1258] Step 1:

[1259] The user accesses the dedicated terminal, opens the account creation screen, enters required information such as name, school name, grade, and email address, and presses the "Register" button.

[1260] Step 2:

[1261] The terminal sends the input information to the server.

[1262] Step 3:

[1263] The server stores the received user information in a database and generates authentication information such as a user ID and password.

[1264] Step 4:

[1265] The server sends the generated authentication information to the terminal.

[1266] Step 5:

[1267] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[1268] Step 6:

[1269] To log in to the online platform, the user enters their authentication information and presses the login button.

[1270] Step 7:

[1271] The device sends the login information to the server.

[1272] Step 8:

[1273] The server checks the authentication information, and if authentication is successful, it sends the dashboard screen data (learning progress, recommended courses, etc.) to the device.

[1274] Step 9:

[1275] The terminal displays the dashboard screen, and the user opens the "Course Selection" menu.

[1276] Step 10:

[1277] The terminal sends a request for a course list to the server.

[1278] Step 11:

[1279] The server sends a list of available STEAM courses to the device.

[1280] Step 12:

[1281] The user selects the desired course from the course list.

[1282] Step 13:

[1283] The terminal transmits the selected course information to the server.

[1284] Step 14:

[1285] The server generates a learning plan based on the selected course and transmits it to the terminal.

[1286] Step 15:

[1287] The device displays the learning plan, and the user clicks the "Start" button to display the first learning module.

[1288] Step 16:

[1289] Users watch learning content (e.g., video tutorials) and complete experiments and quizzes.

[1290] Step 17:

[1291] The device records the user's learning data (experiment results, quiz answers, etc.) in real time and sends it to the server.

[1292] Step 18:

[1293] The server analyzes the data it receives and evaluates the user's learning progress and level of understanding.

[1294] Step 19:

[1295] The server generates feedback based on the evaluation results and sends it to the terminal.

[1296] Step 20:

[1297] The device displays feedback to the user and prompts them to relearn or proceed to the next module as needed.

[1298] Step 21:

[1299] A user uses the mobile app to post a question via the LINE chat function.

[1300] Step 22:

[1301] The terminal sends the user's question to the server.

[1302] Step 23:

[1303] The server analyzes the question, generates an appropriate answer, and sends it to the terminal.

[1304] Step 24:

[1305] The terminal displays the answer received from the server to the user.

[1306] Step 25:

[1307] If the user is not satisfied with the answer, they can post the question again and the server will escalate the question to an expert or employee.

[1308] Step 26:

[1309] The server periodically generates and sends offline event notifications to the terminal.

[1310] Step 27:

[1311] Users receive event notifications and view detailed information on mobile apps and online platforms.

[1312] Step 28:

[1313] A user registers to participate in an offline event, enters information in the application form, and presses the "Register" button.

[1314] Step 29:

[1315] The terminal transmits the participation registration information to the server.

[1316] Step 30:

[1317] The server stores the registration information in a database and generates a confirmation message.

[1318] Step 31:

[1319] The server sends a participation confirmation message to the terminal and notifies the user.

[1320] Example 1

[1321] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1322] Current STEAM education support systems struggle to effectively manage users' learning progress and comprehension, and provide individually customized feedback. They also lack systems that allow users to ask questions in real time and receive immediate answers. Furthermore, they lack a means to seamlessly link online and offline learning environments. This makes it difficult for users to progress efficiently.

[1323] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1324] In this invention, the server includes means for recording and analyzing a user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, and means for the user to post questions via a mobile app, analyze the questions, and provide answers. This makes it possible to instantly check a user's learning progress and suggest areas for improvement, thereby providing individually customized learning support. Furthermore, the real-time exchange of questions and answers via the mobile app allows users to quickly resolve their doubts and enable efficient learning. Furthermore, seamless integration of online and offline events can increase users' motivation to learn and encourage continuous learning.

[1325] "User" refers to a person who uses the system to carry out learning activities.

[1326] "Dedicated Terminal" refers to a specific electronic device used to access the system.

[1327] "Mobile app" refers to application software that runs on smartphones and tablets.

[1328] "Server" refers to a central computer system that stores, processes, and serves data.

[1329] "User interface" refers to the screen and operating means that allow a user to directly interact with a system.

[1330] "Learning curriculum" refers to various learning contents and courses selected by the user.

[1331] "Study progress" refers to the progress of a user in a learning curriculum selected by the user.

[1332] "Learning Content" refers to information and materials such as videos, texts, and quizzes provided to support learning.

[1333] "Feedback" refers to information that provides an evaluation of a user's learning activities and suggestions for improvement.

[1334] "Analysis" refers to the process of processing collected data to derive useful information or results.

[1335] "Non-online events" refer to learning or education-related events that take place in a physical location.

[1336] A "generative AI model" refers to an artificial intelligence algorithm that generates data-based feedback to track a user's learning progress and provide adaptive feedback.

[1337] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system provides a user interface for users to register an account and enter personal information and learning objectives. Specific hardware includes dedicated terminals, mobile terminals, and servers. Software includes database management software (e.g., MySQL or PostgreSQL), real-time data analysis software (e.g., Apache Kafka), front-end frameworks (e.g., React or Angular), and mobile app development frameworks (e.g., Flutter or React Native).

[1338] System configuration

[1339] 1. User Registration and Authentication

[1340] A user accesses a dedicated terminal and creates an account by entering profile information. The terminal sends the user's input information to the server, which stores the received information in a database. The server generates authentication information and sends it to the terminal. The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[1341] 2. Course Selection and Learning Management

[1342] The user logs in to a dedicated terminal using authentication information, and the server verifies the authentication information. If authentication is successful, the server sends the user's dedicated dashboard data to the terminal. The user selects the desired learning curriculum from the dashboard, and the terminal sends the selection information to the server. The server generates an optimal learning plan based on the user's selection and sends it to the terminal. The terminal displays the generated learning plan, and the user begins learning.

[1343] 3. Learning and Feedback

[1344] Users progress through learning content (e.g., video tutorials, textbooks, quizzes) provided on dedicated devices. The devices record the user's learning progress and quiz answers in real time and send them to the server. The server analyzes the received data and evaluates the user's level of understanding and progress. Feedback generated based on the evaluation is sent to the device, which then displays it to the user.

[1345] 4. Real-time Q&A

[1346] Users use a mobile app to post questions. The questions are sent from the device to a server, which analyzes them. An appropriate answer is generated and sent to the device. The user can review the answer and ask the question again if necessary. Complex questions are answered by experts or employees.

[1347] 5. Participating in offline events

[1348] The server periodically generates notifications for non-online events and sends them to the device. The user receives the notifications and views the details. When registering for an event, the user enters the required information and the device sends the registration information to the server. The server stores the information in a database and sends a participation confirmation message to the device.

[1349] Specific examples

[1350] For example, if a junior high school student user selects the "Science Experiment" course and uses the system, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. The user can continue learning by using this as a reference when reviewing the course or moving on to the next module.

[1351] Prompt Sentence Examples

[1352] "Please tell me the specific steps to provide lesson plans and video tutorials for a science lab course for middle school students to learn basic scientific principles."

[1353] The present invention allows users to learn STEAM education interactively and effectively, and receive customized feedback based on their progress and level of understanding, maximizing learning efficiency.

[1354] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1355] Step 1: User registers for an account

[1356] The user accesses the dedicated terminal and opens the account creation screen.

[1357] Input: The user enters required information such as name, school name, grade, and email address.

[1358] The terminal sends the input information to the server.

[1359] Output: User information is saved in the database on the server side. Authentication information (user ID, password) is generated and sent to the terminal.

[1360] Specific operation: The terminal sends the input information as an HTTP POST request, and the server executes an SQL INSERT statement against the database.

[1361] Step 2: User enters credentials and logs in

[1362] The user enters authentication information (user ID, password) on the dedicated terminal and presses the login button.

[1363] Input: The user enters the user ID and password.

[1364] The device sends the login information to the server.

[1365] Output: The server verifies the authentication information and, if successful, sends the dashboard data to the device.

[1366] Specific operation: The terminal sends an HTTP POST request, and the server searches for user information in the database and compares the password.

[1367] Step 3: User makes course selection

[1368] After logging in, the user selects the "Course Selection" menu from the dashboard and chooses the desired learning curriculum.

[1369] Input: The user selects the desired learning curriculum.

[1370] The terminal transmits the course selection information to the server.

[1371] Output: The server generates the optimal learning plan for the user and sends it to the device.

[1372] How it works: The device sends an HTTP POST request to the server containing the selected course information, and the server uses an algorithm to generate a learning plan.

[1373] Step 4: Users progress through the learning content

[1374] The user starts the provided learning content on a dedicated device.

[1375] Input: User watches video tutorials, performs experiments and quizzes.

[1376] The device records the user's progress and quiz answers and sends them to the server.

[1377] Output: The server receives the data in real time and generates feedback based on the analysis results.

[1378] Specific operation: The device records user operation data in real time and sends it to the server via HTTP POST request. The server then runs an analysis algorithm to evaluate the data.

[1379] Step 5: Provide feedback based on the analysis results

[1380] The server analyzes the user's learning data and evaluates their level of understanding and progress.

[1381] Input: User learning progress data and quiz answer data.

[1382] The server generates the analysis results and sends feedback to the device.

[1383] Output: The device displays feedback to the user.

[1384] Specific operation: The server analyzes the data in the database, generates a feedback message based on the evaluation criteria, and sends the generated feedback to the terminal as an HTTP response.

[1385] Step 6: Users post questions and get answers

[1386] A user posts a question using a mobile app.

[1387] Input: The user enters the question and clicks the post button.

[1388] The device sends the question to the server.

[1389] Output: The server analyzes the question, generates an appropriate answer, and sends it to the device.

[1390] Specific operation: The device sends an HTTP POST request containing a question to the server, and the server analyzes the question using a natural language processing algorithm, generates an answer, and sends the answer to the device as an HTTP response.

[1391] Step 7: User registers for a non-online event

[1392] The server periodically generates and sends notifications of non-online events to the terminal.

[1393] Input: Event details and registration information.

[1394] Users receive notifications, view details, and register to attend.

[1395] Output: The terminal sends the participation registration information to the server, which stores the information in a database. The server generates a participation confirmation message and sends it to the terminal.

[1396] Specific operation: The server periodically generates an event notification and sends it to the terminal as an HTTP response. The user registers for participation, and the terminal sends it to the server as an HTTP POST request. The server executes an SQL INSERT statement in the database, generates a participation confirmation message, and sends it to the terminal.

[1397] (Application example 1)

[1398] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1399] Conventional STEAM education systems have limited functionality for managing learners' progress and providing feedback, making real-time analysis and customized instruction difficult. They also lack sufficient management of interactive virtual learning experiences and offline events. Furthermore, the adoption of new learning methods utilizing wearable devices has been slow, creating a need for ways to maximize the effectiveness of education.

[1400] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1401] In this invention, the server includes: means for providing a user interface for users to register an account and input personal information and learning objectives; means for managing the user's selected course and learning progress via a dedicated terminal; means for recording and analyzing the user's learning progress and results in real time; means for providing feedback to the user based on the analysis results; means for users to post questions via a mobile app or a wearable device (smart glasses or a head-mounted display), analyze the questions, and provide answers; means for conducting experiments in a virtual space, simulating the experiment results, and providing feedback in real time; and event management means for displaying regular offline event notifications and detailed information and registering participants. This enables an interactive learning experience using a variety of devices, realizes real-time analysis of learning progress, and provides customized feedback, thereby improving the quality of education and maximizing learning effectiveness.

[1402] "A user interface for users to register an account and enter personal information and learning objectives" is a means of providing an operation screen for users to create their own account in an educational system and enter personal information and learning objectives.

[1403] "Means for managing a user's selected course and learning progress via a dedicated terminal" refers to a means in an educational system that provides a user with the ability to track and manage their selected learning course and learning progress using a dedicated device.

[1404] "Means for recording and analyzing a user's learning progress and results in real time" refers to a means for providing a function in an educational system to collect a user's learning progress and results as data in real time and analyze them using statistical or machine learning methods.

[1405] "Means for providing feedback to the user based on the analysis results" refers to means for providing the user with a function for suggesting areas for improvement and next learning steps based on the learning data analyzed in the educational system.

[1406] "A means for users to post questions via a mobile app or wearable device, and for the content of the question to be analyzed and an answer to be provided" refers to a means in an educational system that allows users to send questions using a smartphone app or a wearable device such as smart glasses, and provides a function that analyzes the content of the question using a natural language processing model or the like and provides an appropriate answer.

[1407] "Means for conducting experiments in a virtual space, simulating the experimental results, and providing feedback in real time" refers to a means for realizing the function in an educational system in which a user conducts an experiment in a virtual reality or mixed reality space, analyzes the results using computer simulation, and provides feedback in real time.

[1408] "Event management means for displaying regular offline event notifications and detailed information, and registering for participation" refers to a means for providing users in an educational system with regular notifications of offline event information, allowing them to view detailed information, and providing the functionality for registering for participation.

[1409] "Means for analyzing data in real time using cloud computing and providing optimal feedback using generative artificial intelligence" refers to a means in an educational system that uses cloud services via the internet to process and analyze collected data in real time and generate and provide optimal feedback to users using machine learning models.

[1410] "Means for providing an interactive learning experience using mixed reality technology to create a virtual space" refers to a means for using mixed reality (MR) technology in an educational system to create a learning environment that combines reality and virtual space, and to provide users with an interactive and immersive learning experience.

[1411] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students.

[1412] Overall structure

[1413] 1. User Registration:

[1414] The server provides a user interface for users to enter personal information such as name, school name, grade, and email address, as well as their learning goals. When a user registers an account from a dedicated terminal, the terminal sends the information to the server, which stores it in a database. After registration is complete, the server generates a user ID and password and sends them to the terminal.

[1415] 2. Course Selection and Learning Management:

[1416] The server manages the courses selected by the user via a dedicated terminal and their learning progress. When the user logs in to the online platform and selects a course, the server generates an optimal learning plan and sends the data to the terminal. The terminal displays the learning plan, and the user starts the learning module.

[1417] 3. Learning progression and feedback:

[1418] Users progress through learning content on their devices. The devices record the user's learning progress and results (e.g., scientific experiment results and quiz answers) in real time and send them to a cloud server. The server analyzes the data, generates feedback based on the analysis results, and sends it to the device. Users can also conduct experiments in virtual space, simulate the results, and receive feedback in real time.

[1419] 4. Real-time communication and support:

[1420] When a user posts a question using a mobile app or a wearable device (smart glasses or a head-mounted display), the device sends the question to a server, which then analyzes the question using a generative AI model (e.g., an NLP model), generates an appropriate answer, and sends it to the device.

[1421] 5. Offline event notification and participation management:

[1422] The server periodically generates notifications for offline events and sends them to the terminal. When a user receives the notification, views detailed information, and registers for participation, the terminal sends the participation registration information to the server, which stores the information in a database. A participation confirmation message is sent to the terminal to notify the user.

[1423] Hardware and Software Use

[1424] Hardware:

[1425] Dedicated devices (PC, smartphone, smart glasses, head-mounted display)

[1426] Server (cloud-based, e.g. AWS, Google Cloud)

[1427] software:

[1428] Frontend: React Native, Flutter, Unity engine

[1429] Backend: Express.js, Firebase Functions

[1430] Database: MongoDB, Firestore database

[1431] Analysis: TensorFlow, PyTorch, AWS Lambda functions

[1432] Communication: LINE API, Twilio Chat API

[1433] Specific examples

[1434] Examples of how the virtual laboratory can be used:

[1435] 1. Scenario: A middle school student user selects the "Virtual Lab" course.

[1436] 2. Procedure: Users simulate scientific experiments in a virtual space built with the Unity engine. The results are analyzed in real time in the Azure cloud, and feedback is generated using Azure ML.

[1437] 3. Questions: When a user asks a question about something they don't understand, the question is sent using the LINE chat function, and an answer analyzed by the BERT model is provided in real time.

[1438] 4. Offline Events: Users receive offline event notifications, view details, and register to attend.

[1439] Example prompts to input to a generative AI model:

[1440] Prompt: We are developing a STEAM education application for elementary, middle, and high school students. The app allows users to conduct experiments in a virtual space, analyzes the results in real time, and provides feedback. Additionally, the app uses an NLP model to generate answers when users post questions. It also periodically notifies users of offline events. Please explain in detail the use case for this app and the technologies required.

[1441] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1442] Step 1:

[1443] User Registration:

[1444] The user accesses the account registration screen from a dedicated terminal and enters personal information such as name, school name, grade, and email address. The terminal sends this input information to the server. The server stores the received information in a database, generates a user ID and password, and sends them to the terminal. The terminal then displays a notification to the user that registration is complete. The input is the user's personal information, and the output is the user information stored in the database and the generated user ID and password.

[1445] Step 2:

[1446] Course Selection and Learning Management:

[1447] The user enters authentication information to log in to the online platform from a dedicated terminal and presses the login button. The terminal sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. The user opens the "Course Selection" menu from the dashboard and selects the desired course. The terminal sends the course selection information to the server, and the server generates an optimal learning plan for the user and sends it to the terminal. The input is the user's authentication information and course selection information, and the output is the dashboard screen data and learning plan.

[1448] Step 3:

[1449] Learning progression and feedback:

[1450] The user progresses through the learning content on the device. The device records the user's learning progress and results (e.g., scientific experiment results or quiz answers) in real time and sends them to a cloud server. The server analyzes the data in real time (e.g., using a machine learning model), generates feedback based on the analysis results, and sends it to the device. The device displays the feedback to the user. The input is the user's learning progress and results, and the output is feedback based on the analysis results.

[1451] Step 4:

[1452] Real-time communication and support:

[1453] A user posts a question using a mobile app or a wearable device (smart glasses or a head-mounted display). The device sends the question to the server, which analyzes the question using a generative AI model (e.g., an NLP model). An appropriate answer is generated and sent to the device. The device displays the answer to the user. The input is the user's question, and the output is the analyzed answer.

[1454] Step 5:

[1455] Virtual experiments and simulations:

[1456] Users conduct scientific experiments in a virtual space using dedicated terminals or wearable devices. The experimental results are recorded in real time on the terminal and sent to a cloud server. The server simulates the data and generates analytical results. Feedback is generated in real time and sent to the terminal. The terminal displays the feedback to the user. The input is the results of the virtual experiment, and the output is feedback based on the simulation and analytical results.

[1457] Step 6:

[1458] Offline event notification and participation management:

[1459] The server periodically generates offline event notifications and sends them to the terminal. When a user receives the notification, views detailed information, and registers to participate, the terminal sends the registration information to the server, which stores the information in a database. A participation confirmation message is sent to the terminal, notifying the user. The input is the offline event notification and the user's registration information, and the output is the registration information stored in the database and the participation confirmation message.

[1460] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1461] This invention provides a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives; a function for managing the user's selected courses and learning progress via a dedicated terminal; a function for recording and analyzing the user's learning progress and results in real time; a function for providing feedback based on the analysis results; a function for posting questions via the mobile app, analyzing them, and providing answers; and an event management function for participating in offline events. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system provides a more personalized learning experience and more effective educational support.

[1462] The overall system flow is as follows:

[1463] User Registration

[1464] First, the user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal then sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[1465] Course Selection and Learning Management

[1466] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[1467] Learning progression and feedback

[1468] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[1469] Communication and Support

[1470] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[1471] Participating in offline events

[1472] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[1473] Customizing the learning experience with an emotional engine

[1474] While the user is studying, the emotion engine collects emotional data in real time. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their studies, it presents more difficult tasks. This ensures that the user's learning experience is always optimal.

[1475] Specific examples

[1476] For example, if a junior high school student user selects a "science experiment" course and proceeds with the program, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then actually performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. Furthermore, the emotion engine detects stress from the user's facial expressions and sends instructions to the server to ease some of the learning as necessary. As a result, the user can proceed with their learning while reducing stress.

[1477] This system allows users to learn STEAM education interactively and effectively, providing an optimal learning environment based on their emotions and customized feedback according to their progress and level of understanding.

[1478] The processing flow will be explained below.

[1479] Step 1:

[1480] The user accesses the dedicated terminal, opens the account creation screen, enters required information such as name, school name, grade, and email address, and presses the "Register" button.

[1481] Step 2:

[1482] The terminal sends the input information to the server.

[1483] Step 3:

[1484] The server stores the received user information in a database and generates authentication information such as a user ID and password.

[1485] Step 4:

[1486] The server sends the generated authentication information to the terminal.

[1487] Step 5:

[1488] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[1489] Step 6:

[1490] To log in to the online platform, the user enters their authentication information and presses the login button.

[1491] Step 7:

[1492] The device sends the login information to the server.

[1493] Step 8:

[1494] The server checks the authentication information, and if authentication is successful, it sends the dashboard screen data (learning progress, recommended courses, etc.) to the device.

[1495] Step 9:

[1496] The terminal displays the dashboard screen, and the user opens the "Course Selection" menu.

[1497] Step 10:

[1498] The terminal sends a request for a course list to the server.

[1499] Step 11:

[1500] The server sends a list of available STEAM courses to the device.

[1501] Step 12:

[1502] The user selects the desired course from the course list.

[1503] Step 13:

[1504] The terminal transmits the selected course information to the server.

[1505] Step 14:

[1506] The server generates a learning plan based on the selected course and transmits it to the terminal.

[1507] Step 15:

[1508] The device displays the learning plan, and the user clicks the "Start" button to display the first learning module.

[1509] Step 16:

[1510] Users watch learning content (e.g., video tutorials) and complete experiments and quizzes.

[1511] Step 17:

[1512] The device records the user's learning data (experiment results, quiz answers, etc.) in real time and sends it to the server.

[1513] Step 18:

[1514] The server analyzes the data it receives and evaluates the user's learning progress and level of understanding.

[1515] Step 19:

[1516] The server generates feedback based on the evaluation results and sends it to the terminal.

[1517] Step 20:

[1518] The device displays feedback to the user and prompts them to relearn or proceed to the next module as needed.

[1519] Step 21:

[1520] A user uses the mobile app to post a question via the LINE chat function.

[1521] Step 22:

[1522] The terminal sends the user's question to the server.

[1523] Step 23:

[1524] The server analyzes the question, generates an appropriate answer, and sends it to the terminal.

[1525] Step 24:

[1526] The terminal displays the answer received from the server to the user.

[1527] Step 25:

[1528] If the user is not satisfied with the answer, they can post the question again and the server will escalate the question to an expert or employee.

[1529] Step 26:

[1530] The server periodically generates and sends offline event notifications to the terminal.

[1531] Step 27:

[1532] Users receive event notifications and view detailed information on mobile apps and online platforms.

[1533] Step 28:

[1534] A user registers to participate in an offline event, enters information in the application form, and presses the "Register" button.

[1535] Step 29:

[1536] The terminal transmits the participation registration information to the server.

[1537] Step 30:

[1538] The server stores the registration information in a database and generates a confirmation message.

[1539] Step 31:

[1540] The server sends a participation confirmation message to the terminal and notifies the user.

[1541] Form using emotion engine

[1542] Step 32:

[1543] During learning, the device collects facial and voice data in real time through the device's camera and microphone.

[1544] Step 33:

[1545] The device transmits the collected emotion data to a server.

[1546] Step 34:

[1547] The server uses an emotion engine to analyze the user's emotion data and evaluate the user's stress level and concentration.

[1548] Step 35:

[1549] The server adjusts the difficulty and progress speed of the learning content based on the results of emotion analysis.

[1550] Step 36:

[1551] The server transmits the adjusted learning content to the terminal.

[1552] Step 37:

[1553] The terminal displays the adjusted learning content to the user, and the user continues learning.

[1554] As a concrete example, when a junior high school student user is working through a "science experiment" course, if the user is concentrating during the experiment, the server will present more difficult experimental tasks, and conversely, if the user is feeling stressed, the server will present easier auxiliary tasks and provide relaxation techniques, allowing the user to always enjoy an optimal learning experience.

[1555] Example 2

[1556] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1557] Conventional educational systems have difficulty providing appropriate feedback in real time based on each user's learning progress and level of understanding, resulting in a decline in user learning efficiency.In addition, they are unable to grasp the stress and concentration levels felt by users in real time and adjust the learning environment accordingly, making it impossible to optimize each individual learning experience.

[1558] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1559] In this invention, the server includes means for providing a user interface for users to register an account and input personal information and learning objectives, means for managing the user's selected course and learning progress via a dedicated terminal, means for recording and analyzing the user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, means for users to post questions via a mobile app and provide answers by analyzing the questions, event management means for users to register for offline events, and means for collecting emotional data while the user is learning and adjusting the learning experience. This enables users to receive appropriate feedback in real time according to their individual progress and level of understanding, and to have their learning environment adjusted based on the emotional data.

[1560] "User" refers to a general user who uses the system to access learning content and manage their progress.

[1561] "Account Registration" refers to the process by which a User registers with the System by providing personal information and authentication information required to access the System.

[1562] "Personal information" refers to information that can be used to identify an individual, such as the user's name, school name, grade, and email address.

[1563] "Learning objectives" refer to the educational goals or needs that users want to achieve by using the system.

[1564] "User interface" refers to the screens and means by which a user logs into a system or enters information.

[1565] "Dedicated Terminal" refers to a device used to access the system and view learning content.

[1566] "Course" refers to a program of study or curriculum aligned with a specific educational objective.

[1567] "Study progress" refers to the progress of a user's studies in a course selected by the user.

[1568] "Results" refers to the achievements and evaluations that users obtain through learning and quizzes.

[1569] "Real-time" refers to immediate responses to user operations and actions.

[1570] "Analysis" refers to the process of creating evaluations and feedback based on collected data.

[1571] "Feedback" refers to evaluations and advice provided based on a user's learning progress and results.

[1572] "Mobile app" refers to software that runs on mobile devices such as smartphones and tablets.

[1573] "Posting a question" refers to the act of a user submitting a question that arises during study to the system.

[1574] "Event management" refers to the process of registering for and providing information about offline events.

[1575] "Emotional data" refers to information about emotions collected from the user's facial expressions, tone of voice, etc.

[1576] "Learning experience" refers to the entire process and environment in which a user learns using a system.

[1577] "Generative artificial intelligence" refers to technology that uses natural language processing and machine learning to analyze data and automatically generate feedback and answers.

[1578] "Educational organization" refers to an organization that carries out educational activities, such as a school or educational institution.

[1579] "High-speed internet" refers to an internet connection that can send and receive large amounts of data quickly.

[1580] "Modern learning equipment" refers to advanced educational devices that utilize modern technology.

[1581] System Overview

[1582] This invention provides a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives. It also includes functions for managing the user's selected courses and learning progress via a dedicated device, recording and analyzing the user's learning progress and results in real time, providing feedback based on the analysis results, posting questions via the mobile app, analyzing them, and providing answers, and event management for participating in offline events. In addition, by combining it with an emotion engine that recognizes the user's emotions, the learning experience is more personalized, providing effective educational support.

[1583] User Registration

[1584] The user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal sends the entered information to the server, which stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[1585] Course Selection and Learning Management

[1586] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[1587] Learning progression and feedback

[1588] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[1589] Communication and Support

[1590] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[1591] Participating in offline events

[1592] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[1593] Customizing the learning experience with an emotional engine

[1594] While the user is studying, the emotion engine collects emotional data in real time. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their studies, it presents more difficult tasks. This ensures that the user's learning experience is always optimal.

[1595] Examples of prompt statements

[1596] 1. "Select a science experiment course and generate feedback after completing the video tutorial."

[1597] 2. "What are some examples of relaxation content for users when they are feeling stressed?"

[1598] 3. "Generate notification messages for offline events."

[1599] This system allows users to learn STEAM education interactively and effectively, and provides an optimal learning environment based on their emotions and customized feedback according to their progress and level of understanding.

[1600] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1601] Step 1:

[1602] The user accesses the dedicated terminal and opens the account creation screen.

[1603] (Input): None

[1604] (Output): Account creation screen

[1605] (Specific operation): The user opens the browser on the dedicated device, enters the system URL, and displays the account creation screen.

[1606] Step 2:

[1607] The user enters their name, school name, grade, and email address and presses the "Register" button.

[1608] (Input): User's name, school name, grade, email address

[1609] (Output): Personal information entered

[1610] (Specific operation): The user enters the required information into the form on the account creation screen and clicks the "Register" button.

[1611] Step 3:

[1612] The terminal sends the input information to the server.

[1613] (Input): User's personal information

[1614] (Output): User information sent to the server

[1615] (Specific operation): The device sends the data entered in the form to the server via an HTTPS POST request. The server receives the data.

[1616] Step 4:

[1617] The server stores the input information in a database.

[1618] (Input): User's personal information

[1619] (Output): User information stored in the database

[1620] (Specific operation): The server saves the received user information in a database (e.g., MySQL or PostgreSQL) using an INSERT statement.

[1621] Step 5:

[1622] The server generates authentication information such as a user ID and password and sends it to the terminal.

[1623] (Input): User's personal information

[1624] (Output): Generated authentication information (user ID, password)

[1625] (Specific operation): The server generates a unique user ID from the user information, sets a password in a secure manner, and sends the generated authentication information to the terminal as an HTTPS response.

[1626] Step 6:

[1627] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[1628] (Input): Generated credentials

[1629] (Output): Registration completion notification and displayed authentication information

[1630] (Specific operation): The terminal displays the received authentication information on the screen in a format that is easy for the user to understand, and notifies the user that "registration is complete."

[1631] Step 7:

[1632] A user enters their credentials to log into an online platform.

[1633] (Input): User ID, password

[1634] (Output): Login information

[1635] (Specific operation): The user enters the user ID and password on the login screen and clicks the "Login" button.

[1636] Step 8:

[1637] The device sends the login information to the server.

[1638] (Input): User ID, password

[1639] (Output): Login information sent to the server

[1640] (Specific operation): The device sends the entered login information to the server via an HTTPS POST request.

[1641] Step 9:

[1642] The server verifies the authentication information.

[1643] (Input): Login information

[1644] (Output): Authentication result (success or failure)

[1645] (Specific operation): The server checks the received login information against the database to verify its validity. If successful, it returns an authentication success message to the terminal, and if unsuccessful, it returns an error message.

[1646] Step 10:

[1647] If the authentication is successful, the server sends the dashboard data to the terminal.

[1648] (Input): Authentication success result

[1649] (Output): Dashboard data

[1650] (Specific operation): After successful authentication, the server generates the necessary information for the user's dashboard screen (recommended courses, learning progress, etc.) and sends it to the device as an HTTPS response.

[1651] Step 11:

[1652] The device displays the dashboard.

[1653] (Input): Dashboard data

[1654] (Output): Dashboard displayed

[1655] (Specific operation): The dashboard data received by the device is displayed on the screen.

[1656] Step 12:

[1657] The user opens the "Course Selection" menu and selects the desired course.

[1658] (Input): User selection

[1659] (Output): Selected course information

[1660] (Specific operation): The user clicks on the "Course Selection" menu on the dashboard screen and selects the desired course from the displayed list of courses.

[1661] Step 13:

[1662] The terminal transmits the course selection information to the server.

[1663] (Input): Selected course information

[1664] (Output): Course selection information sent to the server

[1665] (Specific operation): The device sends the selected course information to the server via an HTTPS POST request.

[1666] Step 14:

[1667] The server generates the optimal learning plan for the user and sends it to the device.

[1668] (Input): Selected course information, user information

[1669] (Output): Generated lesson plan

[1670] (Specific operation): The server uses an AI model to generate an optimal learning plan based on the selected course information and the user's learning history, and sends it to the device as an HTTPS response.

[1671] Step 15:

[1672] The device displays the learning plan and the user begins the learning module.

[1673] (Input): Generated lesson plan

[1674] (Output): Displayed learning plan, start of learning module

[1675] (Specific operation): The device displays the received learning plan on the screen, and the user selects and starts a learning module.

[1676] Step 16:

[1677] Users progress through the learning content on dedicated devices.

[1678] (Input): Learning plan content

[1679] (Output): Learning progress data

[1680] (Specific actions): The user watches a video tutorial and proceeds through experiments and quizzes according to the learning content.

[1681] Step 17:

[1682] The device records experimental results and quiz answers in real time and sends them to the server.

[1683] (Input): Experiment results, quiz answers

[1684] (Output): Submitted experiment results, quiz answers

[1685] (Specific operation): The device records the user's input and selections in real time and sends them to the server via an HTTPS POST request.

[1686] Step 18:

[1687] The server analyzes the received data and evaluates the user's level of understanding and progress.

[1688] (Input): Experiment results, quiz answers

[1689] (Output): Analysis results (level of understanding, progress evaluation)

[1690] (Specific operation): The server analyzes the data received using an AI model to evaluate the user's learning comprehension and progress.

[1691] Step 19:

[1692] The server generates feedback based on the evaluation results and sends it to the terminal.

[1693] (Input): Understanding level, progress evaluation

[1694] (Output): Generated feedback

[1695] (Specific operation): Based on the evaluation results, the server generates a specific feedback message and sends it to the terminal as an HTTPS response.

[1696] Step 20:

[1697] The device displays feedback to the user, who can use it to improve and take next steps as they progress through their learning.

[1698] (Input): Feedback message

[1699] (Output): The displayed feedback message

[1700] (Specific operation): The device displays the received feedback message on the screen and notifies the user of the next learning step or areas for improvement.

[1701] Step 21:

[1702] A user posts a question using a mobile app.

[1703] (Input): Question

[1704] (Output): Question sent

[1705] (Specific action): A user types a question using the chat function of a mobile app and presses the "Send" button.

[1706] Step 22:

[1707] The device sends the question to the server.

[1708] (Input): Question

[1709] (Output): The question sent to the server

[1710] (Specific operation): The device sends the entered question to the server via an HTTPS POST request.

[1711] Step 23:

[1712] The server analyzes the question, generates an appropriate answer, and sends it to the device.

[1713] (Input): Question

[1714] (Output): Generated answer

[1715] (Specific operation): The server analyzes the received question using natural language processing (NLP), generates an appropriate answer using an AI model, and sends it to the device.

[1716] Step 24:

[1717] The device displays the answer for the user to review, and if they are not satisfied, they can post the question again.

[1718] (Input): Generated answer

[1719] (Output): Displayed answer, re-ask if necessary

[1720] (Specific operation): The device displays the received answer on the screen for the user to confirm. If the user is not satisfied, they can re-enter the question and submit it.

[1721] Step 25:

[1722] The server periodically generates and sends offline event notifications to the terminal.

[1723] (Input): None

[1724] (Output): Generated notification

[1725] (Specific operation): The server generates detailed information about offline events based on a pre-set schedule and sends notifications to the device.

[1726] Step 26:

[1727] The user receives a notification and registers for the event.

[1728] (Input): Notification content

[1729] (Output): Participant registration information

[1730] (Specific actions): The user checks the notification sent to them, and if they wish to participate, they enter the necessary information into the dedicated form and press the "Register" button.

[1731] Step 27:

[1732] The terminal transmits the participation registration information to the server.

[1733] (Input): Registration information

[1734] (Output): Registration information sent to the server

[1735] (Specific operation): The device sends the entered participation registration information to the server via an HTTPS POST request.

[1736] Step 28:

[1737] The server generates a participation confirmation message and sends it to the terminal to notify the user.

[1738] (Input): Registration information

[1739] (Output): The generated confirmation message

[1740] (Specific operation): The server stores the received registration information in a database, generates a participation confirmation message and sends it to the terminal, which then displays the confirmation message to the user.

[1741] Step 29:

[1742] As the user learns, the emotion engine collects emotion data in real time.

[1743] (Input): User's facial expression, tone of voice

[1744] (Output): Real-time emotion data

[1745] (Specific operation): Using the camera and microphone of the dedicated device, the user's facial expressions and tone of voice are sensed in real time and collected as emotional data.

[1746] Step 30:

[1747] The server analyzes the emotional data and adjusts the learning content according to the user's emotional state.

[1748] (Input): Emotion data

[1749] (Output): Tailored learning content

[1750] (Specific operation): Based on the emotional data received by the server, the system evaluates the user's stress level and concentration level, and adjusts the difficulty of the learning content. It may also add relaxation content.

[1751] Step 31:

[1752] The terminal displays the adjusted content to the user.

[1753] (Input): Tailored learning content

[1754] (Output): The adjusted content displayed

[1755] (Specific operation): The device displays tailored learning and relaxation content on the screen and provides it to the user.

[1756] Through the above processing steps, the system can provide users with interactive and effective STEAM education, providing appropriate feedback and adjusting the learning environment according to individual progress and level of understanding.

[1757] (Application example 2)

[1758] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1759] Traditional STEAM education platforms and mobile apps focus on managing users' learning progress and providing feedback, but lack the emotional analysis capabilities to personalize the learning experience. As a result, they often provide uniform content without considering users' stress levels or concentration levels, making effective learning difficult. They also struggle to efficiently respond to questions in real time or manage offline events. A comprehensive system that can resolve these issues and provide a more personalized learning experience is needed.

[1760] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1761] In this invention, the server includes means for recording and analyzing the user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, and emotion analysis means for collecting the user's emotion data and analyzing their stress level and concentration level to optimize the learning experience. This makes it possible to analyze the user's learning progress in real time and provide optimal content based on the emotion data.

[1762] 1. "User interface" refers to the screens and operating means through which users register an account and enter personal information and learning objectives.

[1763] 2. "Dedicated Device" means a hardware device or software application used to manage the user's selected course and learning progress.

[1764] 3. "Means for recording and analyzing learning progress and results in real time" refers to algorithms or computer programs for instantly storing and analyzing a user's learning progress and results in a database.

[1765] 4. "Means for providing feedback" refers to a method or system for returning advice or instructions regarding the user's learning based on the analysis results.

[1766] 5. "Means for posting questions, analyzing the content of the questions, and providing answers" refers to a system that allows users to submit questions via a mobile app, analyzes the questions using natural language processing technology, and generates appropriate answers.

[1767] 6. "Event Management Means" means a method or software that allows a user to register to participate in an offline event and manage detailed information about that event.

[1768] 7. "Emotion analysis means" refers to technologies and systems that collect users' emotional data, analyze their stress levels and concentration levels, and optimize learning content.

[1769] This invention is a system that realizes a comprehensive online platform and mobile application that supports STEAM education for elementary, middle, and high school students. The entire system is composed as follows.

[1770] User Registration

[1771] The user accesses the dedicated terminal and first opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and then presses the "Register" button. The terminal sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal displays this authentication information on the screen and notifies the user that registration is complete.

[1772] Course Selection and Learning Management

[1773] The user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal then sends the course selection information to the server, which generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[1774] Learning progression and feedback

[1775] Users progress through learning content on a dedicated device. For example, they watch video tutorials and conduct scientific experiments. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. Based on the evaluation results, the server generates feedback and sends it to the device. The device displays the feedback to the user, who can use it as a reference for areas for improvement and next steps to continue their learning.

[1776] Question and Answer Support

[1777] When a user posts a question using the mobile app, the application sends the question to a server. The server uses a generative AI model to analyze the question and generate an appropriate answer. This answer is then sent to the device, where it is displayed to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to an expert.

[1778] Participating in offline events

[1779] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which stores it in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[1780] Sentiment analysis and customised learning experiences

[1781] During learning, the server uses emotion analysis to collect the user's emotional data. For example, it evaluates the user's stress level and concentration level from their facial expressions and voice data. The server analyzes the emotional data and, if the user is highly stressed, adjusts the difficulty of the learning content and provides relaxation content. Conversely, if the user is concentrating on their learning, it presents more difficult tasks.

[1782] Hardware and software used

[1783] The system uses the following elements:

[1784] React Native is used for the user interface of the dedicated terminal.

[1785] The backend development uses the Django framework and PostgreSQL.

[1786] TensorFlow is used for sentiment analysis.

[1787] Generative AI models such as ChatGPT API are used to analyze questions and generate answers.

[1788] Examples of concrete examples and prompts

[1789] For example, if a junior high school student user selects the "Science Experiment" course, the server generates a learning plan including a video tutorial and experiment procedures. The user watches the video and records the experiment results on their device. The server receives and analyzes the results in real time and generates appropriate feedback. At the same time, an emotion analysis means detects the user's stress level from facial expression data and adjusts the learning content as needed.

[1790] Example prompt sentence:

[1791] "Analyze the stress level of the user from facial expression data."

[1792] "Please analyze the level of concentration from user voice data."

[1793] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1794] Step 1: User Registration

[1795] The user opens the account creation screen using a dedicated terminal. They enter required information such as name, school name, grade, and email address, and press the "Register" button. The input data is sent from the terminal to the server. The server receives it, stores it in a database, and generates authentication information such as a user ID and password. The generated authentication information is sent from the server to the terminal, and the terminal notifies the user.

[1796] Input data: Name, school name, grade, email address

[1797] Output data: User ID, password

[1798] Step 2: Log in and view the dashboard

[1799] The user enters authentication information on the dedicated terminal and presses the login button. The terminal sends the login information to the server, which then verifies the authentication information. If authentication is successful, the server sends data about the user's learning course and progress to the terminal as dashboard screen data. The terminal displays this data.

[1800] Input data: User ID, password

[1801] Output data: Dashboard screen data

[1802] Step 3: Select your courses and create your study plan

[1803] The user opens the "Course Selection" menu from the dashboard and selects the desired course. The device sends the course selection information to the server. The server generates an optimal learning plan for the user and sends the learning plan data to the device. The device displays the learning plan, and the user starts the learning module.

[1804] Input data: Course selection information

[1805] Output data: Learning plan data

[1806] Step 4: Learning progression and data recording

[1807] Users progress through learning content on a dedicated device. When watching video tutorials or conducting scientific experiments, the device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress.

[1808] Input data: Experiment results, quiz answers

[1809] Output data: Comprehension assessment, progress data

[1810] Step 5: Generate and display feedback

[1811] The server generates feedback based on the understanding assessment and progress data. The generated feedback is sent to the device, which displays it to the user. The user can then refer to the feedback to identify areas for improvement and next steps.

[1812] Input data: Comprehension assessment, progress data

[1813] Output Data: Feedback

[1814] Step 6: Sentiment Analysis

[1815] While learning, a dedicated device records the user's facial expressions and voice in real time to collect emotional data. This data is sent from the device to a server, which uses an emotion analysis model to evaluate the user's stress level and concentration. Based on the evaluation results, the server adjusts the difficulty of the learning content.

[1816] Input data: facial expression data, voice data

[1817] Output data: stress level, concentration evaluation

[1818] Step 7: Questions and Answers

[1819] When a user posts a question using a mobile app, the device sends the question to the server, which uses a generative AI model to analyze the question and generate an appropriate answer. The generated answer is then sent to the device, which displays it to the user.

[1820] Input data: Question content

[1821] Output data: Answer

[1822] Step 8: Offline event management

[1823] The server periodically generates notifications for offline events and sends them to the device. The user receives the notification, enters the necessary information in the application form, and presses the "Register" button. The device sends the participation registration information to the server, which stores it in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[1824] Input data: Event details, application information

[1825] Output data: Participation confirmation message

[1826] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1827] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1828] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1829] [Fourth embodiment]

[1830] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1831] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1832] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1833] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1834] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1835] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1836] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1837] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1838] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1839] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1840] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1841] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1842] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1843] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system includes a user interface for users to register an account and enter personal information and learning objectives, a function for managing the user's selected courses and learning progress via a dedicated terminal, a function for recording and analyzing the user's learning progress and results in real time, a function for providing feedback based on the analysis results, a function for posting questions via the mobile app, analyzing them, and providing answers, and an event management function for participating in offline events. However, this does not involve writing program code.

[1844] The overall system flow is as follows:

[1845] User Registration

[1846] First, the user accesses the dedicated terminal and opens the account creation screen. The user enters required information such as name, school name, grade, and email address, and presses the "Register" button. The terminal then sends the entered information to the server, which receives it and stores it in a database. Based on the stored information, the server generates authentication information such as a user ID and password and sends it to the terminal. The terminal notifies the user that registration is complete by displaying this authentication information on the screen.

[1847] Course Selection and Learning Management

[1848] A user enters authentication information to log in to the online platform and presses the login button. The terminal then sends the login information to the server, which verifies the authentication information. If authentication is successful, the server sends the dashboard screen data to the terminal, which displays it. From this dashboard, the user can open the "Course Selection" menu and select the desired course. The terminal sends the course selection information to the server, which then generates a learning plan optimized for the user and sends it to the terminal. The terminal displays this learning plan, and the user begins the learning module.

[1849] Learning progression and feedback

[1850] The user progresses through learning content on a dedicated device. For example, they watch a video tutorial and conduct a scientific experiment. The device records the user's experiment results and quiz answers in real time and sends them to a server. The server analyzes the received data and evaluates the user's level of understanding and progress. The server generates feedback based on the evaluation results and sends it to the device. The device displays the feedback to the user, who uses it as a reference for areas for improvement and next steps to continue their learning.

[1851] Communication and Support

[1852] When a user posts a question using the mobile app, they use the LINE chat function. The device sends the question to the server, which analyzes it. An appropriate answer is generated and sent to the device. The device displays the answer and provides information to the user. If the answer is not satisfactory, the user can post the question again, and complex questions are escalated to be answered by experts or employees.

[1853] Participating in offline events

[1854] The server periodically generates notifications for offline events and sends them to the device. The user receives the notifications and views the details. To register for an event, the user enters the required information in the application form and presses the "Register" button. The device sends the registration information to the server, which saves the information in a database. The server generates a participation confirmation message and sends it to the device to notify the user.

[1855] Specific examples

[1856] For example, if a junior high school student user selects the "Science Experiment" course and proceeds through the program, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. The user can continue learning by using this as a reference when reviewing the course or moving on to the next module.

[1857] This system allows users to learn STEAM education interactively and effectively, and provides customized feedback based on their progress and level of understanding.

[1858] The processing flow will be explained below.

[1859] Step 1:

[1860] The user accesses the dedicated terminal, opens the account creation screen, enters required information such as name, school name, grade, and email address, and presses the "Register" button.

[1861] Step 2:

[1862] The terminal sends the input information to the server.

[1863] Step 3:

[1864] The server stores the received user information in a database and generates authentication information such as a user ID and password.

[1865] Step 4:

[1866] The server sends the generated authentication information to the terminal.

[1867] Step 5:

[1868] The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[1869] Step 6:

[1870] To log in to the online platform, the user enters their authentication information and presses the login button.

[1871] Step 7:

[1872] The device sends the login information to the server.

[1873] Step 8:

[1874] The server checks the authentication information, and if authentication is successful, it sends the dashboard screen data (learning progress, recommended courses, etc.) to the device.

[1875] Step 9:

[1876] The terminal displays the dashboard screen, and the user opens the "Course Selection" menu.

[1877] Step 10:

[1878] The terminal sends a request for a course list to the server.

[1879] Step 11:

[1880] The server sends a list of available STEAM courses to the device.

[1881] Step 12:

[1882] The user selects the desired course from the course list.

[1883] Step 13:

[1884] The terminal transmits the selected course information to the server.

[1885] Step 14:

[1886] The server generates a learning plan based on the selected course and transmits it to the terminal.

[1887] Step 15:

[1888] The device displays the learning plan, and the user clicks the "Start" button to display the first learning module.

[1889] Step 16:

[1890] Users watch learning content (e.g., video tutorials) and complete experiments and quizzes.

[1891] Step 17:

[1892] The device records the user's learning data (experiment results, quiz answers, etc.) in real time and sends it to the server.

[1893] Step 18:

[1894] The server analyzes the data it receives and evaluates the user's learning progress and level of understanding.

[1895] Step 19:

[1896] The server generates feedback based on the evaluation results and sends it to the terminal.

[1897] Step 20:

[1898] The device displays feedback to the user and prompts them to relearn or proceed to the next module as needed.

[1899] Step 21:

[1900] A user uses the mobile app to post a question via the LINE chat function.

[1901] Step 22:

[1902] The terminal sends the user's question to the server.

[1903] Step 23:

[1904] The server analyzes the question, generates an appropriate answer, and sends it to the terminal.

[1905] Step 24:

[1906] The terminal displays the answer received from the server to the user.

[1907] Step 25:

[1908] If the user is not satisfied with the answer, they can post the question again and the server will escalate the question to an expert or employee.

[1909] Step 26:

[1910] The server periodically generates and sends offline event notifications to the terminal.

[1911] Step 27:

[1912] Users receive event notifications and view detailed information on mobile apps and online platforms.

[1913] Step 28:

[1914] A user registers to participate in an offline event, enters information in the application form, and presses the "Register" button.

[1915] Step 29:

[1916] The terminal transmits the participation registration information to the server.

[1917] Step 30:

[1918] The server stores the registration information in a database and generates a confirmation message.

[1919] Step 31:

[1920] The server sends a participation confirmation message to the terminal and notifies the user.

[1921] Example 1

[1922] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1923] Current STEAM education support systems struggle to effectively manage users' learning progress and comprehension, and provide individually customized feedback. They also lack systems that allow users to ask questions in real time and receive immediate answers. Furthermore, they lack a means to seamlessly link online and offline learning environments. This makes it difficult for users to progress efficiently.

[1924] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1925] In this invention, the server includes means for recording and analyzing a user's learning progress and results in real time, means for providing feedback to the user based on the analysis results, and means for the user to post questions via a mobile app, analyze the questions, and provide answers. This makes it possible to instantly check a user's learning progress and suggest areas for improvement, thereby providing individually customized learning support. Furthermore, the real-time exchange of questions and answers via the mobile app allows users to quickly resolve their doubts and enable efficient learning. Furthermore, seamless integration of online and offline events can increase users' motivation to learn and encourage continuous learning.

[1926] "User" refers to a person who uses the system to carry out learning activities.

[1927] "Dedicated Terminal" refers to a specific electronic device used to access the system.

[1928] "Mobile app" refers to application software that runs on smartphones and tablets.

[1929] "Server" refers to a central computer system that stores, processes, and serves data.

[1930] "User interface" refers to the screen and operating means that allow a user to directly interact with a system.

[1931] "Learning curriculum" refers to various learning contents and courses selected by the user.

[1932] "Study progress" refers to the progress of a user in a learning curriculum selected by the user.

[1933] "Learning Content" refers to information and materials such as videos, texts, and quizzes provided to support learning.

[1934] "Feedback" refers to information that provides an evaluation of a user's learning activities and suggestions for improvement.

[1935] "Analysis" refers to the process of processing collected data to derive useful information or results.

[1936] "Non-online events" refer to learning or education-related events that take place in a physical location.

[1937] A "generative AI model" refers to an artificial intelligence algorithm that generates data-based feedback to track a user's learning progress and provide adaptive feedback.

[1938] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students. The system provides a user interface for users to register an account and enter personal information and learning objectives. Specific hardware includes dedicated terminals, mobile terminals, and servers. Software includes database management software (e.g., MySQL or PostgreSQL), real-time data analysis software (e.g., Apache Kafka), front-end frameworks (e.g., React or Angular), and mobile app development frameworks (e.g., Flutter or React Native).

[1939] System configuration

[1940] 1. User Registration and Authentication

[1941] A user accesses a dedicated terminal and creates an account by entering profile information. The terminal sends the user's input information to the server, which stores the received information in a database. The server generates authentication information and sends it to the terminal. The terminal displays the authentication information on the screen and notifies the user that registration is complete.

[1942] 2. Course Selection and Learning Management

[1943] The user logs in to a dedicated terminal using authentication information, and the server verifies the authentication information. If authentication is successful, the server sends the user's dedicated dashboard data to the terminal. The user selects the desired learning curriculum from the dashboard, and the terminal sends the selection information to the server. The server generates an optimal learning plan based on the user's selection and sends it to the terminal. The terminal displays the generated learning plan, and the user begins learning.

[1944] 3. Learning and Feedback

[1945] Users progress through learning content (e.g., video tutorials, textbooks, quizzes) provided on dedicated devices. The devices record the user's learning progress and quiz answers in real time and send them to the server. The server analyzes the received data and evaluates the user's level of understanding and progress. Feedback generated based on the evaluation is sent to the device, which then displays it to the user.

[1946] 4. Real-time Q&A

[1947] Users use a mobile app to post questions. The questions are sent from the device to a server, which analyzes them. An appropriate answer is generated and sent to the device. The user can review the answer and ask the question again if necessary. Complex questions are answered by experts or employees.

[1948] 5. Participating in offline events

[1949] The server periodically generates notifications for non-online events and sends them to the device. The user receives the notifications and views the details. When registering for an event, the user enters the required information and the device sends the registration information to the server. The server stores the information in a database and sends a participation confirmation message to the device.

[1950] Specific examples

[1951] For example, if a junior high school student user selects the "Science Experiment" course and uses the system, the server first generates a learning plan including a video tutorial and experiment procedures. The user watches the video and then performs the experiment. The results of the experiment are recorded on the device, and the server receives and analyzes them in real time. Based on the analysis results, the server generates appropriate feedback and displays it to the user via the device. The user can continue learning by using this as a reference when reviewing the course or moving on to the next module.

[1952] Prompt Sentence Examples

[1953] "Please tell me the specific steps to provide lesson plans and video tutorials for a science lab course for middle school students to learn basic scientific principles."

[1954] The present invention allows users to learn STEAM education interactively and effectively, and receive customized feedback based on their progress and level of understanding, maximizing learning efficiency.

[1955] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1956] Step 1: User registers for an account

[1957] The user accesses the dedicated terminal and opens the account creation screen.

[1958] Input: The user enters required information such as name, school name, grade, and email address.

[1959] The terminal sends the input information to the server.

[1960] Output: User information is saved in the database on the server side. Authentication information (user ID, password) is generated and sent to the terminal.

[1961] Specific operation: The terminal sends the input information as an HTTP POST request, and the server executes an SQL INSERT statement against the database.

[1962] Step 2: User enters credentials and logs in

[1963] The user enters authentication information (user ID, password) on the dedicated terminal and presses the login button.

[1964] Input: The user enters the user ID and password.

[1965] The device sends the login information to the server.

[1966] Output: The server verifies the authentication information and, if successful, sends the dashboard data to the device.

[1967] Specific operation: The terminal sends an HTTP POST request, and the server searches for user information in the database and compares the password.

[1968] Step 3: User makes course selection

[1969] After logging in, the user selects the "Course Selection" menu from the dashboard and chooses the desired learning curriculum.

[1970] Input: The user selects the desired learning curriculum.

[1971] The terminal transmits the course selection information to the server.

[1972] Output: The server generates the optimal learning plan for the user and sends it to the device.

[1973] How it works: The device sends an HTTP POST request to the server containing the selected course information, and the server uses an algorithm to generate a learning plan.

[1974] Step 4: Users progress through the learning content

[1975] The user starts the provided learning content on a dedicated device.

[1976] Input: User watches video tutorials, performs experiments and quizzes.

[1977] The device records the user's progress and quiz answers and sends them to the server.

[1978] Output: The server receives the data in real time and generates feedback based on the analysis results.

[1979] Specific operation: The device records user operation data in real time and sends it to the server via HTTP POST request. The server then runs an analysis algorithm to evaluate the data.

[1980] Step 5: Provide feedback based on the analysis results

[1981] The server analyzes the user's learning data and evaluates their level of understanding and progress.

[1982] Input: User learning progress data and quiz answer data.

[1983] The server generates the analysis results and sends feedback to the device.

[1984] Output: The device displays feedback to the user.

[1985] Specific operation: The server analyzes the data in the database, generates a feedback message based on the evaluation criteria, and sends the generated feedback to the terminal as an HTTP response.

[1986] Step 6: Users post questions and get answers

[1987] A user posts a question using a mobile app.

[1988] Input: The user enters the question and clicks the post button.

[1989] The device sends the question to the server.

[1990] Output: The server analyzes the question, generates an appropriate answer, and sends it to the device.

[1991] Specific operation: The device sends an HTTP POST request containing a question to the server, and the server analyzes the question using a natural language processing algorithm, generates an answer, and sends the answer to the device as an HTTP response.

[1992] Step 7: User registers for a non-online event

[1993] The server periodically generates and sends notifications of non-online events to the terminal.

[1994] Input: Event details and registration information.

[1995] Users receive notifications, view details, and register to attend.

[1996] Output: The terminal sends the participation registration information to the server, which stores the information in a database. The server generates a participation confirmation message and sends it to the terminal.

[1997] Specific operation: The server periodically generates an event notification and sends it to the terminal as an HTTP response. The user registers for participation, and the terminal sends it to the server as an HTTP POST request. The server executes an SQL INSERT statement in the database, generates a participation confirmation message, and sends it to the terminal.

[1998] (Application example 1)

[1999] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2000] Conventional STEAM education systems have limited functionality for tracking learners' progress and providing feedback, making real-time analysis and customized instruction difficult. They also lack the ability to adequately manage interactive virtual learning experiences and offline events. Furthermore, the adoption of new learning methods utilizing wearable devices has been slow, creating a need for ways to maximize the effectiveness of education.

[2001] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2002] In this invention, the server includes: means for providing a user interface for users to register an account and input personal information and learning objectives; means for managing the user's selected course and learning progress via a dedicated terminal; means for recording and analyzing the user's learning progress and results in real time; means for providing feedback to the user based on the analysis results; means for users to post questions via a mobile app or a wearable device (smart glasses or a head-mounted display), analyze the questions, and provide answers; means for conducting experiments in a virtual space, simulating the experiment results, and providing feedback in real time; and event management means for displaying regular offline event notifications and detailed information and registering participants. This enables an interactive learning experience using a variety of devices, realizes real-time analysis of learning progress, and provides customized feedback, thereby improving the quality of education and maximizing learning effectiveness.

[2003] "A user interface for users to register an account and enter personal information and learning objectives" is a means of providing an operation screen for users to create their own account in an educational system and enter personal information and learning objectives.

[2004] "Means for managing a user's selected course and learning progress via a dedicated terminal" refers to a means in an educational system that provides a user with the ability to track and manage their selected learning course and learning progress using a dedicated device.

[2005] "Means for recording and analyzing a user's learning progress and results in real time" refers to a means for providing a function in an educational system to collect a user's learning progress and results as data in real time and analyze them using statistical or machine learning methods.

[2006] "Means for providing feedback to the user based on the analysis results" refers to means for providing the user with a function for suggesting areas for improvement and next learning steps based on the learning data analyzed in the educational system.

[2007] "A means for users to post questions via a mobile app or wearable device, and for the content of the question to be analyzed and an answer to be provided" refers to a means in an educational system that allows users to send questions using a smartphone app or a wearable device such as smart glasses, and provides a function that analyzes the content of the question using a natural language processing model or the like and provides an appropriate answer.

[2008] "Means for conducting experiments in a virtual space, simulating the experimental results, and providing feedback in real time" refers to a means for realizing the function in an educational system in which a user conducts an experiment in a virtual reality or mixed reality space, analyzes the results using computer simulation, and provides feedback in real time.

[2009] "Event management means for displaying regular offline event notifications and detailed information, and registering for participation" refers to a means for providing users in an educational system with regular notifications of offline event information, allowing them to view detailed information, and providing the functionality for registering for participation.

[2010] "Means for analyzing data in real time using cloud computing and providing optimal feedback using generative artificial intelligence" refers to a means in an educational system that uses cloud services via the internet to process and analyze collected data in real time and generate and provide optimal feedback to users using machine learning models.

[2011] "Means for providing an interactive learning experience using mixed reality technology to create a virtual space" refers to a means for using mixed reality (MR) technology in an educational system to create a learning environment that combines reality and virtual space, and to provide users with an interactive and immersive learning experience.

[2012] This invention is a system that provides a comprehensive online platform and mobile app to support STEAM education for elementary, junior high, and high school students.

[2013] Overall structure

[2014] 1. User Registration:

[2015] The server provides a user interface for users to enter personal information such as name, school name, grade, and email address, as well as their learning goals. When a user registers an account from a dedicated terminal, the terminal sends the information to the server, which stores it in a database. After registration is complete, the server generates a user ID and password and sends them to the terminal.

[2016] 2. Cour...

Claims

1. means for providing a user interface for a user to register for an account and enter personal information and learning objectives; A means for managing the user's selected course and learning progress via a dedicated terminal; A means for recording and analyzing the user's learning progress and results in real time; means for providing feedback to the user based on the analysis results; A means for users to post questions via a mobile app, analyze the questions, and provide answers; an event management means for allowing users to register to participate in offline events; A system including:

2. The system of claim 1 , further comprising means for using generative artificial intelligence to customize a user's learning progress and generate automated feedback.

3. 10. The system of claim 1, further comprising means for providing the educational institution with high speed internet and modern learning equipment to optimize the learning environment.

Images