system

Abstract

We provide the system. [Solution] A means for collecting learner data acquired from individual information processing devices, A means for analyzing collected learner data to generate an optimized learning plan for each learner, A means for distributing the generated learning plan to individual information processing devices, A means of providing real-time feedback to learners based on the distributed learning plan, A means of strengthening learner motivation by visually providing rewards and progress for learning activities, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional learning systems, it is difficult to provide learning methods optimized for each of a variety of learners, and there is a problem in that there is a lack of individualized support for realizing particularly efficient and effective learning. For this reason, there is a need for a system that generates a plan according to the needs of each learner within a limited time and immediately reflects the results.

Means for Solving the Problems

[0005] This invention provides a means for generating an optimized learning plan for each learner by collecting learner information from individual information processing devices and analyzing that data in detail. By delivering this plan to the information processing devices in real time, the immediacy and effectiveness of learning are enhanced. Furthermore, by introducing a reward system for feedback and motivation obtained during learning, it is possible to continuously strengthen the learner's motivation.

[0006] "Learner data" refers to a collection of data that learners generate through their learning activities, including their learning history, learning time, and points where they are prone to making mistakes.

[0007] "Information processing equipment" refers to devices that process data electrically, such as computers and smartphones.

[0008] A "learning plan" is a plan that outlines the learning content and schedule to be achieved, based on the individual needs and goals of the learner.

[0009] "Analysis" is the act of analyzing data in detail to clarify its meaning and characteristics.

[0010] "Real-time feedback" refers to information provided immediately during a learning activity for evaluation and improvement.

[0011] A "reward system" is a mechanism that promotes motivation, such as badges or points that learners can earn when they achieve specific goals. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0014] First, the language used in the following description will be explained.

[0015] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, the labeled communication I / F (Interface) is an interface that includes a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention relates to an individualized learning system using an information processing device. The aim of this system is to improve learning efficiency by providing an optimal learning plan using each learner's data.

[0034] Server operation:

[0035] The server collects learner data from the user's information processing device. This includes the user's learning history, learning time, and information about past errors. The server analyzes the collected data and uses generative AI to identify learner patterns. Based on this analysis, the server creates a learning plan tailored to each learner. This learning plan includes recommended learning content and time, and incorporates a variety of content formats. The created plan is transmitted by the server to the user's information processing device in real time.

[0036] Device operation:

[0037] The terminal, i.e., the user's device, visually displays the learning plan received from the server. The user can proceed with their learning according to the plan provided through the terminal. The terminal also has the function of sending progress data generated during learning to the server in real time, and receives immediate feedback from the server. The terminal displays this feedback to the user in an easy-to-understand manner, helping them to understand the learning content.

[0038] User experience:

[0039] Users follow a learning plan received through their device, completing tasks such as vocabulary exercises and listening practice. Real-time feedback is displayed on the device as they progress, allowing users to immediately identify their weaknesses and use that information to improve. Furthermore, the device incorporates a gamification function, enabling users to earn badges based on their achieved goals. This helps maintain user motivation for learning.

[0040] Specific example:

[0041] For example, if a user wants to improve their English listening skills, the server analyzes their past listening test results and delivers a learning plan, including specific audio materials, to their device. The user then answers specific listening questions according to this plan and receives feedback from the server on their specific listening weaknesses based on their results. This feedback is immediately displayed on the device, allowing the user to instantly understand what needs improvement and apply it to their next learning session.

[0042] This embodiment of the invention allows learners to flexibly and efficiently pursue learning according to their individual needs.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The server collects learner data from the user's device. This includes recent learning history, usage time, and questions answered and their results. This data is stored in a database on the server.

[0046] Step 2:

[0047] The server analyzes learner data retrieved from the database. Here, a generative AI model is used to analyze user characteristics and learning patterns. In particular, a detailed analysis is performed to identify weaknesses and strengths in answering questions.

[0048] Step 3:

[0049] The server creates an optimized learning plan for each user based on the analysis results. The plan includes what to learn, learning priorities, and recommended content formats (e.g., video, audio, tests).

[0050] Step 4:

[0051] The server sends the generated learning plan to the user's device. After sending, the device receives it and displays it to the user. The user is then ready to begin learning based on this plan.

[0052] Step 5:

[0053] Users follow the learning plan displayed on their device and carry out the corresponding learning activities. For example, they may take listening practice or vocabulary tests, and the results are promptly fed back to the server.

[0054] Step 6:

[0055] The server evaluates the received learning results in real time and generates immediate feedback for each user. The feedback includes specific points about areas where understanding is lacking and suggestions for improvement.

[0056] Step 7:

[0057] After feedback is generated, the server sends that information to the user's device. The device immediately displays the received feedback, allowing the user to review it and use it to improve their next learning session.

[0058] Step 8:

[0059] Depending on the user's progress, the device displays badges and provides level-up notifications. This stimulates the user's motivation to learn and encourages further learning.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] Problems with traditional education systems include insufficient individualized learning and a lack of real-time feedback based on learning progress. This makes it difficult for learners to progress efficiently, potentially leading to a decline in motivation.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for collecting learner history information obtained from individual information processing devices, means for analyzing the collected history information using a generative model to generate an optimized learning plan for each learner, and means for transmitting the generated learning plan to the individual information processing devices. This enables the provision of an optimized learning plan for each learner and immediate suggestions for improvements based on their progress.

[0065] An "information processing device" refers to hardware or software used for collecting, analyzing, and transmitting data.

[0066] "History information" refers to data about a learner's past learning history, study time, and tendency to make incorrect answers.

[0067] A "generative model" refers to artificial intelligence technology that analyzes collected data and generates an optimal educational plan for each learner.

[0068] An "educational plan" refers to a learning guideline that includes recommended learning content and study time, created based on the individual needs of the learner.

[0069] "Immediate improvement suggestions" refer to specific advice on evaluations and areas for improvement based on learning outcomes, which learners can receive in real time.

[0070] "Rewards" refer to means of motivation provided to learners in visual or other forms in accordance with their achievement of learning activities.

[0071] Modes for carrying out the invention

[0072] This invention is an information processing system that provides learners with individually optimized educational plans and supports efficient learning. The following describes how to specifically implement the program of this system.

[0073] The server collects historical information from each learner's individual information processing device. This historical information includes learning history, learning time, and error tendencies. The server analyzes the collected data using a generative AI model to generate an optimized learning plan for each learner. Specifically, this generative AI model identifies the learner's performance patterns based on past learning data and determines what learning content is most effective.

[0074] The generated learning plan is transmitted in real time by the server to individual information processing devices. This plan includes details tailored to the learner's needs, such as recommended learning content and study time. The user's device visually displays this learning plan, providing clear guidance to the learner. It also records progress in real time and sends feedback to the server, allowing the user to receive immediate improvement suggestions from the server.

[0075] Users can progress through their learning by following the educational plan provided via their device. For example, if a user wants to improve their English listening skills, the server generates an educational plan, including specific audio materials, based on the user's past listening test results, and delivers it to the user. The user answers listening questions according to this plan and receives feedback from the server on specific listening weaknesses based on the results. This feedback is immediately displayed on the device, allowing the user to quickly understand what they need to improve.

[0076] As an example of implementing this system, one could input the following prompt into the generative AI model: "Generate the optimal listening learning plan based on the user's learning history and error patterns." Based on this prompt, the generative AI model would generate the most suitable educational plan.

[0077] In this way, the present invention can provide an educational plan tailored to each learner, thereby enabling efficient learning.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The server retrieves historical information from the user's terminal, such as learning history, learning time, and error tendencies. This input data reflects each learner's individual learning tendencies. The server securely collects this data and stores it in storage for later analysis.

[0081] Step 2:

[0082] The server inputs the collected historical information into a generating AI model for data analysis. During this analysis process, the AI ​​model identifies learner performance patterns and automatically discovers learning challenges. As a result, data is output that forms the basis for an optimized educational plan for each user.

[0083] Step 3:

[0084] The server generates an optimized learning plan for each learner based on the data output from the generated AI model. This plan includes specific learning content, recommended study time, and areas to focus on. This information is customized based on the analysis results obtained in Step 2.

[0085] Step 4:

[0086] The server sends the generated lesson plan to the user's device in real time. The device receives this plan and displays it visually to the learner. The plan displayed on the device includes specific learning tasks and links to related content, allowing the user to start learning immediately.

[0087] Step 5:

[0088] Users engage in learning activities according to the educational plan displayed on the device. Specifically, they practice listening using audio materials and answer questions. As the user progresses through the learning process, the device records their progress.

[0089] Step 6:

[0090] The device sends data generated based on learning progress to the server in real time. The server then immediately analyzes the progress data and generates instant improvement suggestions as needed. This immediate feedback is provided individually based on the analyzed learner's weaknesses.

[0091] Step 7:

[0092] The device receives immediate improvement suggestions sent from the server and displays them clearly to the user. Based on this feedback, the user can modify their learning and apply those improvements to future learning opportunities.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] In recent years, there has been a growing demand for personalized education tailored to individual learners. However, providing each learner with an optimal learning plan in real time, along with effective assessment and feedback, is extremely difficult. Furthermore, appropriate methods for maintaining learner motivation are also necessary. This invention aims to solve these problems.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for collecting learner information obtained from individual computer devices, means for analyzing the collected learner information to generate a learning plan optimized for each learner, and means for distributing the generated learning plan to the individual computer devices. This makes it possible to distribute a learning plan optimized for each learner and to provide timely evaluation and feedback.

[0098] An "individualized computer device" is a user-specific computer device used to provide personalized information for learners.

[0099] "Learner information" refers to a collection of various data related to the learner, such as their past learning history, study time, and common areas of error.

[0100] A "learning plan" is an educational plan optimized according to each learner's learning style and history, and includes customized learning materials and schedules.

[0101] "Real-time response" refers to a process of evaluation and feedback provided immediately during learning, enabling learners to immediately recognize and address areas for improvement.

[0102] "Means of strengthening motivation" refer to visual rewards and features that foster a sense of accomplishment, provided to increase learners' motivation to continue learning.

[0103] "Performance evaluation" is a process for quantifying or visualizing the performance skills acquired by learners and providing suggestions for improvement.

[0104] "Adjusting audio content to match skill level" refers to the activity of changing the difficulty level and content of audio materials according to the learner's skill level to provide an optimal learning experience.

[0105] "Delivering content" refers to the process of transferring generated learning plans and audio content to learners' devices, making them accessible to them.

[0106] To implement this invention, the process begins with the server collecting learner data. This data is obtained from individual computing devices such as the user's smartphone or tablet, including past learning history, learning time, and points where the user is prone to errors. This data is then analyzed by an AI model running on a cloud server, generating a learning plan optimized for each individual learner.

[0107] The generated learning plan is delivered to the user's computer in real time. The receiving computer visually presents the learning plan to the user and provides feedback based on their progress. This allows the user to receive immediate evaluations and improvement suggestions through their device, and the optimal audio content and materials are delivered based on their skill level through performance evaluations. Furthermore, visual rewards and motivational elements are incorporated to enhance a sense of accomplishment.

[0108] A concrete example is a user who wants to improve their musical performance skills using this system. The server analyzes the user's past practiced songs and performance quality, and proposes an optimal practice plan. The user learns new songs and techniques according to this plan and receives immediate feedback on their device. At that time, detailed areas for improvement regarding the quality of their performance are also pointed out, which they can use to improve their next performance.

[0109] An example of a prompt to input into a generative AI model is, "Please suggest the optimal plan based on the user's history and errors in order to generate personalized learning content." This wording is used to create a plan that adapts to the different needs and abilities of each learner.

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The server collects learner information from the user's terminal via individual computer devices. Input data includes past learning history, learning time, and common error points. The server aggregates and organizes this input data to create a detailed profile of the target learner. The output is a dataset saved as individual learner profiles.

[0113] Step 2:

[0114] The server analyzes the learner information collected using a generative AI model. The input is the learner profile obtained from step 1. The server inputs the prompt "Propose the optimal plan based on the user's history and errors to generate the personalized learning content to be provided" to the AI ​​model and performs data analysis and prediction. The output is the optimized learning plan.

[0115] Step 3:

[0116] The server delivers the generated learning plan to the user's terminal. The input is the learning plan generated in step 2. The server sends the plan data to each user's terminal, making it available for real-time reception. The output is the learning plan received and displayed on the user's terminal.

[0117] Step 4:

[0118] The terminal visually displays the received learning plan to the user. The input is the learning plan data sent in step 3. The terminal presents the learning plan to the user in an easy-to-understand format through a graphical user interface. The output is a visual display of the learning plan that the user can review.

[0119] Step 5:

[0120] The user engages in learning activities based on a learning plan, and the device monitors their progress. Input consists of user actions and progress data on the learning content. The device sends progress data to the server in real time. Output is data reflecting the learner's progress.

[0121] Step 6:

[0122] The server analyzes progress data sent from the terminal and generates real-time feedback. The input is the learner's progress data received from step 5. The server uses an AI model to evaluate the progress data and makes necessary improvement suggestions and sets achievable goals. The output is immediate evaluation and feedback information.

[0123] Step 7:

[0124] The terminal presents the user with the feedback received from the server. The input is the feedback information generated in step 6. The terminal displays the feedback in a visual and easy-to-understand format so that the user can understand what to do in the next step. The output is the feedback information displayed to the user.

[0125] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0126] This invention is an individually optimized learning system that takes into account the learner's emotional state. In particular, the system aims to maintain the learner's concentration and motivation optimally by recognizing the learner's emotions in real time and reflecting that information in the learning plan and feedback.

[0127] Server operation:

[0128] The server collects emotional data, including learner data, from the user's device. Using the device's built-in camera and sensors, data related to emotions, such as the user's facial expressions, voice tone, and typing speed, is acquired. The server analyzes this data to understand the user's emotional state during learning in real time. The emotion engine categorizes these emotional states into specific categories such as "joy," "surprise," "anger," and "anxiety." This information is used to generate and adjust learning plans optimized for each user.

[0129] Device operation:

[0130] The device quickly displays a pre-adjusted learning plan received from the server. The user then engages in normal learning activities based on this plan, while the device continuously monitors their emotional state. If an emotional change is detected, the device sends this information to the server, triggering a real-time adjustment of the plan. This ensures the user can always learn in the optimal psychological state.

[0131] User experience:

[0132] Users practice English listening, reading, and other skills through learning plans suggested via their devices. For example, if a user shows frustration with a complex problem, the server uses that information to modify the feedback and adjust the difficulty level of the plan as needed. This process is repeated to reduce the learner's burden and support effective skill improvement.

[0133] Specific example:

[0134] As a concrete example, suppose the emotion engine collects data indicating "surprise" or "anxiety" while a user is taking an online English test. The server can analyze this emotional state and adjust the feedback, such as replacing some parts of the learning plan with easier questions or suggesting a break. This emotional data can also be used to generate future plans, enabling users to continue learning in a less stressful environment.

[0135] This invention makes it possible to understand learners' emotions as much as possible and provide a learning environment that is tailored to them, which is expected to improve learning effectiveness and maintain sustained motivation.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The user starts a learning session using the device. At this time, the device's built-in camera and microphone are activated and begin collecting data on the user's facial expressions and voice.

[0139] Step 2:

[0140] The device transmits collected emotional data to the server in real time. This data includes facial expressions, voice tone, and typing speed.

[0141] Step 3:

[0142] The server analyzes the received emotional data using an emotion engine. This analysis classifies the user's current emotional state into categories such as "joy," "surprise," "anxiety," and "concentration."

[0143] Step 4:

[0144] The server uses the analysis results to create or adjust a learning plan optimized for the user. If the user's emotional state is unstable, adjustments will be made, such as lowering the difficulty of the tasks or suggesting a break.

[0145] Step 5:

[0146] The server sends the adjusted learning plan to the user's device, which then displays it. The user continues learning according to the newly adjusted plan.

[0147] Step 6:

[0148] As users continue their learning activities, their emotional state may change. In such cases, the device continues to collect emotional data and send it to the server.

[0149] Step 7:

[0150] The server re-analyzes the received sentiment data and makes further adjustments to the learning plan as needed. It also instantly generates and sends appropriate feedback to the device.

[0151] Step 8:

[0152] The device displays feedback and adjustments sent from the server to the user. Based on this, the user continues learning and aims for efficient skill improvement.

[0153] In this way, by continuously optimizing the learning environment, users can always learn in the optimal psychological state.

[0154] (Example 2)

[0155] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0156] Despite the significant impact learners' emotional states on learning effectiveness and motivation, traditional learning systems have not adequately considered this. As a result, learners often face stress and decreased motivation, leading to reduced learning efficiency. This issue is particularly pronounced in online and remote learning environments, and a solution has been urgently needed.

[0157] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0158] In this invention, the server includes means for collecting learner data, including emotional data acquired from individual terminals; means for analyzing the collected learner data and generating a learning plan optimized for each learner based on the emotional information; and means for providing real-time feedback to learners based on the delivered learning plan and adjusting the plan according to changes in their emotions. This makes it possible to provide flexible learning plans that take into account the learner's emotional state, thereby maximizing the learner's motivation and efficiency.

[0159] An "individual terminal" is an information processing device used by a user, equipped with a camera and sensors, and capable of collecting user emotional data.

[0160] "Emotional data" refers to data that indicates a user's emotional state, including information such as facial expressions, voice, and typing speed.

[0161] "Learner data" refers to all information about the user, including past learning history, learning time, and sentiment data.

[0162] A "learning plan" refers to a learning plan and content optimized according to each learner's emotional state and individual characteristics.

[0163] "Real-time feedback" refers to the provision of immediate responses and information generated while the user is engaged in learning activities.

[0164] "Means of collection" refers to methods and devices that include the process of acquiring data from a terminal and transferring it to a server.

[0165] "Means of analysis" refers to the processes and devices used to analyze collected data and classify emotional states and learning patterns.

[0166] "Means of delivery" refers to methods or devices for sending learning plans, generated or adjusted on the server side, to the terminal.

[0167] Modes for carrying out the invention

[0168] This invention is an individually optimized learning system that takes into account the emotional state of the learner, and is mainly composed of three subjects: server, terminal, and user.

[0169] Server operation

[0170] The server collects learner data, including emotional data, from individual terminals. This data includes facial expressions, voice tone, and typing speed captured by the terminal's built-in cameras and sensors. The collected data is analyzed in real time using an emotional AI model and classified into specific emotional categories such as "joy," "surprise," "anger," and "anxiety." This analysis information is used by a generative AI model to generate learning plans optimized for each learner.

[0171] Terminal operation

[0172] The device quickly displays the adjusted learning plan received from the server. While the user progresses through learning activities based on this plan, the device continuously monitors their emotional state. If a change in emotion is detected, the device notifies the server, immediately triggering a readjustment of the plan.

[0173] User experience

[0174] Users practice skills such as listening and reading using learning plans suggested through their devices. For example, if a user shows frustration when faced with a difficult problem, the server uses that information to modify the feedback, either lowering the difficulty of the plan or suggesting a break. This allows users to improve their skills effectively while reducing their burden.

[0175] Examples of specific cases and prompt statements

[0176] For example, while a user is taking an online English test, emotional data such as "surprise" or "anxiety" may be collected. The server analyzes this emotional state and either replaces parts of the learning plan with easier questions or suggests a break. This data is also used to generate future plans, providing a learning environment that reduces stress.

[0177] An example of a prompt could be input to the generative AI model as, "How would you optimize the learning plan when a user expresses frustration with a complex problem?" Based on this prompt, it is possible to suggest more efficient feedback.

[0178] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0179] Step 1:

[0180] The device collects user emotion data. This data collection is done by capturing the user's facial expressions with a built-in camera and analyzing their voice tone with a microphone. It also records keyboard typing speed. This provides raw data about the user's emotions. The input is this emotion-related data, which is output as formatted data for the next step.

[0181] Step 2:

[0182] The server receives emotional data collected from the terminal and begins data analysis. Using an emotional AI model, it classifies the data into specific categories such as "joy," "surprise," "anger," and "anxiety." In this process, the input is formatted emotional data, and the output is the result of the analyzed emotional state.

[0183] Step 3:

[0184] The server utilizes a generated AI model based on the analysis results to create an optimized learning plan for each learner. The data processing performed here involves generating and adjusting the learning plan to reflect the learner's emotional state. The input is the analyzed emotional state, and the output is the optimized learning plan.

[0185] Step 4:

[0186] The server sends the generated learning plan to the terminal. The terminal displays the received learning plan to the user and prompts them to begin learning activities. The input is the optimized learning plan, and the output is the presentation of the learning plan to the user.

[0187] Step 5:

[0188] The user engages in learning activities based on a learning plan received through the device. During this time, the device continuously monitors emotional data in real time and sends new data to the server if any changes occur. The input is the user's learning activities, and the output is additional emotional data.

[0189] Step 6:

[0190] The server analyzes new emotion data transmitted in real time and updates the learning plan as needed. This allows for flexible adjustments tailored to the user's emotional state. The input is real-time emotion data, and the output is the learning plan adjusted as needed.

[0191] (Application Example 2)

[0192] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0193] The present invention aims to provide a system that can enhance the user's shopping experience in a virtual environment by recognizing the user's emotional state in real time, providing individually optimized product suggestions based on that information, and thereby increasing their desire to purchase. Specifically, it solves the problem of instantly presenting relevant information and content according to the products and categories that the user has shown interest in, thereby supporting more intuitive and effective purchasing activities.

[0194] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0195] In this invention, the server includes means for collecting user information obtained from individual devices, means for analyzing the collected user information to generate personalized recommendations for each user, and means for distributing the generated recommendations to individual devices to improve the purchasing experience in the virtual environment. This enables personalized recommendations that reflect the user's emotional state, thereby effectively increasing the user's willingness to purchase.

[0196] A "device" refers to an electronic device used for information processing and communication. It plays a role in directly interacting with the user through a user interface and is capable of inputting and outputting data.

[0197] "User information" refers to the data obtained from device usage and user behavior. This information is analyzed to understand the characteristics of individual users.

[0198] "Emotional state" refers to the state in which a user expresses their emotions. It is usually determined in real time based on data obtained from facial expressions, tone of voice, body movements, etc.

[0199] "Optimized recommendations" refer to product and service recommendations tailored to the user's emotional state and interests. This provides information that improves user satisfaction.

[0200] A "virtual environment" refers to a virtual space constructed using digital technology that is different from the real world. Users can have visual and auditory experiences within this environment through their devices.

[0201] "Purchase intent" refers to a user's desire or motivation to buy a product or service. This intent can be strengthened by individually tailored information and suggestions.

[0202] The system that realizes this invention has a structure in which a server, a terminal, and a user cooperate to function.

[0203] The server receives information collected from the user's device in real time and analyzes their emotional state. The server uses emotion recognition software such as OpenCV and TENSORFLOW® to capture image and audio data and understand the user's current emotional state. The analyzed data is used to build individually optimized product recommendations using a generative AI model. These product recommendations are sent to the user's device and used to improve the user experience.

[0204] The terminal displays product suggestions received from the server in a way that users can visually confirm within a virtual environment. This information is displayed through a user interface on terminals such as smart glasses or head-mounted displays. The device also has the ability to send new data to the server that may reflect changes in the user's emotional state. This ensures that the system operates in a way that optimized suggestions are constantly updated.

[0205] Users can view product suggestions and content presented through their devices and research items that interest them in more detail. User behavior and reactions are continuously recorded and transmitted to the server, allowing the system to adapt to the user environment more effectively.

[0206] As a concrete example, suppose a user is browsing a virtual store and finds a gadget that interests them. The device detects the user's "excitement" and sends that information to a server. The server performs appropriate data analysis and presents similar products or content of interest to the user's glasses. In this case, possible prompts for the generative AI model might be as follows:

[0207] "If a user's emotion indicates excitement, generate a list of gadget products corresponding to that emotion and display it along with detailed reviews."

[0208] In this way, the system can always provide a customized purchasing experience that responds to the user's emotions.

[0209] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0210] Step 1:

[0211] The server receives information from the user's device in real time. It takes video and audio data transmitted from the device as input. This data is then used with emotion recognition software such as OpenCV or TensorFlow to perform calculations to detect the user's current emotional state. The output is data representing the analyzed emotional state.

[0212] Step 2:

[0213] The server uses a generative AI model to build individually optimized product recommendations based on the emotional state data obtained in Step 1. The inputs used are emotional state data and past user behavior data. The AI ​​model generates a list of products and content tailored to the user's interests and emotions. The output includes a set of individually tailored product recommendations.

[0214] Step 3:

[0215] The server sends generated product suggestions to the user's device. The device receives the product suggestions from the server as input and presents them to the user visually in a virtual environment. Interactive feedback from the user can also be incorporated. As output, the products selected by the user and content of interest are displayed on the device.

[0216] Step 4:

[0217] Users view product suggestions displayed through their devices and search for detailed information on items that interest them. The user's actions and responses are then sent back to the server via the device. This input includes user selections and browsing history, which the server further analyzes to provide foundational data for updating subsequent suggestions. The output generates new suggestions and feedback tailored to the user's actions.

[0218] Step 5:

[0219] The server continuously tracks user behavior based on updated suggestions, feeding appropriate data and prompt messages to an AI model, and then provides continuous service based on the results obtained. An example of such a prompt message is: "If the user's emotion indicates excitement, generate a list of gadget products corresponding to that emotion and display them along with detailed reviews."

[0220] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0221] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0222] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0223] [Second Embodiment]

[0224] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0225] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0226] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0227] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0228] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0229] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0230] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0231] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0232] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0233] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0234] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0235] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0236] This invention relates to an individualized learning system using an information processing device. The aim of this system is to improve learning efficiency by providing an optimal learning plan using each learner's data.

[0237] Server operation:

[0238] The server collects learner data from the user's information processing device. This includes the user's learning history, learning time, and information about past errors. The server analyzes the collected data and uses generative AI to identify learner patterns. Based on this analysis, the server creates a learning plan tailored to each learner. This learning plan includes recommended learning content and time, and incorporates a variety of content formats. The created plan is transmitted by the server to the user's information processing device in real time.

[0239] Device operation:

[0240] The terminal, i.e., the user's device, visually displays the learning plan received from the server. The user can proceed with their learning according to the plan provided through the terminal. The terminal also has the function of sending progress data generated during learning to the server in real time, and receives immediate feedback from the server. The terminal displays this feedback to the user in an easy-to-understand manner, helping them to understand the learning content.

[0241] User experience:

[0242] Users follow a learning plan received through their device, completing tasks such as vocabulary exercises and listening practice. Real-time feedback is displayed on the device as they progress, allowing users to immediately identify their weaknesses and use that information to improve. Furthermore, the device incorporates a gamification function, enabling users to earn badges based on their achieved goals. This helps maintain user motivation for learning.

[0243] Specific example:

[0244] For example, if a user wants to improve their English listening skills, the server analyzes their past listening test results and delivers a learning plan, including specific audio materials, to their device. The user then answers specific listening questions according to this plan and receives feedback from the server on their specific listening weaknesses based on their results. This feedback is immediately displayed on the device, allowing the user to instantly understand what needs improvement and apply it to their next learning session.

[0245] This embodiment of the invention allows learners to flexibly and efficiently pursue learning according to their individual needs.

[0246] The following describes the processing flow.

[0247] Step 1:

[0248] The server collects learner data from the user's device. This includes recent learning history, usage time, and questions answered and their results. This data is stored in a database on the server.

[0249] Step 2:

[0250] The server analyzes learner data retrieved from the database. Here, a generative AI model is used to analyze user characteristics and learning patterns. In particular, a detailed analysis is performed to identify weaknesses and strengths in answering questions.

[0251] Step 3:

[0252] The server creates an optimized learning plan for each user based on the analysis results. The plan includes what to learn, learning priorities, and recommended content formats (e.g., video, audio, tests).

[0253] Step 4:

[0254] The server sends the generated learning plan to the user's device. After sending, the device receives it and displays it to the user. The user is then ready to begin learning based on this plan.

[0255] Step 5:

[0256] Users follow the learning plan displayed on their device and carry out the corresponding learning activities. For example, they may take listening practice or vocabulary tests, and the results are promptly fed back to the server.

[0257] Step 6:

[0258] The server evaluates the received learning results in real time and generates immediate feedback for each user. The feedback includes specific points about areas where understanding is lacking and suggestions for improvement.

[0259] Step 7:

[0260] After feedback is generated, the server sends that information to the user's device. The device immediately displays the received feedback, allowing the user to review it and use it to improve their next learning session.

[0261] Step 8:

[0262] Depending on the user's progress, the device displays badges and provides level-up notifications. This stimulates the user's motivation to learn and encourages further learning.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] Problems with traditional education systems include insufficient individualized learning and a lack of real-time feedback based on learning progress. This makes it difficult for learners to progress efficiently, potentially leading to a decline in motivation.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for collecting learner history information obtained from individual information processing devices, means for analyzing the collected history information using a generative model to generate an optimized learning plan for each learner, and means for transmitting the generated learning plan to the individual information processing devices. This enables the provision of an optimized learning plan for each learner and immediate suggestions for improvements based on their progress.

[0268] An "information processing device" refers to hardware or software used for collecting, analyzing, and transmitting data.

[0269] "History information" refers to data about a learner's past learning history, study time, and tendency to make incorrect answers.

[0270] A "generative model" refers to artificial intelligence technology that analyzes collected data and generates an optimal educational plan for each learner.

[0271] An "educational plan" refers to a learning guideline that includes recommended learning content and study time, created based on the individual needs of the learner.

[0272] "Immediate improvement suggestions" refer to specific advice on evaluations and areas for improvement based on learning outcomes, which learners can receive in real time.

[0273] "Rewards" refer to means of motivation provided to learners in visual or other forms in accordance with their achievement of learning activities.

[0274] Modes for carrying out the invention

[0275] This invention is an information processing system that provides learners with individually optimized educational plans and supports efficient learning. The following describes how to specifically implement the program of this system.

[0276] The server collects historical information from each learner's individual information processing device. This historical information includes learning history, learning time, and error tendencies. The server analyzes the collected data using a generative AI model to generate an optimized learning plan for each learner. Specifically, this generative AI model identifies the learner's performance patterns based on past learning data and determines what learning content is most effective.

[0277] The generated learning plan is transmitted in real time by the server to individual information processing devices. This plan includes details tailored to the learner's needs, such as recommended learning content and study time. The user's device visually displays this learning plan, providing clear guidance to the learner. It also records progress in real time and sends feedback to the server, allowing the user to receive immediate improvement suggestions from the server.

[0278] Users can progress through their learning by following the educational plan provided via their device. For example, if a user wants to improve their English listening skills, the server generates an educational plan, including specific audio materials, based on the user's past listening test results, and delivers it to the user. The user answers listening questions according to this plan and receives feedback from the server on specific listening weaknesses based on the results. This feedback is immediately displayed on the device, allowing the user to quickly understand what they need to improve.

[0279] As an example of the implementation of this system, it is conceivable to input the following prompt sentence into the generative AI model. "Please generate an optimal listening learning plan based on the user's learning history and tendency to make mistakes." Based on this prompt sentence, the generative AI model generates the most suitable educational plan.

[0280] In this way, the present invention can provide an educational plan suitable for each learner and realize efficient learning.

[0281] The flow of the specific process in Example 1 will be described using FIG. 11.

[0282] Step 1:

[0283] The server acquires history information such as the learning history, learning time, and tendency to answer incorrectly from the user's terminal. This input data indicates the individual learning tendencies of each learner. The server securely collects this data and stores it in storage for later analysis.

[0284] Step 2:

[0285] The server inputs the collected history information into the generative AI model for data analysis. In this analysis process, the AI model identifies the learner's performance pattern and automatically discovers learning issues. As a result, data serving as the basis for an optimized educational plan for each user is output.

[0286] Step 3:

[0287] The server generates an optimized educational plan for each learner based on the data output from the generative AI model. This plan includes specific learning content, recommended learning time, and learning content to be emphasized. This information is customized based on the analysis results obtained in Step 2.

[0288] Step 4:

[0289] The server sends the generated lesson plan to the user's device in real time. The device receives this plan and displays it visually to the learner. The plan displayed on the device includes specific learning tasks and links to related content, allowing the user to start learning immediately.

[0290] Step 5:

[0291] Users engage in learning activities according to the educational plan displayed on the device. Specifically, they practice listening using audio materials and answer questions. As the user progresses through the learning process, the device records their progress.

[0292] Step 6:

[0293] The device sends data generated based on learning progress to the server in real time. The server then immediately analyzes the progress data and generates instant improvement suggestions as needed. This immediate feedback is provided individually based on the analyzed learner's weaknesses.

[0294] Step 7:

[0295] The device receives immediate improvement suggestions sent from the server and displays them clearly to the user. Based on this feedback, the user can modify their learning and apply those improvements to future learning opportunities.

[0296] (Application Example 1)

[0297] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0298] In recent years, there has been a growing demand for personalized education tailored to individual learners. However, providing each learner with an optimal learning plan in real time, along with effective assessment and feedback, is extremely difficult. Furthermore, appropriate methods for maintaining learner motivation are also necessary. This invention aims to solve these problems.

[0299] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0300] In this invention, the server includes means for collecting learner information obtained from individual computer devices, means for analyzing the collected learner information to generate a learning plan optimized for each learner, and means for distributing the generated learning plan to the individual computer devices. This makes it possible to distribute a learning plan optimized for each learner and to provide timely evaluation and feedback.

[0301] An "individualized computer device" is a user-specific computer device used to provide personalized information for learners.

[0302] "Learner information" refers to a collection of various data related to the learner, such as their past learning history, study time, and common areas of error.

[0303] A "learning plan" is an educational plan optimized according to each learner's learning style and history, and includes customized learning materials and schedules.

[0304] "Real-time response" refers to a process of evaluation and feedback provided immediately during learning, enabling learners to immediately recognize and address areas for improvement.

[0305] "Means of strengthening motivation" refer to visual rewards and features that foster a sense of accomplishment, provided to increase learners' motivation to continue learning.

[0306] "Performance evaluation" is a process for quantifying or visualizing the performance skills acquired by learners, evaluating them, and providing improvement suggestions.

[0307] "Adjusting audio content according to skills" is an activity of changing the difficulty level and content of audio teaching materials according to the skill level of learners to provide an optimal learning experience.

[0308] "Delivering content" is a process of transferring the generated learning plan and audio content to the learner's device so that the learner can access it.

[0309] To implement this invention, it first starts with the server collecting learners' data. The individual computer devices used are the user's smartphones, tablets, etc., from which data on past learning history, learning time, or error-prone points are obtained. This data is analyzed by an AI model operating on a cloud server, and it is structured to generate a learning plan optimized for each learner.

[0310] The generated learning plan is delivered to the user's computer device in real time. The receiving computer device visually presents the learning plan to the user and provides feedback according to the progress. As a result, the user can immediately receive evaluations and improvement suggestions through the device, and optimal audio content and content according to their skills are delivered through performance evaluation. Furthermore, visual rewards and elements promoting motivation are incorporated to enhance the sense of achievement.

[0311] As a specific example, it is for a user who wants to improve their music performance skills to use this system. The server analyzes based on the music practiced in the past and the quality of the performance, and proposes an optimal practice plan. The user learns new music and techniques according to this plan and immediately receives feedback on the device. At that time, detailed improvement points regarding the quality of the performance are also presented, and these can be utilized in the next performance.

[0312] An example of a prompt to input into a generative AI model is, "Please suggest the optimal plan based on the user's history and errors in order to generate personalized learning content." This wording is used to create a plan that adapts to the different needs and abilities of each learner.

[0313] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0314] Step 1:

[0315] The server collects learner information from the user's terminal via individual computer devices. Input data includes past learning history, learning time, and common error points. The server aggregates and organizes this input data to create a detailed profile of the target learner. The output is a dataset saved as individual learner profiles.

[0316] Step 2:

[0317] The server analyzes the learner information collected using a generative AI model. The input is the learner profile obtained from step 1. The server inputs the prompt "Propose the optimal plan based on the user's history and errors to generate the personalized learning content to be provided" to the AI ​​model and performs data analysis and prediction. The output is the optimized learning plan.

[0318] Step 3:

[0319] The server delivers the generated learning plan to the user's terminal. The input is the learning plan generated in step 2. The server sends the plan data to each user's terminal, making it available for real-time reception. The output is the learning plan received and displayed on the user's terminal.

[0320] Step 4:

[0321] The terminal visually displays the received learning plan to the user. The input is the learning plan data sent in step 3. The terminal presents the learning plan to the user in an easy-to-understand format through a graphical user interface. The output is a visual display of the learning plan that the user can review.

[0322] Step 5:

[0323] The user engages in learning activities based on a learning plan, and the device monitors their progress. Input consists of user actions and progress data on the learning content. The device sends progress data to the server in real time. Output is data reflecting the learner's progress.

[0324] Step 6:

[0325] The server analyzes progress data sent from the terminal and generates real-time feedback. The input is the learner's progress data received from step 5. The server uses an AI model to evaluate the progress data and makes necessary improvement suggestions and sets achievable goals. The output is immediate evaluation and feedback information.

[0326] Step 7:

[0327] The terminal presents the user with the feedback received from the server. The input is the feedback information generated in step 6. The terminal displays the feedback in a visual and easy-to-understand format so that the user can understand what to do in the next step. The output is the feedback information displayed to the user.

[0328] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0329] This invention is an individually optimized learning system that takes into account the learner's emotional state. In particular, the system aims to maintain the learner's concentration and motivation optimally by recognizing the learner's emotions in real time and reflecting that information in the learning plan and feedback.

[0330] Server operation:

[0331] The server collects emotional data, including learner data, from the user's device. Using the device's built-in camera and sensors, data related to emotions, such as the user's facial expressions, voice tone, and typing speed, is acquired. The server analyzes this data to understand the user's emotional state during learning in real time. The emotion engine categorizes these emotional states into specific categories such as "joy," "surprise," "anger," and "anxiety." This information is used to generate and adjust learning plans optimized for each user.

[0332] Device operation:

[0333] The device quickly displays a pre-adjusted learning plan received from the server. The user then engages in normal learning activities based on this plan, while the device continuously monitors their emotional state. If an emotional change is detected, the device sends this information to the server, triggering a real-time adjustment of the plan. This ensures the user can always learn in the optimal psychological state.

[0334] User experience:

[0335] Users practice English listening, reading, and other skills through learning plans suggested via their devices. For example, if a user shows frustration with a complex problem, the server uses that information to modify the feedback and adjust the difficulty level of the plan as needed. This process is repeated to reduce the learner's burden and support effective skill improvement.

[0336] Specific example:

[0337] As a concrete example, suppose the emotion engine collects data indicating "surprise" or "anxiety" while a user is taking an online English test. The server can analyze this emotional state and adjust the feedback, such as replacing some parts of the learning plan with easier questions or suggesting a break. This emotional data can also be used to generate future plans, enabling users to continue learning in a less stressful environment.

[0338] This invention makes it possible to understand learners' emotions as much as possible and provide a learning environment that is tailored to them, which is expected to improve learning effectiveness and maintain sustained motivation.

[0339] The following describes the processing flow.

[0340] Step 1:

[0341] The user starts a learning session using the device. At this time, the device's built-in camera and microphone are activated and begin collecting data on the user's facial expressions and voice.

[0342] Step 2:

[0343] The device transmits collected emotional data to the server in real time. This data includes facial expressions, voice tone, and typing speed.

[0344] Step 3:

[0345] The server analyzes the received emotional data using an emotion engine. This analysis classifies the user's current emotional state into categories such as "joy," "surprise," "anxiety," and "concentration."

[0346] Step 4:

[0347] The server uses the analysis results to create or adjust a learning plan optimized for the user. If the user's emotional state is unstable, adjustments will be made, such as lowering the difficulty of the tasks or suggesting a break.

[0348] Step 5:

[0349] The server sends the adjusted learning plan to the user's device, which then displays it. The user continues learning according to the newly adjusted plan.

[0350] Step 6:

[0351] As users continue their learning activities, their emotional state may change. In such cases, the device continues to collect emotional data and send it to the server.

[0352] Step 7:

[0353] The server re-analyzes the received sentiment data and makes further adjustments to the learning plan as needed. It also instantly generates and sends appropriate feedback to the device.

[0354] Step 8:

[0355] The device displays feedback and adjustments sent from the server to the user. Based on this, the user continues learning and aims for efficient skill improvement.

[0356] In this way, by continuously optimizing the learning environment, users can always learn in the optimal psychological state.

[0357] (Example 2)

[0358] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0359] Despite the significant impact learners' emotional states on learning effectiveness and motivation, traditional learning systems have not adequately considered this. As a result, learners often face stress and decreased motivation, leading to reduced learning efficiency. This issue is particularly pronounced in online and remote learning environments, and a solution has been urgently needed.

[0360] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0361] In this invention, the server includes means for collecting learner data, including emotional data acquired from individual terminals; means for analyzing the collected learner data and generating a learning plan optimized for each learner based on the emotional information; and means for providing real-time feedback to learners based on the delivered learning plan and adjusting the plan according to changes in their emotions. This makes it possible to provide flexible learning plans that take into account the learner's emotional state, thereby maximizing the learner's motivation and efficiency.

[0362] An "individual terminal" is an information processing device used by a user, equipped with a camera and sensors, and capable of collecting user emotional data.

[0363] "Emotional data" refers to data that indicates a user's emotional state, including information such as facial expressions, voice, and typing speed.

[0364] "Learner data" refers to all information about the user, including past learning history, learning time, and sentiment data.

[0365] A "learning plan" refers to a learning plan and content optimized according to each learner's emotional state and individual characteristics.

[0366] "Real-time feedback" refers to the provision of immediate responses and information generated while the user is engaged in learning activities.

[0367] "Means of collection" refers to methods and devices that include the process of acquiring data from a terminal and transferring it to a server.

[0368] "Means of analysis" refers to the processes and devices used to analyze collected data and classify emotional states and learning patterns.

[0369] "Means of delivery" refers to methods or devices for sending learning plans, generated or adjusted on the server side, to the terminal.

[0370] Modes for carrying out the invention

[0371] This invention is an individually optimized learning system that takes into account the emotional state of the learner, and is mainly composed of three subjects: server, terminal, and user.

[0372] Server operation

[0373] The server collects learner data, including emotional data, from individual terminals. This data includes facial expressions, voice tone, and typing speed captured by the terminal's built-in cameras and sensors. The collected data is analyzed in real time using an emotional AI model and classified into specific emotional categories such as "joy," "surprise," "anger," and "anxiety." This analysis information is used by a generative AI model to generate learning plans optimized for each learner.

[0374] Terminal operation

[0375] The device quickly displays the adjusted learning plan received from the server. While the user progresses through learning activities based on this plan, the device continuously monitors their emotional state. If a change in emotion is detected, the device notifies the server, immediately triggering a readjustment of the plan.

[0376] User experience

[0377] Users practice skills such as listening and reading using learning plans suggested through their devices. For example, if a user shows frustration when faced with a difficult problem, the server uses that information to modify the feedback, either lowering the difficulty of the plan or suggesting a break. This allows users to improve their skills effectively while reducing their burden.

[0378] Examples of specific cases and prompt statements

[0379] For example, while a user is taking an online English test, emotional data such as "surprise" or "anxiety" may be collected. The server analyzes this emotional state and either replaces parts of the learning plan with easier questions or suggests a break. This data is also used to generate future plans, providing a learning environment that reduces stress.

[0380] An example of a prompt could be input to the generative AI model as, "How would you optimize the learning plan when a user expresses frustration with a complex problem?" Based on this prompt, it is possible to suggest more efficient feedback.

[0381] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0382] Step 1:

[0383] The device collects user emotion data. This data collection is done by capturing the user's facial expressions with a built-in camera and analyzing their voice tone with a microphone. It also records keyboard typing speed. This provides raw data about the user's emotions. The input is this emotion-related data, which is output as formatted data for the next step.

[0384] Step 2:

[0385] The server receives emotional data collected from the terminal and begins data analysis. Using an emotional AI model, it classifies the data into specific categories such as "joy," "surprise," "anger," and "anxiety." In this process, the input is formatted emotional data, and the output is the result of the analyzed emotional state.

[0386] Step 3:

[0387] The server utilizes a generated AI model based on the analysis results to create an optimized learning plan for each learner. The data processing performed here involves generating and adjusting the learning plan to reflect the learner's emotional state. The input is the analyzed emotional state, and the output is the optimized learning plan.

[0388] Step 4:

[0389] The server sends the generated learning plan to the terminal. The terminal displays the received learning plan to the user and prompts them to begin learning activities. The input is the optimized learning plan, and the output is the presentation of the learning plan to the user.

[0390] Step 5:

[0391] The user engages in learning activities based on a learning plan received through the device. During this time, the device continuously monitors emotional data in real time and sends new data to the server if any changes occur. The input is the user's learning activities, and the output is additional emotional data.

[0392] Step 6:

[0393] The server analyzes new emotion data transmitted in real time and updates the learning plan as needed. This allows for flexible adjustments tailored to the user's emotional state. The input is real-time emotion data, and the output is the learning plan adjusted as needed.

[0394] (Application Example 2)

[0395] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0396] The present invention aims to provide a system that can enhance the user's shopping experience in a virtual environment by recognizing the user's emotional state in real time, providing individually optimized product suggestions based on that information, and thereby increasing their desire to purchase. Specifically, it solves the problem of instantly presenting relevant information and content according to the products and categories that the user has shown interest in, thereby supporting more intuitive and effective purchasing activities.

[0397] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0398] In this invention, the server includes means for collecting user information obtained from individual devices, means for analyzing the collected user information to generate personalized recommendations for each user, and means for distributing the generated recommendations to individual devices to improve the purchasing experience in the virtual environment. This enables personalized recommendations that reflect the user's emotional state, thereby effectively increasing the user's willingness to purchase.

[0399] A "device" refers to an electronic device used for information processing and communication. It plays a role in directly interacting with the user through a user interface and is capable of inputting and outputting data.

[0400] "User information" refers to the data obtained from device usage and user behavior. This information is analyzed to understand the characteristics of individual users.

[0401] "Emotional state" refers to the state in which a user expresses their emotions. It is usually determined in real time based on data obtained from facial expressions, tone of voice, body movements, etc.

[0402] "Optimized recommendations" refer to product and service recommendations tailored to the user's emotional state and interests. This provides information that improves user satisfaction.

[0403] A "virtual environment" refers to a virtual space constructed using digital technology that is different from the real world. Users can have visual and auditory experiences within this environment through their devices.

[0404] "Purchase intent" refers to a user's desire or motivation to buy a product or service. This intent can be strengthened by individually tailored information and suggestions.

[0405] The system that realizes this invention has a structure in which a server, a terminal, and a user cooperate to function.

[0406] The server receives information collected from the user's device in real time and analyzes their emotional state. The server uses emotion recognition software such as OpenCV or TensorFlow to capture image and audio data and understand the user's current emotional state. The analyzed data is used to build personalized product recommendations using a generative AI model. These product recommendations are sent to the user's device and used to improve the user experience.

[0407] The terminal displays product suggestions received from the server in a way that users can visually confirm within a virtual environment. This information is displayed through a user interface on terminals such as smart glasses or head-mounted displays. The device also has the ability to send new data to the server that may reflect changes in the user's emotional state. This ensures that the system operates in a way that optimized suggestions are constantly updated.

[0408] Users can view product suggestions and content presented through their devices and research items that interest them in more detail. User behavior and reactions are continuously recorded and transmitted to the server, allowing the system to adapt to the user environment more effectively.

[0409] As a concrete example, suppose a user is browsing a virtual store and finds a gadget that interests them. The device detects the user's "excitement" and sends that information to a server. The server performs appropriate data analysis and presents similar products or content of interest to the user's glasses. In this case, possible prompts for the generative AI model might be as follows:

[0410] "If a user's emotion indicates excitement, generate a list of gadget products corresponding to that emotion and display it along with detailed reviews."

[0411] In this way, the system can always provide a customized purchasing experience that responds to the user's emotions.

[0412] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0413] Step 1:

[0414] The server receives information from the user's device in real time. It takes video and audio data transmitted from the device as input. This data is then used with emotion recognition software such as OpenCV or TensorFlow to perform calculations to detect the user's current emotional state. The output is data representing the analyzed emotional state.

[0415] Step 2:

[0416] The server uses a generative AI model to build individually optimized product recommendations based on the emotional state data obtained in Step 1. The inputs used are emotional state data and past user behavior data. The AI ​​model generates a list of products and content tailored to the user's interests and emotions. The output includes a set of individually tailored product recommendations.

[0417] Step 3:

[0418] The server sends generated product suggestions to the user's device. The device receives the product suggestions from the server as input and presents them to the user visually in a virtual environment. Interactive feedback from the user can also be incorporated. As output, the products selected by the user and content of interest are displayed on the device.

[0419] Step 4:

[0420] Users view product suggestions displayed through their devices and search for detailed information on items that interest them. The user's actions and responses are then sent back to the server via the device. This input includes user selections and browsing history, which the server further analyzes to provide foundational data for updating subsequent suggestions. The output generates new suggestions and feedback tailored to the user's actions.

[0421] Step 5:

[0422] The server continuously tracks user behavior based on updated suggestions, feeding appropriate data and prompt messages to an AI model, and then provides continuous service based on the results obtained. An example of such a prompt message is: "If the user's emotion indicates excitement, generate a list of gadget products corresponding to that emotion and display them along with detailed reviews."

[0423] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0424] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0425] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0426] [Third Embodiment]

[0427] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0428] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0429] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0430] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0431] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0432] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0433] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0434] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0435] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0436] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0437] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0438] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0439] This invention relates to an individualized learning system using an information processing device. The aim of this system is to improve learning efficiency by providing an optimal learning plan using each learner's data.

[0440] Server operation:

[0441] The server collects learner data from the user's information processing device. This includes the user's learning history, learning time, and information about past errors. The server analyzes the collected data and uses generative AI to identify learner patterns. Based on this analysis, the server creates a learning plan tailored to each learner. This learning plan includes recommended learning content and time, and incorporates a variety of content formats. The created plan is transmitted by the server to the user's information processing device in real time.

[0442] Device operation:

[0443] The terminal, i.e., the user's device, visually displays the learning plan received from the server. The user can proceed with their learning according to the plan provided through the terminal. The terminal also has the function of sending progress data generated during learning to the server in real time, and receives immediate feedback from the server. The terminal displays this feedback to the user in an easy-to-understand manner, helping them to understand the learning content.

[0444] User experience:

[0445] Users follow a learning plan received through their device, completing tasks such as vocabulary exercises and listening practice. Real-time feedback is displayed on the device as they progress, allowing users to immediately identify their weaknesses and use that information to improve. Furthermore, the device incorporates a gamification function, enabling users to earn badges based on their achieved goals. This helps maintain user motivation for learning.

[0446] Specific example:

[0447] For example, if a user wants to improve their English listening skills, the server analyzes their past listening test results and delivers a learning plan, including specific audio materials, to their device. The user then answers specific listening questions according to this plan and receives feedback from the server on their specific listening weaknesses based on their results. This feedback is immediately displayed on the device, allowing the user to instantly understand what needs improvement and apply it to their next learning session.

[0448] This embodiment of the invention allows learners to flexibly and efficiently pursue learning according to their individual needs.

[0449] The following describes the processing flow.

[0450] Step 1:

[0451] The server collects learner data from the user's device. This includes recent learning history, usage time, and questions answered and their results. This data is stored in a database on the server.

[0452] Step 2:

[0453] The server analyzes learner data retrieved from the database. Here, a generative AI model is used to analyze user characteristics and learning patterns. In particular, a detailed analysis is performed to identify weaknesses and strengths in answering questions.

[0454] Step 3:

[0455] The server creates an optimized learning plan for each user based on the analysis results. The plan includes what to learn, learning priorities, and recommended content formats (e.g., video, audio, tests).

[0456] Step 4:

[0457] The server sends the generated learning plan to the user's device. After sending, the device receives it and displays it to the user. The user is then ready to begin learning based on this plan.

[0458] Step 5:

[0459] Users follow the learning plan displayed on their device and carry out the corresponding learning activities. For example, they may take listening practice or vocabulary tests, and the results are promptly fed back to the server.

[0460] Step 6:

[0461] The server evaluates the received learning results in real time and generates immediate feedback for each user. The feedback includes specific points about areas where understanding is lacking and suggestions for improvement.

[0462] Step 7:

[0463] After feedback is generated, the server sends that information to the user's device. The device immediately displays the received feedback, allowing the user to review it and use it to improve their next learning session.

[0464] Step 8:

[0465] Depending on the user's progress, the device displays badges and provides level-up notifications. This stimulates the user's motivation to learn and encourages further learning.

[0466] (Example 1)

[0467] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0468] Problems with traditional education systems include insufficient individualized learning and a lack of real-time feedback based on learning progress. This makes it difficult for learners to progress efficiently, potentially leading to a decline in motivation.

[0469] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0470] In this invention, the server includes means for collecting learner history information obtained from individual information processing devices, means for analyzing the collected history information using a generative model to generate an optimized learning plan for each learner, and means for transmitting the generated learning plan to the individual information processing devices. This enables the provision of an optimized learning plan for each learner and immediate suggestions for improvements based on their progress.

[0471] An "information processing device" refers to hardware or software used for collecting, analyzing, and transmitting data.

[0472] "History information" refers to data about a learner's past learning history, study time, and tendency to make incorrect answers.

[0473] A "generative model" refers to artificial intelligence technology that analyzes collected data and generates an optimal educational plan for each learner.

[0474] An "educational plan" refers to a learning guideline that includes recommended learning content and study time, created based on the individual needs of the learner.

[0475] "Immediate improvement suggestions" refer to specific advice on evaluations and areas for improvement based on learning outcomes, which learners can receive in real time.

[0476] "Rewards" refer to means of motivation provided to learners in visual or other forms in accordance with their achievement of learning activities.

[0477] Modes for carrying out the invention

[0478] This invention is an information processing system that provides learners with individually optimized educational plans and supports efficient learning. The following describes how to specifically implement the program of this system.

[0479] The server collects historical information from each learner's individual information processing device. This historical information includes learning history, learning time, and error tendencies. The server analyzes the collected data using a generative AI model to generate an optimized learning plan for each learner. Specifically, this generative AI model identifies the learner's performance patterns based on past learning data and determines what learning content is most effective.

[0480] The generated learning plan is transmitted in real time by the server to individual information processing devices. This plan includes details tailored to the learner's needs, such as recommended learning content and study time. The user's device visually displays this learning plan, providing clear guidance to the learner. It also records progress in real time and sends feedback to the server, allowing the user to receive immediate improvement suggestions from the server.

[0481] Users can progress through their learning by following the educational plan provided via their device. For example, if a user wants to improve their English listening skills, the server generates an educational plan, including specific audio materials, based on the user's past listening test results, and delivers it to the user. The user answers listening questions according to this plan and receives feedback from the server on specific listening weaknesses based on the results. This feedback is immediately displayed on the device, allowing the user to quickly understand what they need to improve.

[0482] As an example of implementing this system, one could input the following prompt into the generative AI model: "Generate the optimal listening learning plan based on the user's learning history and error patterns." Based on this prompt, the generative AI model would generate the most suitable educational plan.

[0483] In this way, the present invention can provide an educational plan tailored to each learner, thereby enabling efficient learning.

[0484] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0485] Step 1:

[0486] The server retrieves historical information from the user's terminal, such as learning history, learning time, and error tendencies. This input data reflects each learner's individual learning tendencies. The server securely collects this data and stores it in storage for later analysis.

[0487] Step 2:

[0488] The server inputs the collected historical information into a generating AI model for data analysis. During this analysis process, the AI ​​model identifies learner performance patterns and automatically discovers learning challenges. As a result, data is output that forms the basis for an optimized educational plan for each user.

[0489] Step 3:

[0490] The server generates an optimized learning plan for each learner based on the data output from the generated AI model. This plan includes specific learning content, recommended study time, and areas to focus on. This information is customized based on the analysis results obtained in Step 2.

[0491] Step 4:

[0492] The server sends the generated lesson plan to the user's device in real time. The device receives this plan and displays it visually to the learner. The plan displayed on the device includes specific learning tasks and links to related content, allowing the user to start learning immediately.

[0493] Step 5:

[0494] Users engage in learning activities according to the educational plan displayed on the device. Specifically, they practice listening using audio materials and answer questions. As the user progresses through the learning process, the device records their progress.

[0495] Step 6:

[0496] The device sends data generated based on learning progress to the server in real time. The server then immediately analyzes the progress data and generates instant improvement suggestions as needed. This immediate feedback is provided individually based on the analyzed learner's weaknesses.

[0497] Step 7:

[0498] The device receives immediate improvement suggestions sent from the server and displays them clearly to the user. Based on this feedback, the user can modify their learning and apply those improvements to future learning opportunities.

[0499] (Application Example 1)

[0500] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0501] In recent years, there has been a growing demand for personalized education tailored to individual learners. However, providing each learner with an optimal learning plan in real time, along with effective assessment and feedback, is extremely difficult. Furthermore, appropriate methods for maintaining learner motivation are also necessary. This invention aims to solve these problems.

[0502] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0503] In this invention, the server includes means for collecting learner information obtained from individual computer devices, means for analyzing the collected learner information to generate a learning plan optimized for each learner, and means for distributing the generated learning plan to the individual computer devices. This makes it possible to distribute a learning plan optimized for each learner and to provide timely evaluation and feedback.

[0504] An "individualized computer device" is a user-specific computer device used to provide personalized information for learners.

[0505] "Learner information" refers to a collection of various data related to the learner, such as their past learning history, study time, and common areas of error.

[0506] A "learning plan" is an educational plan optimized according to each learner's learning style and history, and includes customized learning materials and schedules.

[0507] "Real-time response" refers to a process of evaluation and feedback provided immediately during learning, enabling learners to immediately recognize and address areas for improvement.

[0508] "Means of strengthening motivation" refer to visual rewards and features that foster a sense of accomplishment, provided to increase learners' motivation to continue learning.

[0509] "Performance evaluation" is a process for quantifying or visualizing the performance skills acquired by learners and providing suggestions for improvement.

[0510] "Adjusting audio content to match skill level" refers to the activity of changing the difficulty level and content of audio materials according to the learner's skill level to provide an optimal learning experience.

[0511] "Delivering content" refers to the process of transferring generated learning plans and audio content to learners' devices, making them accessible to them.

[0512] To implement this invention, the process begins with the server collecting learner data. This data is obtained from individual computing devices such as the user's smartphone or tablet, including past learning history, learning time, and points where the user is prone to errors. This data is then analyzed by an AI model running on a cloud server, generating a learning plan optimized for each individual learner.

[0513] The generated learning plan is delivered to the user's computer in real time. The receiving computer visually presents the learning plan to the user and provides feedback based on their progress. This allows the user to receive immediate evaluations and improvement suggestions through their device, and the optimal audio content and materials are delivered based on their skill level through performance evaluations. Furthermore, visual rewards and motivational elements are incorporated to enhance a sense of accomplishment.

[0514] A concrete example is a user who wants to improve their musical performance skills using this system. The server analyzes the user's past practiced songs and performance quality, and proposes an optimal practice plan. The user learns new songs and techniques according to this plan and receives immediate feedback on their device. At that time, detailed areas for improvement regarding the quality of their performance are also pointed out, which they can use to improve their next performance.

[0515] An example of a prompt to input into a generative AI model is, "Please suggest the optimal plan based on the user's history and errors in order to generate personalized learning content." This wording is used to create a plan that adapts to the different needs and abilities of each learner.

[0516] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0517] Step 1:

[0518] The server collects learner information from the user's terminal via individual computer devices. Input data includes past learning history, learning time, and common error points. The server aggregates and organizes this input data to create a detailed profile of the target learner. The output is a dataset saved as individual learner profiles.

[0519] Step 2:

[0520] The server analyzes the learner information collected using a generative AI model. The input is the learner profile obtained from step 1. The server inputs the prompt "Propose the optimal plan based on the user's history and errors to generate the personalized learning content to be provided" to the AI ​​model and performs data analysis and prediction. The output is the optimized learning plan.

[0521] Step 3:

[0522] The server delivers the generated learning plan to the user's terminal. The input is the learning plan generated in step 2. The server sends the plan data to each user's terminal, making it available for real-time reception. The output is the learning plan received and displayed on the user's terminal.

[0523] Step 4:

[0524] The terminal visually displays the received learning plan to the user. The input is the learning plan data sent in step 3. The terminal presents the learning plan to the user in an easy-to-understand format through a graphical user interface. The output is a visual display of the learning plan that the user can review.

[0525] Step 5:

[0526] The user engages in learning activities based on a learning plan, and the device monitors their progress. Input consists of user actions and progress data on the learning content. The device sends progress data to the server in real time. Output is data reflecting the learner's progress.

[0527] Step 6:

[0528] The server analyzes progress data sent from the terminal and generates real-time feedback. The input is the learner's progress data received from step 5. The server uses an AI model to evaluate the progress data and makes necessary improvement suggestions and sets achievable goals. The output is immediate evaluation and feedback information.

[0529] Step 7:

[0530] The terminal presents the user with the feedback received from the server. The input is the feedback information generated in step 6. The terminal displays the feedback in a visual and easy-to-understand format so that the user can understand what to do in the next step. The output is the feedback information displayed to the user.

[0531] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0532] This invention is an individually optimized learning system that takes into account the learner's emotional state. In particular, the system aims to maintain the learner's concentration and motivation optimally by recognizing the learner's emotions in real time and reflecting that information in the learning plan and feedback.

[0533] Server operation:

[0534] The server collects emotional data, including learner data, from the user's device. Using the device's built-in camera and sensors, data related to emotions, such as the user's facial expressions, voice tone, and typing speed, is acquired. The server analyzes this data to understand the user's emotional state during learning in real time. The emotion engine categorizes these emotional states into specific categories such as "joy," "surprise," "anger," and "anxiety." This information is used to generate and adjust learning plans optimized for each user.

[0535] Device operation:

[0536] The device quickly displays a pre-adjusted learning plan received from the server. The user then engages in normal learning activities based on this plan, while the device continuously monitors their emotional state. If an emotional change is detected, the device sends this information to the server, triggering a real-time adjustment of the plan. This ensures the user can always learn in the optimal psychological state.

[0537] User experience:

[0538] Users practice English listening, reading, and other skills through learning plans suggested via their devices. For example, if a user shows frustration with a complex problem, the server uses that information to modify the feedback and adjust the difficulty level of the plan as needed. This process is repeated to reduce the learner's burden and support effective skill improvement.

[0539] Specific example:

[0540] As a concrete example, suppose the emotion engine collects data indicating "surprise" or "anxiety" while a user is taking an online English test. The server can analyze this emotional state and adjust the feedback, such as replacing some parts of the learning plan with easier questions or suggesting a break. This emotional data can also be used to generate future plans, enabling users to continue learning in a less stressful environment.

[0541] This invention makes it possible to understand learners' emotions as much as possible and provide a learning environment that is tailored to them, which is expected to improve learning effectiveness and maintain sustained motivation.

[0542] The following describes the processing flow.

[0543] Step 1:

[0544] The user starts a learning session using the device. At this time, the device's built-in camera and microphone are activated and begin collecting data on the user's facial expressions and voice.

[0545] Step 2:

[0546] The device transmits collected emotional data to the server in real time. This data includes facial expressions, voice tone, and typing speed.

[0547] Step 3:

[0548] The server analyzes the received emotional data using an emotion engine. This analysis classifies the user's current emotional state into categories such as "joy," "surprise," "anxiety," and "concentration."

[0549] Step 4:

[0550] The server uses the analysis results to create or adjust a learning plan optimized for the user. If the user's emotional state is unstable, adjustments will be made, such as lowering the difficulty of the tasks or suggesting a break.

[0551] Step 5:

[0552] The server sends the adjusted learning plan to the user's device, which then displays it. The user continues learning according to the newly adjusted plan.

[0553] Step 6:

[0554] As users continue their learning activities, their emotional state may change. In such cases, the device continues to collect emotional data and send it to the server.

[0555] Step 7:

[0556] The server re-analyzes the received sentiment data and makes further adjustments to the learning plan as needed. It also instantly generates and sends appropriate feedback to the device.

[0557] Step 8:

[0558] The device displays feedback and adjustments sent from the server to the user. Based on this, the user continues learning and aims for efficient skill improvement.

[0559] In this way, by continuously optimizing the learning environment, users can always learn in the optimal psychological state.

[0560] (Example 2)

[0561] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0562] Despite the significant impact learners' emotional states on learning effectiveness and motivation, traditional learning systems have not adequately considered this. As a result, learners often face stress and decreased motivation, leading to reduced learning efficiency. This issue is particularly pronounced in online and remote learning environments, and a solution has been urgently needed.

[0563] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0564] In this invention, the server includes means for collecting learner data, including emotional data acquired from individual terminals; means for analyzing the collected learner data and generating a learning plan optimized for each learner based on the emotional information; and means for providing real-time feedback to learners based on the delivered learning plan and adjusting the plan according to changes in their emotions. This makes it possible to provide flexible learning plans that take into account the learner's emotional state, thereby maximizing the learner's motivation and efficiency.

[0565] An "individual terminal" is an information processing device used by a user, equipped with a camera and sensors, and capable of collecting user emotional data.

[0566] "Emotional data" refers to data that indicates a user's emotional state, including information such as facial expressions, voice, and typing speed.

[0567] "Learner data" refers to all information about the user, including past learning history, learning time, and sentiment data.

[0568] A "learning plan" refers to a learning plan and content optimized according to each learner's emotional state and individual characteristics.

[0569] "Real-time feedback" refers to the provision of immediate responses and information generated while the user is engaged in learning activities.

[0570] "Means of collection" refers to methods and devices that include the process of acquiring data from a terminal and transferring it to a server.

[0571] "Means of analysis" refers to the processes and devices used to analyze collected data and classify emotional states and learning patterns.

[0572] "Means of delivery" refers to methods or devices for sending learning plans, generated or adjusted on the server side, to the terminal.

[0573] Modes for carrying out the invention

[0574] This invention is an individually optimized learning system that takes into account the emotional state of the learner, and is mainly composed of three subjects: server, terminal, and user.

[0575] Server operation

[0576] The server collects learner data, including emotional data, from individual terminals. This data includes facial expressions, voice tone, and typing speed captured by the terminal's built-in cameras and sensors. The collected data is analyzed in real time using an emotional AI model and classified into specific emotional categories such as "joy," "surprise," "anger," and "anxiety." This analysis information is used by a generative AI model to generate learning plans optimized for each learner.

[0577] Terminal operation

[0578] The device quickly displays the adjusted learning plan received from the server. While the user progresses through learning activities based on this plan, the device continuously monitors their emotional state. If a change in emotion is detected, the device notifies the server, immediately triggering a readjustment of the plan.

[0579] User experience

[0580] Users practice skills such as listening and reading using learning plans suggested through their devices. For example, if a user shows frustration when faced with a difficult problem, the server uses that information to modify the feedback, either lowering the difficulty of the plan or suggesting a break. This allows users to improve their skills effectively while reducing their burden.

[0581] Examples of specific cases and prompt statements

[0582] For example, while a user is taking an online English test, emotional data such as "surprise" or "anxiety" may be collected. The server analyzes this emotional state and either replaces parts of the learning plan with easier questions or suggests a break. This data is also used to generate future plans, providing a learning environment that reduces stress.

[0583] An example of a prompt could be input to the generative AI model as, "How would you optimize the learning plan when a user expresses frustration with a complex problem?" Based on this prompt, it is possible to suggest more efficient feedback.

[0584] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0585] Step 1:

[0586] The device collects user emotion data. This data collection is done by capturing the user's facial expressions with a built-in camera and analyzing their voice tone with a microphone. It also records keyboard typing speed. This provides raw data about the user's emotions. The input is this emotion-related data, which is output as formatted data for the next step.

[0587] Step 2:

[0588] The server receives emotional data collected from the terminal and begins data analysis. Using an emotional AI model, it classifies the data into specific categories such as "joy," "surprise," "anger," and "anxiety." In this process, the input is formatted emotional data, and the output is the result of the analyzed emotional state.

[0589] Step 3:

[0590] The server utilizes a generated AI model based on the analysis results to create an optimized learning plan for each learner. The data processing performed here involves generating and adjusting the learning plan to reflect the learner's emotional state. The input is the analyzed emotional state, and the output is the optimized learning plan.

[0591] Step 4:

[0592] The server sends the generated learning plan to the terminal. The terminal displays the received learning plan to the user and prompts them to begin learning activities. The input is the optimized learning plan, and the output is the presentation of the learning plan to the user.

[0593] Step 5:

[0594] The user engages in learning activities based on a learning plan received through the device. During this time, the device continuously monitors emotional data in real time and sends new data to the server if any changes occur. The input is the user's learning activities, and the output is additional emotional data.

[0595] Step 6:

[0596] The server analyzes new emotion data transmitted in real time and updates the learning plan as needed. This allows for flexible adjustments tailored to the user's emotional state. The input is real-time emotion data, and the output is the learning plan adjusted as needed.

[0597] (Application Example 2)

[0598] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0599] The present invention aims to provide a system that can enhance the user's shopping experience in a virtual environment by recognizing the user's emotional state in real time, providing individually optimized product suggestions based on that information, and thereby increasing their desire to purchase. Specifically, it solves the problem of instantly presenting relevant information and content according to the products and categories that the user has shown interest in, thereby supporting more intuitive and effective purchasing activities.

[0600] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0601] In this invention, the server includes means for collecting user information obtained from individual devices, means for analyzing the collected user information to generate personalized recommendations for each user, and means for distributing the generated recommendations to individual devices to improve the purchasing experience in the virtual environment. This enables personalized recommendations that reflect the user's emotional state, thereby effectively increasing the user's willingness to purchase.

[0602] A "device" refers to an electronic device used for information processing and communication. It plays a role in directly interacting with the user through a user interface and is capable of inputting and outputting data.

[0603] "User information" refers to the data obtained from device usage and user behavior. This information is analyzed to understand the characteristics of individual users.

[0604] "Emotional state" refers to the state in which a user expresses their emotions. It is usually determined in real time based on data obtained from facial expressions, tone of voice, body movements, etc.

[0605] "Optimized recommendations" refer to product and service recommendations tailored to the user's emotional state and interests. This provides information that improves user satisfaction.

[0606] A "virtual environment" refers to a virtual space constructed using digital technology that is different from the real world. Users can have visual and auditory experiences within this environment through their devices.

[0607] "Purchase intent" refers to a user's desire or motivation to buy a product or service. This intent can be strengthened by individually tailored information and suggestions.

[0608] The system that realizes this invention has a structure in which a server, a terminal, and a user cooperate to function.

[0609] The server receives information collected from the user's device in real time and analyzes their emotional state. The server uses emotion recognition software such as OpenCV or TensorFlow to capture image and audio data and understand the user's current emotional state. The analyzed data is used to build personalized product recommendations using a generative AI model. These product recommendations are sent to the user's device and used to improve the user experience.

[0610] The terminal displays product suggestions received from the server in a way that users can visually confirm within a virtual environment. This information is displayed through a user interface on terminals such as smart glasses or head-mounted displays. The device also has the ability to send new data to the server that may reflect changes in the user's emotional state. This ensures that the system operates in a way that optimized suggestions are constantly updated.

[0611] Users can view product suggestions and content presented through their devices and research items that interest them in more detail. User behavior and reactions are continuously recorded and transmitted to the server, allowing the system to adapt to the user environment more effectively.

[0612] As a concrete example, suppose a user is browsing a virtual store and finds a gadget that interests them. The device detects the user's "excitement" and sends that information to a server. The server performs appropriate data analysis and presents similar products or content of interest to the user's glasses. In this case, possible prompts for the generative AI model might be as follows:

[0613] "If a user's emotion indicates excitement, generate a list of gadget products corresponding to that emotion and display it along with detailed reviews."

[0614] In this way, the system can always provide a customized purchasing experience that responds to the user's emotions.

[0615] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0616] Step 1:

[0617] The server receives information from the user's device in real time. It takes video and audio data transmitted from the device as input. This data is then used with emotion recognition software such as OpenCV or TensorFlow to perform calculations to detect the user's current emotional state. The output is data representing the analyzed emotional state.

[0618] Step 2:

[0619] The server uses a generative AI model to build individually optimized product recommendations based on the emotional state data obtained in Step 1. The inputs used are emotional state data and past user behavior data. The AI ​​model generates a list of products and content tailored to the user's interests and emotions. The output includes a set of individually tailored product recommendations.

[0620] Step 3:

[0621] The server sends generated product suggestions to the user's device. The device receives the product suggestions from the server as input and presents them to the user visually in a virtual environment. Interactive feedback from the user can also be incorporated. As output, the products selected by the user and content of interest are displayed on the device.

[0622] Step 4:

[0623] Users view product suggestions displayed through their devices and search for detailed information on items that interest them. The user's actions and responses are then sent back to the server via the device. This input includes user selections and browsing history, which the server further analyzes to provide foundational data for updating subsequent suggestions. The output generates new suggestions and feedback tailored to the user's actions.

[0624] Step 5:

[0625] The server continuously tracks user behavior based on updated suggestions, feeding appropriate data and prompt messages to an AI model, and then provides continuous service based on the results obtained. An example of such a prompt message is: "If the user's emotion indicates excitement, generate a list of gadget products corresponding to that emotion and display them along with detailed reviews."

[0626] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0627] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0628] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0629] [Fourth Embodiment]

[0630] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0631] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0632] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0633] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0634] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0635] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0636] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0637] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0638] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0639] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0640] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0641] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0642] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0643] This invention relates to an individualized learning system using an information processing device. The aim of this system is to improve learning efficiency by providing an optimal learning plan using each learner's data.

[0644] Server operation:

[0645] The server collects learner data from the user's information processing device. This includes the user's learning history, learning time, and information about past errors. The server analyzes the collected data and uses generative AI to identify learner patterns. Based on this analysis, the server creates a learning plan tailored to each learner. This learning plan includes recommended learning content and time, and incorporates a variety of content formats. The created plan is transmitted by the server to the user's information processing device in real time.

[0646] Device operation:

[0647] The terminal, i.e., the user's device, visually displays the learning plan received from the server. The user can proceed with their learning according to the plan provided through the terminal. The terminal also has the function of sending progress data generated during learning to the server in real time, and receives immediate feedback from the server. The terminal displays this feedback to the user in an easy-to-understand manner, helping them to understand the learning content.

[0648] User experience:

[0649] Users follow a learning plan received through their device, completing tasks such as vocabulary exercises and listening practice. Real-time feedback is displayed on the device as they progress, allowing users to immediately identify their weaknesses and use that information to improve. Furthermore, the device incorporates a gamification function, enabling users to earn badges based on their achieved goals. This helps maintain user motivation for learning.

[0650] Specific example:

[0651] For example, if a user wants to improve their English listening skills, the server analyzes their past listening test results and delivers a learning plan, including specific audio materials, to their device. The user then answers specific listening questions according to this plan and receives feedback from the server on their specific listening weaknesses based on their results. This feedback is immediately displayed on the device, allowing the user to instantly understand what needs improvement and apply it to their next learning session.

[0652] This embodiment of the invention allows learners to flexibly and efficiently pursue learning according to their individual needs.

[0653] The following describes the processing flow.

[0654] Step 1:

[0655] The server collects learner data from the user's device. This includes recent learning history, usage time, and questions answered and their results. This data is stored in a database on the server.

[0656] Step 2:

[0657] The server analyzes learner data retrieved from the database. Here, a generative AI model is used to analyze user characteristics and learning patterns. In particular, a detailed analysis is performed to identify weaknesses and strengths in answering questions.

[0658] Step 3:

[0659] The server creates an optimized learning plan for each user based on the analysis results. The plan includes what to learn, learning priorities, and recommended content formats (e.g., video, audio, tests).

[0660] Step 4:

[0661] The server sends the generated learning plan to the user's device. After sending, the device receives it and displays it to the user. The user is then ready to begin learning based on this plan.

[0662] Step 5:

[0663] Users follow the learning plan displayed on their device and carry out the corresponding learning activities. For example, they may take listening practice or vocabulary tests, and the results are promptly fed back to the server.

[0664] Step 6:

[0665] The server evaluates the received learning results in real time and generates immediate feedback for each user. The feedback includes specific points about areas where understanding is lacking and suggestions for improvement.

[0666] Step 7:

[0667] After feedback is generated, the server sends that information to the user's device. The device immediately displays the received feedback, allowing the user to review it and use it to improve their next learning session.

[0668] Step 8:

[0669] Depending on the user's progress, the device displays badges and provides level-up notifications. This stimulates the user's motivation to learn and encourages further learning.

[0670] (Example 1)

[0671] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0672] Problems with traditional education systems include insufficient individualized learning and a lack of real-time feedback based on learning progress. This makes it difficult for learners to progress efficiently, potentially leading to a decline in motivation.

[0673] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0674] In this invention, the server includes means for collecting learner history information obtained from individual information processing devices, means for analyzing the collected history information using a generative model to generate an optimized learning plan for each learner, and means for transmitting the generated learning plan to the individual information processing devices. This enables the provision of an optimized learning plan for each learner and immediate suggestions for improvements based on their progress.

[0675] An "information processing device" refers to hardware or software used for collecting, analyzing, and transmitting data.

[0676] "History information" refers to data about a learner's past learning history, study time, and tendency to make incorrect answers.

[0677] A "generative model" refers to artificial intelligence technology that analyzes collected data and generates an optimal educational plan for each learner.

[0678] An "educational plan" refers to a learning guideline that includes recommended learning content and study time, created based on the individual needs of the learner.

[0679] "Immediate improvement suggestions" refer to specific advice on evaluations and areas for improvement based on learning outcomes, which learners can receive in real time.

[0680] "Rewards" refer to means of motivation provided to learners in visual or other forms in accordance with their achievement of learning activities.

[0681] Modes for carrying out the invention

[0682] This invention is an information processing system that provides learners with individually optimized educational plans and supports efficient learning. The following describes how to specifically implement the program of this system.

[0683] The server collects historical information from each learner's individual information processing device. This historical information includes learning history, learning time, and error tendencies. The server analyzes the collected data using a generative AI model to generate an optimized learning plan for each learner. Specifically, this generative AI model identifies the learner's performance patterns based on past learning data and determines what learning content is most effective.

[0684] The generated learning plan is transmitted in real time by the server to individual information processing devices. This plan includes details tailored to the learner's needs, such as recommended learning content and study time. The user's device visually displays this learning plan, providing clear guidance to the learner. It also records progress in real time and sends feedback to the server, allowing the user to receive immediate improvement suggestions from the server.

[0685] Users can progress through their learning by following the educational plan provided via their device. For example, if a user wants to improve their English listening skills, the server generates an educational plan, including specific audio materials, based on the user's past listening test results, and delivers it to the user. The user answers listening questions according to this plan and receives feedback from the server on specific listening weaknesses based on the results. This feedback is immediately displayed on the device, allowing the user to quickly understand what they need to improve.

[0686] As an example of implementing this system, one could input the following prompt into the generative AI model: "Generate the optimal listening learning plan based on the user's learning history and error patterns." Based on this prompt, the generative AI model would generate the most suitable educational plan.

[0687] In this way, the present invention can provide an educational plan tailored to each learner, thereby enabling efficient learning.

[0688] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0689] Step 1:

[0690] The server retrieves historical information from the user's terminal, such as learning history, learning time, and error tendencies. This input data reflects each learner's individual learning tendencies. The server securely collects this data and stores it in storage for later analysis.

[0691] Step 2:

[0692] The server inputs the collected historical information into a generating AI model for data analysis. During this analysis process, the AI ​​model identifies learner performance patterns and automatically discovers learning challenges. As a result, data is output that forms the basis for an optimized educational plan for each user.

[0693] Step 3:

[0694] The server generates an optimized learning plan for each learner based on the data output from the generated AI model. This plan includes specific learning content, recommended study time, and areas to focus on. This information is customized based on the analysis results obtained in Step 2.

[0695] Step 4:

[0696] The server sends the generated lesson plan to the user's device in real time. The device receives this plan and displays it visually to the learner. The plan displayed on the device includes specific learning tasks and links to related content, allowing the user to start learning immediately.

[0697] Step 5:

[0698] Users engage in learning activities according to the educational plan displayed on the device. Specifically, they practice listening using audio materials and answer questions. As the user progresses through the learning process, the device records their progress.

[0699] Step 6:

[0700] The device sends data generated based on learning progress to the server in real time. The server then immediately analyzes the progress data and generates instant improvement suggestions as needed. This immediate feedback is provided individually based on the analyzed learner's weaknesses.

[0701] Step 7:

[0702] The device receives immediate improvement suggestions sent from the server and displays them clearly to the user. Based on this feedback, the user can modify their learning and apply those improvements to future learning opportunities.

[0703] (Application Example 1)

[0704] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0705] In recent years, there has been a growing demand for personalized education tailored to individual learners. However, providing each learner with an optimal learning plan in real time, along with effective assessment and feedback, is extremely difficult. Furthermore, appropriate methods for maintaining learner motivation are also necessary. This invention aims to solve these problems.

[0706] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0707] In this invention, the server includes means for collecting learner information obtained from individual computer devices, means for analyzing the collected learner information to generate a learning plan optimized for each learner, and means for distributing the generated learning plan to the individual computer devices. This makes it possible to distribute a learning plan optimized for each learner and to provide timely evaluation and feedback.

[0708] An "individualized computer device" is a user-specific computer device used to provide personalized information for learners.

[0709] "Learner information" refers to a collection of various data related to the learner, such as their past learning history, study time, and common areas of error.

[0710] A "learning plan" is an educational plan optimized according to each learner's learning style and history, and includes customized learning materials and schedules.

[0711] "Real-time response" refers to a process of evaluation and feedback provided immediately during learning, enabling learners to immediately recognize and address areas for improvement.

[0712] "Means of strengthening motivation" refer to visual rewards and features that foster a sense of accomplishment, provided to increase learners' motivation to continue learning.

[0713] "Performance evaluation" is a process for quantifying or visualizing the performance skills acquired by learners and providing suggestions for improvement.

[0714] "Adjusting audio content to match skill level" refers to the activity of changing the difficulty level and content of audio materials according to the learner's skill level to provide an optimal learning experience.

[0715] "Delivering content" refers to the process of transferring generated learning plans and audio content to learners' devices, making them accessible to them.

[0716] To implement this invention, the process begins with the server collecting learner data. This data is obtained from individual computing devices such as the user's smartphone or tablet, including past learning history, learning time, and points where the user is prone to errors. This data is then analyzed by an AI model running on a cloud server, generating a learning plan optimized for each individual learner.

[0717] The generated learning plan is delivered to the user's computer in real time. The receiving computer visually presents the learning plan to the user and provides feedback based on their progress. This allows the user to receive immediate evaluations and improvement suggestions through their device, and the optimal audio content and materials are delivered based on their skill level through performance evaluations. Furthermore, visual rewards and motivational elements are incorporated to enhance a sense of accomplishment.

[0718] A concrete example is a user who wants to improve their musical performance skills using this system. The server analyzes the user's past practiced songs and performance quality, and proposes an optimal practice plan. The user learns new songs and techniques according to this plan and receives immediate feedback on their device. At that time, detailed areas for improvement regarding the quality of their performance are also pointed out, which they can use to improve their next performance.

[0719] An example of a prompt to input into a generative AI model is, "Please suggest the optimal plan based on the user's history and errors in order to generate personalized learning content." This wording is used to create a plan that adapts to the different needs and abilities of each learner.

[0720] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0721] Step 1:

[0722] The server collects learner information from the user's terminal via individual computer devices. Input data includes past learning history, learning time, and common error points. The server aggregates and organizes this input data to create a detailed profile of the target learner. The output is a dataset saved as individual learner profiles.

[0723] Step 2:

[0724] The server analyzes the learner information collected using a generative AI model. The input is the learner profile obtained from step 1. The server inputs the prompt "Propose the optimal plan based on the user's history and errors to generate the personalized learning content to be provided" to the AI ​​model and performs data analysis and prediction. The output is the optimized learning plan.

[0725] Step 3:

[0726] The server delivers the generated learning plan to the user's terminal. The input is the learning plan generated in step 2. The server sends the plan data to each user's terminal, making it available for real-time reception. The output is the learning plan received and displayed on the user's terminal.

[0727] Step 4:

[0728] The terminal visually displays the received learning plan to the user. The input is the learning plan data sent in step 3. The terminal presents the learning plan to the user in an easy-to-understand format through a graphical user interface. The output is a visual display of the learning plan that the user can review.

[0729] Step 5:

[0730] The user engages in learning activities based on a learning plan, and the device monitors their progress. Input consists of user actions and progress data on the learning content. The device sends progress data to the server in real time. Output is data reflecting the learner's progress.

[0731] Step 6:

[0732] The server analyzes progress data sent from the terminal and generates real-time feedback. The input is the learner's progress data received from step 5. The server uses an AI model to evaluate the progress data and makes necessary improvement suggestions and sets achievable goals. The output is immediate evaluation and feedback information.

[0733] Step 7:

[0734] The terminal presents the user with the feedback received from the server. The input is the feedback information generated in step 6. The terminal displays the feedback in a visual and easy-to-understand format so that the user can understand what to do in the next step. The output is the feedback information displayed to the user.

[0735] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0736] This invention is an individually optimized learning system that takes into account the learner's emotional state. In particular, the system aims to maintain the learner's concentration and motivation optimally by recognizing the learner's emotions in real time and reflecting that information in the learning plan and feedback.

[0737] Server operation:

[0738] The server collects emotional data, including learner data, from the user's device. Using the device's built-in camera and sensors, data related to emotions, such as the user's facial expressions, voice tone, and typing speed, is acquired. The server analyzes this data to understand the user's emotional state during learning in real time. The emotion engine categorizes these emotional states into specific categories such as "joy," "surprise," "anger," and "anxiety." This information is used to generate and adjust learning plans optimized for each user.

[0739] Device operation:

[0740] The device quickly displays a pre-adjusted learning plan received from the server. The user then engages in normal learning activities based on this plan, while the device continuously monitors their emotional state. If an emotional change is detected, the device sends this information to the server, triggering a real-time adjustment of the plan. This ensures the user can always learn in the optimal psychological state.

[0741] User experience:

[0742] Users practice English listening, reading, and other skills through learning plans suggested via their devices. For example, if a user shows frustration with a complex problem, the server uses that information to modify the feedback and adjust the difficulty level of the plan as needed. This process is repeated to reduce the learner's burden and support effective skill improvement.

[0743] Specific example:

[0744] As a concrete example, suppose the emotion engine collects data indicating "surprise" or "anxiety" while a user is taking an online English test. The server can analyze this emotional state and adjust the feedback, such as replacing some parts of the learning plan with easier questions or suggesting a break. This emotional data can also be used to generate future plans, enabling users to continue learning in a less stressful environment.

[0745] This invention makes it possible to understand learners' emotions as much as possible and provide a learning environment that is tailored to them, which is expected to improve learning effectiveness and maintain sustained motivation.

[0746] The following describes the processing flow.

[0747] Step 1:

[0748] The user starts a learning session using the device. At this time, the device's built-in camera and microphone are activated and begin collecting data on the user's facial expressions and voice.

[0749] Step 2:

[0750] The device transmits collected emotional data to the server in real time. This data includes facial expressions, voice tone, and typing speed.

[0751] Step 3:

[0752] The server analyzes the received emotional data using an emotion engine. This analysis classifies the user's current emotional state into categories such as "joy," "surprise," "anxiety," and "concentration."

[0753] Step 4:

[0754] The server uses the analysis results to create or adjust a learning plan optimized for the user. If the user's emotional state is unstable, adjustments will be made, such as lowering the difficulty of the tasks or suggesting a break.

[0755] Step 5:

[0756] The server sends the adjusted learning plan to the user's device, which then displays it. The user continues learning according to the newly adjusted plan.

[0757] Step 6:

[0758] As users continue their learning activities, their emotional state may change. In such cases, the device continues to collect emotional data and send it to the server.

[0759] Step 7:

[0760] The server re-analyzes the received sentiment data and makes further adjustments to the learning plan as needed. It also instantly generates and sends appropriate feedback to the device.

[0761] Step 8:

[0762] The device displays feedback and adjustments sent from the server to the user. Based on this, the user continues learning and aims for efficient skill improvement.

[0763] In this way, by continuously optimizing the learning environment, users can always learn in the optimal psychological state.

[0764] (Example 2)

[0765] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0766] Despite the significant impact learners' emotional states on learning effectiveness and motivation, traditional learning systems have not adequately considered this. As a result, learners often face stress and decreased motivation, leading to reduced learning efficiency. This issue is particularly pronounced in online and remote learning environments, and a solution has been urgently needed.

[0767] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0768] In this invention, the server includes means for collecting learner data, including emotional data acquired from individual terminals; means for analyzing the collected learner data and generating a learning plan optimized for each learner based on the emotional information; and means for providing real-time feedback to learners based on the delivered learning plan and adjusting the plan according to changes in their emotions. This makes it possible to provide flexible learning plans that take into account the learner's emotional state, thereby maximizing the learner's motivation and efficiency.

[0769] An "individual terminal" is an information processing device used by a user, equipped with a camera and sensors, and capable of collecting user emotional data.

[0770] "Emotional data" refers to data that indicates a user's emotional state, including information such as facial expressions, voice, and typing speed.

[0771] "Learner data" refers to all information about the user, including past learning history, learning time, and sentiment data.

[0772] A "learning plan" refers to a learning plan and content optimized according to each learner's emotional state and individual characteristics.

[0773] "Real-time feedback" refers to the provision of immediate responses and information generated while the user is engaged in learning activities.

[0774] "Means of collection" refers to methods and devices that include the process of acquiring data from a terminal and transferring it to a server.

[0775] "Means of analysis" refers to the processes and devices used to analyze collected data and classify emotional states and learning patterns.

[0776] "Means of delivery" refers to methods or devices for sending learning plans, generated or adjusted on the server side, to the terminal.

[0777] Modes for carrying out the invention

[0778] This invention is an individually optimized learning system that takes into account the emotional state of the learner, and is mainly composed of three subjects: server, terminal, and user.

[0779] Server operation

[0780] The server collects learner data, including emotional data, from individual terminals. This data includes facial expressions, voice tone, and typing speed captured by the terminal's built-in cameras and sensors. The collected data is analyzed in real time using an emotional AI model and classified into specific emotional categories such as "joy," "surprise," "anger," and "anxiety." This analysis information is used by a generative AI model to generate learning plans optimized for each learner.

[0781] Terminal operation

[0782] The device quickly displays the adjusted learning plan received from the server. While the user progresses through learning activities based on this plan, the device continuously monitors their emotional state. If a change in emotion is detected, the device notifies the server, immediately triggering a readjustment of the plan.

[0783] User experience

[0784] Users practice skills such as listening and reading using learning plans suggested through their devices. For example, if a user shows frustration when faced with a difficult problem, the server uses that information to modify the feedback, either lowering the difficulty of the plan or suggesting a break. This allows users to improve their skills effectively while reducing their burden.

[0785] Examples of specific cases and prompt statements

[0786] For example, while a user is taking an online English test, emotional data such as "surprise" or "anxiety" may be collected. The server analyzes this emotional state and either replaces parts of the learning plan with easier questions or suggests a break. This data is also used to generate future plans, providing a learning environment that reduces stress.

[0787] An example of a prompt could be input to the generative AI model as, "How would you optimize the learning plan when a user expresses frustration with a complex problem?" Based on this prompt, it is possible to suggest more efficient feedback.

[0788] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0789] Step 1:

[0790] The device collects user emotion data. This data collection is done by capturing the user's facial expressions with a built-in camera and analyzing their voice tone with a microphone. It also records keyboard typing speed. This provides raw data about the user's emotions. The input is this emotion-related data, which is output as formatted data for the next step.

[0791] Step 2:

[0792] The server receives emotional data collected from the terminal and begins data analysis. Using an emotional AI model, it classifies the data into specific categories such as "joy," "surprise," "anger," and "anxiety." In this process, the input is formatted emotional data, and the output is the result of the analyzed emotional state.

[0793] Step 3:

[0794] The server utilizes a generated AI model based on the analysis results to create an optimized learning plan for each learner. The data processing performed here involves generating and adjusting the learning plan to reflect the learner's emotional state. The input is the analyzed emotional state, and the output is the optimized learning plan.

[0795] Step 4:

[0796] The server sends the generated learning plan to the terminal. The terminal displays the received learning plan to the user and prompts them to begin learning activities. The input is the optimized learning plan, and the output is the presentation of the learning plan to the user.

[0797] Step 5:

[0798] The user engages in learning activities based on a learning plan received through the device. During this time, the device continuously monitors emotional data in real time and sends new data to the server if any changes occur. The input is the user's learning activities, and the output is additional emotional data.

[0799] Step 6:

[0800] The server analyzes new emotion data transmitted in real time and updates the learning plan as needed. This allows for flexible adjustments tailored to the user's emotional state. The input is real-time emotion data, and the output is the learning plan adjusted as needed.

[0801] (Application Example 2)

[0802] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0803] The present invention aims to provide a system that can enhance the user's shopping experience in a virtual environment by recognizing the user's emotional state in real time, providing individually optimized product suggestions based on that information, and thereby increasing their desire to purchase. Specifically, it solves the problem of instantly presenting relevant information and content according to the products and categories that the user has shown interest in, thereby supporting more intuitive and effective purchasing activities.

[0804] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0805] In this invention, the server includes means for collecting user information obtained from individual devices, means for analyzing the collected user information to generate personalized recommendations for each user, and means for distributing the generated recommendations to individual devices to improve the purchasing experience in the virtual environment. This enables personalized recommendations that reflect the user's emotional state, thereby effectively increasing the user's willingness to purchase.

[0806] A "device" refers to an electronic device used for information processing and communication. It plays a role in directly interacting with the user through a user interface and is capable of inputting and outputting data.

[0807] "User information" refers to the data obtained from device usage and user behavior. This information is analyzed to understand the characteristics of individual users.

[0808] "Emotional state" refers to the state in which a user expresses their emotions. It is usually determined in real time based on data obtained from facial expressions, tone of voice, body movements, etc.

[0809] "Optimized recommendations" refer to product and service recommendations tailored to the user's emotional state and interests. This provides information that improves user satisfaction.

[0810] A "virtual environment" refers to a virtual space constructed using digital technology that is different from the real world. Users can have visual and auditory experiences within this environment through their devices.

[0811] "Purchase intent" refers to a user's desire or motivation to buy a product or service. This intent can be strengthened by individually tailored information and suggestions.

[0812] The system that realizes this invention has a structure in which a server, a terminal, and a user cooperate to function.

[0813] The server receives information collected from the user's device in real time and analyzes their emotional state. The server uses emotion recognition software such as OpenCV or TensorFlow to capture image and audio data and understand the user's current emotional state. The analyzed data is used to build personalized product recommendations using a generative AI model. These product recommendations are sent to the user's device and used to improve the user experience.

[0814] The terminal displays product suggestions received from the server in a way that users can visually confirm within a virtual environment. This information is displayed through a user interface on terminals such as smart glasses or head-mounted displays. The device also has the ability to send new data to the server that may reflect changes in the user's emotional state. This ensures that the system operates in a way that optimized suggestions are constantly updated.

[0815] Users can view product suggestions and content presented through their devices and research items that interest them in more detail. User behavior and reactions are continuously recorded and transmitted to the server, allowing the system to adapt to the user environment more effectively.

[0816] As a concrete example, suppose a user is browsing a virtual store and finds a gadget that interests them. The device detects the user's "excitement" and sends that information to a server. The server performs appropriate data analysis and presents similar products or content of interest to the user's glasses. In this case, possible prompts for the generative AI model might be as follows:

[0817] "If a user's emotion indicates excitement, generate a list of gadget products corresponding to that emotion and display it along with detailed reviews."

[0818] In this way, the system can always provide a customized purchasing experience that responds to the user's emotions.

[0819] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0820] Step 1:

[0821] The server receives information from the user's device in real time. It takes video and audio data transmitted from the device as input. This data is then used with emotion recognition software such as OpenCV or TensorFlow to perform calculations to detect the user's current emotional state. The output is data representing the analyzed emotional state.

[0822] Step 2:

[0823] The server uses a generative AI model to build individually optimized product recommendations based on the emotional state data obtained in Step 1. The inputs used are emotional state data and past user behavior data. The AI ​​model generates a list of products and content tailored to the user's interests and emotions. The output includes a set of individually tailored product recommendations.

[0824] Step 3:

[0825] The server sends generated product suggestions to the user's device. The device receives the product suggestions from the server as input and presents them to the user visually in a virtual environment. Interactive feedback from the user can also be incorporated. As output, the products selected by the user and content of interest are displayed on the device.

[0826] Step 4:

[0827] Users view product suggestions displayed through their devices and search for detailed information on items that interest them. The user's actions and responses are then sent back to the server via the device. This input includes user selections and browsing history, which the server further analyzes to provide foundational data for updating subsequent suggestions. The output generates new suggestions and feedback tailored to the user's actions.

[0828] Step 5:

[0829] The server continuously tracks user behavior based on updated suggestions, feeding appropriate data and prompt messages to an AI model, and then provides continuous service based on the results obtained. An example of such a prompt message is: "If the user's emotion indicates excitement, generate a list of gadget products corresponding to that emotion and display them along with detailed reviews."

[0830] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0831] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0832] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0833] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0834] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0835] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0836] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0837] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0838] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0839] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0840] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0841] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0842] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0843] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0844] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0845] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0846] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0847] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0848] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0849] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0850] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0851] The following is further disclosed regarding the embodiments described above.

[0852] (Claim 1)

[0853] A means for collecting learner data acquired from individual information processing devices,

[0854] A means for analyzing collected learner data to generate an optimized learning plan for each learner,

[0855] A means for distributing the generated learning plan to individual information processing devices,

[0856] A means of providing real-time feedback to learners based on the distributed learning plan,

[0857] A means of strengthening learner motivation by visually providing rewards and progress for learning activities,

[0858] A system that includes this.

[0859] (Claim 2)

[0860] The system according to claim 1, wherein the collected learner data includes past learning history, learning time, and points where mistakes are frequently made.

[0861] (Claim 3)

[0862] The system according to claim 1, wherein real-time feedback includes evaluation and improvement suggestions based on the answer results.

[0863] "Example 1"

[0864] (Claim 1)

[0865] A means for collecting learner history information obtained from individual information processing devices,

[0866] A means for analyzing collected historical information using a generative model and generating an optimized educational plan for each learner,

[0867] A means for transmitting the generated educational plan to an individual information processing device,

[0868] A means of providing learners with immediate improvement suggestions based on the submitted educational plan,

[0869] A means of improving learner motivation by visualizing rewards and progress for educational activities,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The system according to claim 1, wherein the collected historical information includes past learning history, learning time, and error tendency.

[0873] (Claim 3)

[0874] The system according to claim 1, in which immediate improvement suggestions include evaluation and improvement instructions based on the answer results.

[0875] "Application Example 1"

[0876] (Claim 1)

[0877] A means for collecting learner information obtained from individual computer devices,

[0878] A means for analyzing collected learner information and generating an optimized learning plan for each learner,

[0879] A means for distributing the generated learning plan to individual computer devices,

[0880] A means of providing learners with immediate responses based on the distributed learning plan,

[0881] A means of strengthening learner motivation by visually providing rewards and progress for learning activities,

[0882] A means of delivering optimized content by adjusting audio content according to the learner's skill level through performance evaluation.

[0883] A system that includes this.

[0884] (Claim 2)

[0885] The system according to claim 1, wherein the collected learner information includes past learning history, learning time, and points where mistakes are likely to occur.

[0886] (Claim 3)

[0887] The system according to claim 1, which includes a process for improving technique by evaluating the quality of performance and providing feedback.

[0888] "Example 2 of combining an emotion engine"

[0889] (Claim 1)

[0890] A means of collecting learner data, including sentiment data, acquired from individual devices,

[0891] A means for analyzing collected learner data and generating an optimized learning plan for each learner based on emotional information,

[0892] A means of distributing the generated learning plan to individual devices,

[0893] A means of providing real-time feedback to learners based on the delivered learning plan and adjusting the plan in response to changes in their emotions,

[0894] A means of strengthening learner motivation by visually providing rewards and progress for learning activities,

[0895] A system that includes this.

[0896] (Claim 2)

[0897] The system according to claim 1, wherein the collected learner data includes past learning history, learning time, sentiment data, and points where mistakes are likely to occur.

[0898] (Claim 3)

[0899] The system according to claim 1, wherein real-time feedback includes evaluation and improvement suggestions based on sentiment analysis results and response results.

[0900] "Application example 2 when combining with an emotional engine"

[0901] (Claim 1)

[0902] A means of collecting user information obtained from individual devices,

[0903] A means of analyzing collected user information to generate personalized suggestions for each user,

[0904] A means of delivering generated suggestions to individual devices to improve the purchasing experience in a virtual environment,

[0905] A means of providing users with real-time feedback, including sentiment analysis, based on the proposal,

[0906] A means of strengthening users' purchasing intent by visually providing rewards and promotional information for purchasing activities,

[0907] A system that includes this.

[0908] (Claim 2)

[0909] The system according to claim 1, wherein the collected user information includes past purchase history, visit frequency, and product categories of interest.

[0910] (Claim 3)

[0911] The system according to claim 1, which includes real-time feedback, product suggestions based on emotional state, and engaging content. [Explanation of Symbols]

[0912] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting learner data acquired from individual information processing devices, A means for analyzing collected learner data to generate an optimized learning plan for each learner, A means for distributing the generated learning plan to individual information processing devices, A means of providing real-time feedback to learners based on the distributed learning plan, A means of strengthening learner motivation by visually providing rewards and progress for learning activities, A system that includes this.

2. The system according to claim 1, wherein the collected learner data includes past learning history, learning time, and points where learners tend to make mistakes.

3. The system according to claim 1, wherein real-time feedback includes evaluation and improvement suggestions based on the answer results.

Images