System

Abstract

A system is provided.SOLUTION: A system, comprising: means for inputting user information; a server for receiving and storing the user information; means for selecting and providing content based on the user information; means for recording reactions and operations of a child in real time; means for analyzing the reaction and operation data and determining next content; and means for generating a learning result and a progress report based on the analysis result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today's parents want to use apps as appropriate learning and play tools for their children, but they face the challenge of deciding which apps to choose. It's particularly difficult to find content that interests children, making it difficult to provide appropriate educational and play experiences. Furthermore, there are limited means of understanding children's concentration and interest in learning and play in real time. The present invention aims to solve these problems and provide children with optimal learning content, allowing parents to effectively support their children's education. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for inputting user information, a server that receives and stores the user information, a means for selecting and providing content based on the user information, a means for recording a child's responses and operations in real time, a means for analyzing the response and operation data to determine the next content, and a means for generating a report of learning results and progress based on the analysis results. This makes it easier for parents to select learning content appropriate for their child, allowing the child to continue learning while maintaining their interest. Specifically, the system requires the child's age, field of interest, learning content, and usage time to be input as user information, and by analyzing the child's facial expressions, line of sight, operation frequency, and reaction speed collected in real time, the system determines the optimal content to be provided next, thereby improving learning effectiveness.

[0006] "User information" refers to information entered into the app by the user (parent), such as the child's age, areas of interest, learning content, and usage time.

[0007] "Server" means a central processing unit for receiving, storing, and analyzing user information.

[0008] "Content" refers to the programs and materials related to learning and play that are provided to children.

[0009] "Real-time" refers to the process of instantly collecting and analyzing children's reaction and action data as they learn and play.

[0010] "Reaction" refers to the behavior and movements that children show in relation to learning and play, such as their facial expressions, gaze, frequency of operations, and reaction speed.

[0011] "Operation" refers to the input actions that children take into the interface within the app, such as tapping or sliding.

[0012] "Data analysis" is the process of processing collected reaction and operation data using AI algorithms, etc., to evaluate children's interest and concentration.

[0013] A "report" is written or digital content that summarizes learning results and progress and provides them to parents. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention relates to an AI educational app that provides optimal learning content according to a child's reactions, developmental status, and learning needs. This system analyzes a child's interest and concentration in real time based on the content set in advance by the parent, and provides appropriate content and a report of the learning results to the parent.

[0036] Program processing overview

[0037] User input of settings

[0038] Through the app, users (parents) enter their child's information (such as name, age, areas of interest, and special attention). They also set the desired learning content category (e.g., animals, numbers, alphabet) and learning period (e.g., 30 minutes per day). The device then sends this information to the server and saves it as a user profile.

[0039] Real-time data collection

[0040] When a child uses the app to engage in learning or playful activities, the device uses its built-in camera and sensors to collect real-time data such as the child's facial expressions, gaze, frequency of operations, and reaction speed, and the device then transmits this data to a server at regular intervals.

[0041] Analyzing the data

[0042] The server uses an AI algorithm to analyze the child's interest and concentration based on the real-time data it receives, and then determines the optimal content to provide next.

[0043] Content provision

[0044] The server then sends the most appropriate content based on the analysis results to the device, which then displays that content to the child, allowing them to continue learning or playing. The device continues to collect real-time data and send it to the server.

[0045] Feedback and report generation

[0046] When a study session ends, the device sends all data (study results, concentration level, etc.) to the server. The server uses this data to generate a report summarizing the study results and progress. The generated report is notified to the user by the server, and the user can view the report through the app.

[0047] Specific examples

[0048] For example, let's say parent A uses this app for their 5-year-old daughter B. Parent A enters B's information into the app and sets animal-related quizzes and a learning period of 30 minutes per day. When B starts using the app, the device's camera analyzes B's facial expressions and gaze, and records the frequency of operations and the speed at which she responds. This data is sent to a server, which uses AI to analyze B's level of interest and concentration.

[0049] If the analysis shows that B-chan is very interested in animal quizzes, the server will send more difficult animal quizzes and videos introducing new animals to the device, which will then provide this new content to B-chan, allowing her to continue learning.

[0050] When the learning session ends, the device sends the session data to the server, which then generates a detailed report based on this data. This report includes information on how long B-chan focused on her learning and which content she showed particular interest in. Person A can check this report to see B-chan's learning progress.

[0051] As described above, the present invention provides a system that allows parents to effectively support their children's education, and enables children to continue learning while maintaining their interest.

[0052] The processing flow will be explained below.

[0053] Step 1: User Input

[0054] The user (parent) starts the app and enters their child's information (such as name, age, areas of interest, learning content, and special attention) into the form on the screen. They also set the categories of learning content they want to provide and the usage time.

[0055] Step 2: Send data

[0056] The device sends the entered user information and learning settings to the server, which receives it and stores the user information in a database.

[0057] Step 3: Providing initial content

[0058] The server selects the most suitable initial content (e.g., a quiz game about animals) based on the user information and sends it to the terminal, which then displays this content to the child.

[0059] Step 4: Collect real-time data

[0060] While the child is working on the content, the device uses its built-in camera and sensors to collect real-time data on the child's facial expressions, gaze, operation frequency, reaction speed, etc. This data is periodically sent to a server.

[0061] Step 5: Analyze the data

[0062] The server uses AI algorithms to analyze the child's interest and concentration based on the received real-time data, and evaluates which content the child is particularly interested in.

[0063] Step 6: Decide on your next content

[0064] Based on the analysis, the server then decides the next best content to serve, such as a more advanced animal quiz or a video introducing a new animal.

[0065] Step 7: Providing content

[0066] The server sends the next content to the device, which continues to display this new content to the child. The device continues to collect real-time data and send it to the server.

[0067] Step 8: Ending the Session

[0068] When the user-specified study time has elapsed, the device ends the study session. After the session ends, the device sends all data (study results, concentration analysis results, etc.) to the server.

[0069] Step 9: Generate reports

[0070] The server generates a report summarizing the learning results and progress based on the data sent, including details on the student's concentration level, response speed, and interest.

[0071] Step 10: Feedback Notification

[0072] The server notifies the user of the generated report, who can then check the report through the app to understand the child's learning progress. Based on this feedback, the user can plan the next lesson.

[0073] Example 1

[0074] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0075] Conventional educational applications have difficulty providing content tailored to individual children's interests and learning progress, and do not provide sufficient feedback for parents to effectively support their children's learning. Furthermore, they are unable to record children's reactions and operations in real time and optimize learning content based on that analysis, which can lead to reduced learning effectiveness.

[0076] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0077] In this invention, the server includes means for inputting user information, means for receiving and storing the user information, means for selecting and providing content based on the user information, means for recording the child's responses and operations in real time, means for analyzing the response and operation data and using an AI algorithm to determine the next content, and means for generating and providing a report of learning results and progress based on the analysis results. This allows parents to provide optimal learning content based on their child's individual interests and learning progress, and also allows them to understand their child's learning status in real time and receive effective feedback.

[0078] "User information" refers to data about the child, such as their name, age, areas of interest, special attention, learning content, and usage time.

[0079] A "server" is a computer system that receives and stores user information and performs analysis.

[0080] "Content" refers to educational or entertainment materials, quizzes, videos, animations, and other data provided based on children's learning and interests.

[0081] "Reactions" refers to behavioral data such as facial expressions, gaze, frequency of operation, and reaction speed when children use an application.

[0082] An "AI algorithm" is a program that uses technologies such as machine learning and deep learning to analyze data and evaluate children's interest and concentration levels.

[0083] "Learning results" refers to data that indicates what a child has learned, their grades, progress, etc.

[0084] "Progress" is data that indicates the progress and achievement of a child in their studies.

[0085] A "report" is a report generated by the server summarizing a student's learning results and progress.

[0086] This invention relates to an AI educational app that provides optimal learning content based on a child's reactions, developmental status, and learning needs. This system analyzes a child's interest and concentration in real time based on the content set in advance by the parent, and provides appropriate content and a report of the learning results to the parent. The specific implementation method of this system is described below.

[0087] User input of settings

[0088] The parent / guardian user first launches the app and enters information about their child. This information includes the child's name, age, areas of interest, and any special attention they may need. They also set the desired learning content category (e.g., animals, numbers, alphabet) and the amount of time they want to study per day (e.g., 30 minutes per day). The device then sends this information to a server, where it is saved as a user profile. The device used is typically a mobile device such as a smartphone or tablet.

[0089] Real-time data collection

[0090] When a child uses the app to engage with learning and playful content, the device uses its built-in camera and sensors to collect real-time data such as the child's facial expressions, gaze, operation frequency, and reaction speed. For example, the camera and sensors on an iPhone or Android smartphone are used to collect this data. The device then transmits this data to a server at regular intervals.

[0091] Analyzing the data

[0092] The server uses AI algorithms to analyze the child's interest and concentration based on the real-time data it receives. TensorFlow and PyTorch are often used as AI models. For example, the server analyzes eye-tracking data to assess the type of content the child is interested in. It also measures the child's concentration based on operation frequency and reaction speed, and determines the optimal content to provide next.

[0093] Content provision

[0094] Based on the analysis results, the server selects the next most appropriate content to provide and sends it to the device. The device then provides the content received from the server to the child, allowing them to continue learning or playing. For example, if a child shows a strong interest in animal quizzes, the server will send new animal quizzes and related videos to the device.

[0095] Feedback and report generation

[0096] Once a learning session is over, the device sends all learning data (such as learning performance, concentration level, and frequency of operation) to the server. The server uses this data to generate a detailed report using an AI algorithm. This report includes information on which content the child showed interest in, how much time they spent, and how focused they were. The server then notifies the user of the generated report, which can be viewed through the app.

[0097] Specific examples

[0098] For example, consider the case where parent A uses this app for their 5-year-old daughter B. Parent A enters B's information into their device, selects animal-related quizzes as the learning content, and sets the learning period to 30 minutes per day. When B begins using the app, the device's camera analyzes B's facial expressions and gaze, and records the frequency of operations and response speed. This data is sent to a server, which uses an AI model to analyze B's level of interest and concentration.

[0099] If the analysis shows that B-chan is very interested in animal quizzes, the server will send more difficult animal quizzes and videos introducing new animals to the device, which will then provide this new content to B-chan, allowing her to continue learning.

[0100] When the learning session ends, the device sends the session data to the server, which then generates a detailed report based on this data. This report includes information on how long B-chan focused on her learning and which content she showed particular interest in. Person A can check this report to see B-chan's learning progress.

[0101] Prompt Sentence Examples

[0102] Below are some examples of specific prompt sentences to input into the generative AI model.

[0103] "I want to provide an animal quiz that will interest 5-year-olds."

[0104] "Please recommend content that is highly relevant"

[0105] "Provide content related to a specific learning category (e.g., the alphabet)"

[0106] As described above, the present invention is a system that allows parents to effectively support their children's education, and is designed to enable children to continue learning while maintaining their interest.

[0107] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0108] Step 1: User inputs settings

[0109] The user launches the app and enters information about their child (such as name, age, areas of interest, and special attention). They also set the desired learning content category (e.g., animals, numbers, alphabet) and the amount of time they want to study per day (e.g., 30 minutes per day). The device then sends this information to the server and saves it as a user profile.

[0110] Specifically, the user enters the required information into the form on the screen of their smartphone or tablet and presses the submit button. The input data is sent from the device to the server in JSON format.

[0111] Input: Child's name, age, area of ​​interest, special attention, learning content category, learning time

[0112] Output: Save user profile

[0113] Step 2: Collect real-time data

[0114] When a user begins learning using the app, the device uses its built-in camera and sensors to collect real-time data, including the child's facial expressions, gaze, frequency of operations, and reaction time. This data is then sent to a server at regular intervals.

[0115] Specifically, the device's camera tracks the child's facial expressions and gaze, and sensors measure the frequency of operations and reaction speed. The data is sent to a server in JSON format at regular intervals.

[0116] Input: Real-time data during use by children (facial expressions, gaze, frequency of operation, reaction speed)

[0117] Output: Real-time data sent to the server

[0118] Step 3: Analyze the data

[0119] The server uses an AI algorithm to analyze the received real-time data, specifically analyzing the child's interest and concentration level to determine the next optimal content to provide.

[0120] Specifically, the server analyzes the data using AI models such as TensorFlow and PyTorch, assessing interest from gaze and facial expression data, and calculating concentration levels from operation frequency and reaction speed.

[0121] Input: Real-time data (facial expressions, gaze, operation frequency, reaction speed)

[0122] Output: Analysis results (interest level, concentration level)

[0123] Step 4: Providing content

[0124] Based on the analysis results, the server selects the most appropriate content and sends it to the device. The device then displays the appropriate content received from the server to the student, allowing them to continue their learning.

[0125] Specifically, the server determines the next content to provide based on the analyzed interest and concentration levels. The determined content is sent to the device in JSON format, and the device displays it.

[0126] Input: Analysis results

[0127] Output: Content sent to the device

[0128] Step 5: Generate feedback and reports

[0129] Once the learning session is over, the device sends all learning data to the server, which generates a detailed report and notifies the user.

[0130] Specifically, at the end of a session, the device sends the collected data (such as learning performance, concentration level, and frequency of operations) to the server, which then uses an AI algorithm to generate a report and send a notification to the user's device.

[0131] Input: Training data

[0132] Output: Generate learning report and notify user

[0133] Through these steps, the system provides optimal content tailored to each child's learning needs, providing effective learning support.

[0134] (Application example 1)

[0135] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0136] While conventional educational systems for children provide optimal learning content based on user information, they are primarily designed for use at home and are not suitable for use in physical stores. Furthermore, there is no system in physical stores that can record children's reactions and actions in real time, instantly analyze that data, and provide the next content. Furthermore, there are limited means of providing parents with feedback on their learning results and progress. Under these circumstances, it has been difficult to expand children's daily learning environments and provide more diverse learning experiences.

[0137] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0138] In this invention, the server includes a means for inputting user information, a means for receiving and storing the user information, a means for selecting and providing content based on the user information, a means for recording the user's responses and operations in real time, a means for analyzing the response and operation data to determine the next content, a means for generating a report of learning results and progress based on the analysis results, a means for providing the content using a robot installed in a physical store and recording the responses and operations, and an input device for the user to input the information at the physical store. This makes it possible to accurately grasp the learning status of children in real time even in the physical store and continue to provide appropriate learning content. Furthermore, by providing parents with immediate feedback on learning results and progress, it is possible to effectively support the children's learning experience.

[0139] "User information" refers to personal information entered by a child or their guardian, such as age, areas of interest, learning content, and usage time, that is necessary for providing learning content.

[0140] "Server" refers to a central processing unit that receives and stores user information, and then selects and provides the most appropriate content based on that information.

[0141] "Content" refers to information such as activities and teaching materials provided to children for the purpose of learning.

[0142] A "robot" is a mechanical device that is installed in a physical store and provides learning content interactively to children.

[0143] "Reactions" refer to the emotions and behaviors that children show toward the learning content, such as changes in facial expressions and body movements.

[0144] "Operation" refers to specific actions that children take with learning content, such as pressing a button or swiping.

[0145] Analyzing the "data" means collecting information about children's reactions and operations, and using that information to evaluate their interest and concentration.

[0146] An "input device" is a device for inputting user information in a physical store, and includes tablet terminals, smartphones, etc.

[0147] A "report" is a report summarizing learning results and progress, and is provided to parents so that they can check their child's learning status.

[0148] This invention is a system that supports children's learning experiences using educational robots installed in brick-and-mortar stores. This system provides optimal learning content based on user information, records and analyzes children's reactions and actions in real time, and determines the next content based on the results. It also generates reports on learning results and progress and provides them to parents, allowing them to check their children's learning progress.

[0149] System Program Overview

[0150] 1. User information input method

[0151] Users (parents) enter their child's information using a tablet or smartphone installed at the reception desk of a physical store. User information includes the child's name, age, areas of interest, learning content, and usage time. This information is entered through a web form interface and stored in a cloud database (e.g., Firebase).

[0152] 2. Real-time data collection methods

[0153] As children engage with learning content through the robot, the robot uses cameras and sensors to record the children's facial expressions, gaze, frequency of operations, and reaction speed in real time, using image recognition algorithms (e.g., OpenCV) and voice recognition software.

[0154] 3. Data Analysis Methods

[0155] The collected real-time data is sent to a cloud server, which uses AI analysis tools (e.g., TensorFlow) to analyze the student's interest and concentration. Based on the analysis results, the next optimal learning content to be provided is determined.

[0156] 4. Means of providing content

[0157] Based on the analysis results, the server determines the optimal learning content and sends it to the robot, which then provides that content to the child and continues to collect real-time data.

[0158] 5. Report Generation Methods

[0159] When the learning session ends, the robot sends data such as learning results and concentration level to the server in bulk. The server then generates a report summarizing the learning results and progress and provides it to parents. The report is displayed in HTML / CSS and is generated using a data processing tool (e.g., Pandas).

[0160] Specific examples

[0161] For example, if parent C uses this system at a physical store for their 7-year-old son, D, they enter D's information at the reception desk and select math puzzles as the learning content. As D begins to solve the math puzzle with the robot, the robot captures D's facial expressions and actions with a camera and sends the data to the server in real time. The server uses AI to analyze D's interest and level of concentration and decides which problem to tackle next. Once the learning is complete, the server generates a report based on all the data, which parent C can check on a tablet or smartphone.

[0162] Prompt Sentence Examples

[0163] "Information about D-kun"

[0164] Name: D-kun

[0165] Age: 7

[0166] Interests: Mathematics, puzzles

[0167] Special Notes: None

[0168] "Learning Content Settings"

[0169] Category: Math Puzzles

[0170] Study duration: 20 minutes per day

[0171] "Study Session Results Report"

[0172] 1. Session duration: 20 minutes

[0173] 2. Interest level: High

[0174] 3. Concentration: High

[0175] 4. Content provided: High-level math puzzles

[0176] 5. Overall rating: Excellent

[0177] The above is a specific embodiment of the present invention. This system effectively supports children's learning experiences even in brick-and-mortar stores, and allows parents to understand their children's learning progress in real time.

[0178] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0179] Step 1:

[0180] Users (parents) use tablets or smartphones installed at the reception desk of physical stores to enter information about their children, such as their name, age, interests, learning content, and usage time. This information is entered through a web form interface and sent to a cloud database (e.g., Firebase) for storage.

[0181] Input: Child's name, age, area of ​​interest, learning content, usage time

[0182] Output: User information stored in a cloud database

[0183] Step 2:

[0184] The device receives user information entered by the user (parent) and sends it to the server. After receiving this information, the server stores it as base data for selecting appropriate learning content for each child.

[0185] Input: User information

[0186] Output: User information stored on the server

[0187] Step 3:

[0188] Students move to the learning area and use the robot to carry out learning content. The robot uses cameras and sensors to record students' facial expressions, gaze, operation frequency, reaction speed, and other information in real time. This real-time data is sent to a server at regular intervals.

[0189] Input: Child's facial expression, gaze, operation frequency, reaction speed

[0190] Output: Data transmitted in real time

[0191] Step 4:

[0192] The server inputs the received real-time data into an AI analysis tool (e.g., TensorFlow) to analyze the child's interest and concentration. Image recognition algorithms (e.g., OpenCV) and voice recognition software are used for the analysis. The analysis results are used to select the next content.

[0193] Input: Real-time data

[0194] Output: Analysis results of children's interest and concentration

[0195] Step 5:

[0196] The server then determines the next optimal learning content to provide based on the analysis results and sends it to the robot, adjusting the difficulty and type of content according to the child's learning progress.

[0197] Input: Analysis results of children's interest and concentration

[0198] Output: Optimal learning content

[0199] Step 6:

[0200] The robot presents the content received from the server to the child and continues to record the child's reactions and actions, thereby continuously collecting real-time data needed for the next cycle.

[0201] Input: Optimal learning content

[0202] Output: Learning content presented to students

[0203] Step 7:

[0204] After the learning session is over, the robot sends all the data from the entire session to the server, which then uses this data to generate a report summarizing the learning results and progress. This report generation is done using a data processing tool (e.g., Pandas).

[0205] Input: Session-wide data

[0206] Output: Generated training result report

[0207] Step 8:

[0208] The server stores the generated report on the cloud and notifies the parents, who can then check the report using a tablet or smartphone located in the store.

[0209] Input: Generated learning result report

[0210] Output: Report provided to parents

[0211] The system's processing consists of the above steps, each of which optimally collects, analyzes, and provides the data required for the next step to optimize the learning experience for children in physical stores.

[0212] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0213] This invention relates to an AI educational app that provides optimal learning content based on user information and children's reaction data collected in real time, and further combines an emotion engine to recognize children's emotions and optimize the learning experience. This system analyzes the content preset by parents and children's reactions in real time, providing appropriate content to improve children's learning effectiveness, and provides parents with a report of the learning results.

[0214] Program processing overview

[0215] User input of settings

[0216] The user (parent) starts the app and enters their child's information (name, age, areas of interest, learning content, special attention, etc.) on the initial setup screen. They also set the desired learning content category and learning period. The device sends this information to the server and saves it in the database as a user profile.

[0217] Providing initial content

[0218] The server selects the most suitable initial content (e.g., a quiz game about animals) based on the user information and sends it to the device, which then displays this content to the child.

[0219] Real-time data collection

[0220] When a child uses the app to engage in learning or playful activities, the device uses its built-in camera and sensors to collect real-time data on the child's facial expressions, gaze, frequency of operations, reaction speed, etc. The device then periodically transmits this data to a server.

[0221] Emotion recognition by emotion engine

[0222] The emotion engine recognizes the child's emotions (joy, excitement, stress, concentration, etc.) based on the collected real-time data. The emotion data recognized by the emotion engine is sent to the server.

[0223] Data analysis

[0224] The server uses AI algorithms to analyze the received real-time data and emotional data, which allows it to assess the child's interest and concentration and determine which content they are most interested in.

[0225] Deciding on the next content

[0226] The server then uses the analysis results and emotional data to determine the best content to provide next, such as a video introducing new animals or a more challenging quiz to increase the fun the child experiences.

[0227] Content provision

[0228] The server then sends the next content to the device, which then displays the new content to the child, allowing them to continue their learning or playing experience. The device continues to collect real-time data and emotional data and transmits it to the server.

[0229] Ending a session and generating a report

[0230] Once the user-specified study time has elapsed, the device will end the study session. After the session ends, the device will send all data (such as learning performance, concentration level, and emotional analysis results) to the server. The server will then generate a detailed report based on this data. This report will include information on the child's emotional changes, concentration level, interest level, etc.

[0231] Feedback Notifications

[0232] The server notifies the user of the generated report. The user can check the report through the app and understand the child's learning progress and emotional changes. Based on this feedback, the user can create the next learning plan.

[0233] Specific examples

[0234] For example, let's say a parent is using this app for their six-year-old child. The parent enters the child's information on the initial setup screen, setting the child's area of ​​interest to "animals" and the learning period to "30 minutes per day." The server selects the optimal initial content based on this information and sends it to the device.

[0235] When a child starts a quiz game about animals, the device uses a camera to record the child's facial expressions, gaze, and frequency of operations. From this data, the emotion engine recognizes the child's joy, excitement, and level of concentration in real time and sends the data to the server.

[0236] The server analyzes the received data and determines whether the next content to provide is a challenging quiz about an animal that the child is particularly interested in, or a video introducing a new animal. The new content is then sent to the device, allowing the child to continue enjoying it.

[0237] After the learning session, the server generates a detailed report of the learning results and emotional changes based on all the data and notifies the parents, who can use this report to check their child's learning progress and emotional changes and plan the next learning session.

[0238] As described above, the present invention supports parents in improving their children's learning effectiveness and helping children to continue learning while maintaining their interest.

[0239] The processing flow will be explained below.

[0240] Step 1: User Input

[0241] The user (parent) starts the app and enters their child's information (name, age, areas of interest, learning content, special attention, etc.) into the form on the screen. They also set the category of learning content they want to provide (e.g., animals, numbers, alphabet) and the learning period (e.g., 30 minutes per day). The device then sends this information to the server.

[0242] Step 2: Save data

[0243] The server stores the received user information in a database, which can then be used for subsequent content selection and data analysis.

[0244] Step 3: Providing initial content

[0245] Based on the user information, the server selects the most suitable initial content for the child (e.g., a quiz game about animals) and sends it to the terminal, which then displays this content to the child.

[0246] Step 4: Collect real-time data

[0247] While children use the app to learn and play, the device uses its built-in camera and sensors to record their facial expressions, gaze, operation frequency, and reaction speed, and this data is sent to a server at regular intervals.

[0248] Step 5: Emotion Recognition with the Emotion Engine

[0249] Based on the data sent from the device, the emotion engine identifies the child's emotions (joy, excitement, stress, concentration, etc.) and sends the recognized emotion data to the server.

[0250] Step 6: Analyze the data

[0251] The server uses AI algorithms to analyze real-time and emotional data, assessing the child's interest and concentration and identifying content and reactions that they are particularly interested in.

[0252] Step 7: Decide on your next content

[0253] Based on the analysis and emotional data, the server then decides what content to provide next, such as a more advanced quiz or a video introducing a new animal.

[0254] Step 8: Providing content

[0255] The server sends the next content to the device. The device displays this new content to the child, allowing them to continue learning or playing. The device then collects real-time data and emotional data and sends them to the server.

[0256] Step 9: Ending the Session

[0257] When the study session ends, the device sends a termination signal to the server, along with all data (study results, concentration level, and emotion analysis results).

[0258] Step 10: Generate reports

[0259] The server uses all the data it receives to generate a detailed report summarizing learning results, progress and emotional changes, including areas of particular interest and concentration of the child.

[0260] Step 11: Feedback Notification

[0261] The server notifies the user of the generated report. The user can check this report through the app to understand the child's learning progress and emotional changes. Based on this feedback, the next lesson plan can be created.

[0262] Example 2

[0263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] Conventional learning applications have the problem of being unable to provide effective learning support because they are unable to properly reflect children's learning effects and interests in real time. Furthermore, because they are unable to provide learning content that takes children's emotions and concentration levels into account, there is a high possibility that children will become bored. Furthermore, it is difficult for parents to properly grasp their children's learning progress and emotional changes, making it difficult to continue learning or provide effective feedback.

[0265] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0266] a means for a user to input information about a child on an initial setup screen;

[0267] means for receiving the child's information and transmitting it to a server;

[0268] means for selecting initial learning content based on the received information and transmitting the content to the terminal;

[0269] A means of collecting data on facial expressions, gaze, operation frequency and reaction speed when children engage with learning content;

[0270] means for transmitting the collected data to a server and recognizing emotions with an emotion engine;

[0271] A means for analyzing the recognized emotion data and real-time data using an AI algorithm to evaluate the interest and concentration of the child;

[0272] a means for determining the next most suitable learning content to be provided based on the evaluation results and transmitting the determined content to the terminal;

[0273] means for generating a report of learning results and emotional changes based on all data after the session is completed;

[0274] means for notifying a user of the generated report;

[0275] This allows the learning content that children receive to be individually optimized over time, improving learning effectiveness, and allows parents to create better learning plans based on detailed learning progress and emotional changes.

[0276] "User information" is individual setting information for using the application, consisting of information such as the child's name, age, areas of interest, learning content, and learning period.

[0277] "Server" means a networked host system that receives, stores, and analyzes user information and selects and provides the most appropriate learning content.

[0278] A "terminal" is a device that provides learning content to children and collects data in real time.

[0279] "Real-time data" is data that is generated and transmitted on the spot, collecting information on children's facial expressions, gaze, operation frequency, and reaction speed.

[0280] The "emotion engine" is a software component that analyzes collected real-time data and recognizes children's emotions (joy, excitement, stress, concentration, etc.).

[0281] "AI algorithm" is an artificial intelligence technology that analyzes real-time and emotional data, finds patterns, and selects the most appropriate learning content.

[0282] A "report" is a document generated after a learning session that contains a detailed record of a child's learning performance, concentration level, emotional changes, etc.

[0283] "Learning content" refers to educational materials such as quizzes, videos, and games provided to help children learn.

[0284] MODE FOR CARRYING OUT THE INVENTION

[0285] The present invention relates to an AI educational app that provides optimal learning content based on user information and children's reaction data collected in real time. Furthermore, by combining an emotion engine, the system recognizes children's emotions and optimizes their learning experience. The following describes in detail the implementation of this system.

[0286] composition

[0287] The system includes the following main elements:

[0288] 1. How to enter user information

[0289] 2. A means of receiving user information and sending it to the server

[0290] 3. A method for selecting optimal initial learning content and sending it to devices

[0291] 4. Means of collecting real-time data (facial expressions, gaze, operation frequency, reaction speed)

[0292] 5. How to Recognize Emotions with an Emotion Engine

[0293] 6. A means of analyzing received data with AI algorithms

[0294] 7. A method for determining the next learning content to be provided and sending it to the device

[0295] 8. A way to generate a report after the learning session is completed

[0296] 9. A means of notifying users of generated reports

[0297] Hardware and Software

[0298] Devices: Devices equipped with cameras and various sensors, such as tablets and smartphones, are used, which enable accurate collection of data on children's facial expressions, gaze, and actions.

[0299] Server: A high-performance cloud server is used to store and analyze user information and select the most appropriate learning content. This is linked to a database system to enable rapid processing of large amounts of data.

[0300] Emotion Engine: A machine learning model is used to recognize children's emotions based on the collected data. The recommended model uses a deep learning framework specialized in emotion analysis.

[0301] AI Algorithm: We use AI algorithms to analyze the received data and select the most suitable learning content. These algorithms incorporate techniques of pattern recognition and predictive analysis.

[0302] Example

[0303] For example, if a parent were to use this app for a 6-year-old child, the steps would be as follows:

[0304] 1. User input of settings: Parents launch the app and enter their child's name, age, area of ​​interest (e.g., animals), learning period (e.g., 30 minutes per day), etc. on the initial setup screen.

[0305] 2. Providing initial content: Based on the received information, the server selects an animal-related quiz game as the optimal initial content and sends it to the device. The device then displays this content to the child.

[0306] 3. Real-time data collection: While the child is working on the content, the device uses its built-in camera and sensors to collect information on the child's facial expressions, gaze, operation frequency, reaction speed, etc. in real time. This data is periodically sent to the server.

[0307] 4. Emotion Recognition and Data Analysis: The emotion engine analyzes the collected data and recognizes the child's emotions (e.g., joy, excitement, stress, concentration). The server then analyzes this data using AI algorithms to evaluate the child's interest and concentration.

[0308] 5. Determining and providing the next content: Based on the analysis results, the server determines the most appropriate learning content to provide next. For example, if a child shows interest in a quiz about animals, a video introducing a new animal will be selected. This information is sent to the device and displayed.

[0309] 6. Post-session report generation and notification: After a learning session, a detailed report is generated based on all data. This report includes academic performance, concentration level, emotional changes, etc. The report is notified to parents and can be viewed through the app.

[0310] Prompt Sentence Examples

[0311] To ask a generative AI model to explain this system in natural language, use a prompt like this:

[0312] "Please explain in natural language in detail the processing of this AI educational app, which provides optimal learning content based on user information and children's reaction data collected in real time, and recognizes and optimizes children's emotions by combining an emotion engine. Please also explain what kind of data processing and calculations are performed using specific hardware (built-in camera, sensors, etc.) and software (emotion engine, AI algorithms, etc.)."

[0313] In this way, by clearly indicating the use of specific hardware and software, and the data processing and calculations based thereon, it is possible to provide a detailed mode for carrying out the present invention.

[0314] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0315] Step 1:

[0316] (User input)

[0317] The user (parent) starts the app and enters the child's name, age, areas of interest, learning content, and learning period on the initial setup screen. The device receives this information and sends it to the server. Based on the input data, the device creates a user profile and sends it to the server, where it is saved in the database.

[0318] Input: Child's name, age, area of ​​interest, study content, study period

[0319] Output: Generates a user profile and sends it to the server

[0320] Specifically, when the user has finished entering each item, he or she presses the send button, and the information is transferred from the terminal to the server. The terminal then displays a message indicating successful transmission.

[0321] Step 2:

[0322] (Providing initial content)

[0323] The server selects the most suitable initial learning content based on the received user information, and the selected content is sent to the terminal, which then displays the content to the student.

[0324] Input: User information

[0325] Output: Selection of initial learning content and transmission to the device

[0326] Specifically, the server searches for an appropriate quiz from the database and sends the data to the device, which then displays the quiz using animation and provides it to the student.

[0327] Step 3:

[0328] (Real-time data collection)

[0329] While students are working on learning content, the device uses its built-in camera and sensors to collect data on their facial expressions, gaze, operation frequency, and reaction speed, which is then periodically sent to a server.

[0330] Input: Learning content usage data (facial expressions, gaze, operation frequency, reaction speed)

[0331] Output: Sending collected real-time data to a server

[0332] Specifically, the camera tracks the child's face and records their eye movements. It also collects information on the frequency of their operations and their reaction speed, and sends this information to a server at regular intervals.

[0333] Step 4:

[0334] (Emotion recognition by emotion engine)

[0335] The server passes the collected real-time data to the emotion engine, which recognizes the child's emotions based on facial expression data and other data, and sends the results to the server.

[0336] Input: Real-time data collected

[0337] Output: Recognized emotion data

[0338] Specifically, the emotion engine analyzes the collected data and identifies the child's emotional state (e.g., joy, concentration, stress). The identified emotion data is sent to the server.

[0339] Step 5:

[0340] (Data Analysis)

[0341] The server uses an AI algorithm to analyze the received real-time data and emotional data, and evaluates the child's level of interest and concentration.

[0342] Input: Real-time data and sentiment data

[0343] Output: Interest and concentration evaluation results

[0344] Specifically, the AI ​​algorithm analyzes data patterns and generates a score based on the child's interest and concentration, which then reveals which content they are interested in.

[0345] Step 6:

[0346] (Determining the next content)

[0347] Based on the analysis results, the server determines the next optimal learning content to provide and sends it to the device, resulting in new learning content suited to the child.

[0348] Input: Evaluation result

[0349] Output: Selecting the most suitable learning content and sending it to the device

[0350] Specifically, the server selects the next appropriate content from the database and sends it to the device, which then displays the new content and continues learning.

[0351] Step 7:

[0352] (Content provided)

[0353] When the server sends the next learning content to the device, the device displays the new content to the child and continues learning. Data collection continues.

[0354] Input: Next learning content

[0355] Output: Display of new learning content

[0356] Specifically, the device will display new quizzes and videos while continuing to collect real-time data.

[0357] Step 8:

[0358] (Ending the session and generating a report)

[0359] After the user-specified learning time has elapsed, the device ends the learning session and then transmits all data to the server, which then generates a report.

[0360] Input: All data from the study session

[0361] Output: Report of learning results, concentration level, and emotional changes

[0362] Specifically, when the session ends, the device displays a message saying the session is over and all data is sent to the server, which then creates a report based on that data.

[0363] Step 9:

[0364] (Feedback Notification)

[0365] The server notifies the user (parent) of the generated report, who can then check the report through the app to understand their child's learning progress and emotional changes.

[0366] Input: Generated report

[0367] Output: Report notification and confirmation

[0368] Specifically, a notification will appear in the app, and when the user taps on the notification, a detailed report will be displayed, including information on study time, concentration level, and emotional changes.

[0369] (Application example 2)

[0370] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0371] Conventional educational apps have difficulty recognizing children's reactions and emotions in real time and optimizing learning content based on that. Furthermore, there are few ways to provide a learning experience tailored to each individual child in physical stores. This makes it difficult to achieve learning results and stimulate children's interest.

[0372] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting user information, a server for receiving and storing the user information, a means for selecting and providing content based on the user information, a means for recording the child's reactions and operations in real time, a means for identifying the child's emotions from the real-time data using an emotion recognition model, a means for analyzing the reaction and operation data and determining the next content, a means for generating a report of learning results and progress based on the analysis results and emotion data, and a tablet terminal for making the system available in the kids' space of a physical store. This makes it possible to provide optimal learning content based on the child's emotions and reactions and improve the learning experience in real time.

[0373] "Means for inputting user information" refers to an interface that allows a parent or user to input information about a child (such as name, age, areas of interest, special instructions, etc.).

[0374] "Server" refers to a computing device that receives and stores user information and analyzes data collected in real time.

[0375] "Means for selecting and providing content" refers to a system that selects and displays learning content appropriate for children based on user information and collected real-time data.

[0376] "Means for recording children's reactions and operations in real time" refers to the function of using cameras and sensors to collect children's facial expressions, gaze, frequency of operations, reaction speed, etc. in real time.

[0377] "Means for identifying children's emotions from the real-time data using an emotion recognition model" refers to a function that uses an AI model to analyze children's emotions such as joy, excitement, and concentration based on the collected data.

[0378] "Means for analyzing the reaction and operation data and determining the next content" refers to the function of analyzing the collected real-time data and emotional data and determining the next optimal content to be provided based on that data.

[0379] "Means for generating reports on learning results and progress" refers to the function of creating a report summarizing the child's learning results and progress based on the analyzed data and providing it to parents.

[0380] "Tablet terminal for making the system available in the kids' space of a physical store" refers to a tablet device installed to make the system available in the kids' space of a physical store.

[0381] The present invention relates to an AI educational app that provides optimal learning content based on user information and real-time collected data on children's reactions. The system combines an emotion engine to recognize children's emotions and optimize their learning experience. Specific embodiments of the present invention are described below.

[0382] First, the user (parent) enters information about their child using a tablet device installed in the kids' area of ​​the physical store. The user information includes the child's name, age, areas of interest, special attention, desired learning content category, and learning period. This information is sent from the tablet device to the server and stored.

[0383] Next, the server selects the most appropriate initial content based on the user information sent and sends it to the tablet device. For example, a quiz game about animals might be selected. As a child plays the quiz game using the tablet device, the device's built-in camera and sensors collect data in real time, such as the child's facial expressions, line of sight, operation frequency, and reaction speed. This data is sent to the server at regular intervals.

[0384] The emotion recognition model uses collected real-time data to identify a child's emotions. This emotion data is also sent to the server. The server then uses an AI algorithm to analyze the received real-time data and emotion data, and performs an analytical process to evaluate the child's level of interest and concentration. Based on the results, it determines the most appropriate content to provide next. For example, if a child shows interest in an animal quiz, a video introducing new animals or a more difficult quiz will be selected.

[0385] The tablet device then displays new content to the child, and the learning or play experience continues. The device continues to collect real-time data and emotional data and transmits it to the server. After the learning session ends, the server generates a detailed report based on all data (such as learning performance, concentration level, and emotional analysis results) and notifies the user. This allows the user to understand the child's learning progress and emotional changes and plan the next lesson.

[0386] For example, a parent might enter information about a six-year-old child on a tablet device and indicate that the child is interested in "animals." In this case, the server selects the initial content of an animal quiz and sends it to the tablet device. While the child is enjoying the quiz, the built-in camera analyzes their facial expressions in real time, and if they smile a lot, the server will present them with a new animal quiz or video.

[0387] Below are some example prompts to input to a generative AI model:

[0388] A 6-year-old boy named Taro is interested in animal quizzes. Write code to analyze Taro's reactions in real time using a facial expression recognition system, and if he seems to be enjoying it, determine the next best learning content to provide him. Also, please provide detailed information about the process of sending and receiving data to the server.

[0389] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0390] Step 1:

[0391] User input:

[0392] Users (parents) use tablet devices installed in the kids' areas of physical stores to enter their child's information (name, age, areas of interest, special attention, desired learning content category, and learning period). The device then sends this information to the server, which then stores the user profile in a database.

[0393] Input: Child's name, age, areas of interest, special notes, desired learning content category and learning period

[0394] Output: User information sent to server, saved user profile

[0395] Step 2:

[0396] Initial content provided:

[0397] Based on the received user information, the server selects the most appropriate initial content (for example, a quiz game about animals) and sends it to the tablet device, which then displays this content to the child.

[0398] Input: User profile stored on the server

[0399] Output: Initial content

[0400] Step 3:

[0401] Real-time data collection:

[0402] When a child uses a tablet device to engage with learning content, the device's built-in camera and sensors collect real-time data on the child's facial expressions, gaze, frequency of operations, reaction speed, etc. This data is then sent to a server at regular intervals.

[0403] Input: Real-time reaction data of children (facial expressions, gaze, frequency of operation, reaction speed)

[0404] Output: Real-time data sent

[0405] Step 4:

[0406] Emotion Recognition with Emotion Engine:

[0407] The server inputs the received real-time data into an emotion recognition model to identify the child's emotions (joy, excitement, concentration, etc.), and this emotion data is also stored on the server.

[0408] Input: Real-time data

[0409] Output: Identified emotion data

[0410] Step 5:

[0411] Data analysis:

[0412] The server uses AI algorithms to analyze real-time data and emotional data to assess the child's level of interest and concentration, and then determines the next most appropriate content to provide.

[0413] Input: Real-time data, emotion data

[0414] Output: Analysis results, next best content to provide

[0415] Step 6:

[0416] Providing the following content:

[0417] The server determines the next content based on the analysis results and sends it to the tablet device (for example, a video introducing a new animal or a high-level quiz). The device then displays this new content to the child.

[0418] Input: The next best thing to serve

[0419] Output: New content

[0420] Step 7:

[0421] End the session and generate a report:

[0422] After the user-specified study time has elapsed, the tablet device ends the study session. The device sends all data (such as study performance, concentration level, and emotion analysis results) to the server, which then generates a detailed report and notifies the user.

[0423] Input: Learning performance, concentration level, emotion analysis results

[0424] Output: Detailed report

[0425] As a result, by using this system, it is possible to draw out children's learning effectiveness and interest, and provide them with optimal learning content in real time.

[0426] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0428] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0429] [Second embodiment]

[0430] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0431] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0432] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0433] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0434] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0436] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0437] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0438] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0439] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0440] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0441] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0442] This invention relates to an AI educational app that provides optimal learning content according to a child's reactions, developmental status, and learning needs. This system analyzes a child's interest and concentration in real time based on the content set in advance by the parent, and provides appropriate content and a report of the learning results to the parent.

[0443] Program processing overview

[0444] User input of settings

[0445] Through the app, users (parents) enter their child's information (such as name, age, areas of interest, and special attention). They also set the desired learning content category (e.g., animals, numbers, alphabet) and learning period (e.g., 30 minutes per day). The device then sends this information to the server and saves it as a user profile.

[0446] Real-time data collection

[0447] When a child uses the app to engage in learning or playful activities, the device uses its built-in camera and sensors to collect real-time data such as the child's facial expressions, gaze, frequency of operations, and reaction speed, and the device then transmits this data to a server at regular intervals.

[0448] Analyzing the data

[0449] The server uses an AI algorithm to analyze the child's interest and concentration based on the real-time data it receives, and then determines the optimal content to provide next.

[0450] Content provision

[0451] The server then sends the most appropriate content based on the analysis results to the device, which then displays that content to the child, allowing them to continue learning or playing. The device continues to collect real-time data and send it to the server.

[0452] Feedback and report generation

[0453] When a study session ends, the device sends all data (study results, concentration level, etc.) to the server. The server uses this data to generate a report summarizing the study results and progress. The generated report is notified to the user by the server, and the user can view the report through the app.

[0454] Specific examples

[0455] For example, let's say parent A uses this app for their 5-year-old daughter B. Parent A enters B's information into the app and sets animal-related quizzes and a learning period of 30 minutes per day. When B starts using the app, the device's camera analyzes B's facial expressions and gaze, and records the frequency of operations and the speed at which she responds. This data is sent to a server, which uses AI to analyze B's level of interest and concentration.

[0456] If the analysis shows that B-chan is very interested in animal quizzes, the server will send more difficult animal quizzes and videos introducing new animals to the device, which will then provide this new content to B-chan, allowing her to continue learning.

[0457] When the learning session ends, the device sends the session data to the server, which then generates a detailed report based on this data. This report includes information on how long B-chan focused on her learning and which content she showed particular interest in. Person A can check this report to see B-chan's learning progress.

[0458] As described above, the present invention provides a system that allows parents to effectively support their children's education, and enables children to continue learning while maintaining their interest.

[0459] The processing flow will be explained below.

[0460] Step 1: User Input

[0461] The user (parent) starts the app and enters their child's information (such as name, age, areas of interest, learning content, and special attention) into the form on the screen. They also set the categories of learning content they want to provide and the usage time.

[0462] Step 2: Send data

[0463] The device sends the entered user information and learning settings to the server, which receives it and stores the user information in a database.

[0464] Step 3: Providing initial content

[0465] The server selects the most suitable initial content (e.g., a quiz game about animals) based on the user information and sends it to the terminal, which then displays this content to the child.

[0466] Step 4: Collect real-time data

[0467] While the child is working on the content, the device uses its built-in camera and sensors to collect real-time data on the child's facial expressions, gaze, operation frequency, reaction speed, etc. This data is periodically sent to a server.

[0468] Step 5: Analyze the data

[0469] The server uses AI algorithms to analyze the child's interest and concentration based on the received real-time data, and evaluates which content the child is particularly interested in.

[0470] Step 6: Decide on your next content

[0471] Based on the analysis, the server then decides the next best content to serve, such as a more advanced animal quiz or a video introducing a new animal.

[0472] Step 7: Providing content

[0473] The server sends the next content to the device, which continues to display this new content to the child. The device continues to collect real-time data and send it to the server.

[0474] Step 8: Ending the Session

[0475] When the user-specified study time has elapsed, the device ends the study session. After the session ends, the device sends all data (study results, concentration analysis results, etc.) to the server.

[0476] Step 9: Generate reports

[0477] The server generates a report summarizing the learning results and progress based on the data sent, including details on the student's concentration level, response speed, and interest.

[0478] Step 10: Feedback Notification

[0479] The server notifies the user of the generated report, who can then check the report through the app to understand the child's learning progress. Based on this feedback, the user can plan the next lesson.

[0480] Example 1

[0481] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0482] Conventional educational applications have difficulty providing content tailored to individual children's interests and learning progress, and do not provide sufficient feedback for parents to effectively support their children's learning. Furthermore, they are unable to record children's reactions and operations in real time and optimize learning content based on that analysis, which can lead to reduced learning effectiveness.

[0483] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0484] In this invention, the server includes means for inputting user information, means for receiving and storing the user information, means for selecting and providing content based on the user information, means for recording the child's responses and operations in real time, means for analyzing the response and operation data and using an AI algorithm to determine the next content, and means for generating and providing a report of learning results and progress based on the analysis results. This allows parents to provide optimal learning content based on their child's individual interests and learning progress, and also allows them to understand their child's learning status in real time and receive effective feedback.

[0485] "User information" refers to data about the child, such as their name, age, areas of interest, special attention, learning content, and usage time.

[0486] A "server" is a computer system that receives and stores user information and performs analysis.

[0487] "Content" refers to educational or entertainment materials, quizzes, videos, animations, and other data provided based on children's learning and interests.

[0488] "Reactions" refers to behavioral data such as facial expressions, gaze, frequency of operation, and reaction speed when children use an application.

[0489] An "AI algorithm" is a program that uses technologies such as machine learning and deep learning to analyze data and evaluate children's interest and concentration levels.

[0490] "Learning results" refers to data that indicates what a child has learned, their grades, progress, etc.

[0491] "Progress" is data that indicates the progress and achievement of a child in their studies.

[0492] A "report" is a report generated by the server summarizing a student's learning results and progress.

[0493] This invention relates to an AI educational app that provides optimal learning content based on a child's reactions, developmental status, and learning needs. This system analyzes a child's interest and concentration in real time based on the content set in advance by the parent, and provides appropriate content and a report of the learning results to the parent. The specific implementation method of this system is described below.

[0494] User input of settings

[0495] The parent / guardian user first launches the app and enters information about their child. This information includes the child's name, age, areas of interest, and any special attention they may need. They also set the desired learning content category (e.g., animals, numbers, alphabet) and the amount of time they want to study per day (e.g., 30 minutes per day). The device then sends this information to a server, where it is saved as a user profile. The device used is typically a mobile device such as a smartphone or tablet.

[0496] Real-time data collection

[0497] When a child uses the app to engage with learning and playful content, the device uses its built-in camera and sensors to collect real-time data such as the child's facial expressions, gaze, operation frequency, and reaction speed. For example, the camera and sensors on an iPhone or Android smartphone are used to collect this data. The device then transmits this data to a server at regular intervals.

[0498] Analyzing the data

[0499] The server uses AI algorithms to analyze the child's interest and concentration based on the real-time data it receives. TensorFlow and PyTorch are often used as AI models. For example, the server analyzes eye-tracking data to assess the type of content the child is interested in. It also measures the child's concentration based on operation frequency and reaction speed, and determines the optimal content to provide next.

[0500] Content provision

[0501] Based on the analysis results, the server selects the next most appropriate content to provide and sends it to the device. The device then provides the content received from the server to the child, allowing them to continue learning or playing. For example, if a child shows a strong interest in animal quizzes, the server will send new animal quizzes and related videos to the device.

[0502] Feedback and report generation

[0503] Once a learning session is over, the device sends all learning data (such as learning performance, concentration level, and frequency of operation) to the server. The server uses this data to generate a detailed report using an AI algorithm. This report includes information on which content the child showed interest in, how much time they spent, and how focused they were. The server then notifies the user of the generated report, which can be viewed through the app.

[0504] Specific examples

[0505] For example, consider the case where parent A uses this app for their 5-year-old daughter B. Parent A enters B's information into their device, selects animal-related quizzes as the learning content, and sets the learning period to 30 minutes per day. When B begins using the app, the device's camera analyzes B's facial expressions and gaze, and records the frequency of operations and response speed. This data is sent to a server, which uses an AI model to analyze B's level of interest and concentration.

[0506] If the analysis shows that B-chan is very interested in animal quizzes, the server will send more difficult animal quizzes and videos introducing new animals to the device, which will then provide this new content to B-chan, allowing her to continue learning.

[0507] When the learning session ends, the device sends the session data to the server, which then generates a detailed report based on this data. This report includes information on how long B-chan focused on her learning and which content she showed particular interest in. Person A can check this report to see B-chan's learning progress.

[0508] Prompt Sentence Examples

[0509] Below are some examples of specific prompt sentences to input into the generative AI model.

[0510] "I want to provide an animal quiz that will interest 5-year-olds."

[0511] "Please recommend content that is highly relevant"

[0512] "Provide content related to a specific learning category (e.g., the alphabet)"

[0513] As described above, the present invention is a system that allows parents to effectively support their children's education, and is designed to enable children to continue learning while maintaining their interest.

[0514] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0515] Step 1: User inputs settings

[0516] The user launches the app and enters information about their child (such as name, age, areas of interest, and special attention). They also set the desired learning content category (e.g., animals, numbers, alphabet) and the amount of time they want to study per day (e.g., 30 minutes per day). The device then sends this information to the server and saves it as a user profile.

[0517] Specifically, the user enters the required information into the form on the screen of their smartphone or tablet and presses the submit button. The input data is sent from the device to the server in JSON format.

[0518] Input: Child's name, age, area of ​​interest, special attention, learning content category, learning time

[0519] Output: Save user profile

[0520] Step 2: Collect real-time data

[0521] When a user begins learning using the app, the device uses its built-in camera and sensors to collect real-time data, including the child's facial expressions, gaze, frequency of operations, and reaction time. This data is then sent to a server at regular intervals.

[0522] Specifically, the device's camera tracks the child's facial expressions and gaze, and sensors measure the frequency of operations and reaction speed. The data is sent to a server in JSON format at regular intervals.

[0523] Input: Real-time data during use by children (facial expressions, gaze, frequency of operation, reaction speed)

[0524] Output: Real-time data sent to the server

[0525] Step 3: Analyze the data

[0526] The server uses an AI algorithm to analyze the received real-time data, specifically analyzing the child's interest and concentration level to determine the next optimal content to provide.

[0527] Specifically, the server analyzes the data using AI models such as TensorFlow and PyTorch, assessing interest from gaze and facial expression data, and calculating concentration levels from operation frequency and reaction speed.

[0528] Input: Real-time data (facial expressions, gaze, operation frequency, reaction speed)

[0529] Output: Analysis results (interest level, concentration level)

[0530] Step 4: Providing content

[0531] Based on the analysis results, the server selects the most appropriate content and sends it to the device. The device then displays the appropriate content received from the server to the student, allowing them to continue their learning.

[0532] Specifically, the server determines the next content to provide based on the analyzed interest and concentration levels. The determined content is sent to the device in JSON format, and the device displays it.

[0533] Input: Analysis results

[0534] Output: Content sent to the device

[0535] Step 5: Generate feedback and reports

[0536] Once the learning session is over, the device sends all learning data to the server, which generates a detailed report and notifies the user.

[0537] Specifically, at the end of a session, the device sends the collected data (such as learning performance, concentration level, and frequency of operations) to the server, which then uses an AI algorithm to generate a report and send a notification to the user's device.

[0538] Input: Training data

[0539] Output: Generate learning report and notify user

[0540] Through these steps, the system provides optimal content tailored to each child's learning needs, providing effective learning support.

[0541] (Application example 1)

[0542] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0543] While conventional educational systems for children provide optimal learning content based on user information, they are primarily designed for use at home and are not suitable for use in physical stores. Furthermore, there is no system in physical stores that can record children's reactions and actions in real time, instantly analyze that data, and provide the next content. Furthermore, there are limited means of providing parents with feedback on their learning results and progress. Under these circumstances, it has been difficult to expand children's daily learning environments and provide more diverse learning experiences.

[0544] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0545] In this invention, the server includes a means for inputting user information, a means for receiving and storing the user information, a means for selecting and providing content based on the user information, a means for recording the user's responses and operations in real time, a means for analyzing the response and operation data to determine the next content, a means for generating a report of learning results and progress based on the analysis results, a means for providing the content using a robot installed in a physical store and recording the responses and operations, and an input device for the user to input the information at the physical store. This makes it possible to accurately grasp the learning status of children in real time even in the physical store and continue to provide appropriate learning content. Furthermore, by providing parents with immediate feedback on learning results and progress, it is possible to effectively support the children's learning experience.

[0546] "User information" refers to personal information entered by a child or their guardian, such as age, areas of interest, learning content, and usage time, that is necessary for providing learning content.

[0547] "Server" refers to a central processing unit that receives and stores user information, and then selects and provides the most appropriate content based on that information.

[0548] "Content" refers to information such as activities and teaching materials provided to children for the purpose of learning.

[0549] A "robot" is a mechanical device that is installed in a physical store and provides learning content interactively to children.

[0550] "Reactions" refer to the emotions and behaviors that children show toward the learning content, such as changes in facial expressions and body movements.

[0551] "Operation" refers to specific actions that children take with learning content, such as pressing a button or swiping.

[0552] Analyzing the "data" means collecting information about children's reactions and operations, and using that information to evaluate their interest and concentration.

[0553] An "input device" is a device for inputting user information in a physical store, and includes tablet terminals, smartphones, etc.

[0554] A "report" is a report summarizing learning results and progress, and is provided to parents so that they can check their child's learning status.

[0555] This invention is a system that supports children's learning experiences using educational robots installed in brick-and-mortar stores. This system provides optimal learning content based on user information, records and analyzes children's reactions and actions in real time, and determines the next content based on the results. It also generates reports on learning results and progress and provides them to parents, allowing them to check their children's learning progress.

[0556] System Program Overview

[0557] 1. User information input method

[0558] Users (parents) enter their child's information using a tablet or smartphone installed at the reception desk of a physical store. User information includes the child's name, age, areas of interest, learning content, and usage time. This information is entered through a web form interface and stored in a cloud database (e.g., Firebase).

[0559] 2. Real-time data collection methods

[0560] As children engage with learning content through the robot, the robot uses cameras and sensors to record the children's facial expressions, gaze, frequency of operations, and reaction speed in real time, using image recognition algorithms (e.g., OpenCV) and voice recognition software.

[0561] 3. Data Analysis Methods

[0562] The collected real-time data is sent to a cloud server, which uses AI analysis tools (e.g., TensorFlow) to analyze the student's interest and concentration. Based on the analysis results, the next optimal learning content to be provided is determined.

[0563] 4. Means of providing content

[0564] Based on the analysis results, the server determines the optimal learning content and sends it to the robot, which then provides that content to the child and continues to collect real-time data.

[0565] 5. Report Generation Methods

[0566] When the learning session ends, the robot sends data such as learning results and concentration level to the server in bulk. The server then generates a report summarizing the learning results and progress and provides it to parents. The report is displayed in HTML / CSS and is generated using a data processing tool (e.g., Pandas).

[0567] Specific examples

[0568] For example, if parent C uses this system at a physical store for their 7-year-old son, D, they enter D's information at the reception desk and select math puzzles as the learning content. As D begins to solve the math puzzle with the robot, the robot captures D's facial expressions and actions with a camera and sends the data to the server in real time. The server uses AI to analyze D's interest and level of concentration and decides which problem to tackle next. Once the learning is complete, the server generates a report based on all the data, which parent C can check on a tablet or smartphone.

[0569] Prompt Sentence Examples

[0570] "Information about D-kun"

[0571] Name: D-kun

[0572] Age: 7

[0573] Interests: Mathematics, puzzles

[0574] Special Notes: None

[0575] "Learning Content Settings"

[0576] Category: Math Puzzles

[0577] Study duration: 20 minutes per day

[0578] "Study Session Results Report"

[0579] 1. Session duration: 20 minutes

[0580] 2. Interest level: High

[0581] 3. Concentration: High

[0582] 4. Content provided: High-level math puzzles

[0583] 5. Overall rating: Excellent

[0584] The above is a specific embodiment of the present invention. This system effectively supports children's learning experiences even in brick-and-mortar stores, and allows parents to understand their children's learning progress in real time.

[0585] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0586] Step 1:

[0587] Users (parents) use tablets or smartphones installed at the reception desk of physical stores to enter information about their children, such as their name, age, interests, learning content, and usage time. This information is entered through a web form interface and sent to a cloud database (e.g., Firebase) for storage.

[0588] Input: Child's name, age, area of ​​interest, learning content, usage time

[0589] Output: User information stored in a cloud database

[0590] Step 2:

[0591] The device receives user information entered by the user (parent) and sends it to the server. After receiving this information, the server stores it as base data for selecting appropriate learning content for each child.

[0592] Input: User information

[0593] Output: User information stored on the server

[0594] Step 3:

[0595] Students move to the learning area and use the robot to carry out learning content. The robot uses cameras and sensors to record students' facial expressions, gaze, operation frequency, reaction speed, and other information in real time. This real-time data is sent to a server at regular intervals.

[0596] Input: Child's facial expression, gaze, operation frequency, reaction speed

[0597] Output: Data transmitted in real time

[0598] Step 4:

[0599] The server inputs the received real-time data into an AI analysis tool (e.g., TensorFlow) to analyze the child's interest and concentration. Image recognition algorithms (e.g., OpenCV) and voice recognition software are used for the analysis. The analysis results are used to select the next content.

[0600] Input: Real-time data

[0601] Output: Analysis results of children's interest and concentration

[0602] Step 5:

[0603] The server then determines the next optimal learning content to provide based on the analysis results and sends it to the robot, adjusting the difficulty and type of content according to the child's learning progress.

[0604] Input: Analysis results of children's interest and concentration

[0605] Output: Optimal learning content

[0606] Step 6:

[0607] The robot presents the content received from the server to the child and continues to record the child's reactions and actions, thereby continuously collecting real-time data needed for the next cycle.

[0608] Input: Optimal learning content

[0609] Output: Learning content presented to students

[0610] Step 7:

[0611] After the learning session is over, the robot sends all the data from the entire session to the server, which then uses this data to generate a report summarizing the learning results and progress. This report generation is done using a data processing tool (e.g., Pandas).

[0612] Input: Session-wide data

[0613] Output: Generated training result report

[0614] Step 8:

[0615] The server stores the generated report on the cloud and notifies the parents, who can then check the report using a tablet or smartphone located in the store.

[0616] Input: Generated learning result report

[0617] Output: Report provided to parents

[0618] The system's processing consists of the above steps, each of which optimally collects, analyzes, and provides the data required for the next step to optimize the learning experience for children in physical stores.

[0619] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0620] This invention relates to an AI educational app that provides optimal learning content based on user information and children's reaction data collected in real time, and further combines an emotion engine to recognize children's emotions and optimize the learning experience. This system analyzes the content preset by parents and children's reactions in real time, providing appropriate content to improve children's learning effectiveness, and provides parents with a report of the learning results.

[0621] Program processing overview

[0622] User input of settings

[0623] The user (parent) starts the app and enters their child's information (name, age, areas of interest, learning content, special attention, etc.) on the initial setup screen. They also set the desired learning content category and learning period. The device sends this information to the server and saves it in the database as a user profile.

[0624] Providing initial content

[0625] The server selects the most suitable initial content (e.g., a quiz game about animals) based on the user information and sends it to the device, which then displays this content to the child.

[0626] Real-time data collection

[0627] When a child uses the app to engage in learning or playful activities, the device uses its built-in camera and sensors to collect real-time data on the child's facial expressions, gaze, frequency of operations, reaction speed, etc. The device then periodically transmits this data to a server.

[0628] Emotion recognition by emotion engine

[0629] The emotion engine recognizes the child's emotions (joy, excitement, stress, concentration, etc.) based on the collected real-time data. The emotion data recognized by the emotion engine is sent to the server.

[0630] Data analysis

[0631] The server uses AI algorithms to analyze the received real-time data and emotional data, which allows it to assess the child's interest and concentration and determine which content they are most interested in.

[0632] Deciding on the next content

[0633] The server then uses the analysis results and emotional data to determine the best content to provide next, such as a video introducing new animals or a more challenging quiz to increase the fun the child experiences.

[0634] Content provision

[0635] The server then sends the next content to the device, which then displays the new content to the child, allowing them to continue their learning or playing experience. The device continues to collect real-time data and emotional data and transmits it to the server.

[0636] Ending a session and generating a report

[0637] Once the user-specified study time has elapsed, the device will end the study session. After the session ends, the device will send all data (such as learning performance, concentration level, and emotional analysis results) to the server. The server will then generate a detailed report based on this data. This report will include information on the child's emotional changes, concentration level, interest level, etc.

[0638] Feedback Notifications

[0639] The server notifies the user of the generated report. The user can check the report through the app and understand the child's learning progress and emotional changes. Based on this feedback, the user can create the next learning plan.

[0640] Specific examples

[0641] For example, let's say a parent is using this app for their six-year-old child. The parent enters the child's information on the initial setup screen, setting the child's area of ​​interest to "animals" and the learning period to "30 minutes per day." The server selects the optimal initial content based on this information and sends it to the device.

[0642] When a child starts a quiz game about animals, the device uses a camera to record the child's facial expressions, gaze, and frequency of operations. From this data, the emotion engine recognizes the child's joy, excitement, and level of concentration in real time and sends the data to the server.

[0643] The server analyzes the received data and determines whether the next content to provide is a challenging quiz about an animal that the child is particularly interested in, or a video introducing a new animal. The new content is then sent to the device, allowing the child to continue enjoying it.

[0644] After the learning session, the server generates a detailed report of the learning results and emotional changes based on all the data and notifies the parents, who can use this report to check their child's learning progress and emotional changes and plan the next learning session.

[0645] As described above, the present invention supports parents in improving their children's learning effectiveness and helping children to continue learning while maintaining their interest.

[0646] The processing flow will be explained below.

[0647] Step 1: User Input

[0648] The user (parent) starts the app and enters their child's information (name, age, areas of interest, learning content, special attention, etc.) into the form on the screen. They also set the category of learning content they want to provide (e.g., animals, numbers, alphabet) and the learning period (e.g., 30 minutes per day). The device then sends this information to the server.

[0649] Step 2: Save data

[0650] The server stores the received user information in a database, which can then be used for subsequent content selection and data analysis.

[0651] Step 3: Providing initial content

[0652] Based on the user information, the server selects the most suitable initial content for the child (e.g., a quiz game about animals) and sends it to the terminal, which then displays this content to the child.

[0653] Step 4: Collect real-time data

[0654] While children use the app to learn and play, the device uses its built-in camera and sensors to record their facial expressions, gaze, operation frequency, and reaction speed, and this data is sent to a server at regular intervals.

[0655] Step 5: Emotion Recognition with the Emotion Engine

[0656] Based on the data sent from the device, the emotion engine identifies the child's emotions (joy, excitement, stress, concentration, etc.) and sends the recognized emotion data to the server.

[0657] Step 6: Analyze the data

[0658] The server uses AI algorithms to analyze real-time and emotional data, assessing the child's interest and concentration and identifying content and reactions that they are particularly interested in.

[0659] Step 7: Decide on your next content

[0660] Based on the analysis and emotional data, the server then decides what content to provide next, such as a more advanced quiz or a video introducing a new animal.

[0661] Step 8: Providing content

[0662] The server sends the next content to the device. The device displays this new content to the child, allowing them to continue learning or playing. The device then collects real-time data and emotional data and sends them to the server.

[0663] Step 9: Ending the Session

[0664] When the study session ends, the device sends a termination signal to the server, along with all data (study results, concentration level, and emotion analysis results).

[0665] Step 10: Generate reports

[0666] The server uses all the data it receives to generate a detailed report summarizing learning results, progress and emotional changes, including areas of particular interest and concentration of the child.

[0667] Step 11: Feedback Notification

[0668] The server notifies the user of the generated report. The user can check this report through the app to understand the child's learning progress and emotional changes. Based on this feedback, the next lesson plan can be created.

[0669] Example 2

[0670] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0671] Conventional learning applications have the problem of being unable to provide effective learning support because they are unable to properly reflect children's learning effects and interests in real time. Furthermore, because they are unable to provide learning content that takes children's emotions and concentration levels into account, there is a high possibility that children will become bored. Furthermore, it is difficult for parents to properly grasp their children's learning progress and emotional changes, making it difficult to continue learning or provide effective feedback.

[0672] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0673] a means for a user to input information about a child on an initial setup screen;

[0674] means for receiving the child's information and transmitting it to a server;

[0675] means for selecting initial learning content based on the received information and transmitting the content to the terminal;

[0676] A means of collecting data on facial expressions, gaze, operation frequency and reaction speed when children engage with learning content;

[0677] means for transmitting the collected data to a server and recognizing emotions with an emotion engine;

[0678] A means for analyzing the recognized emotion data and real-time data using an AI algorithm to evaluate the interest and concentration of the child;

[0679] a means for determining the next most suitable learning content to be provided based on the evaluation results and transmitting the determined content to the terminal;

[0680] means for generating a report of learning results and emotional changes based on all data after the session is completed;

[0681] means for notifying a user of the generated report;

[0682] This allows the learning content that children receive to be individually optimized over time, improving learning effectiveness, and allows parents to create better learning plans based on detailed learning progress and emotional changes.

[0683] "User information" is individual setting information for using the application, consisting of information such as the child's name, age, areas of interest, learning content, and learning period.

[0684] "Server" means a networked host system that receives, stores, and analyzes user information and selects and provides the most appropriate learning content.

[0685] A "terminal" is a device that provides learning content to children and collects data in real time.

[0686] "Real-time data" is data that is generated and transmitted on the spot, collecting information on children's facial expressions, gaze, operation frequency, and reaction speed.

[0687] The "emotion engine" is a software component that analyzes collected real-time data and recognizes children's emotions (joy, excitement, stress, concentration, etc.).

[0688] "AI algorithm" is an artificial intelligence technology that analyzes real-time and emotional data, finds patterns, and selects the most appropriate learning content.

[0689] A "report" is a document generated after a learning session that contains a detailed record of a child's learning performance, concentration level, emotional changes, etc.

[0690] "Learning content" refers to educational materials such as quizzes, videos, and games provided to help children learn.

[0691] MODE FOR CARRYING OUT THE INVENTION

[0692] The present invention relates to an AI educational app that provides optimal learning content based on user information and children's reaction data collected in real time. Furthermore, by combining an emotion engine, the system recognizes children's emotions and optimizes their learning experience. The following describes in detail the implementation of this system.

[0693] composition

[0694] The system includes the following main elements:

[0695] 1. How to enter user information

[0696] 2. A means of receiving user information and sending it to the server

[0697] 3. A method for selecting optimal initial learning content and sending it to devices

[0698] 4. Means of collecting real-time data (facial expressions, gaze, operation frequency, reaction speed)

[0699] 5. How to Recognize Emotions with an Emotion Engine

[0700] 6. A means of analyzing received data with AI algorithms

[0701] 7. A method for determining the next learning content to be provided and sending it to the device

[0702] 8. A way to generate a report after the learning session is completed

[0703] 9. A means of notifying users of generated reports

[0704] Hardware and Software

[0705] Devices: Devices equipped with cameras and various sensors, such as tablets and smartphones, are used, which enable accurate collection of data on children's facial expressions, gaze, and actions.

[0706] Server: A high-performance cloud server is used to store and analyze user information and select the most appropriate learning content. This is linked to a database system to enable rapid processing of large amounts of data.

[0707] Emotion Engine: A machine learning model is used to recognize children's emotions based on the collected data. The recommended model uses a deep learning framework specialized in emotion analysis.

[0708] AI Algorithm: We use AI algorithms to analyze the received data and select the most suitable learning content. These algorithms incorporate techniques of pattern recognition and predictive analysis.

[0709] Example

[0710] For example, if a parent were to use this app for a 6-year-old child, the steps would be as follows:

[0711] 1. User input of settings: Parents launch the app and enter their child's name, age, area of ​​interest (e.g., animals), learning period (e.g., 30 minutes per day), etc. on the initial setup screen.

[0712] 2. Providing initial content: Based on the received information, the server selects an animal-related quiz game as the optimal initial content and sends it to the device. The device then displays this content to the child.

[0713] 3. Real-time data collection: While the child is working on the content, the device uses its built-in camera and sensors to collect information on the child's facial expressions, gaze, operation frequency, reaction speed, etc. in real time. This data is periodically sent to the server.

[0714] 4. Emotion Recognition and Data Analysis: The emotion engine analyzes the collected data and recognizes the child's emotions (e.g., joy, excitement, stress, concentration). The server then analyzes this data using AI algorithms to evaluate the child's interest and concentration.

[0715] 5. Determining and providing the next content: Based on the analysis results, the server determines the most appropriate learning content to provide next. For example, if a child shows interest in a quiz about animals, a video introducing a new animal will be selected. This information is sent to the device and displayed.

[0716] 6. Post-session report generation and notification: After a learning session, a detailed report is generated based on all data. This report includes academic performance, concentration level, emotional changes, etc. The report is notified to parents and can be viewed through the app.

[0717] Prompt Sentence Examples

[0718] To ask a generative AI model to explain this system in natural language, use a prompt like this:

[0719] "Please explain in natural language in detail the processing of this AI educational app, which provides optimal learning content based on user information and children's reaction data collected in real time, and recognizes and optimizes children's emotions by combining an emotion engine. Please also explain what kind of data processing and calculations are performed using specific hardware (built-in camera, sensors, etc.) and software (emotion engine, AI algorithms, etc.)."

[0720] In this way, by clearly indicating the use of specific hardware and software, and the data processing and calculations based thereon, it is possible to provide a detailed mode for carrying out the present invention.

[0721] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0722] Step 1:

[0723] (User input)

[0724] The user (parent) starts the app and enters the child's name, age, areas of interest, learning content, and learning period on the initial setup screen. The device receives this information and sends it to the server. Based on the input data, the device creates a user profile and sends it to the server, where it is saved in the database.

[0725] Input: Child's name, age, area of ​​interest, study content, study period

[0726] Output: Generates a user profile and sends it to the server

[0727] Specifically, when the user has finished entering each item, he or she presses the send button, and the information is transferred from the terminal to the server. The terminal then displays a message indicating successful transmission.

[0728] Step 2:

[0729] (Providing initial content)

[0730] The server selects the most suitable initial learning content based on the received user information, and the selected content is sent to the terminal, which then displays the content to the student.

[0731] Input: User information

[0732] Output: Selection of initial learning content and transmission to the device

[0733] Specifically, the server searches for an appropriate quiz from the database and sends the data to the device, which then displays the quiz using animation and provides it to the student.

[0734] Step 3:

[0735] (Real-time data collection)

[0736] While students are working on learning content, the device uses its built-in camera and sensors to collect data on their facial expressions, gaze, operation frequency, and reaction speed, which is then periodically sent to a server.

[0737] Input: Learning content usage data (facial expressions, gaze, operation frequency, reaction speed)

[0738] Output: Sending collected real-time data to a server

[0739] Specifically, the camera tracks the child's face and records their eye movements. It also collects information on the frequency of their operations and their reaction speed, and sends this information to a server at regular intervals.

[0740] Step 4:

[0741] (Emotion recognition by emotion engine)

[0742] The server passes the collected real-time data to the emotion engine, which recognizes the child's emotions based on facial expression data and other data, and sends the results to the server.

[0743] Input: Real-time data collected

[0744] Output: Recognized emotion data

[0745] Specifically, the emotion engine analyzes the collected data and identifies the child's emotional state (e.g., joy, concentration, stress). The identified emotion data is sent to the server.

[0746] Step 5:

[0747] (Data Analysis)

[0748] The server uses an AI algorithm to analyze the received real-time data and emotional data, and evaluates the child's level of interest and concentration.

[0749] Input: Real-time data and sentiment data

[0750] Output: Interest and concentration evaluation results

[0751] Specifically, the AI ​​algorithm analyzes data patterns and generates a score based on the child's interest and concentration, which then reveals which content they are interested in.

[0752] Step 6:

[0753] (Determining the next content)

[0754] Based on the analysis results, the server determines the next optimal learning content to provide and sends it to the device, resulting in new learning content suited to the child.

[0755] Input: Evaluation result

[0756] Output: Selecting the most suitable learning content and sending it to the device

[0757] Specifically, the server selects the next appropriate content from the database and sends it to the device, which then displays the new content and continues learning.

[0758] Step 7:

[0759] (Content provided)

[0760] When the server sends the next learning content to the device, the device displays the new content to the child and continues learning. Data collection continues.

[0761] Input: Next learning content

[0762] Output: Display of new learning content

[0763] Specifically, the device will display new quizzes and videos while continuing to collect real-time data.

[0764] Step 8:

[0765] (Ending the session and generating a report)

[0766] After the user-specified learning time has elapsed, the device ends the learning session and then transmits all data to the server, which then generates a report.

[0767] Input: All data from the study session

[0768] Output: Report of learning results, concentration level, and emotional changes

[0769] Specifically, when the session ends, the device displays a message saying the session is over and all data is sent to the server, which then creates a report based on that data.

[0770] Step 9:

[0771] (Feedback Notification)

[0772] The server notifies the user (parent) of the generated report, who can then check the report through the app to understand their child's learning progress and emotional changes.

[0773] Input: Generated report

[0774] Output: Report notification and confirmation

[0775] Specifically, a notification will appear in the app, and when the user taps on the notification, a detailed report will be displayed, including information on study time, concentration level, and emotional changes.

[0776] (Application example 2)

[0777] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0778] Conventional educational apps have difficulty recognizing children's reactions and emotions in real time and optimizing learning content based on that. Furthermore, there are few ways to provide a learning experience tailored to each individual child in physical stores. This makes it difficult to achieve learning results and stimulate children's interest.

[0779] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting user information, a server for receiving and storing the user information, a means for selecting and providing content based on the user information, a means for recording the child's reactions and operations in real time, a means for identifying the child's emotions from the real-time data using an emotion recognition model, a means for analyzing the reaction and operation data and determining the next content, a means for generating a report of learning results and progress based on the analysis results and emotion data, and a tablet terminal for making the system available in the kids' space of a physical store. This makes it possible to provide optimal learning content based on the child's emotions and reactions and improve the learning experience in real time.

[0780] "Means for inputting user information" refers to an interface that allows a parent or user to input information about a child (such as name, age, areas of interest, special instructions, etc.).

[0781] "Server" refers to a computing device that receives and stores user information and analyzes data collected in real time.

[0782] "Means for selecting and providing content" refers to a system that selects and displays learning content appropriate for children based on user information and collected real-time data.

[0783] "Means for recording children's reactions and operations in real time" refers to the function of using cameras and sensors to collect children's facial expressions, gaze, frequency of operations, reaction speed, etc. in real time.

[0784] "Means for identifying children's emotions from the real-time data using an emotion recognition model" refers to a function that uses an AI model to analyze children's emotions such as joy, excitement, and concentration based on the collected data.

[0785] "Means for analyzing the reaction and operation data and determining the next content" refers to the function of analyzing the collected real-time data and emotional data and determining the next optimal content to be provided based on that data.

[0786] "Means for generating reports on learning results and progress" refers to the function of creating a report summarizing the child's learning results and progress based on the analyzed data and providing it to parents.

[0787] "Tablet terminal for making the system available in the kids' space of a physical store" refers to a tablet device installed to make the system available in the kids' space of a physical store.

[0788] The present invention relates to an AI educational app that provides optimal learning content based on user information and real-time collected data on children's reactions. The system combines an emotion engine to recognize children's emotions and optimize their learning experience. Specific embodiments of the present invention are described below.

[0789] First, the user (parent) enters information about their child using a tablet device installed in the kids' area of ​​the physical store. The user information includes the child's name, age, areas of interest, special attention, desired learning content category, and learning period. This information is sent from the tablet device to the server and stored.

[0790] Next, the server selects the most appropriate initial content based on the user information sent and sends it to the tablet device. For example, a quiz game about animals might be selected. As a child plays the quiz game using the tablet device, the device's built-in camera and sensors collect data in real time, such as the child's facial expressions, line of sight, operation frequency, and reaction speed. This data is sent to the server at regular intervals.

[0791] The emotion recognition model uses collected real-time data to identify a child's emotions. This emotion data is also sent to the server. The server then uses an AI algorithm to analyze the received real-time data and emotion data, and performs an analytical process to evaluate the child's level of interest and concentration. Based on the results, it determines the most appropriate content to provide next. For example, if a child shows interest in an animal quiz, a video introducing new animals or a more difficult quiz will be selected.

[0792] The tablet device then displays new content to the child, and the learning or play experience continues. The device continues to collect real-time data and emotional data and transmits it to the server. After the learning session ends, the server generates a detailed report based on all data (such as learning performance, concentration level, and emotional analysis results) and notifies the user. This allows the user to understand the child's learning progress and emotional changes and plan the next lesson.

[0793] For example, a parent might enter information about a six-year-old child on a tablet device and indicate that the child is interested in "animals." In this case, the server selects the initial content of an animal quiz and sends it to the tablet device. While the child is enjoying the quiz, the built-in camera analyzes their facial expressions in real time, and if they smile a lot, the server will present them with a new animal quiz or video.

[0794] Below are some example prompts to input to a generative AI model:

[0795] A 6-year-old boy named Taro is interested in animal quizzes. Write code to analyze Taro's reactions in real time using a facial expression recognition system, and if he seems to be enjoying it, determine the next best learning content to provide him. Also, please provide detailed information about the process of sending and receiving data to the server.

[0796] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0797] Step 1:

[0798] User input:

[0799] Users (parents) use tablet devices installed in the kids' areas of physical stores to enter their child's information (name, age, areas of interest, special attention, desired learning content category, and learning period). The device then sends this information to the server, which then stores the user profile in a database.

[0800] Input: Child's name, age, areas of interest, special notes, desired learning content category and learning period

[0801] Output: User information sent to server, saved user profile

[0802] Step 2:

[0803] Initial content provided:

[0804] Based on the received user information, the server selects the most appropriate initial content (for example, a quiz game about animals) and sends it to the tablet device, which then displays this content to the child.

[0805] Input: User profile stored on the server

[0806] Output: Initial content

[0807] Step 3:

[0808] Real-time data collection:

[0809] When a child uses a tablet device to engage with learning content, the device's built-in camera and sensors collect real-time data on the child's facial expressions, gaze, frequency of operations, reaction speed, etc. This data is then sent to a server at regular intervals.

[0810] Input: Real-time reaction data of children (facial expressions, gaze, frequency of operation, reaction speed)

[0811] Output: Real-time data sent

[0812] Step 4:

[0813] Emotion Recognition with Emotion Engine:

[0814] The server inputs the received real-time data into an emotion recognition model to identify the child's emotions (joy, excitement, concentration, etc.), and this emotion data is also stored on the server.

[0815] Input: Real-time data

[0816] Output: Identified emotion data

[0817] Step 5:

[0818] Data analysis:

[0819] The server uses AI algorithms to analyze real-time data and emotional data to assess the child's level of interest and concentration, and then determines the next most appropriate content to provide.

[0820] Input: Real-time data, emotion data

[0821] Output: Analysis results, next best content to provide

[0822] Step 6:

[0823] Providing the following content:

[0824] The server determines the next content based on the analysis results and sends it to the tablet device (for example, a video introducing a new animal or a high-level quiz). The device then displays this new content to the child.

[0825] Input: The next best thing to serve

[0826] Output: New content

[0827] Step 7:

[0828] End the session and generate a report:

[0829] After the user-specified study time has elapsed, the tablet device ends the study session. The device sends all data (such as study performance, concentration level, and emotion analysis results) to the server, which then generates a detailed report and notifies the user.

[0830] Input: Learning performance, concentration level, emotion analysis results

[0831] Output: Detailed report

[0832] As a result, by using this system, it is possible to draw out children's learning effectiveness and interest, and provide them with optimal learning content in real time.

[0833] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0834] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0835] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0836] [Third embodiment]

[0837] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0838] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0839] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0840] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0841] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0842] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0843] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0844] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0845] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0846] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0847] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0848] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0849] This invention relates to an AI educational app that provides optimal learning content according to a child's reactions, developmental status, and learning needs. This system analyzes a child's interest and concentration in real time based on the content set in advance by the parent, and provides appropriate content and a report of the learning results to the parent.

[0850] Program processing overview

[0851] User input of settings

[0852] Through the app, users (parents) enter their child's information (such as name, age, areas of interest, and special attention). They also set the desired learning content category (e.g., animals, numbers, alphabet) and learning period (e.g., 30 minutes per day). The device then sends this information to the server and saves it as a user profile.

[0853] Real-time data collection

[0854] When a child uses the app to engage in learning or playful activities, the device uses its built-in camera and sensors to collect real-time data such as the child's facial expressions, gaze, frequency of operations, and reaction speed, and the device then transmits this data to a server at regular intervals.

[0855] Analyzing the data

[0856] The server uses an AI algorithm to analyze the child's interest and concentration based on the real-time data it receives, and then determines the optimal content to provide next.

[0857] Content provision

[0858] The server then sends the most appropriate content based on the analysis results to the device, which then displays that content to the child, allowing them to continue learning or playing. The device continues to collect real-time data and send it to the server.

[0859] Feedback and report generation

[0860] When a study session ends, the device sends all data (study results, concentration level, etc.) to the server. The server uses this data to generate a report summarizing the study results and progress. The generated report is notified to the user by the server, and the user can view the report through the app.

[0861] Specific examples

[0862] For example, let's say parent A uses this app for their 5-year-old daughter B. Parent A enters B's information into the app and sets animal-related quizzes and a learning period of 30 minutes per day. When B starts using the app, the device's camera analyzes B's facial expressions and gaze, and records the frequency of operations and the speed at which she responds. This data is sent to a server, which uses AI to analyze B's level of interest and concentration.

[0863] If the analysis shows that B-chan is very interested in animal quizzes, the server will send more difficult animal quizzes and videos introducing new animals to the device, which will then provide this new content to B-chan, allowing her to continue learning.

[0864] When the learning session ends, the device sends the session data to the server, which then generates a detailed report based on this data. This report includes information on how long B-chan focused on her learning and which content she showed particular interest in. Person A can check this report to see B-chan's learning progress.

[0865] As described above, the present invention provides a system that allows parents to effectively support their children's education, and enables children to continue learning while maintaining their interest.

[0866] The processing flow will be explained below.

[0867] Step 1: User Input

[0868] The user (parent) starts the app and enters their child's information (such as name, age, areas of interest, learning content, and special attention) into the form on the screen. They also set the categories of learning content they want to provide and the usage time.

[0869] Step 2: Send data

[0870] The device sends the entered user information and learning settings to the server, which receives it and stores the user information in a database.

[0871] Step 3: Providing initial content

[0872] The server selects the most suitable initial content (e.g., a quiz game about animals) based on the user information and sends it to the terminal, which then displays this content to the child.

[0873] Step 4: Collect real-time data

[0874] While the child is working on the content, the device uses its built-in camera and sensors to collect real-time data on the child's facial expressions, gaze, operation frequency, reaction speed, etc. This data is periodically sent to a server.

[0875] Step 5: Analyze the data

[0876] The server uses AI algorithms to analyze the child's interest and concentration based on the received real-time data, and evaluates which content the child is particularly interested in.

[0877] Step 6: Decide on your next content

[0878] Based on the analysis, the server then decides the next best content to serve, such as a more advanced animal quiz or a video introducing a new animal.

[0879] Step 7: Providing content

[0880] The server sends the next content to the device, which continues to display this new content to the child. The device continues to collect real-time data and send it to the server.

[0881] Step 8: Ending the Session

[0882] When the user-specified study time has elapsed, the device ends the study session. After the session ends, the device sends all data (study results, concentration analysis results, etc.) to the server.

[0883] Step 9: Generate reports

[0884] The server generates a report summarizing the learning results and progress based on the data sent, including details on the student's concentration level, response speed, and interest.

[0885] Step 10: Feedback Notification

[0886] The server notifies the user of the generated report, who can then check the report through the app to understand the child's learning progress. Based on this feedback, the user can plan the next lesson.

[0887] Example 1

[0888] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0889] Conventional educational applications have difficulty providing content tailored to individual children's interests and learning progress, and do not provide sufficient feedback for parents to effectively support their children's learning. Furthermore, they are unable to record children's reactions and operations in real time and optimize learning content based on that analysis, which can lead to reduced learning effectiveness.

[0890] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0891] In this invention, the server includes means for inputting user information, means for receiving and storing the user information, means for selecting and providing content based on the user information, means for recording the child's responses and operations in real time, means for analyzing the response and operation data and using an AI algorithm to determine the next content, and means for generating and providing a report of learning results and progress based on the analysis results. This allows parents to provide optimal learning content based on their child's individual interests and learning progress, and also allows them to understand their child's learning status in real time and receive effective feedback.

[0892] "User information" refers to data about the child, such as their name, age, areas of interest, special attention, learning content, and usage time.

[0893] A "server" is a computer system that receives and stores user information and performs analysis.

[0894] "Content" refers to educational or entertainment materials, quizzes, videos, animations, and other data provided based on children's learning and interests.

[0895] "Reactions" refers to behavioral data such as facial expressions, gaze, frequency of operation, and reaction speed when children use an application.

[0896] An "AI algorithm" is a program that uses technologies such as machine learning and deep learning to analyze data and evaluate children's interest and concentration levels.

[0897] "Learning results" refers to data that indicates what a child has learned, their grades, progress, etc.

[0898] "Progress" is data that indicates the progress and achievement of a child in their studies.

[0899] A "report" is a report generated by the server summarizing a student's learning results and progress.

[0900] This invention relates to an AI educational app that provides optimal learning content based on a child's reactions, developmental status, and learning needs. This system analyzes a child's interest and concentration in real time based on the content set in advance by the parent, and provides appropriate content and a report of the learning results to the parent. The specific implementation method of this system is described below.

[0901] User input of settings

[0902] The parent / guardian user first launches the app and enters information about their child. This information includes the child's name, age, areas of interest, and any special attention they may need. They also set the desired learning content category (e.g., animals, numbers, alphabet) and the amount of time they want to study per day (e.g., 30 minutes per day). The device then sends this information to a server, where it is saved as a user profile. The device used is typically a mobile device such as a smartphone or tablet.

[0903] Real-time data collection

[0904] When a child uses the app to engage with learning and playful content, the device uses its built-in camera and sensors to collect real-time data such as the child's facial expressions, gaze, operation frequency, and reaction speed. For example, the camera and sensors on an iPhone or Android smartphone are used to collect this data. The device then transmits this data to a server at regular intervals.

[0905] Analyzing the data

[0906] The server uses AI algorithms to analyze the child's interest and concentration based on the real-time data it receives. TensorFlow and PyTorch are often used as AI models. For example, the server analyzes eye-tracking data to assess the type of content the child is interested in. It also measures the child's concentration based on operation frequency and reaction speed, and determines the optimal content to provide next.

[0907] Content provision

[0908] Based on the analysis results, the server selects the next most appropriate content to provide and sends it to the device. The device then provides the content received from the server to the child, allowing them to continue learning or playing. For example, if a child shows a strong interest in animal quizzes, the server will send new animal quizzes and related videos to the device.

[0909] Feedback and report generation

[0910] Once a learning session is over, the device sends all learning data (such as learning performance, concentration level, and frequency of operation) to the server. The server uses this data to generate a detailed report using an AI algorithm. This report includes information on which content the child showed interest in, how much time they spent, and how focused they were. The server then notifies the user of the generated report, which can be viewed through the app.

[0911] Specific examples

[0912] For example, consider the case where parent A uses this app for their 5-year-old daughter B. Parent A enters B's information into their device, selects animal-related quizzes as the learning content, and sets the learning period to 30 minutes per day. When B begins using the app, the device's camera analyzes B's facial expressions and gaze, and records the frequency of operations and response speed. This data is sent to a server, which uses an AI model to analyze B's level of interest and concentration.

[0913] If the analysis shows that B-chan is very interested in animal quizzes, the server will send more difficult animal quizzes and videos introducing new animals to the device, which will then provide this new content to B-chan, allowing her to continue learning.

[0914] When the learning session ends, the device sends the session data to the server, which then generates a detailed report based on this data. This report includes information on how long B-chan focused on her learning and which content she showed particular interest in. Person A can check this report to see B-chan's learning progress.

[0915] Prompt Sentence Examples

[0916] Below are some examples of specific prompt sentences to input into the generative AI model.

[0917] "I want to provide an animal quiz that will interest 5-year-olds."

[0918] "Please recommend content that is highly relevant"

[0919] "Provide content related to a specific learning category (e.g., the alphabet)"

[0920] As described above, the present invention is a system that allows parents to effectively support their children's education, and is designed to enable children to continue learning while maintaining their interest.

[0921] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0922] Step 1: User inputs settings

[0923] The user launches the app and enters information about their child (such as name, age, areas of interest, and special attention). They also set the desired learning content category (e.g., animals, numbers, alphabet) and the amount of time they want to study per day (e.g., 30 minutes per day). The device then sends this information to the server and saves it as a user profile.

[0924] Specifically, the user enters the required information into the form on the screen of their smartphone or tablet and presses the submit button. The input data is sent from the device to the server in JSON format.

[0925] Input: Child's name, age, area of ​​interest, special attention, learning content category, learning time

[0926] Output: Save user profile

[0927] Step 2: Collect real-time data

[0928] When a user begins learning using the app, the device uses its built-in camera and sensors to collect real-time data, including the child's facial expressions, gaze, frequency of operations, and reaction time. This data is then sent to a server at regular intervals.

[0929] Specifically, the device's camera tracks the child's facial expressions and gaze, and sensors measure the frequency of operations and reaction speed. The data is sent to a server in JSON format at regular intervals.

[0930] Input: Real-time data during use by children (facial expressions, gaze, frequency of operation, reaction speed)

[0931] Output: Real-time data sent to the server

[0932] Step 3: Analyze the data

[0933] The server uses an AI algorithm to analyze the received real-time data, specifically analyzing the child's interest and concentration level to determine the next optimal content to provide.

[0934] Specifically, the server analyzes the data using AI models such as TensorFlow and PyTorch, assessing interest from gaze and facial expression data, and calculating concentration levels from operation frequency and reaction speed.

[0935] Input: Real-time data (facial expressions, gaze, operation frequency, reaction speed)

[0936] Output: Analysis results (interest level, concentration level)

[0937] Step 4: Providing content

[0938] Based on the analysis results, the server selects the most appropriate content and sends it to the device. The device then displays the appropriate content received from the server to the student, allowing them to continue their learning.

[0939] Specifically, the server determines the next content to provide based on the analyzed interest and concentration levels. The determined content is sent to the device in JSON format, and the device displays it.

[0940] Input: Analysis results

[0941] Output: Content sent to the device

[0942] Step 5: Generate feedback and reports

[0943] Once the learning session is over, the device sends all learning data to the server, which generates a detailed report and notifies the user.

[0944] Specifically, at the end of a session, the device sends the collected data (such as learning performance, concentration level, and frequency of operations) to the server, which then uses an AI algorithm to generate a report and send a notification to the user's device.

[0945] Input: Training data

[0946] Output: Generate learning report and notify user

[0947] Through these steps, the system provides optimal content tailored to each child's learning needs, providing effective learning support.

[0948] (Application example 1)

[0949] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0950] While conventional educational systems for children provide optimal learning content based on user information, they are primarily designed for use at home and are not suitable for use in physical stores. Furthermore, there is no system in physical stores that can record children's reactions and actions in real time, instantly analyze that data, and provide the next content. Furthermore, there are limited means of providing parents with feedback on their learning results and progress. Under these circumstances, it has been difficult to expand children's daily learning environments and provide more diverse learning experiences.

[0951] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0952] In this invention, the server includes a means for inputting user information, a means for receiving and storing the user information, a means for selecting and providing content based on the user information, a means for recording the user's responses and operations in real time, a means for analyzing the response and operation data to determine the next content, a means for generating a report of learning results and progress based on the analysis results, a means for providing the content using a robot installed in a physical store and recording the responses and operations, and an input device for the user to input the information at the physical store. This makes it possible to accurately grasp the learning status of children in real time even in the physical store and continue to provide appropriate learning content. Furthermore, by providing parents with immediate feedback on learning results and progress, it is possible to effectively support the children's learning experience.

[0953] "User information" refers to personal information entered by a child or their guardian, such as age, areas of interest, learning content, and usage time, that is necessary for providing learning content.

[0954] "Server" refers to a central processing unit that receives and stores user information, and then selects and provides the most appropriate content based on that information.

[0955] "Content" refers to information such as activities and teaching materials provided to children for the purpose of learning.

[0956] A "robot" is a mechanical device that is installed in a physical store and provides learning content interactively to children.

[0957] "Reactions" refer to the emotions and behaviors that children show toward the learning content, such as changes in facial expressions and body movements.

[0958] "Operation" refers to specific actions that children take with learning content, such as pressing a button or swiping.

[0959] Analyzing the "data" means collecting information about children's reactions and operations, and using that information to evaluate their interest and concentration.

[0960] An "input device" is a device for inputting user information in a physical store, and includes tablet terminals, smartphones, etc.

[0961] A "report" is a report summarizing learning results and progress, and is provided to parents so that they can check their child's learning status.

[0962] This invention is a system that supports children's learning experiences using educational robots installed in brick-and-mortar stores. This system provides optimal learning content based on user information, records and analyzes children's reactions and actions in real time, and determines the next content based on the results. It also generates reports on learning results and progress and provides them to parents, allowing them to check their children's learning progress.

[0963] System Program Overview

[0964] 1. User information input method

[0965] Users (parents) enter their child's information using a tablet or smartphone installed at the reception desk of a physical store. User information includes the child's name, age, areas of interest, learning content, and usage time. This information is entered through a web form interface and stored in a cloud database (e.g., Firebase).

[0966] 2. Real-time data collection methods

[0967] As children engage with learning content through the robot, the robot uses cameras and sensors to record the children's facial expressions, gaze, frequency of operations, and reaction speed in real time, using image recognition algorithms (e.g., OpenCV) and voice recognition software.

[0968] 3. Data Analysis Methods

[0969] The collected real-time data is sent to a cloud server, which uses AI analysis tools (e.g., TensorFlow) to analyze the student's interest and concentration. Based on the analysis results, the next optimal learning content to be provided is determined.

[0970] 4. Means of providing content

[0971] Based on the analysis results, the server determines the optimal learning content and sends it to the robot, which then provides that content to the child and continues to collect real-time data.

[0972] 5. Report Generation Methods

[0973] When the learning session ends, the robot sends data such as learning results and concentration level to the server in bulk. The server then generates a report summarizing the learning results and progress and provides it to parents. The report is displayed in HTML / CSS and is generated using a data processing tool (e.g., Pandas).

[0974] Specific examples

[0975] For example, if parent C uses this system at a physical store for their 7-year-old son, D, they enter D's information at the reception desk and select math puzzles as the learning content. As D begins to solve the math puzzle with the robot, the robot captures D's facial expressions and actions with a camera and sends the data to the server in real time. The server uses AI to analyze D's interest and level of concentration and decides which problem to tackle next. Once the learning is complete, the server generates a report based on all the data, which parent C can check on a tablet or smartphone.

[0976] Prompt Sentence Examples

[0977] "Information about D-kun"

[0978] Name: D-kun

[0979] Age: 7

[0980] Interests: Mathematics, puzzles

[0981] Special Notes: None

[0982] "Learning Content Settings"

[0983] Category: Math Puzzles

[0984] Study duration: 20 minutes per day

[0985] "Study Session Results Report"

[0986] 1. Session duration: 20 minutes

[0987] 2. Interest level: High

[0988] 3. Concentration: High

[0989] 4. Content provided: High-level math puzzles

[0990] 5. Overall rating: Excellent

[0991] The above is a specific embodiment of the present invention. This system effectively supports children's learning experiences even in brick-and-mortar stores, and allows parents to understand their children's learning progress in real time.

[0992] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0993] Step 1:

[0994] Users (parents) use tablets or smartphones installed at the reception desk of physical stores to enter information about their children, such as their name, age, interests, learning content, and usage time. This information is entered through a web form interface and sent to a cloud database (e.g., Firebase) for storage.

[0995] Input: Child's name, age, area of ​​interest, learning content, usage time

[0996] Output: User information stored in a cloud database

[0997] Step 2:

[0998] The device receives user information entered by the user (parent) and sends it to the server. After receiving this information, the server stores it as base data for selecting appropriate learning content for each child.

[0999] Input: User information

[1000] Output: User information stored on the server

[1001] Step 3:

[1002] Students move to the learning area and use the robot to carry out learning content. The robot uses cameras and sensors to record students' facial expressions, gaze, operation frequency, reaction speed, and other information in real time. This real-time data is sent to a server at regular intervals.

[1003] Input: Child's facial expression, gaze, operation frequency, reaction speed

[1004] Output: Data transmitted in real time

[1005] Step 4:

[1006] The server inputs the received real-time data into an AI analysis tool (e.g., TensorFlow) to analyze the child's interest and concentration. Image recognition algorithms (e.g., OpenCV) and voice recognition software are used for the analysis. The analysis results are used to select the next content.

[1007] Input: Real-time data

[1008] Output: Analysis results of children's interest and concentration

[1009] Step 5:

[1010] The server then determines the next optimal learning content to provide based on the analysis results and sends it to the robot, adjusting the difficulty and type of content according to the child's learning progress.

[1011] Input: Analysis results of children's interest and concentration

[1012] Output: Optimal learning content

[1013] Step 6:

[1014] The robot presents the content received from the server to the child and continues to record the child's reactions and actions, thereby continuously collecting real-time data needed for the next cycle.

[1015] Input: Optimal learning content

[1016] Output: Learning content presented to students

[1017] Step 7:

[1018] After the learning session is over, the robot sends all the data from the entire session to the server, which then uses this data to generate a report summarizing the learning results and progress. This report generation is done using a data processing tool (e.g., Pandas).

[1019] Input: Session-wide data

[1020] Output: Generated training result report

[1021] Step 8:

[1022] The server stores the generated report on the cloud and notifies the parents, who can then check the report using a tablet or smartphone located in the store.

[1023] Input: Generated learning result report

[1024] Output: Report provided to parents

[1025] The system's processing consists of the above steps, each of which optimally collects, analyzes, and provides the data required for the next step to optimize the learning experience for children in physical stores.

[1026] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1027] This invention relates to an AI educational app that provides optimal learning content based on user information and children's reaction data collected in real time, and further combines an emotion engine to recognize children's emotions and optimize the learning experience. This system analyzes the content preset by parents and children's reactions in real time, providing appropriate content to improve children's learning effectiveness, and provides parents with a report of the learning results.

[1028] Program processing overview

[1029] User input of settings

[1030] The user (parent) starts the app and enters their child's information (name, age, areas of interest, learning content, special attention, etc.) on the initial setup screen. They also set the desired learning content category and learning period. The device sends this information to the server and saves it in the database as a user profile.

[1031] Providing initial content

[1032] The server selects the most suitable initial content (e.g., a quiz game about animals) based on the user information and sends it to the device, which then displays this content to the child.

[1033] Real-time data collection

[1034] When a child uses the app to engage in learning or playful activities, the device uses its built-in camera and sensors to collect real-time data on the child's facial expressions, gaze, frequency of operations, reaction speed, etc. The device then periodically transmits this data to a server.

[1035] Emotion recognition by emotion engine

[1036] The emotion engine recognizes the child's emotions (joy, excitement, stress, concentration, etc.) based on the collected real-time data. The emotion data recognized by the emotion engine is sent to the server.

[1037] Data analysis

[1038] The server uses AI algorithms to analyze the received real-time data and emotional data, which allows it to assess the child's interest and concentration and determine which content they are most interested in.

[1039] Deciding on the next content

[1040] The server then uses the analysis results and emotional data to determine the best content to provide next, such as a video introducing new animals or a more challenging quiz to increase the fun the child experiences.

[1041] Content provision

[1042] The server then sends the next content to the device, which then displays the new content to the child, allowing them to continue their learning or playing experience. The device continues to collect real-time data and emotional data and transmits it to the server.

[1043] Ending a session and generating a report

[1044] Once the user-specified study time has elapsed, the device will end the study session. After the session ends, the device will send all data (such as learning performance, concentration level, and emotional analysis results) to the server. The server will then generate a detailed report based on this data. This report will include information on the child's emotional changes, concentration level, interest level, etc.

[1045] Feedback Notifications

[1046] The server notifies the user of the generated report. The user can check the report through the app and understand the child's learning progress and emotional changes. Based on this feedback, the user can create the next learning plan.

[1047] Specific examples

[1048] For example, let's say a parent is using this app for their six-year-old child. The parent enters the child's information on the initial setup screen, setting the child's area of ​​interest to "animals" and the learning period to "30 minutes per day." The server selects the optimal initial content based on this information and sends it to the device.

[1049] When a child starts a quiz game about animals, the device uses a camera to record the child's facial expressions, gaze, and frequency of operations. From this data, the emotion engine recognizes the child's joy, excitement, and level of concentration in real time and sends the data to the server.

[1050] The server analyzes the received data and determines whether the next content to provide is a challenging quiz about an animal that the child is particularly interested in, or a video introducing a new animal. The new content is then sent to the device, allowing the child to continue enjoying it.

[1051] After the learning session, the server generates a detailed report of the learning results and emotional changes based on all the data and notifies the parents, who can use this report to check their child's learning progress and emotional changes and plan the next learning session.

[1052] As described above, the present invention supports parents in improving their children's learning effectiveness and helping children to continue learning while maintaining their interest.

[1053] The processing flow will be explained below.

[1054] Step 1: User Input

[1055] The user (parent) starts the app and enters their child's information (name, age, areas of interest, learning content, special attention, etc.) into the form on the screen. They also set the category of learning content they want to provide (e.g., animals, numbers, alphabet) and the learning period (e.g., 30 minutes per day). The device then sends this information to the server.

[1056] Step 2: Save data

[1057] The server stores the received user information in a database, which can then be used for subsequent content selection and data analysis.

[1058] Step 3: Providing initial content

[1059] Based on the user information, the server selects the most suitable initial content for the child (e.g., a quiz game about animals) and sends it to the terminal, which then displays this content to the child.

[1060] Step 4: Collect real-time data

[1061] While children use the app to learn and play, the device uses its built-in camera and sensors to record their facial expressions, gaze, operation frequency, and reaction speed, and this data is sent to a server at regular intervals.

[1062] Step 5: Emotion Recognition with the Emotion Engine

[1063] Based on the data sent from the device, the emotion engine identifies the child's emotions (joy, excitement, stress, concentration, etc.) and sends the recognized emotion data to the server.

[1064] Step 6: Analyze the data

[1065] The server uses AI algorithms to analyze real-time and emotional data, assessing the child's interest and concentration and identifying content and reactions that they are particularly interested in.

[1066] Step 7: Decide on your next content

[1067] Based on the analysis and emotional data, the server then decides what content to provide next, such as a more advanced quiz or a video introducing a new animal.

[1068] Step 8: Providing content

[1069] The server sends the next content to the device. The device displays this new content to the child, allowing them to continue learning or playing. The device then collects real-time data and emotional data and sends them to the server.

[1070] Step 9: Ending the Session

[1071] When the study session ends, the device sends a termination signal to the server, along with all data (study results, concentration level, and emotion analysis results).

[1072] Step 10: Generate reports

[1073] The server uses all the data it receives to generate a detailed report summarizing learning results, progress and emotional changes, including areas of particular interest and concentration of the child.

[1074] Step 11: Feedback Notification

[1075] The server notifies the user of the generated report. The user can check this report through the app to understand the child's learning progress and emotional changes. Based on this feedback, the next lesson plan can be created.

[1076] Example 2

[1077] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1078] Conventional learning applications have the problem of being unable to provide effective learning support because they are unable to properly reflect children's learning effects and interests in real time. Furthermore, because they are unable to provide learning content that takes children's emotions and concentration levels into account, there is a high possibility that children will become bored. Furthermore, it is difficult for parents to properly grasp their children's learning progress and emotional changes, making it difficult to continue learning or provide effective feedback.

[1079] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1080] a means for a user to input information about a child on an initial setup screen;

[1081] means for receiving the child's information and transmitting it to a server;

[1082] means for selecting initial learning content based on the received information and transmitting the content to the terminal;

[1083] A means of collecting data on facial expressions, gaze, operation frequency and reaction speed when children engage with learning content;

[1084] means for transmitting the collected data to a server and recognizing emotions with an emotion engine;

[1085] A means for analyzing the recognized emotion data and real-time data using an AI algorithm to evaluate the interest and concentration of the child;

[1086] a means for determining the next most suitable learning content to be provided based on the evaluation results and transmitting the determined content to the terminal;

[1087] means for generating a report of learning results and emotional changes based on all data after the session is completed;

[1088] means for notifying a user of the generated report;

[1089] This allows the learning content that children receive to be individually optimized over time, improving learning effectiveness, and allows parents to create better learning plans based on detailed learning progress and emotional changes.

[1090] "User information" is individual setting information for using the application, consisting of information such as the child's name, age, areas of interest, learning content, and learning period.

[1091] "Server" means a networked host system that receives, stores, and analyzes user information and selects and provides the most appropriate learning content.

[1092] A "terminal" is a device that provides learning content to children and collects data in real time.

[1093] "Real-time data" is data that is generated and transmitted on the spot, collecting information on children's facial expressions, gaze, operation frequency, and reaction speed.

[1094] The "emotion engine" is a software component that analyzes collected real-time data and recognizes children's emotions (joy, excitement, stress, concentration, etc.).

[1095] "AI algorithm" is an artificial intelligence technology that analyzes real-time and emotional data, finds patterns, and selects the most appropriate learning content.

[1096] A "report" is a document generated after a learning session that contains a detailed record of a child's learning performance, concentration level, emotional changes, etc.

[1097] "Learning content" refers to educational materials such as quizzes, videos, and games provided to help children learn.

[1098] MODE FOR CARRYING OUT THE INVENTION

[1099] The present invention relates to an AI educational app that provides optimal learning content based on user information and children's reaction data collected in real time. Furthermore, by combining an emotion engine, the system recognizes children's emotions and optimizes their learning experience. The following describes in detail the implementation of this system.

[1100] composition

[1101] The system includes the following main elements:

[1102] 1. How to enter user information

[1103] 2. A means of receiving user information and sending it to the server

[1104] 3. A method for selecting optimal initial learning content and sending it to devices

[1105] 4. Means of collecting real-time data (facial expressions, gaze, operation frequency, reaction speed)

[1106] 5. How to Recognize Emotions with an Emotion Engine

[1107] 6. A means of analyzing received data with AI algorithms

[1108] 7. A method for determining the next learning content to be provided and sending it to the device

[1109] 8. A way to generate a report after the learning session is completed

[1110] 9. A means of notifying users of generated reports

[1111] Hardware and Software

[1112] Devices: Devices equipped with cameras and various sensors, such as tablets and smartphones, are used, which enable accurate collection of data on children's facial expressions, gaze, and actions.

[1113] Server: A high-performance cloud server is used to store and analyze user information and select the most appropriate learning content. This is linked to a database system to enable rapid processing of large amounts of data.

[1114] Emotion Engine: A machine learning model is used to recognize children's emotions based on the collected data. The recommended model uses a deep learning framework specialized in emotion analysis.

[1115] AI Algorithm: We use AI algorithms to analyze the received data and select the most suitable learning content. These algorithms incorporate techniques of pattern recognition and predictive analysis.

[1116] Example

[1117] For example, if a parent were to use this app for a 6-year-old child, the steps would be as follows:

[1118] 1. User input of settings: Parents launch the app and enter their child's name, age, area of ​​interest (e.g., animals), learning period (e.g., 30 minutes per day), etc. on the initial setup screen.

[1119] 2. Providing initial content: Based on the received information, the server selects an animal-related quiz game as the optimal initial content and sends it to the device. The device then displays this content to the child.

[1120] 3. Real-time data collection: While the child is working on the content, the device uses its built-in camera and sensors to collect information on the child's facial expressions, gaze, operation frequency, reaction speed, etc. in real time. This data is periodically sent to the server.

[1121] 4. Emotion Recognition and Data Analysis: The emotion engine analyzes the collected data and recognizes the child's emotions (e.g., joy, excitement, stress, concentration). The server then analyzes this data using AI algorithms to evaluate the child's interest and concentration.

[1122] 5. Determining and providing the next content: Based on the analysis results, the server determines the most appropriate learning content to provide next. For example, if a child shows interest in a quiz about animals, a video introducing a new animal will be selected. This information is sent to the device and displayed.

[1123] 6. Post-session report generation and notification: After a learning session, a detailed report is generated based on all data. This report includes academic performance, concentration level, emotional changes, etc. The report is notified to parents and can be viewed through the app.

[1124] Prompt Sentence Examples

[1125] To ask a generative AI model to explain this system in natural language, use a prompt like this:

[1126] "Please explain in natural language in detail the processing of this AI educational app, which provides optimal learning content based on user information and children's reaction data collected in real time, and recognizes and optimizes children's emotions by combining an emotion engine. Please also explain what kind of data processing and calculations are performed using specific hardware (built-in camera, sensors, etc.) and software (emotion engine, AI algorithms, etc.)."

[1127] In this way, by clearly indicating the use of specific hardware and software, and the data processing and calculations based thereon, it is possible to provide a detailed mode for carrying out the present invention.

[1128] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1129] Step 1:

[1130] (User input)

[1131] The user (parent) starts the app and enters the child's name, age, areas of interest, learning content, and learning period on the initial setup screen. The device receives this information and sends it to the server. Based on the input data, the device creates a user profile and sends it to the server, where it is saved in the database.

[1132] Input: Child's name, age, area of ​​interest, study content, study period

[1133] Output: Generates a user profile and sends it to the server

[1134] Specifically, when the user has finished entering each item, he or she presses the send button, and the information is transferred from the terminal to the server. The terminal then displays a message indicating successful transmission.

[1135] Step 2:

[1136] (Providing initial content)

[1137] The server selects the most suitable initial learning content based on the received user information, and the selected content is sent to the terminal, which then displays the content to the student.

[1138] Input: User information

[1139] Output: Selection of initial learning content and transmission to the device

[1140] Specifically, the server searches for an appropriate quiz from the database and sends the data to the device, which then displays the quiz using animation and provides it to the student.

[1141] Step 3:

[1142] (Real-time data collection)

[1143] While students are working on learning content, the device uses its built-in camera and sensors to collect data on their facial expressions, gaze, operation frequency, and reaction speed, which is then periodically sent to a server.

[1144] Input: Learning content usage data (facial expressions, gaze, operation frequency, reaction speed)

[1145] Output: Sending collected real-time data to a server

[1146] Specifically, the camera tracks the child's face and records their eye movements. It also collects information on the frequency of their operations and their reaction speed, and sends this information to a server at regular intervals.

[1147] Step 4:

[1148] (Emotion recognition by emotion engine)

[1149] The server passes the collected real-time data to the emotion engine, which recognizes the child's emotions based on facial expression data and other data, and sends the results to the server.

[1150] Input: Real-time data collected

[1151] Output: Recognized emotion data

[1152] Specifically, the emotion engine analyzes the collected data and identifies the child's emotional state (e.g., joy, concentration, stress). The identified emotion data is sent to the server.

[1153] Step 5:

[1154] (Data Analysis)

[1155] The server uses an AI algorithm to analyze the received real-time data and emotional data, and evaluates the child's level of interest and concentration.

[1156] Input: Real-time data and sentiment data

[1157] Output: Interest and concentration evaluation results

[1158] Specifically, the AI ​​algorithm analyzes data patterns and generates a score based on the child's interest and concentration, which then reveals which content they are interested in.

[1159] Step 6:

[1160] (Determining the next content)

[1161] Based on the analysis results, the server determines the next optimal learning content to provide and sends it to the device, resulting in new learning content suited to the child.

[1162] Input: Evaluation result

[1163] Output: Selecting the most suitable learning content and sending it to the device

[1164] Specifically, the server selects the next appropriate content from the database and sends it to the device, which then displays the new content and continues learning.

[1165] Step 7:

[1166] (Content provided)

[1167] When the server sends the next learning content to the device, the device displays the new content to the child and continues learning. Data collection continues.

[1168] Input: Next learning content

[1169] Output: Display of new learning content

[1170] Specifically, the device will display new quizzes and videos while continuing to collect real-time data.

[1171] Step 8:

[1172] (Ending the session and generating a report)

[1173] After the user-specified learning time has elapsed, the device ends the learning session and then transmits all data to the server, which then generates a report.

[1174] Input: All data from the study session

[1175] Output: Report of learning results, concentration level, and emotional changes

[1176] Specifically, when the session ends, the device displays a message saying the session is over and all data is sent to the server, which then creates a report based on that data.

[1177] Step 9:

[1178] (Feedback Notification)

[1179] The server notifies the user (parent) of the generated report, who can then check the report through the app to understand their child's learning progress and emotional changes.

[1180] Input: Generated report

[1181] Output: Report notification and confirmation

[1182] Specifically, a notification will appear in the app, and when the user taps on the notification, a detailed report will be displayed, including information on study time, concentration level, and emotional changes.

[1183] (Application example 2)

[1184] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1185] Conventional educational apps have difficulty recognizing children's reactions and emotions in real time and optimizing learning content based on that. Furthermore, there are few ways to provide a learning experience tailored to each individual child in physical stores. This makes it difficult to achieve learning results and stimulate children's interest.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting user information, a server for receiving and storing the user information, a means for selecting and providing content based on the user information, a means for recording the child's reactions and operations in real time, a means for identifying the child's emotions from the real-time data using an emotion recognition model, a means for analyzing the reaction and operation data and determining the next content, a means for generating a report of learning results and progress based on the analysis results and emotion data, and a tablet terminal for making the system available in the kids' space of a physical store. This makes it possible to provide optimal learning content based on the child's emotions and reactions and improve the learning experience in real time.

[1187] "Means for inputting user information" refers to an interface that allows a parent or user to input information about a child (such as name, age, areas of interest, special instructions, etc.).

[1188] "Server" refers to a computing device that receives and stores user information and analyzes data collected in real time.

[1189] "Means for selecting and providing content" refers to a system that selects and displays learning content appropriate for children based on user information and collected real-time data.

[1190] "Means for recording children's reactions and operations in real time" refers to the function of using cameras and sensors to collect children's facial expressions, gaze, frequency of operations, reaction speed, etc. in real time.

[1191] "Means for identifying children's emotions from the real-time data using an emotion recognition model" refers to a function that uses an AI model to analyze children's emotions such as joy, excitement, and concentration based on the collected data.

[1192] "Means for analyzing the reaction and operation data and determining the next content" refers to the function of analyzing the collected real-time data and emotional data and determining the next optimal content to be provided based on that data.

[1193] "Means for generating reports on learning results and progress" refers to the function of creating a report summarizing the child's learning results and progress based on the analyzed data and providing it to parents.

[1194] "Tablet terminal for making the system available in the kids' space of a physical store" refers to a tablet device installed to make the system available in the kids' space of a physical store.

[1195] The present invention relates to an AI educational app that provides optimal learning content based on user information and real-time collected data on children's reactions. The system combines an emotion engine to recognize children's emotions and optimize their learning experience. Specific embodiments of the present invention are described below.

[1196] First, the user (parent) enters information about their child using a tablet device installed in the kids' area of ​​the physical store. The user information includes the child's name, age, areas of interest, special attention, desired learning content category, and learning period. This information is sent from the tablet device to the server and stored.

[1197] Next, the server selects the most appropriate initial content based on the user information sent and sends it to the tablet device. For example, a quiz game about animals might be selected. As a child plays the quiz game using the tablet device, the device's built-in camera and sensors collect data in real time, such as the child's facial expressions, line of sight, operation frequency, and reaction speed. This data is sent to the server at regular intervals.

[1198] The emotion recognition model uses collected real-time data to identify a child's emotions. This emotion data is also sent to the server. The server then uses an AI algorithm to analyze the received real-time data and emotion data, and performs an analytical process to evaluate the child's level of interest and concentration. Based on the results, it determines the most appropriate content to provide next. For example, if a child shows interest in an animal quiz, a video introducing new animals or a more difficult quiz will be selected.

[1199] The tablet device then displays new content to the child, and the learning or play experience continues. The device continues to collect real-time data and emotional data and transmits it to the server. After the learning session ends, the server generates a detailed report based on all data (such as learning performance, concentration level, and emotional analysis results) and notifies the user. This allows the user to understand the child's learning progress and emotional changes and plan the next lesson.

[1200] For example, a parent might enter information about a six-year-old child on a tablet device and indicate that the child is interested in "animals." In this case, the server selects the initial content of an animal quiz and sends it to the tablet device. While the child is enjoying the quiz, the built-in camera analyzes their facial expressions in real time, and if they smile a lot, the server will present them with a new animal quiz or video.

[1201] Below are some example prompts to input to a generative AI model:

[1202] A 6-year-old boy named Taro is interested in animal quizzes. Write code to analyze Taro's reactions in real time using a facial expression recognition system, and if he seems to be enjoying it, determine the next best learning content to provide him. Also, please provide detailed information about the process of sending and receiving data to the server.

[1203] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1204] Step 1:

[1205] User input:

[1206] Users (parents) use tablet devices installed in the kids' areas of physical stores to enter their child's information (name, age, areas of interest, special attention, desired learning content category, and learning period). The device then sends this information to the server, which then stores the user profile in a database.

[1207] Input: Child's name, age, areas of interest, special notes, desired learning content category and learning period

[1208] Output: User information sent to server, saved user profile

[1209] Step 2:

[1210] Initial content provided:

[1211] Based on the received user information, the server selects the most appropriate initial content (for example, a quiz game about animals) and sends it to the tablet device, which then displays this content to the child.

[1212] Input: User profile stored on the server

[1213] Output: Initial content

[1214] Step 3:

[1215] Real-time data collection:

[1216] When a child uses a tablet device to engage with learning content, the device's built-in camera and sensors collect real-time data on the child's facial expressions, gaze, frequency of operations, reaction speed, etc. This data is then sent to a server at regular intervals.

[1217] Input: Real-time reaction data of children (facial expressions, gaze, frequency of operation, reaction speed)

[1218] Output: Real-time data sent

[1219] Step 4:

[1220] Emotion Recognition with Emotion Engine:

[1221] The server inputs the received real-time data into an emotion recognition model to identify the child's emotions (joy, excitement, concentration, etc.), and this emotion data is also stored on the server.

[1222] Input: Real-time data

[1223] Output: Identified emotion data

[1224] Step 5:

[1225] Data analysis:

[1226] The server uses AI algorithms to analyze real-time data and emotional data to assess the child's level of interest and concentration, and then determines the next most appropriate content to provide.

[1227] Input: Real-time data, emotion data

[1228] Output: Analysis results, next best content to provide

[1229] Step 6:

[1230] Providing the following content:

[1231] The server determines the next content based on the analysis results and sends it to the tablet device (for example, a video introducing a new animal or a high-level quiz). The device then displays this new content to the child.

[1232] Input: The next best thing to serve

[1233] Output: New content

[1234] Step 7:

[1235] End the session and generate a report:

[1236] After the user-specified study time has elapsed, the tablet device ends the study session. The device sends all data (such as study performance, concentration level, and emotion analysis results) to the server, which then generates a detailed report and notifies the user.

[1237] Input: Learning performance, concentration level, emotion analysis results

[1238] Output: Detailed report

[1239] As a result, by using this system, it is possible to draw out children's learning effectiveness and interest, and provide them with optimal learning content in real time.

[1240] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1241] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1242] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1243] [Fourth embodiment]

[1244] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1245] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1246] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1247] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1248] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1249] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1250] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1251] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1252] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1253] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1254] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1255] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1256] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1257] This invention relates to an AI educational app that provides optimal learning content according to a child's reactions, developmental status, and learning needs. This system analyzes a child's interest and concentration in real time based on the content set in advance by the parent, and provides appropriate content and a report of the learning results to the parent.

[1258] Program processing overview

[1259] User input of settings

[1260] Through the app, users (parents) enter their child's information (such as name, age, areas of interest, and special attention). They also set the desired learning content category (e.g., animals, numbers, alphabet) and learning period (e.g., 30 minutes per day). The device then sends this information to the server and saves it as a user profile.

[1261] Real-time data collection

[1262] When a child uses the app to engage in learning or playful activities, the device uses its built-in camera and sensors to collect real-time data such as the child's facial expressions, gaze, frequency of operations, and reaction speed, and the device then transmits this data to a server at regular intervals.

[1263] Analyzing the data

[1264] The server uses an AI algorithm to analyze the child's interest and concentration based on the real-time data it receives, and then determines the optimal content to provide next.

[1265] Content provision

[1266] The server then sends the most appropriate content based on the analysis results to the device, which then displays that content to the child, allowing them to continue learning or playing. The device continues to collect real-time data and send it to the server.

[1267] Feedback and report generation

[1268] When a study session ends, the device sends all data (study results, concentration level, etc.) to the server. The server uses this data to generate a report summarizing the study results and progress. The generated report is notified to the user by the server, and the user can view the report through the app.

[1269] Specific examples

[1270] For example, let's say parent A uses this app for their 5-year-old daughter B. Parent A enters B's information into the app and sets animal-related quizzes and a learning period of 30 minutes per day. When B starts using the app, the device's camera analyzes B's facial expressions and gaze, and records the frequency of operations and the speed at which she responds. This data is sent to a server, which uses AI to analyze B's level of interest and concentration.

[1271] If the analysis shows that B-chan is very interested in animal quizzes, the server will send more difficult animal quizzes and videos introducing new animals to the device, which will then provide this new content to B-chan, allowing her to continue learning.

[1272] When the learning session ends, the device sends the session data to the server, which then generates a detailed report based on this data. This report includes information on how long B-chan focused on her learning and which content she showed particular interest in. Person A can check this report to see B-chan's learning progress.

[1273] As described above, the present invention provides a system that allows parents to effectively support their children's education, and enables children to continue learning while maintaining their interest.

[1274] The processing flow will be explained below.

[1275] Step 1: User Input

[1276] The user (parent) starts the app and enters their child's information (such as name, age, areas of interest, learning content, and special attention) into the form on the screen. They also set the categories of learning content they want to provide and the usage time.

[1277] Step 2: Send data

[1278] The device sends the entered user information and learning settings to the server, which receives it and stores the user information in a database.

[1279] Step 3: Providing initial content

[1280] The server selects the most suitable initial content (e.g., a quiz game about animals) based on the user information and sends it to the terminal, which then displays this content to the child.

[1281] Step 4: Collect real-time data

[1282] While the child is working on the content, the device uses its built-in camera and sensors to collect real-time data on the child's facial expressions, gaze, operation frequency, reaction speed, etc. This data is periodically sent to a server.

[1283] Step 5: Analyze the data

[1284] The server uses AI algorithms to analyze the child's interest and concentration based on the received real-time data, and evaluates which content the child is particularly interested in.

[1285] Step 6: Decide on your next content

[1286] Based on the analysis, the server then decides the next best content to serve, such as a more advanced animal quiz or a video introducing a new animal.

[1287] Step 7: Providing content

[1288] The server sends the next content to the device, which continues to display this new content to the child. The device continues to collect real-time data and send it to the server.

[1289] Step 8: Ending the Session

[1290] When the user-specified study time has elapsed, the device ends the study session. After the session ends, the device sends all data (study results, concentration analysis results, etc.) to the server.

[1291] Step 9: Generate reports

[1292] The server generates a report summarizing the learning results and progress based on the data sent, including details on the student's concentration level, response speed, and interest.

[1293] Step 10: Feedback Notification

[1294] The server notifies the user of the generated report, who can then check the report through the app to understand the child's learning progress. Based on this feedback, the user can plan the next lesson.

[1295] Example 1

[1296] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1297] Conventional educational applications have difficulty providing content tailored to individual children's interests and learning progress, and do not provide sufficient feedback for parents to effectively support their children's learning. Furthermore, they are unable to record children's reactions and operations in real time and optimize learning content based on that analysis, which can lead to reduced learning effectiveness.

[1298] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1299] In this invention, the server includes means for inputting user information, means for receiving and storing the user information, means for selecting and providing content based on the user information, means for recording the child's responses and operations in real time, means for analyzing the response and operation data and using an AI algorithm to determine the next content, and means for generating and providing a report of learning results and progress based on the analysis results. This allows parents to provide optimal learning content based on their child's individual interests and learning progress, and also allows them to understand their child's learning status in real time and receive effective feedback.

[1300] "User information" refers to data about the child, such as their name, age, areas of interest, special attention, learning content, and usage time.

[1301] A "server" is a computer system that receives and stores user information and performs analysis.

[1302] "Content" refers to educational or entertainment materials, quizzes, videos, animations, and other data provided based on children's learning and interests.

[1303] "Reactions" refers to behavioral data such as facial expressions, gaze, frequency of operation, and reaction speed when children use an application.

[1304] An "AI algorithm" is a program that uses technologies such as machine learning and deep learning to analyze data and evaluate children's interest and concentration levels.

[1305] "Learning results" refers to data that indicates what a child has learned, their grades, progress, etc.

[1306] "Progress" is data that indicates the progress and achievement of a child in their studies.

[1307] A "report" is a report generated by the server summarizing a student's learning results and progress.

[1308] This invention relates to an AI educational app that provides optimal learning content based on a child's reactions, developmental status, and learning needs. This system analyzes a child's interest and concentration in real time based on the content set in advance by the parent, and provides appropriate content and a report of the learning results to the parent. The specific implementation method of this system is described below.

[1309] User input of settings

[1310] The parent / guardian user first launches the app and enters information about their child. This information includes the child's name, age, areas of interest, and any special attention they may need. They also set the desired learning content category (e.g., animals, numbers, alphabet) and the amount of time they want to study per day (e.g., 30 minutes per day). The device then sends this information to a server, where it is saved as a user profile. The device used is typically a mobile device such as a smartphone or tablet.

[1311] Real-time data collection

[1312] When a child uses the app to engage with learning and playful content, the device uses its built-in camera and sensors to collect real-time data such as the child's facial expressions, gaze, operation frequency, and reaction speed. For example, the camera and sensors on an iPhone or Android smartphone are used to collect this data. The device then transmits this data to a server at regular intervals.

[1313] Analyzing the data

[1314] The server uses AI algorithms to analyze the child's interest and concentration based on the real-time data it receives. TensorFlow and PyTorch are often used as AI models. For example, the server analyzes eye-tracking data to assess the type of content the child is interested in. It also measures the child's concentration based on operation frequency and reaction speed, and determines the optimal content to provide next.

[1315] Content provision

[1316] Based on the analysis results, the server selects the next most appropriate content to provide and sends it to the device. The device then provides the content received from the server to the child, allowing them to continue learning or playing. For example, if a child shows a strong interest in animal quizzes, the server will send new animal quizzes and related videos to the device.

[1317] Feedback and report generation

[1318] Once a learning session is over, the device sends all learning data (such as learning performance, concentration level, and frequency of operation) to the server. The server uses this data to generate a detailed report using an AI algorithm. This report includes information on which content the child showed interest in, how much time they spent, and how focused they were. The server then notifies the user of the generated report, which can be viewed through the app.

[1319] Specific examples

[1320] For example, consider the case where parent A uses this app for their 5-year-old daughter B. Parent A enters B's information into their device, selects animal-related quizzes as the learning content, and sets the learning period to 30 minutes per day. When B begins using the app, the device's camera analyzes B's facial expressions and gaze, and records the frequency of operations and response speed. This data is sent to a server, which uses an AI model to analyze B's level of interest and concentration.

[1321] If the analysis shows that B-chan is very interested in animal quizzes, the server will send more difficult animal quizzes and videos introducing new animals to the device, which will then provide this new content to B-chan, allowing her to continue learning.

[1322] When the learning session ends, the device sends the session data to the server, which then generates a detailed report based on this data. This report includes information on how long B-chan focused on her learning and which content she showed particular interest in. Person A can check this report to see B-chan's learning progress.

[1323] Prompt Sentence Examples

[1324] Below are some examples of specific prompt sentences to input into the generative AI model.

[1325] "I want to provide an animal quiz that will interest 5-year-olds."

[1326] "Please recommend content that is highly relevant"

[1327] "Provide content related to a specific learning category (e.g., the alphabet)"

[1328] As described above, the present invention is a system that allows parents to effectively support their children's education, and is designed to enable children to continue learning while maintaining their interest.

[1329] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1330] Step 1: User inputs settings

[1331] The user launches the app and enters information about their child (such as name, age, areas of interest, and special attention). They also set the desired learning content category (e.g., animals, numbers, alphabet) and the amount of time they want to study per day (e.g., 30 minutes per day). The device then sends this information to the server and saves it as a user profile.

[1332] Specifically, the user enters the required information into the form on the screen of their smartphone or tablet and presses the submit button. The input data is sent from the device to the server in JSON format.

[1333] Input: Child's name, age, area of ​​interest, special attention, learning content category, learning time

[1334] Output: Save user profile

[1335] Step 2: Collect real-time data

[1336] When a user begins learning using the app, the device uses its built-in camera and sensors to collect real-time data, including the child's facial expressions, gaze, frequency of operations, and reaction time. This data is then sent to a server at regular intervals.

[1337] Specifically, the device's camera tracks the child's facial expressions and gaze, and sensors measure the frequency of operations and reaction speed. The data is sent to a server in JSON format at regular intervals.

[1338] Input: Real-time data during use by children (facial expressions, gaze, frequency of operation, reaction speed)

[1339] Output: Real-time data sent to the server

[1340] Step 3: Analyze the data

[1341] The server uses an AI algorithm to analyze the received real-time data, specifically analyzing the child's interest and concentration level to determine the next optimal content to provide.

[1342] Specifically, the server analyzes the data using AI models such as TensorFlow and PyTorch, assessing interest from gaze and facial expression data, and calculating concentration levels from operation frequency and reaction speed.

[1343] Input: Real-time data (facial expressions, gaze, operation frequency, reaction speed)

[1344] Output: Analysis results (interest level, concentration level)

[1345] Step 4: Providing content

[1346] Based on the analysis results, the server selects the most appropriate content and sends it to the device. The device then displays the appropriate content received from the server to the student, allowing them to continue their learning.

[1347] Specifically, the server determines the next content to provide based on the analyzed interest and concentration levels. The determined content is sent to the device in JSON format, and the device displays it.

[1348] Input: Analysis results

[1349] Output: Content sent to the device

[1350] Step 5: Generate feedback and reports

[1351] Once the learning session is over, the device sends all learning data to the server, which generates a detailed report and notifies the user.

[1352] Specifically, at the end of a session, the device sends the collected data (such as learning performance, concentration level, and frequency of operations) to the server, which then uses an AI algorithm to generate a report and send a notification to the user's device.

[1353] Input: Training data

[1354] Output: Generate learning report and notify user

[1355] Through these steps, the system provides optimal content tailored to each child's learning needs, providing effective learning support.

[1356] (Application example 1)

[1357] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1358] While conventional educational systems for children provide optimal learning content based on user information, they are primarily designed for use at home and are not suitable for use in physical stores. Furthermore, there is no system in physical stores that can record children's reactions and actions in real time, instantly analyze that data, and provide the next content. Furthermore, there are limited means of providing parents with feedback on their learning results and progress. Under these circumstances, it has been difficult to expand children's daily learning environments and provide more diverse learning experiences.

[1359] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1360] In this invention, the server includes a means for inputting user information, a means for receiving and storing the user information, a means for selecting and providing content based on the user information, a means for recording the user's responses and operations in real time, a means for analyzing the response and operation data to determine the next content, a means for generating a report of learning results and progress based on the analysis results, a means for providing the content using a robot installed in a physical store and recording the responses and operations, and an input device for the user to input the information at the physical store. This makes it possible to accurately grasp the learning status of children in real time even in the physical store and continue to provide appropriate learning content. Furthermore, by providing parents with immediate feedback on learning results and progress, it is possible to effectively support the children's learning experience.

[1361] "User information" refers to personal information entered by a child or their guardian, such as age, areas of interest, learning content, and usage time, that is necessary for providing learning content.

[1362] "Server" refers to a central processing unit that receives and stores user information, and then selects and provides the most appropriate content based on that information.

[1363] "Content" refers to information such as activities and teaching materials provided to children for the purpose of learning.

[1364] A "robot" is a mechanical device that is installed in a physical store and provides learning content interactively to children.

[1365] "Reactions" refer to the emotions and behaviors that children show toward the learning content, such as changes in facial expressions and body movements.

[1366] "Operation" refers to specific actions that children take with learning content, such as pressing a button or swiping.

[1367] Analyzing the "data" means collecting information about children's reactions and operations, and using that information to evaluate their interest and concentration.

[1368] An "input device" is a device for inputting user information in a physical store, and includes tablet terminals, smartphones, etc.

[1369] A "report" is a report summarizing learning results and progress, and is provided to parents so that they can check their child's learning status.

[1370] This invention is a system that supports children's learning experiences using educational robots installed in brick-and-mortar stores. This system provides optimal learning content based on user information, records and analyzes children's reactions and actions in real time, and determines the next content based on the results. It also generates reports on learning results and progress and provides them to parents, allowing them to check their children's learning progress.

[1371] System Program Overview

[1372] 1. User information input method

[1373] Users (parents) enter their child's information using a tablet or smartphone installed at the reception desk of a physical store. User information includes the child's name, age, areas of interest, learning content, and usage time. This information is entered through a web form interface and stored in a cloud database (e.g., Firebase).

[1374] 2. Real-time data collection methods

[1375] As children engage with learning content through the robot, the robot uses cameras and sensors to record the children's facial expressions, gaze, frequency of operations, and reaction speed in real time, using image recognition algorithms (e.g., OpenCV) and voice recognition software.

[1376] 3. Data Analysis Methods

[1377] The collected real-time data is sent to a cloud server, which uses AI analysis tools (e.g., TensorFlow) to analyze the student's interest and concentration. Based on the analysis results, the next optimal learning content to be provided is determined.

[1378] 4. Means of providing content

[1379] Based on the analysis results, the server determines the optimal learning content and sends it to the robot, which then provides that content to the child and continues to collect real-time data.

[1380] 5. Report Generation Methods

[1381] When the learning session ends, the robot sends data such as learning results and concentration level to the server in bulk. The server then generates a report summarizing the learning results and progress and provides it to parents. The report is displayed in HTML / CSS and is generated using a data processing tool (e.g., Pandas).

[1382] Specific examples

[1383] For example, if parent C uses this system at a physical store for their 7-year-old son, D, they enter D's information at the reception desk and select math puzzles as the learning content. As D begins to solve the math puzzle with the robot, the robot captures D's facial expressions and actions with a camera and sends the data to the server in real time. The server uses AI to analyze D's interest and level of concentration and decides which problem to tackle next. Once the learning is complete, the server generates a report based on all the data, which parent C can check on a tablet or smartphone.

[1384] Prompt Sentence Examples

[1385] "Information about D-kun"

[1386] Name: D-kun

[1387] Age: 7

[1388] Interests: Mathematics, puzzles

[1389] Special Notes: None

[1390] "Learning Content Settings"

[1391] Category: Math Puzzles

[1392] Study duration: 20 minutes per day

[1393] "Study Session Results Report"

[1394] 1. Session duration: 20 minutes

[1395] 2. Interest level: High

[1396] 3. Concentration: High

[1397] 4. Content provided: High-level math puzzles

[1398] 5. Overall rating: Excellent

[1399] The above is a specific embodiment of the present invention. This system effectively supports children's learning experiences even in brick-and-mortar stores, and allows parents to understand their children's learning progress in real time.

[1400] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1401] Step 1:

[1402] Users (parents) use tablets or smartphones installed at the reception desk of physical stores to enter information about their children, such as their name, age, interests, learning content, and usage time. This information is entered through a web form interface and sent to a cloud database (e.g., Firebase) for storage.

[1403] Input: Child's name, age, area of ​​interest, learning content, usage time

[1404] Output: User information stored in a cloud database

[1405] Step 2:

[1406] The device receives user information entered by the user (parent) and sends it to the server. After receiving this information, the server stores it as base data for selecting appropriate learning content for each child.

[1407] Input: User information

[1408] Output: User information stored on the server

[1409] Step 3:

[1410] Students move to the learning area and use the robot to carry out learning content. The robot uses cameras and sensors to record students' facial expressions, gaze, operation frequency, reaction speed, and other information in real time. This real-time data is sent to a server at regular intervals.

[1411] Input: Child's facial expression, gaze, operation frequency, reaction speed

[1412] Output: Data transmitted in real time

[1413] Step 4:

[1414] The server inputs the received real-time data into an AI analysis tool (e.g., TensorFlow) to analyze the child's interest and concentration. Image recognition algorithms (e.g., OpenCV) and voice recognition software are used for the analysis. The analysis results are used to select the next content.

[1415] Input: Real-time data

[1416] Output: Analysis results of children's interest and concentration

[1417] Step 5:

[1418] The server then determines the next optimal learning content to provide based on the analysis results and sends it to the robot, adjusting the difficulty and type of content according to the child's learning progress.

[1419] Input: Analysis results of children's interest and concentration

[1420] Output: Optimal learning content

[1421] Step 6:

[1422] The robot presents the content received from the server to the child and continues to record the child's reactions and actions, thereby continuously collecting real-time data needed for the next cycle.

[1423] Input: Optimal learning content

[1424] Output: Learning content presented to students

[1425] Step 7:

[1426] After the learning session is over, the robot sends all the data from the entire session to the server, which then uses this data to generate a report summarizing the learning results and progress. This report generation is done using a data processing tool (e.g., Pandas).

[1427] Input: Session-wide data

[1428] Output: Generated training result report

[1429] Step 8:

[1430] The server stores the generated report on the cloud and notifies the parents, who can then check the report using a tablet or smartphone located in the store.

[1431] Input: Generated learning result report

[1432] Output: Report provided to parents

[1433] The system's processing consists of the above steps, each of which optimally collects, analyzes, and provides the data required for the next step to optimize the learning experience for children in physical stores.

[1434] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1435] This invention relates to an AI educational app that provides optimal learning content based on user information and children's reaction data collected in real time, and further combines an emotion engine to recognize children's emotions and optimize the learning experience. This system analyzes the content preset by parents and children's reactions in real time, providing appropriate content to improve children's learning effectiveness, and provides parents with a report of the learning results.

[1436] Program processing overview

[1437] User input of settings

[1438] The user (parent) starts the app and enters their child's information (name, age, areas of interest, learning content, special attention, etc.) on the initial setup screen. They also set the desired learning content category and learning period. The device sends this information to the server and saves it in the database as a user profile.

[1439] Providing initial content

[1440] The server selects the most suitable initial content (e.g., a quiz game about animals) based on the user information and sends it to the device, which then displays this content to the child.

[1441] Real-time data collection

[1442] When a child uses the app to engage in learning or playful activities, the device uses its built-in camera and sensors to collect real-time data on the child's facial expressions, gaze, frequency of operations, reaction speed, etc. The device then periodically transmits this data to a server.

[1443] Emotion recognition by emotion engine

[1444] The emotion engine recognizes the child's emotions (joy, excitement, stress, concentration, etc.) based on the collected real-time data. The emotion data recognized by the emotion engine is sent to the server.

[1445] Data analysis

[1446] The server uses AI algorithms to analyze the received real-time data and emotional data, which allows it to assess the child's interest and concentration and determine which content they are most interested in.

[1447] Deciding on the next content

[1448] The server then uses the analysis results and emotional data to determine the best content to provide next, such as a video introducing new animals or a more challenging quiz to increase the fun the child experiences.

[1449] Content provision

[1450] The server then sends the next content to the device, which then displays the new content to the child, allowing them to continue their learning or playing experience. The device continues to collect real-time data and emotional data and transmits it to the server.

[1451] Ending a session and generating a report

[1452] Once the user-specified study time has elapsed, the device will end the study session. After the session ends, the device will send all data (such as learning performance, concentration level, and emotional analysis results) to the server. The server will then generate a detailed report based on this data. This report will include information on the child's emotional changes, concentration level, interest level, etc.

[1453] Feedback Notifications

[1454] The server notifies the user of the generated report. The user can check the report through the app and understand the child's learning progress and emotional changes. Based on this feedback, the user can create the next learning plan.

[1455] Specific examples

[1456] For example, let's say a parent is using this app for their six-year-old child. The parent enters the child's information on the initial setup screen, setting the child's area of ​​interest to "animals" and the learning period to "30 minutes per day." The server selects the optimal initial content based on this information and sends it to the device.

[1457] When a child starts a quiz game about animals, the device uses a camera to record the child's facial expressions, gaze, and frequency of operations. From this data, the emotion engine recognizes the child's joy, excitement, and level of concentration in real time and sends the data to the server.

[1458] The server analyzes the received data and determines whether the next content to provide is a challenging quiz about an animal that the child is particularly interested in, or a video introducing a new animal. The new content is then sent to the device, allowing the child to continue enjoying it.

[1459] After the learning session, the server generates a detailed report of the learning results and emotional changes based on all the data and notifies the parents, who can use this report to check their child's learning progress and emotional changes and plan the next learning session.

[1460] As described above, the present invention supports parents in improving their children's learning effectiveness and helping children to continue learning while maintaining their interest.

[1461] The processing flow will be explained below.

[1462] Step 1: User Input

[1463] The user (parent) starts the app and enters their child's information (name, age, areas of interest, learning content, special attention, etc.) into the form on the screen. They also set the category of learning content they want to provide (e.g., animals, numbers, alphabet) and the learning period (e.g., 30 minutes per day). The device then sends this information to the server.

[1464] Step 2: Save data

[1465] The server stores the received user information in a database, which can then be used for subsequent content selection and data analysis.

[1466] Step 3: Providing initial content

[1467] Based on the user information, the server selects the most suitable initial content for the child (e.g., a quiz game about animals) and sends it to the terminal, which then displays this content to the child.

[1468] Step 4: Collect real-time data

[1469] While children use the app to learn and play, the device uses its built-in camera and sensors to record their facial expressions, gaze, operation frequency, and reaction speed, and this data is sent to a server at regular intervals.

[1470] Step 5: Emotion Recognition with the Emotion Engine

[1471] Based on the data sent from the device, the emotion engine identifies the child's emotions (joy, excitement, stress, concentration, etc.) and sends the recognized emotion data to the server.

[1472] Step 6: Analyze the data

[1473] The server uses AI algorithms to analyze real-time and emotional data, assessing the child's interest and concentration and identifying content and reactions that they are particularly interested in.

[1474] Step 7: Decide on your next content

[1475] Based on the analysis and emotional data, the server then decides what content to provide next, such as a more advanced quiz or a video introducing a new animal.

[1476] Step 8: Providing content

[1477] The server sends the next content to the device. The device displays this new content to the child, allowing them to continue learning or playing. The device then collects real-time data and emotional data and sends them to the server.

[1478] Step 9: Ending the Session

[1479] When the study session ends, the device sends a termination signal to the server, along with all data (study results, concentration level, and emotion analysis results).

[1480] Step 10: Generate reports

[1481] The server uses all the data it receives to generate a detailed report summarizing learning results, progress and emotional changes, including areas of particular interest and concentration of the child.

[1482] Step 11: Feedback Notification

[1483] The server notifies the user of the generated report. The user can check this report through the app to understand the child's learning progress and emotional changes. Based on this feedback, the next lesson plan can be created.

[1484] Example 2

[1485] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1486] Conventional learning applications have the problem of being unable to provide effective learning support because they are unable to properly reflect children's learning effects and interests in real time. Furthermore, because they are unable to provide learning content that takes children's emotions and concentration levels into account, there is a high possibility that children will become bored. Furthermore, it is difficult for parents to properly grasp their children's learning progress and emotional changes, making it difficult to continue learning or provide effective feedback.

[1487] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1488] a means for a user to input information about a child on an initial setup screen;

[1489] means for receiving the child's information and transmitting it to a server;

[1490] means for selecting initial learning content based on the received information and transmitting the content to the terminal;

[1491] A means of collecting data on facial expressions, gaze, operation frequency and reaction speed when children engage with learning content;

[1492] means for transmitting the collected data to a server and recognizing emotions with an emotion engine;

[1493] A means for analyzing the recognized emotion data and real-time data using an AI algorithm to evaluate the interest and concentration of the child;

[1494] a means for determining the next most suitable learning content to be provided based on the evaluation results and transmitting the determined content to the terminal;

[1495] means for generating a report of learning results and emotional changes based on all data after the session is completed;

[1496] means for notifying a user of the generated report;

[1497] This allows the learning content that children receive to be individually optimized over time, improving learning effectiveness, and allows parents to create better learning plans based on detailed learning progress and emotional changes.

[1498] "User information" is individual setting information for using the application, consisting of information such as the child's name, age, areas of interest, learning content, and learning period.

[1499] "Server" means a networked host system that receives, stores, and analyzes user information and selects and provides the most appropriate learning content.

[1500] A "terminal" is a device that provides learning content to children and collects data in real time.

[1501] "Real-time data" is data that is generated and transmitted on the spot, collecting information on children's facial expressions, gaze, operation frequency, and reaction speed.

[1502] The "emotion engine" is a software component that analyzes collected real-time data and recognizes children's emotions (joy, excitement, stress, concentration, etc.).

[1503] "AI algorithm" is an artificial intelligence technology that analyzes real-time and emotional data, finds patterns, and selects the most appropriate learning content.

[1504] A "report" is a document generated after a learning session that contains a detailed record of a child's learning performance, concentration level, emotional changes, etc.

[1505] "Learning content" refers to educational materials such as quizzes, videos, and games provided to help children learn.

[1506] MODE FOR CARRYING OUT THE INVENTION

[1507] The present invention relates to an AI educational app that provides optimal learning content based on user information and children's reaction data collected in real time. Furthermore, by combining an emotion engine, the system recognizes children's emotions and optimizes their learning experience. The following describes in detail the implementation of this system.

[1508] composition

[1509] The system includes the following main elements:

[1510] 1. How to enter user information

[1511] 2. A means of receiving user information and sending it to the server

[1512] 3. A method for selecting optimal initial learning content and sending it to devices

[1513] 4. Means of collecting real-time data (facial expressions, gaze, operation frequency, reaction speed)

[1514] 5. How to Recognize Emotions with an Emotion Engine

[1515] 6. A means of analyzing received data with AI algorithms

[1516] 7. A method for determining the next learning content to be provided and sending it to the device

[1517] 8. A way to generate a report after the learning session is completed

[1518] 9. A means of notifying users of generated reports

[1519] Hardware and Software

[1520] Devices: Devices equipped with cameras and various sensors, such as tablets and smartphones, are used, which enable accurate collection of data on children's facial expressions, gaze, and actions.

[1521] Server: A high-performance cloud server is used to store and analyze user information and select the most appropriate learning content. This is linked to a database system to enable rapid processing of large amounts of data.

[1522] Emotion Engine: A machine learning model is used to recognize children's emotions based on the collected data. The recommended model uses a deep learning framework specialized in emotion analysis.

[1523] AI Algorithm: We use AI algorithms to analyze the received data and select the most suitable learning content. These algorithms incorporate techniques of pattern recognition and predictive analysis.

[1524] Example

[1525] For example, if a parent were to use this app for a 6-year-old child, the steps would be as follows:

[1526] 1. User input of settings: Parents launch the app and enter their child's name, age, area of ​​interest (e.g., animals), learning period (e.g., 30 minutes per day), etc. on the initial setup screen.

[1527] 2. Providing initial content: Based on the received information, the server selects an animal-related quiz game as the optimal initial content and sends it to the device. The device then displays this content to the child.

[1528] 3. Real-time data collection: While the child is working on the content, the device uses its built-in camera and sensors to collect information on the child's facial expressions, gaze, operation frequency, reaction speed, etc. in real time. This data is periodically sent to the server.

[1529] 4. Emotion Recognition and Data Analysis: The emotion engine analyzes the collected data and recognizes the child's emotions (e.g., joy, excitement, stress, concentration). The server then analyzes this data using AI algorithms to evaluate the child's interest and concentration.

[1530] 5. Determining and providing the next content: Based on the analysis results, the server determines the most appropriate learning content to provide next. For example, if a child shows interest in a quiz about animals, a video introducing a new animal will be selected. This information is sent to the device and displayed.

[1531] 6. Post-session report generation and notification: After a learning session, a detailed report is generated based on all data. This report includes academic performance, concentration level, emotional changes, etc. The report is notified to parents and can be viewed through the app.

[1532] Prompt Sentence Examples

[1533] To ask a generative AI model to explain this system in natural language, use a prompt like this:

[1534] "Please explain in natural language in detail the processing of this AI educational app, which provides optimal learning content based on user information and children's reaction data collected in real time, and recognizes and optimizes children's emotions by combining an emotion engine. Please also explain what kind of data processing and calculations are performed using specific hardware (built-in camera, sensors, etc.) and software (emotion engine, AI algorithms, etc.)."

[1535] In this way, by clearly indicating the use of specific hardware and software, and the data processing and calculations based thereon, it is possible to provide a detailed mode for carrying out the present invention.

[1536] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1537] Step 1:

[1538] (User input)

[1539] The user (parent) starts the app and enters the child's name, age, areas of interest, learning content, and learning period on the initial setup screen. The device receives this information and sends it to the server. Based on the input data, the device creates a user profile and sends it to the server, where it is saved in the database.

[1540] Input: Child's name, age, area of ​​interest, study content, study period

[1541] Output: Generates a user profile and sends it to the server

[1542] Specifically, when the user has finished entering each item, he or she presses the send button, and the information is transferred from the terminal to the server. The terminal then displays a message indicating successful transmission.

[1543] Step 2:

[1544] (Providing initial content)

[1545] The server selects the most suitable initial learning content based on the received user information, and the selected content is sent to the terminal, which then displays the content to the student.

[1546] Input: User information

[1547] Output: Selection of initial learning content and transmission to the device

[1548] Specifically, the server searches for an appropriate quiz from the database and sends the data to the device, which then displays the quiz using animation and provides it to the student.

[1549] Step 3:

[1550] (Real-time data collection)

[1551] While students are working on learning content, the device uses its built-in camera and sensors to collect data on their facial expressions, gaze, operation frequency, and reaction speed, which is then periodically sent to a server.

[1552] Input: Learning content usage data (facial expressions, gaze, operation frequency, reaction speed)

[1553] Output: Sending collected real-time data to a server

[1554] Specifically, the camera tracks the child's face and records their eye movements. It also collects information on the frequency of their operations and their reaction speed, and sends this information to a server at regular intervals.

[1555] Step 4:

[1556] (Emotion recognition by emotion engine)

[1557] The server passes the collected real-time data to the emotion engine, which recognizes the child's emotions based on facial expression data and other data, and sends the results to the server.

[1558] Input: Real-time data collected

[1559] Output: Recognized emotion data

[1560] Specifically, the emotion engine analyzes the collected data and identifies the child's emotional state (e.g., joy, concentration, stress). The identified emotion data is sent to the server.

[1561] Step 5:

[1562] (Data Analysis)

[1563] The server uses an AI algorithm to analyze the received real-time data and emotional data, and evaluates the child's level of interest and concentration.

[1564] Input: Real-time data and sentiment data

[1565] Output: Interest and concentration evaluation results

[1566] Specifically, the AI ​​algorithm analyzes data patterns and generates a score based on the child's interest and concentration, which then reveals which content they are interested in.

[1567] Step 6:

[1568] (Determining the next content)

[1569] Based on the analysis results, the server determines the next optimal learning content to provide and sends it to the device, resulting in new learning content suited to the child.

[1570] Input: Evaluation result

[1571] Output: Selecting the most suitable learning content and sending it to the device

[1572] Specifically, the server selects the next appropriate content from the database and sends it to the device, which then displays the new content and continues learning.

[1573] Step 7:

[1574] (Content provided)

[1575] When the server sends the next learning content to the device, the device displays the new content to the child and continues learning. Data collection continues.

[1576] Input: Next learning content

[1577] Output: Display of new learning content

[1578] Specifically, the device will display new quizzes and videos while continuing to collect real-time data.

[1579] Step 8:

[1580] (Ending the session and generating a report)

[1581] After the user-specified learning time has elapsed, the device ends the learning session and then transmits all data to the server, which then generates a report.

[1582] Input: All data from the study session

[1583] Output: Report of learning results, concentration level, and emotional changes

[1584] Specifically, when the session ends, the device displays a message saying the session is over and all data is sent to the server, which then creates a report based on that data.

[1585] Step 9:

[1586] (Feedback Notification)

[1587] The server notifies the user (parent) of the generated report, who can then check the report through the app to understand their child's learning progress and emotional changes.

[1588] Input: Generated report

[1589] Output: Report notification and confirmation

[1590] Specifically, a notification will appear in the app, and when the user taps on the notification, a detailed report will be displayed, including information on study time, concentration level, and emotional changes.

[1591] (Application example 2)

[1592] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1593] Conventional educational apps have difficulty recognizing children's reactions and emotions in real time and optimizing learning content based on that. Furthermore, there are few ways to provide a learning experience tailored to each individual child in physical stores. This makes it difficult to achieve learning results and stimulate children's interest.

[1594] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for inputting user information, a server for receiving and storing the user information, a means for selecting and providing content based on the user information, a means for recording the child's reactions and operations in real time, a means for identifying the child's emotions from the real-time data using an emotion recognition model, a means for analyzing the reaction and operation data and determining the next content, a means for generating a report of learning results and progress based on the analysis results and emotion data, and a tablet terminal for making the system available in the kids' space of a physical store. This makes it possible to provide optimal learning content based on the child's emotions and reactions and improve the learning experience in real time.

[1595] "Means for inputting user information" refers to an interface that allows a parent or user to input information about a child (such as name, age, areas of interest, special instructions, etc.).

[1596] "Server" refers to a computing device that receives and stores user information and analyzes data collected in real time.

[1597] "Means for selecting and providing content" refers to a system that selects and displays learning content appropriate for children based on user information and collected real-time data.

[1598] "Means for recording children's reactions and operations in real time" refers to the function of using cameras and sensors to collect children's facial expressions, gaze, frequency of operations, reaction speed, etc. in real time.

[1599] "Means for identifying children's emotions from the real-time data using an emotion recognition model" refers to a function that uses an AI model to analyze children's emotions such as joy, excitement, and concentration based on the collected data.

[1600] "Means for analyzing the reaction and operation data and determining the next content" refers to the function of analyzing the collected real-time data and emotional data and determining the next optimal content to be provided based on that data.

[1601] "Means for generating reports on learning results and progress" refers to the function of creating a report summarizing the child's learning results and progress based on the analyzed data and providing it to parents.

[1602] "Tablet terminal for making the system available in the kids' space of a physical store" refers to a tablet device installed to make the system available in the kids' space of a physical store.

[1603] The present invention relates to an AI educational app that provides optimal learning content based on user information and real-time collected data on children's reactions. The system combines an emotion engine to recognize children's emotions and optimize their learning experience. Specific embodiments of the present invention are described below.

[1604] First, the user (parent) enters information about their child using a tablet device installed in the kids' area of ​​the physical store. The user information includes the child's name, age, areas of interest, special attention, desired learning content category, and learning period. This information is sent from the tablet device to the server and stored.

[1605] Next, the server selects the most appropriate initial content based on the user information sent and sends it to the tablet device. For example, a quiz game about animals might be selected. As a child plays the quiz game using the tablet device, the device's built-in camera and sensors collect data in real time, such as the child's facial expressions, line of sight, operation frequency, and reaction speed. This data is sent to the server at regular intervals.

[1606] The emotion recognition model uses collected real-time data to identify a child's emotions. This emotion data is also sent to the server. The server then uses an AI algorithm to analyze the received real-time data and emotion data, and performs an analytical process to evaluate the child's level of interest and concentration. Based on the results, it determines the most appropriate content to provide next. For example, if a child shows interest in an animal quiz, a video introducing new animals or a more difficult quiz will be selected.

[1607] The tablet device then displays new content to the child, and the learning or play experience continues. The device continues to collect real-time data and emotional data and transmits it to the server. After the learning session ends, the server generates a detailed report based on all data (such as learning performance, concentration level, and emotional analysis results) and notifies the user. This allows the user to understand the child's learning progress and emotional changes and plan the next lesson.

[1608] For example, a parent might enter information about a six-year-old child on a tablet device and indicate that the child is interested in "animals." In this case, the server selects the initial content of an animal quiz and sends it to the tablet device. While the child is enjoying the quiz, the built-in camera analyzes their facial expressions in real time, and if they smile a lot, the server will present them with a new animal quiz or video.

[1609] Below are some example prompts to input to a generative AI model:

[1610] A 6-year-old boy named Taro is interested in animal quizzes. Write code to analyze Taro's reactions in real time using a facial expression recognition system, and if he seems to be enjoying it, determine the next best learning content to provide him. Also, please provide detailed information about the process of sending and receiving data to the server.

[1611] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1612] Step 1:

[1613] User input:

[1614] Users (parents) use tablet devices installed in the kids' areas of physical stores to enter their child's information (name, age, areas of interest, special attention, desired learning content category, and learning period). The device then sends this information to the server, which then stores the user profile in a database.

[1615] Input: Child's name, age, areas of interest, special notes, desired learning content category and learning period

[1616] Output: User information sent to server, saved user profile

[1617] Step 2:

[1618] Initial content provided:

[1619] Based on the received user information, the server selects the most appropriate initial content (for example, a quiz game about animals) and sends it to the tablet device, which then displays this content to the child.

[1620] Input: User profile stored on the server

[1621] Output: Initial content

[1622] Step 3:

[1623] Real-time data collection:

[1624] When a child uses a tablet device to engage with learning content, the device's built-in camera and sensors collect real-time data on the child's facial expressions, gaze, frequency of operations, reaction speed, etc. This data is then sent to a server at regular intervals.

[1625] Input: Real-time reaction data of children (facial expressions, gaze, frequency of operation, reaction speed)

[1626] Output: Real-time data sent

[1627] Step 4:

[1628] Emotion Recognition with Emotion Engine:

[1629] The server inputs the received real-time data into an emotion recognition model to identify the child's emotions (joy, excitement, concentration, etc.), and this emotion data is also stored on the server.

[1630] Input: Real-time data

[1631] Output: Identified emotion data

[1632] Step 5:

[1633] Data analysis:

[1634] The server uses AI algorithms to analyze real-time data and emotional data to assess the child's level of interest and concentration, and then determines the next most appropriate content to provide.

[1635] Input: Real-time data, emotion data

[1636] Output: Analysis results, next best content to provide

[1637] Step 6:

[1638] Providing the following content:

[1639] The server determines the next content based on the analysis results and sends it to the tablet device (for example, a video introducing a new animal or a high-level quiz). The device then displays this new content to the child.

[1640] Input: The next best thing to serve

[1641] Output: New content

[1642] Step 7:

[1643] End the session and generate a report:

[1644] After the user-specified study time has elapsed, the tablet device ends the study session. The device sends all data (such as study performance, concentration level, and emotion analysis results) to the server, which then generates a detailed report and notifies the user.

[1645] Input: Learning performance, concentration level, emotion analysis results

[1646] Output: Detailed report

[1647] As a result, by using this system, it is possible to draw out children's learning effectiveness and interest, and provide them with optimal learning content in real time.

[1648] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1649] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1650] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1651] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1652] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1653] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1654] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1655] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1656] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1657] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1658] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1659] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1660] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1661] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1662] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1663] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1664] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1665] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1666] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1667] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1668] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1669] The following is further disclosed regarding the above embodiment.

[1670] (Claim 1)

[1671] a means for inputting user information;

[1672] a server that receives and stores the user information;

[1673] means for selecting and providing content based on the user information;

[1674] a means for recording the child's reactions and actions in real time;

[1675] means for analyzing the reaction and operation data and determining next content;

[1676] means for generating a report of learning results and progress based on the analysis results;

[1677] A system including:

[1678] (Claim 2)

[1679] The system of claim 1, wherein the user information includes the child's age, areas of interest, learning content, and usage time.

[1680] (Claim 3)

[1681] The system of claim 1, wherein the data recorded in real time includes the child's facial expression, gaze, operation frequency, and reaction speed.

[1682] "Example 1"

[1683] (Claim 1)

[1684] a means for inputting user information;

[1685] a server that receives and stores the user information;

[1686] means for selecting and providing content based on the user information;

[1687] a means for recording the child's reactions and actions in real time;

[1688] A means for analyzing the reaction and operation data and using an AI algorithm to determine next content;

[1689] means for generating and providing a report of learning results and progress based on the analysis results;

[1690] A system including:

[1691] (Claim 2)

[1692] The system of claim 1, wherein the user information includes the child's name, age, areas of interest, special attention, learning content, and usage time.

[1693] (Claim 3)

[1694] The system of claim 1, wherein the data recorded in real time includes the child's facial expression, gaze, operation frequency, and reaction speed.

[1695] "Application Example 1"

[1696] (Claim 1)

[1697] a means for inputting user information;

[1698] a server that receives and stores the user information;

[1699] means for selecting and providing content based on the user information;

[1700] a means for recording user reactions and operations in real time;

[1701] means for analyzing the reaction and operation data and determining next content;

[1702] means for generating a report of learning results and progress based on the analysis results;

[1703] A means for providing the content using a robot installed in a physical store and recording the reactions and operations;

[1704] an input device for a user to input the information at a physical store;

[1705] A system including:

[1706] (Claim 2)

[1707] The system of claim 1, wherein the user information includes the child's age, areas of interest, learning content, and usage time.

[1708] (Claim 3)

[1709] The system of claim 1, wherein the data recorded in real time includes the user's facial expression, gaze, operation frequency, and reaction speed.

[1710] "Example 2: Combining Emotion Engines"

[1711] (Claim 1)

[1712] a means for a user to input information about a child on an initial setup screen;

[1713] means for receiving the child's information and transmitting it to a server;

[1714] means for selecting initial learning content based on the received information and transmitting the content to the terminal;

[1715] A means of collecting data on facial expressions, gaze, operation frequency and reaction speed when children engage with learning content;

[1716] means for transmitting the collected data to a server and recognizing emotions with an emotion engine;

[1717] A means for analyzing the recognized emotion data and real-time data using an AI algorithm to evaluate the interest and concentration of the child;

[1718] a means for determining the next most suitable learning content to be provided based on the evaluation results and transmitting the determined content to the terminal;

[1719] means for generating a report of learning results and emotional changes based on all data after the session is completed;

[1720] means for notifying a user of the generated report;

[1721] A system including:

[1722] (Claim 2)

[1723] 2. The system of claim 1, wherein the child's information includes age, area of ​​interest, learning content, and learning period.

[1724] (Claim 3)

[1725] The system of claim 1, wherein the collected data includes the child's facial expression, gaze, operation frequency, and reaction speed.

[1726] "Application example 2 when combining emotion engines"

[1727] (Claim 1)

[1728] a means for inputting user information;

[1729] a server that receives and stores the user information;

[1730] means for selecting and providing content based on the user information;

[1731] a means for recording the child's reactions and actions in real time;

[1732] means for analyzing the reaction and operation data and determining next content;

[1733] means for identifying a child's emotion from the real-time data using an emotion recognition model;

[1734] means for generating a report of learning results and progress based on the analysis results and emotion data;

[1735] A system including a tablet device that enables the system to be used in the kids' space of a physical store.

[1736] (Claim 2)

[1737] The system of claim 1, wherein the user information includes the child's age, areas of interest, learning content, and usage time.

[1738] (Claim 3)

[1739] The system of claim 1, wherein the data recorded in real time includes the child's facial expression, gaze, operation frequency, and reaction speed. [Explanation of symbols]

[1740] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for inputting user information; a server that receives and stores the user information; means for selecting and providing content based on the user information; a means for recording the child's reactions and actions in real time; means for analyzing the reaction and operation data and determining next content; means for generating a report of learning results and progress based on the analysis results; A system including:

2. The system according to claim 1, wherein the user information includes the child's age, field of interest, learning content, and usage time.

3. The system of claim 1 , wherein the data recorded in real time includes the child's facial expression, gaze, operation frequency, and reaction speed.

Images