system

Abstract

Provide a system. A means for inputting and saving a child's goals and progress data; A means of building and updating generative models based on the collected data; a means for generating feedback based on the constructed generative model; a means of providing generated feedback to parents and teachers; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the traditional education system, it was difficult for children to visualize their future and visualize the steps to achieve it, making it difficult for them to maintain motivation for learning and activities. It was also difficult for parents and teachers to accurately grasp a child's progress and aptitude and provide appropriate support. This placed a heavy burden on children's education, making it difficult for them to maximize their potential. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means. First, it provides a means for inputting and saving a child's goals and progress data, allowing the child to clarify his or her goals and regularly record the degree of achievement. Next, it provides a means for constructing and updating a generative model based on the collected data, and the generated model is used to analyze the current situation and provide feedback on goal achievement. Furthermore, it presents a visual progress status to the child based on the generated feedback, improving motivation for learning and activities. An interface is provided so that parents and teachers can also check this feedback, and a system is built that can provide appropriate support after understanding the child's aptitude and progress.

[0006] "Goals" refer to specific objectives such as the occupation or skills that the child user wants to achieve in the future.

[0007] "Progress Data" refers to information about the results and grades a child achieves in their daily learning and activity records.

[0008] A "generative model" is an artificial intelligence model that makes future predictions and analyses based on collected user goals and progress data.

[0009] "Feedback" refers to information that provides the user with advice on their current progress and next steps based on the results of analysis by the generative model.

[0010] "Parents and teachers" refers to adults who review the feedback provided by the system and provide appropriate advice and guidance to support a child's learning and development. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0012] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0013] First, the terms used in the following description will be explained.

[0014] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0015] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0016] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0017] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0019] [First embodiment]

[0020] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0021] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0024] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0027] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0031] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0032] The present invention can be implemented in the following manner.

[0033] System Overview

[0034] This invention provides a learning support system that allows children to visually grasp their goals and visualize their progress through feedback using a generative model. This system aims to improve children's motivation for learning and activities, with children, parents, and teachers each playing their respective roles. The system is primarily composed of a server and a terminal (tablet).

[0035] Server-side program

[0036] The server performs the following process:

[0037] 1. Managing your user profile

[0038] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[0039] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[0040] 2. Receiving progress data

[0041] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0042] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0043] 3. Building and updating the generative model

[0044] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[0045] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0046] 4. Generate feedback

[0047] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0048] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0049] 5. Storing and Providing Feedback

[0050] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0051] Example: Parents and teachers can view children's feedback through a web interface.

[0052] Terminal (tablet) side program

[0053] The terminal performs the following process:

[0054] 1. Data Entry

[0055] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0056] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0057] 2. Sending progress data

[0058] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0059] Example: When a child enters the test results, the device immediately sends the data to the server.

[0060] 3. Collaboration with generative models

[0061] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0062] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0063] 4. Viewing Feedback

[0064] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0065] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0066] 5. Providing information to parents and teachers

[0067] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0068] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0069] User (child, parent, teacher) usage

[0070] The user uses the system as follows:

[0071] 1. Use by Children

[0072] Children enter their daily goal achievements into a tablet and check their progress data.

[0073] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[0074] 2. Use by parents and teachers

[0075] Parents and teachers use a tablet or web interface to view a child's progress data and the feedback provided by the generative model.

[0076] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[0077] The above is an embodiment of the present invention.

[0078] The processing flow will be explained below.

[0079] Server-side program processing steps

[0080] Step 1: Manage your user profile

[0081] When a user registers, the server stores the user's profile information (name, age, grade, goals) in a database.

[0082] Example: If a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this information in a database.

[0083] Step 2: Receiving progress data

[0084] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0085] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0086] Step 3: Building and updating the generative model

[0087] The server builds a generative model based on the collected data and updates it as new progress data is received.

[0088] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0089] Step 4: Generate feedback

[0090] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0091] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0092] Step 5: Save and provide feedback

[0093] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0094] Example: Parents and teachers can view children's feedback through a web interface.

[0095] Processing steps of the terminal (tablet) program

[0096] Step 1: Data entry

[0097] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0098] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0099] Step 2: Sending progress data

[0100] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0101] Example: When a child enters the test results, the device immediately sends the data to the server.

[0102] Step 3: Working with the generative model

[0103] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0104] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0105] Step 4: View your feedback

[0106] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0107] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0108] Step 5: Inform parents and teachers

[0109] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0110] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0111] User (child, parent, teacher) usage processing steps

[0112] Step 1: Child access

[0113] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback.

[0114] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[0115] Step 2: Use by parents and teachers

[0116] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and the feedback provided by the generative model.

[0117] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[0118] Example 1

[0119] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] In today's educational environment, children lack the means to visually grasp their goals and effectively manage their progress. It is also difficult for parents and teachers to grasp children's learning progress in real time and provide appropriate feedback. This creates a problem of lowering children's motivation to learn.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0122] In this invention, the server includes means for inputting and saving a child's goal and progress information, means for receiving and saving progress data sent from the terminal, means for building and updating a generative AI model based on the collected data, means for generating individual feedback messages based on the generative AI model, and means for providing the generated feedback to parents and teachers. This allows parents and teachers to visually grasp the child's goal achievement status, check the progress in real time, and provide appropriate feedback.

[0123] "Goals" are specific outcomes or targets for learning or activities that a child wants to achieve.

[0124] "Progress information" refers to the process and results of a child's learning and activities toward a goal.

[0125] A "storage means" is a method or device for permanently recording data and making it available for later reference.

[0126] "Means for receiving" refers to a method or device for receiving information or data sent from outside.

[0127] A "generative AI model" is an algorithm that learns from collected data and makes future predictions and analyses.

[0128] "Individual feedback messages" are messages that provide specific advice or next steps based on the user's progress and goals.

[0129] "Means for providing" refers to a method or device for delivering the generated information or message to users or related parties.

[0130] A "visual display means" is a method or apparatus for displaying data or information on a screen or device in a manner that is easy for a user to understand.

[0131] A "Web interface" is a collection of screens and input devices that allow users to access and operate a system via the Internet.

[0132] This invention provides a learning support system that allows children to visually grasp their goals and visualize their progress through feedback generated by a generative AI model. The system is primarily composed of a server and a terminal (tablet).

[0133] Server-side program

[0134] The server performs the following process.

[0135] 1. Managing your user profile

[0136] When a user registers, the server stores their profile information, such as their name, age, grade, and goals, in a database. This information serves as the basic data for building a generative model.

[0137] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[0138] 2. Receiving progress data

[0139] The server periodically receives progress data (e.g., test scores and assignment completion rates) sent from the device and stores it in a database.

[0140] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0141] 3. Building and updating the generative model

[0142] The server builds a generative AI model based on the collected data, and when new progress data is received, it updates the generative model, enabling more accurate predictions.

[0143] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0144] 4. Generate feedback

[0145] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0146] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0147] 5. Storing and Providing Feedback

[0148] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0149] Example: Parents and teachers can view children's feedback through a web interface.

[0150] Terminal (tablet) side program

[0151] The terminal performs the following process:

[0152] 1. Data Entry

[0153] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0154] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0155] 2. Sending progress data

[0156] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0157] Example: When a child enters the test results, the device immediately sends the data to the server.

[0158] 3. Collaboration with generative models

[0159] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0160] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0161] 4. Viewing Feedback

[0162] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0163] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0164] 5. Providing information to parents and teachers

[0165] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0166] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0167] User (child, parent, teacher) usage

[0168] The user uses the system as follows:

[0169] 1. Use by Children

[0170] Children enter their daily goal achievements into a tablet and check their progress data.

[0171] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[0172] 2. Use by parents and teachers

[0173] Parents and teachers use a tablet or web interface to view a child's progress data and the feedback provided by the generative model.

[0174] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[0175] Specific examples

[0176] A user registers "Yamada Taro" in the system, stating that "my goal is to become a scientist." One week later, Yamada Taro enters "Test results: 90 points in math, 85 points in English, 92 points in science" into his device. The device immediately sends the data to the server. The server receives the new data and stores it in a database. The server updates the generative model, analysing that "to achieve the next goal, you need to improve your English grade by 5 points," and stores this feedback in the database. The device receives the feedback from the server and displays on the dashboard, "Your current science grade is 92 points. Your next goal is to improve your English grade to 90 points." Parents can view all the feedback using a web interface.

[0177] Prompt Sentence Examples

[0178] "Register a new user profile. Enter your name, age, grade, and goals."

[0179] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0180] Step 1:

[0181] Register a new user profile

[0182] Input: The user enters their name, age, grade, and goal.

[0183] Action: The user enters the required information into the web form (e.g., "Name: Yamada Taro, Age: 12, Grade: 1st year of junior high school, Goal: Scientist") and presses the submit button.

[0184] Data processing: The server receives the entered data, checks the format, and then saves it in the user table in the database.

[0185] Output: The profile information is saved and a success message is displayed to the user.

[0186] Step 2:

[0187] Entering and submitting progress data

[0188] Input: Children input their daily progress data, such as test results, into the tablet.

[0189] Action: The child enters their test results in the input field: "Math: 90 points, English: 85 points, Science: 92 points" and presses the save button.

[0190] Data processing: The device saves the input data in local storage and immediately sends it to the server.

[0191] Output: Progress data is sent to the server and saved to local storage.

[0192] Step 3:

[0193] Receiving and saving progress data

[0194] Input: Progress data sent from the device.

[0195] Operation: The server receives HTTP requests from the terminal, interprets the data, and inserts it into a progress table in the database.

[0196] Data processing: Convert the received progress data and add it to the database.

[0197] Output: The progress data is saved in the database and a success response is returned to the device.

[0198] Step 4:

[0199] Updating the generative model

[0200] Input: New progress data stored in the database.

[0201] How it works: The server starts the learning process for the generative AI model based on new progress data. It retrains the existing model with the latest data.

[0202] Data processing: The generative AI model learns from all collected data to improve prediction accuracy.

[0203] Output: The updated generative model is saved and used for the next feedback generation.

[0204] Step 5:

[0205] Generate feedback

[0206] Input: Updated generative model and new progress data.

[0207] How it works: The server uses the generative model to analyze the progress data and create personalized feedback messages.

[0208] Data processing: A generative AI model compares the user's progress with their goals and calculates the necessary advice (e.g., "You need to improve your English grade by 5 points to achieve your next goal").

[0209] Output: The generated feedback messages are stored in a database.

[0210] Step 6:

[0211] Receiving and viewing feedback

[0212] Input: The feedback message sent by the server.

[0213] Operation: The device receives the feedback sent from the server, stores it in local storage, and visually displays it to the user.

[0214] Data processing: Visualize the received feedback in the form of a dashboard on the device.

[0215] Output: A message like "Your current science grade is 92. Your next goal is to get your English grade to 90." will be displayed.

[0216] Step 7:

[0217] Providing information to parents and teachers

[0218] Input: A request for parents and teachers to review feedback.

[0219] How it works: A parent or teacher submits a request to review information via a web interface or tablet.

[0220] Data processing: The server receives the request, retrieves the relevant feedback information from the database, and displays it.

[0221] Output: Parents and teachers can view progress and feedback information in a dashboard format.

[0222] (Application example 1)

[0223] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] Conventional shopping support systems have difficulty monitoring users' purchasing behavior in real time and providing individualized feedback. This has resulted in insufficient support for users to achieve their shopping goals. It has also made it difficult for parents and teachers to effectively monitor their children's learning progress and provide appropriate advice. In light of these circumstances, there is a need for a more comprehensive system that can help users achieve both their purchasing and learning goals.

[0225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0226] In this invention, the server includes means for inputting and saving a child's goals and progress data, means for constructing and updating a generative model based on the collected data, means for generating feedback based on the constructed generative model, means for providing the generated feedback to parents and teachers, means for inputting and saving a user's purchasing goals and purchasing data, means for constructing and updating a generative model based on the collected purchasing data, means for generating purchasing feedback based on the constructed generative model, and means for providing the generated feedback to the user. This enables real-time support for users to achieve their shopping goals, and allows parents and teachers to properly understand their children's learning progress and provide appropriate advice.

[0227] "Progress data" is data that indicates the progress of the goals set by the user.

[0228] A "generative model" is a model built based on collected data to predict and analyze user behavior and progress.

[0229] "Feedback" refers to advice or information provided to users, parents, or teachers based on the generative model.

[0230] "Tools provided to parents and teachers" refers to functions that allow parents and teachers to check information about their children's learning progress and goal achievement.

[0231] "Purchase Goals" refers to specific shopping goals or plans set by a user.

[0232] "Purchase data" refers to data such as information about the products actually purchased by the user, the amount, and the date and time of purchase.

[0233] "Purchase feedback" refers to advice and information provided to users regarding their purchasing behavior based on generative models.

[0234] This invention is a system that helps users set shopping goals and track their progress in real time to help them achieve them. Using a server and a device (smartphone), the system collects user purchasing data and provides feedback using a generative AI model.

[0235] The server performs the following processes: When a user registers, their purchasing goals and personal information are saved in a database. This will later become the basic data for building a generative model. For example, if a user enters "keep my monthly shopping budget under 30,000 yen," the server records this in the database.

[0236] Next, the purchase data entered on the terminal (purchase date and time, product name, price, etc.) is periodically sent to the server. For example, if a user enters data such as "Purchase a T-shirt for 2,500 yen," the server receives this and stores it in a database.

[0237] The server builds and updates a generative AI model based on the collected data. This model is built using Tensorflow (registered trademark) and is updated each time new purchase data is entered. The model predicts the user's progress based on the latest data and provides specific feedback for achieving the next goal. For example, feedback such as "To achieve next month's budget, be careful not to exceed 10,000 yen remaining this month" may be generated.

[0238] The generated feedback is stored in a database and provided to the user. In addition to receiving feedback via smartphone, users also have access to a dashboard that allows them to visually check their purchasing goals and progress. The screen displays the remaining amount of a pre-set budget and a list of recently purchased items.

[0239] The generated feedback may also be provided in a form that can be viewed by parents, teachers, and other relevant parties. In this case, parents and teachers can view the feedback through a web interface and provide appropriate advice to the user.

[0240] For example, if a user sets a monthly shopping budget of ¥30,000, the app collects purchasing data, visually displays progress, and the generative AI model provides appropriate advice, such as, "Your remaining budget for this month is ¥10,000. We recommend you be careful with your next purchase."

[0241] An example of a prompt to be input to a generative AI model could be, "Based on the user's current purchasing data, estimate the next purchase budget and generate feedback."

[0242] This allows users to efficiently manage their purchasing goals, check their progress in real time, and support them in achieving their goals. Parents and teachers can also keep track of their children's learning progress and provide appropriate advice.

[0243] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0244] Step 1:

[0245] The user enters their shopping goals and personal information.

[0246] Input: Shopping goal set by the user (e.g., less than 30,000 yen per month) and personal information (e.g., name, age, budget)

[0247] Output: User profile information stored in the database

[0248] Specific operation: The user enters their goals and personal information into the smartphone application, which is then sent to the server and recorded in a database.

[0249] Step 2:

[0250] The terminal inputs the purchase data.

[0251] Input: Information about the product purchased by the user (e.g., T-shirt, 2,500 yen, purchase date and time, etc.)

[0252] Output: Purchasing data stored in a database

[0253] Specific operation: When a user enters information about a purchased item into the terminal, this information is sent to the server and stored in a database.

[0254] Step 3:

[0255] The server collects purchasing data and builds and updates the generative AI model.

[0256] Input: Purchasing data stored in the database

[0257] Output: An updated generative AI model

[0258] How it works: The server periodically collects purchasing data from a database and builds and updates a generative AI model using TensorFlow, etc. The accuracy of the model improves with each new piece of data.

[0259] Step 4:

[0260] The server generates feedback based on the generative AI model.

[0261] Input: Updated generative AI model and the user's latest purchase data

[0262] Output: The generated feedback message

[0263] Specific operation: The server uses the generated AI model to analyze the user's purchasing behavior and generate feedback such as, "To achieve next month's budget, you need to be careful not to exceed 10,000 yen remaining."

[0264] Step 5:

[0265] The server generates feedback, stores it in a database, and provides it to the terminal.

[0266] Input: The generated feedback message

[0267] Output: Feedback displayed on the user's smartphone

[0268] Specific operation: The server stores the generated feedback in a database and sends it to the user's device for display. For example, the user's application screen might display "This month's remaining budget is 10,000 yen."

[0269] Step 6:

[0270] The device displays visual feedback to the user.

[0271] Input: Feedback message sent by the server

[0272] Output: Feedback information displayed on the dashboard

[0273] What it does: The device displays the feedback it receives in real time in a dashboard format. By opening the app, users can check their current remaining budget, recent purchase history, and more at a glance.

[0274] Step 7:

[0275] Access an interface where parents and teachers can review feedback.

[0276] Input: Parent or teacher request

[0277] Output: Child's purchasing feedback displayed in the web interface

[0278] Specific operation: When a parent or teacher sends a request to check feedback information through a web interface, the server provides the feedback data, allowing the parent or teacher to check their child's purchasing status and goal achievement status.

[0279] These steps enable users to receive real-time support to efficiently achieve their purchasing goals, while parents and teachers can effectively track learning progress and provide appropriate advice.

[0280] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0281] The present invention can be implemented in the following manner.

[0282] System Overview

[0283] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback from a generative model. It also combines an emotion engine that recognizes the user's emotions to further increase children's motivation. The system aims to increase children's motivation for learning and activities, with the users (children, parents, and teachers) participating in their respective roles. The system primarily consists of a server, a terminal (tablet), and an emotion engine.

[0284] Server-side program

[0285] The server performs the following process:

[0286] 1. Managing your user profile

[0287] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[0288] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[0289] 2. Receiving progress data

[0290] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0291] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0292] 3. Building and updating the generative model

[0293] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[0294] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0295] 4. Generate feedback

[0296] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0297] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0298] 5. Storing and Providing Feedback

[0299] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0300] Example: Parents and teachers can view children's feedback through a web interface.

[0301] Terminal (tablet) side program

[0302] The terminal performs the following process:

[0303] 1. Data Entry

[0304] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0305] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0306] 2. Sending progress data

[0307] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0308] Example: When a child enters the test results, the device immediately sends the data to the server.

[0309] 3. Collaboration with generative models

[0310] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0311] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0312] 4. Viewing Feedback

[0313] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0314] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0315] 5. Providing information to parents and teachers

[0316] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0317] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0318] Emotion Engine

[0319] The emotion engine performs the following processing:

[0320] 1. User Emotion Recognition

[0321] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[0322] Example: Using a camera while a child is learning, the system can recognize emotional states such as "happy" or "tired" from their facial expressions.

[0323] 2. Sending Emotional Data

[0324] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[0325] Example: If the emotion engine recognizes the emotion "fun," it sends that data to the server, which then generates an encouraging message in the next feedback, saying, "You're doing your best, so keep it up next time!"

[0326] 3. Providing emotional information to parents and teachers

[0327] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[0328] Example: A parent learns through an emotion interface that their child is perceived as "tired" and advises them to moderate their learning for the day.

[0329] User (child, parent, teacher) usage

[0330] The user uses the system as follows:

[0331] 1. Use by Children

[0332] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback. At the same time, the emotion engine analyzes their facial expressions and voice to understand their emotional state.

[0333] Example: A child enters "I will complete all my math homework today" and sees the result in feedback. If the emotion engine recognizes that they are "tired," that data will also be reflected in the feedback.

[0334] 2. Use by parents and teachers

[0335] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and feedback provided by the generative model, as well as the emotional state recognized by the emotion engine.

[0336] Example: A parent may discover that their child has a strong aptitude for science and provide specific advice to further support them. Then, through the emotion engine, they may learn that their child is enjoying the activity and encourage them to continue learning at this rate.

[0337] The above is an embodiment of the present invention.

[0338] The processing flow will be explained below.

[0339] Server-side program processing steps

[0340] Step 1: Manage your user profile

[0341] When a user registers, the server stores the user's profile information (name, age, grade, goals) in a database.

[0342] Example: If a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this information in a database.

[0343] Step 2: Receiving progress data

[0344] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0345] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0346] Step 3: Building and updating the generative model

[0347] The server builds a generative model based on the collected data and updates it as new progress data is received.

[0348] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0349] Step 4: Generate feedback

[0350] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0351] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0352] Step 5: Save and provide feedback

[0353] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0354] Example: Parents and teachers can view children's feedback through a web interface.

[0355] Processing steps of the terminal (tablet) program

[0356] Step 1: Data entry

[0357] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0358] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0359] Step 2: Sending progress data

[0360] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0361] Example: When a child enters the test results, the device immediately sends the data to the server.

[0362] Step 3: Working with the generative model

[0363] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0364] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0365] Step 4: View your feedback

[0366] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0367] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0368] Step 5: Inform parents and teachers

[0369] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0370] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0371] Emotion Engine Processing Steps

[0372] Step 1: Recognizing user emotions

[0373] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[0374] Example: Using a camera while a child is learning, the system can recognize emotional states such as "happy" or "tired" from their facial expressions.

[0375] Step 2: Sending emotion data

[0376] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[0377] Example: If the emotion engine recognizes the emotion "fun," it sends that data to the server, which then generates an encouraging message in the next feedback, saying, "You're doing your best, so keep it up next time!"

[0378] Step 3: Providing emotional information to parents and teachers

[0379] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[0380] Example: A parent learns through an emotion interface that their child is perceived as "tired" and advises them to moderate their learning for the day.

[0381] User (child, parent, teacher) processing steps

[0382] Step 1: Child access

[0383] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback. At the same time, the emotion engine analyzes their facial expressions and voice to understand their emotional state.

[0384] Example: A child enters "I will complete all my math homework today" and sees the result in feedback. If the emotion engine recognizes that they are "tired," that data will also be reflected in the feedback.

[0385] Step 2: Use by parents and teachers

[0386] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and feedback provided by the generative model, as well as the emotional state recognized by the emotion engine.

[0387] Example: A parent may discover that their child has a strong aptitude for science and provide specific advice to further support them. Then, through the emotion engine, they may learn that their child is enjoying the activity and encourage them to continue learning at this rate.

[0388] Example 2

[0389] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0390] In today's educational environment, it is important for children to set goals independently and manage their progress toward them in order to improve their learning outcomes. However, it is difficult for children to accurately grasp their own progress and decide on their next actions based on that. Furthermore, there are few ways for parents and teachers to grasp children's learning situation and emotional state in real time, making it difficult to provide appropriate feedback and support. Furthermore, a lack of feedback that takes emotional changes into account can lead to a decline in children's motivation. There is a need for a system that can solve these issues and effectively promote learning while maintaining children's motivation.

[0391] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0392] In this invention, the server includes: means for inputting and saving a child's goals and progress data; means for constructing and updating a generative model based on the collected data; means for generating feedback based on the constructed generative model; means for providing the generated feedback to parents and teachers; means for recognizing and digitizing the user's emotions; means for transmitting the recognized emotional data to the server; and means for providing an interface through which parents and teachers can check the feedback. This facilitates children's goal setting and progress management, and enables children to maintain their learning motivation through accurate feedback using the generative AI model. Furthermore, feedback that takes emotional data into account allows parents and teachers to grasp a child's learning situation and emotional state in real time and provide appropriate support.

[0393] "Goals" refer to the specific objectives of learning or activities that a child is trying to achieve.

[0394] "Progress data" refers to information that shows a child's achievement level and learning results toward the goals they have set.

[0395] "Generative model" refers to a machine learning model that is built and updated based on collected progress data.

[0396] "Feedback" refers to messages based on generative models that provide advice and evaluation of the user's progress and next goals.

[0397] An "emotion engine" is a system that analyzes information obtained from the device's sensors and camera, and recognizes and digitizes the user's emotional state.

[0398] "User" refers to all users of this system, including children, parents, and teachers.

[0399] "Interface" refers to the GUI (graphical user interface) or web interface that allows users to interact with the system.

[0400] MODE FOR CARRYING OUT THE INVENTION

[0401] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback from a generative model. It also combines an emotion engine that recognizes the user's emotions to further increase children's motivation. This system involves users (children, parents, and teachers) in their respective roles, aiming to increase children's motivation for learning and activities.

[0402] System Configuration

[0403] The system mainly consists of a server, a terminal (tablet), and an emotion engine. The specific roles of each component are as follows:

[0404] server

[0405] The server manages the child's goals and progress data, and generates and provides feedback using a generative AI model. The server manages user profiles, receives progress data, builds and updates generative models, generates feedback, and stores and provides the feedback. The specific hardware used is a general-purpose server machine, and the software used is Python, MySQL (registered trademark), TensorFlow, and Flask.

[0406] User profile management: The server stores profile information in a MySQL database when a new user registers.

[0407] Receiving progress data: The server periodically receives progress data sent from the device and stores it in a MySQL database.

[0408] Building and updating generative models: The server builds and updates generative AI models using TensorFlow based on the collected data.

[0409] Feedback generation: The server uses the generative AI model to generate feedback messages and save them in JSON format.

[0410] Storing and serving feedback: The server stores the generated feedback in a MySQL database and serves it in a web interface via Flask.

[0411] Device (tablet)

[0412] The terminal is a device that allows users to input goals and progress data and display feedback. Specifically, we will use a tablet terminal and develop applications for ANDROID (registered trademark) and iOS.

[0413] Data Entry: The device stores user-entered goals and daily progress information in local storage. For example, if a child enters "my goal for today is to complete all my math homework," that information is stored in the device's SQLite database.

[0414] Sending progress data: The device automatically sends progress data to the server at regular intervals. For example, when test results are entered, the device sends the data to the server.

[0415] Interaction with generative models: The device receives feedback from the server and stores it in local storage. For example, if a message is received saying, "Your next goal is to get 90 points in math," it will be stored on the device.

[0416] Displaying feedback: The device visually displays the received feedback to the user, for example, "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0417] Providing information to parents and teachers: The device will display appropriate feedback information when requested by a parent or teacher.

[0418] Emotion Engine

[0419] The emotion engine is responsible for recognizing and digitizing the user's emotions. Specifically, it uses the device's camera and microphone to analyze the user's facial expressions and tone of voice, and uses software modules (such as OpenCV and TensorFlow) to recognize the user's emotional state.

[0420] User emotion recognition: The emotion engine analyzes the user's facial expressions and tone of voice to recognize their current emotional state. For example, a camera can recognize a child's facial expression while they are studying and analyze their emotions, such as "happy" or "tired."

[0421] Emotion data transmission: The emotion engine transmits the recognized emotion data to the server via the device, and the server integrates this data into the generative AI model. For example, if the emotion "fun" is recognized, the data is transmitted to the server and reflected in the feedback.

[0422] Providing emotional information to parents and teachers: The emotional state recognized by the emotion engine is also displayed in an interface used by parents and teachers. Parents can see through the interface that their child is recognized as tired and can advise them to moderate their studying for the day.

[0423] Examples and prompts

[0424] Examples of use for children:

[0425] Daily goal achievement and progress are entered into the tablet, and the generated feedback is then confirmed. At the same time, the emotion engine analyzes facial expressions and voice to understand the child's emotional state. A child may enter "I will complete all my math homework today," and the result is confirmed in the feedback. If the emotion engine recognizes that the child is "tired," this data is also reflected in the feedback.

[0426] Examples of parent and teacher use cases:

[0427] Using a tablet or web interface, parents can view their child's progress data, feedback provided by the generative model, and the emotional state recognized by the emotion engine. The parent can see that their child has a strong aptitude for science and provide specific advice to further support them. The emotion engine can also tell them that their child is having fun and encourage them to continue learning at this pace.

[0428] Example prompt:

[0429] Child prompt: "What is your goal for today (e.g., complete all math assignments)?"

[0430] Child prompt: "Enter your test scores (e.g., Math: 90, English: 85)."

[0431] Parent prompt: "Do you want to check in on your child's progress? (Yes / No)"

[0432] Parent prompt: "Would you like to check in with your child's emotional state? (Yes / No)"

[0433] Teacher prompt: "Would you like to review student progress data? (Yes / No)"

[0434] Teacher prompt: "Would you like to review student feedback? (Yes / No)"

[0435] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0436] Server-side program

[0437] Step 1: Manage your user profile

[0438] Specific operation: The server receives profile information (name, age, grade, goals) submitted through the new user registration form and stores it in an SQL database (e.g., MySQL).

[0439] Input: New user registration information (e.g. name, age, grade, goal)

[0440] Data processing: Analyzes the received information and executes INSERT queries to the database to store the information.

[0441] Output: User information saved in the database

[0442] Step 1:

[0443] The server saves the profile information received from the new user registration form into a MySQL database using an INSERT query, which stores the user's basic information in the database.

[0444] Step 2: Receiving progress data

[0445] Specific operation: The server periodically receives progress data (e.g., test scores and task completion levels) sent from the device and stores it in an SQL database.

[0446] Input: Progress data sent from the device (e.g. Math: 90 points, English: 85 points, Science: 92 points)

[0447] Data processing: Parsing the received data and running INSERT or UPDATE queries in the database to store the information.

[0448] Output: Updated progress data saved to the database

[0449] Step 2:

[0450] The server receives the progress data sent from the device and saves it in a MySQL database using INSERT or UPDATE queries, which stores the progress data in the database.

[0451] Step 3: Building and updating the generative model

[0452] Specific operation: The server builds and updates a generative model using a Python machine learning library (e.g., TensorFlow) based on the collected progress data.

[0453] Input: Progress data stored in the database

[0454] Data processing: Preprocessing data, training machine learning models, and building and updating generative models.

[0455] Output: Latest generated model file

[0456] Step 3:

[0457] The server uses TensorFlow to build a generative AI model based on the collected progress data and updates the model based on new data, resulting in an up-to-date generative model.

[0458] Step 4: Generate feedback

[0459] Specific behavior: The server generates feedback messages based on the user's progress and goals using the latest generative model.

[0460] Input: The latest generative model and user progress data

[0461] Data processing: Use the model to make predictions and generate appropriate feedback messages.

[0462] Output: Feedback message (e.g., "You need to improve your English grade by 5 points to achieve your next goal")

[0463] Step 4:

[0464] The server analyzes the progress and goals based on the latest generative model and uses natural language processing to construct feedback messages, which are then provided to the user.

[0465] Step 5: Save and provide feedback

[0466] What it does: The server stores the generated feedback in a database and makes it available for parents and teachers to review via a web interface.

[0467] Input: The generated feedback message

[0468] Data processing: storing feedback messages in a database and preparing the data for display in the web interface.

[0469] Output: Feedback stored in the database and displayed in the web interface

[0470] Step 5:

[0471] The server stores the generated feedback in a MySQL database using INSERT queries and makes it available to parents and teachers through a web interface using Flask, allowing them to review the generated feedback.

[0472] Terminal (tablet) side program

[0473] Step 1: Data entry

[0474] What it does: The device stores the goals and daily progress information entered by the user in local storage (e.g., SQLite).

[0475] Input: User-supplied goals and progress data (e.g., today's goal is to complete all math assignments)

[0476] Data processing: Analyzes input information and saves it to local storage.

[0477] Output: Goal and progress data stored in local storage

[0478] Step 1:

[0479] The device receives the goal and progress data entered by the user and stores it in an SQLite database, which stores the user-entered data in local storage.

[0480] Step 2: Sending progress data

[0481] Specific operation: The device automatically sends progress data to the server at regular intervals.

[0482] Input: Progress data stored in local storage

[0483] Data processing: Format the data and send it to the server as an HTTP request.

[0484] Output: Progress data sent to the server

[0485] Step 2:

[0486] The device periodically sends the progress data stored in the local storage to the server, whereby the progress data is uploaded to the server.

[0487] Step 3: Working with the generative model

[0488] Specific operation: The device receives the latest feedback message from the server and stores it in local storage.

[0489] Input: Feedback message sent by the server

[0490] Data processing: Parse the feedback messages and save them to local storage.

[0491] Output: Feedback message saved to local storage

[0492] Step 3:

[0493] The terminal receives the latest feedback message sent by the server and stores it in local storage, thereby storing the feedback on the terminal.

[0494] Step 4: View your feedback

[0495] Specific behavior: The device visually displays the received feedback to the user.

[0496] Input: Feedback message stored in local storage

[0497] Data processing: Formatting feedback messages for visual display.

[0498] Output: Feedback message displayed on the screen

[0499] Step 4:

[0500] The device visually displays the feedback messages stored in the local storage in the form of a dashboard, allowing the user to review the feedback.

[0501] Step 5: Inform parents and teachers

[0502] Specific behavior: The device will display feedback information appropriately when a parent or teacher requests confirmation.

[0503] Input: Parent and teacher requests

[0504] Data processing: Retrieve feedback information from local storage and display it in an appropriate format.

[0505] Output: Feedback information displayed on the screen

[0506] Step 5:

[0507] Upon receiving a request from a parent or teacher, the device will display the feedback information in an appropriate format, which can then be viewed by the parent or teacher on their dashboard.

[0508] Emotion Engine

[0509] Step 1: Recognizing user emotions

[0510] Specific operation: The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone to recognize their current emotional state.

[0511] Input: User facial and voice data captured by camera and microphone

[0512] Data processing: Analyze facial and voice data to classify and recognize emotions.

[0513] Output: Perceived emotional state (e.g., happy, tired)

[0514] Step 1:

[0515] The emotion engine recognizes the user's current emotional state by capturing and analyzing the user's facial expressions and voice using a camera and microphone, and the user's emotions are then recognized as data.

[0516] Step 2: Sending emotion data

[0517] Specific operation: The emotion engine sends the recognized emotion data to the server via the device.

[0518] Input: Recognized emotion data

[0519] Data processing: Emotion data is formatted and sent to the server via an HTTP request.

[0520] Output: Emotion data sent to the server

[0521] Step 2:

[0522] The emotion engine sends the recognized emotion data to the server via the device, whereby the emotion data is uploaded to the server.

[0523] Step 3: Providing emotional information to parents and teachers

[0524] Specific operation: The emotional state recognized by the emotion engine is displayed on an interface used by parents and teachers.

[0525] Input: Emotion data sent from the server

[0526] Data processing: Formatting emotion data for display on the interface.

[0527] Output: Emotion information displayed on the interface

[0528] Step 3:

[0529] The emotional data recognized by the emotion engine is displayed on the interface for parents and teachers, allowing them to understand the child's mental state, and this allows parents and teachers to check emotional information in real time.

[0530] (Application example 2)

[0531] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0532] Conventional learning support systems only provide feedback based on a child's progress data and do not take into account the child's emotional state. As a result, children's motivation tends to decline, and there are issues with their ability to continue learning. There was also a need for a system that would allow parents and teachers to support children's emotional state as well as their academic progress.

[0533] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0534] In this invention, the server includes means for inputting and saving a child's goals and progress data, means for constructing and updating a generative model based on the collected data, means for generating feedback based on the constructed generative model, means for providing the generated feedback to parents and teachers, means for analyzing emotional states and incorporating them as data, and means for reflecting the emotional data in the feedback. This allows for the provision of feedback from both the perspectives of learning progress and emotional states, improving children's motivation and providing continuous learning support.

[0535] "Means for inputting and saving children's goals and progress data" refers to a function that allows children to input their learning goals and daily progress, and to electronically record and save that data.

[0536] "Means for building and updating generative models based on collected data" refers to a function that collects data on children's progress, creates a generative model based on this data to predict learning progress and outcomes, and keeps the model up to date every time the data is updated.

[0537] "Means for generating feedback based on the constructed generative model" is a function that generates advice and feedback regarding a child's learning progress and next goals based on information predicted and analyzed by the generative model.

[0538] "Means for providing generated feedback to parents and teachers" is a function for providing feedback on learning generated by the system so that it can be viewed by parents and teachers.

[0539] "Means for analyzing emotional state and capturing it as data" is a function that uses sensors such as a camera and microphone to analyze a child's current emotional state from their facial expressions and tone of voice, and obtains the results as data.

[0540] "Means of using emotional data to reflect in feedback" is a function that adjusts the feedback content of the generative model based on the acquired emotional data, and provides appropriate advice and support from an emotional perspective.

[0541] The "means for visually displaying" is a function for displaying the generated feedback and progress status to the user in a visually easy-to-understand format (graphs, messages, etc.).

[0542] "Means for providing an interface" refers to the function of providing a user interface that allows parents and teachers to access the system and check feedback and progress.

[0543] MODE FOR CARRYING OUT THE INVENTION

[0544] System Overview

[0545] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback using a generative model. The system aims to improve children's motivation for learning and activities, with users (children, parents, and teachers) participating in their respective roles. The system is primarily composed of a server, a device (smartphone), and an emotion engine.

[0546] Server-side program

[0547] The server does the following:

[0548] 1. Managing your user profile

[0549] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[0550] 2. Receiving progress data

[0551] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0552] 3. Building and updating the generative model

[0553] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[0554] 4. Generate feedback

[0555] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals, and also incorporates emotional data into the feedback.

[0556] 5. Storing and Providing Feedback

[0557] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0558] Terminal (smartphone) side program

[0559] The terminal does the following:

[0560] 1. Data Entry

[0561] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0562] 2. Sending progress data

[0563] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0564] 3. Collaboration with generative models

[0565] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0566] 4. Viewing Feedback

[0567] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0568] 5. Providing information to parents and teachers

[0569] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0570] Emotion Engine

[0571] The emotion engine does the following:

[0572] 1. User Emotion Recognition

[0573] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[0574] 2. Sending Emotional Data

[0575] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[0576] 3. Providing emotional information to parents and teachers

[0577] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[0578] Examples and prompts

[0579] Examples:

[0580] Children take English tests at brick-and-mortar stores. Test results are entered using a smartphone, and emotions during learning are monitored via a camera. Parents can view test results and emotional data in real time using a dedicated app.

[0581] Prompt statement:

[0582] "With the real-time learning assistance assistant, you set your goals and manually input your test results. Then, it uses the camera to collect emotional data and send it to a server for feedback."

[0583] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0584] Step 1:

[0585] Enter your child's goals and progress data

[0586] The user (child) uses a smartphone application to input goals and progress data. For example, the user might input, "Today's goal is to complete all of my math homework." The input data is saved in the device's local storage. The input data includes goal information and progress information.

[0587] Step 2:

[0588] Sending progress data to the server

[0589] The terminal periodically sends the progress data entered by the user to the server. In particular, when new data is entered, it is automatically sent to the server. The server stores the received data in a database. The input data is progress information, and the output is progress data stored in the server-side database.

[0590] Step 3:

[0591] Building and updating generative models

[0592] The server builds and updates a generative model based on the collected data. For example, it creates a generative model that says, "Your math grade is 90, and your next goal should be 95." The input data is the saved progress data, and the output is the updated generative model.

[0593] Step 4:

[0594] Generate feedback

[0595] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals, for example, "Your next goal is to get a 95 in math." The input data are the generative model and progress data, and the output is the generated feedback message.

[0596] Step 5:

[0597] Storing and Providing Feedback

[0598] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review. For example, parents and teachers can review their child's feedback through the web interface. The input data are the generated feedback messages, and the output are the feedback messages stored in the database and those viewable via the web interface.

[0599] Step 6:

[0600] Acquiring emotion data

[0601] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone. For example, a child may use the camera while studying, and the engine can recognize emotional states such as "happy" or "tired" from their facial expressions. The input data is real-time video and audio from the camera and microphone, and the output is analyzed emotional data.

[0602] Step 7:

[0603] Sending emotion data to the server

[0604] The emotion engine sends the recognized emotion data to the server via the device. The server integrates this data into the generative model and reflects it in the feedback. For example, it generates an encouraging message such as "You're doing well, so keep it up next time!" The input data is emotion data, and the output is the integrated generative model and an updated feedback message.

[0605] Step 8:

[0606] View Feedback

[0607] The device visually displays the received feedback to the user, for example, "Your current science grade is 92. Your next goal is to get your English grade to 90." The input data is the generated feedback message, and the output is the feedback visually displayed to the user.

[0608] Step 9:

[0609] Providing information to parents and teachers

[0610] When a device receives a request from a parent or teacher to check, it displays the appropriate feedback and emotional information. For example, when a parent sends a request, the device displays the user's progress and feedback in a dashboard format. The input data is the request from the parent or teacher and the saved feedback and emotional information, and the output is the visually displayed information.

[0611] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0612] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0613] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0614] [Second embodiment]

[0615] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0616] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0617] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0618] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0619] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0620] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0621] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0622] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0623] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0624] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0625] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0626] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0627] The present invention can be implemented in the following manner.

[0628] System Overview

[0629] This invention provides a learning support system that allows children to visually grasp their goals and visualize their progress through feedback using a generative model. This system aims to improve children's motivation for learning and activities, with children, parents, and teachers each playing their respective roles. The system is primarily composed of a server and a terminal (tablet).

[0630] Server-side program

[0631] The server performs the following process:

[0632] 1. Managing your user profile

[0633] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[0634] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[0635] 2. Receiving progress data

[0636] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0637] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0638] 3. Building and updating the generative model

[0639] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[0640] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0641] 4. Generate feedback

[0642] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0643] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0644] 5. Storing and Providing Feedback

[0645] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0646] Example: Parents and teachers can view children's feedback through a web interface.

[0647] Terminal (tablet) side program

[0648] The terminal performs the following process:

[0649] 1. Data Entry

[0650] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0651] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0652] 2. Sending progress data

[0653] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0654] Example: When a child enters the test results, the device immediately sends the data to the server.

[0655] 3. Collaboration with generative models

[0656] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0657] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0658] 4. Viewing Feedback

[0659] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0660] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0661] 5. Providing information to parents and teachers

[0662] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0663] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0664] User (child, parent, teacher) usage

[0665] The user uses the system as follows:

[0666] 1. Use by Children

[0667] Children enter their daily goal achievements into a tablet and check their progress data.

[0668] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[0669] 2. Use by parents and teachers

[0670] Parents and teachers use a tablet or web interface to view a child's progress data and the feedback provided by the generative model.

[0671] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[0672] The above is an embodiment of the present invention.

[0673] The processing flow will be explained below.

[0674] Server-side program processing steps

[0675] Step 1: Manage your user profile

[0676] When a user registers, the server stores the user's profile information (name, age, grade, goals) in a database.

[0677] Example: If a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this information in a database.

[0678] Step 2: Receiving progress data

[0679] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0680] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0681] Step 3: Building and updating the generative model

[0682] The server builds a generative model based on the collected data and updates it as new progress data is received.

[0683] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0684] Step 4: Generate feedback

[0685] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0686] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0687] Step 5: Save and provide feedback

[0688] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0689] Example: Parents and teachers can view children's feedback through a web interface.

[0690] Processing steps of the terminal (tablet) program

[0691] Step 1: Data entry

[0692] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0693] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0694] Step 2: Sending progress data

[0695] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0696] Example: When a child enters the test results, the device immediately sends the data to the server.

[0697] Step 3: Working with the generative model

[0698] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0699] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0700] Step 4: View your feedback

[0701] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0702] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0703] Step 5: Inform parents and teachers

[0704] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0705] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0706] User (child, parent, teacher) usage processing steps

[0707] Step 1: Child access

[0708] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback.

[0709] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[0710] Step 2: Use by parents and teachers

[0711] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and the feedback provided by the generative model.

[0712] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[0713] Example 1

[0714] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0715] In today's educational environment, children lack the means to visually grasp their goals and effectively manage their progress. It is also difficult for parents and teachers to grasp children's learning progress in real time and provide appropriate feedback. This creates a problem of lowering children's motivation to learn.

[0716] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0717] In this invention, the server includes means for inputting and saving a child's goal and progress information, means for receiving and saving progress data sent from the terminal, means for building and updating a generative AI model based on the collected data, means for generating individual feedback messages based on the generative AI model, and means for providing the generated feedback to parents and teachers. This allows parents and teachers to visually grasp the child's goal achievement status, check the progress in real time, and provide appropriate feedback.

[0718] "Goals" are specific outcomes or targets for learning or activities that a child wants to achieve.

[0719] "Progress information" refers to the process and results of a child's learning and activities toward a goal.

[0720] A "storage means" is a method or device for permanently recording data and making it available for later reference.

[0721] "Means for receiving" refers to a method or device for receiving information or data sent from outside.

[0722] A "generative AI model" is an algorithm that learns from collected data and makes future predictions and analyses.

[0723] "Individual feedback messages" are messages that provide specific advice or next steps based on the user's progress and goals.

[0724] "Means for providing" refers to a method or device for delivering the generated information or message to users or related parties.

[0725] A "visual display means" is a method or apparatus for displaying data or information on a screen or device in a manner that is easy for a user to understand.

[0726] A "Web interface" is a collection of screens and input devices that allow users to access and operate a system via the Internet.

[0727] This invention provides a learning support system that allows children to visually grasp their goals and visualize their progress through feedback generated by a generative AI model. The system is primarily composed of a server and a terminal (tablet).

[0728] Server-side program

[0729] The server performs the following process.

[0730] 1. Managing your user profile

[0731] When a user registers, the server stores their profile information, such as their name, age, grade, and goals, in a database. This information serves as the basic data for building a generative model.

[0732] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[0733] 2. Receiving progress data

[0734] The server periodically receives progress data (e.g., test scores and assignment completion rates) sent from the device and stores it in a database.

[0735] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0736] 3. Building and updating the generative model

[0737] The server builds a generative AI model based on the collected data, and when new progress data is received, it updates the generative model, enabling more accurate predictions.

[0738] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0739] 4. Generate feedback

[0740] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0741] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0742] 5. Storing and Providing Feedback

[0743] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0744] Example: Parents and teachers can view children's feedback through a web interface.

[0745] Terminal (tablet) side program

[0746] The terminal performs the following process:

[0747] 1. Data Entry

[0748] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0749] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0750] 2. Sending progress data

[0751] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0752] Example: When a child enters the test results, the device immediately sends the data to the server.

[0753] 3. Collaboration with generative models

[0754] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0755] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0756] 4. Viewing Feedback

[0757] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0758] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0759] 5. Providing information to parents and teachers

[0760] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0761] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0762] User (child, parent, teacher) usage

[0763] The user uses the system as follows:

[0764] 1. Use by Children

[0765] Children enter their daily goal achievements into a tablet and check their progress data.

[0766] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[0767] 2. Use by parents and teachers

[0768] Parents and teachers use a tablet or web interface to view a child's progress data and the feedback provided by the generative model.

[0769] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[0770] Specific examples

[0771] A user registers "Yamada Taro" in the system, stating that "my goal is to become a scientist." One week later, Yamada Taro enters "Test results: 90 points in math, 85 points in English, 92 points in science" into his device. The device immediately sends the data to the server. The server receives the new data and stores it in a database. The server updates the generative model, analysing that "to achieve the next goal, you need to improve your English grade by 5 points," and stores this feedback in the database. The device receives the feedback from the server and displays on the dashboard, "Your current science grade is 92 points. Your next goal is to improve your English grade to 90 points." Parents can view all the feedback using a web interface.

[0772] Prompt Sentence Examples

[0773] "Register a new user profile. Enter your name, age, grade, and goals."

[0774] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0775] Step 1:

[0776] Register a new user profile

[0777] Input: The user enters their name, age, grade, and goal.

[0778] Action: The user enters the required information into the web form (e.g., "Name: Yamada Taro, Age: 12, Grade: 1st year of junior high school, Goal: Scientist") and presses the submit button.

[0779] Data processing: The server receives the entered data, checks the format, and then saves it in the user table in the database.

[0780] Output: The profile information is saved and a success message is displayed to the user.

[0781] Step 2:

[0782] Entering and submitting progress data

[0783] Input: Children input their daily progress data, such as test results, into the tablet.

[0784] Action: The child enters their test results in the input field: "Math: 90 points, English: 85 points, Science: 92 points" and presses the save button.

[0785] Data processing: The device saves the input data in local storage and immediately sends it to the server.

[0786] Output: Progress data is sent to the server and saved to local storage.

[0787] Step 3:

[0788] Receiving and saving progress data

[0789] Input: Progress data sent from the device.

[0790] Operation: The server receives HTTP requests from the terminal, interprets the data, and inserts it into a progress table in the database.

[0791] Data processing: Convert the received progress data and add it to the database.

[0792] Output: The progress data is saved in the database and a success response is returned to the device.

[0793] Step 4:

[0794] Updating the generative model

[0795] Input: New progress data stored in the database.

[0796] How it works: The server starts the learning process for the generative AI model based on new progress data. It retrains the existing model with the latest data.

[0797] Data processing: The generative AI model learns from all collected data to improve prediction accuracy.

[0798] Output: The updated generative model is saved and used for the next feedback generation.

[0799] Step 5:

[0800] Generate feedback

[0801] Input: Updated generative model and new progress data.

[0802] How it works: The server uses the generative model to analyze the progress data and create personalized feedback messages.

[0803] Data processing: A generative AI model compares the user's progress with their goals and calculates the necessary advice (e.g., "You need to improve your English grade by 5 points to achieve your next goal").

[0804] Output: The generated feedback messages are stored in a database.

[0805] Step 6:

[0806] Receiving and viewing feedback

[0807] Input: The feedback message sent by the server.

[0808] Operation: The device receives the feedback sent from the server, stores it in local storage, and visually displays it to the user.

[0809] Data processing: Visualize the received feedback in the form of a dashboard on the device.

[0810] Output: A message like "Your current science grade is 92. Your next goal is to get your English grade to 90." will be displayed.

[0811] Step 7:

[0812] Providing information to parents and teachers

[0813] Input: A request for parents and teachers to review feedback.

[0814] How it works: A parent or teacher submits a request to review information via a web interface or tablet.

[0815] Data processing: The server receives the request, retrieves the relevant feedback information from the database, and displays it.

[0816] Output: Parents and teachers can view progress and feedback information in a dashboard format.

[0817] (Application example 1)

[0818] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0819] Conventional shopping support systems have difficulty monitoring users' purchasing behavior in real time and providing individualized feedback. This has resulted in insufficient support for users to achieve their shopping goals. It has also made it difficult for parents and teachers to effectively monitor their children's learning progress and provide appropriate advice. In light of these circumstances, there is a need for a more comprehensive system that can help users achieve both their purchasing and learning goals.

[0820] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0821] In this invention, the server includes means for inputting and saving a child's goals and progress data, means for constructing and updating a generative model based on the collected data, means for generating feedback based on the constructed generative model, means for providing the generated feedback to parents and teachers, means for inputting and saving a user's purchasing goals and purchasing data, means for constructing and updating a generative model based on the collected purchasing data, means for generating purchasing feedback based on the constructed generative model, and means for providing the generated feedback to the user. This enables real-time support for users to achieve their shopping goals, and allows parents and teachers to properly understand their children's learning progress and provide appropriate advice.

[0822] "Progress data" is data that indicates the progress of the goals set by the user.

[0823] A "generative model" is a model built based on collected data to predict and analyze user behavior and progress.

[0824] "Feedback" refers to advice or information provided to users, parents, or teachers based on the generative model.

[0825] "Tools provided to parents and teachers" refers to functions that allow parents and teachers to check information about their children's learning progress and goal achievement.

[0826] "Purchase Goals" refers to specific shopping goals or plans set by a user.

[0827] "Purchase data" refers to data such as information about the products actually purchased by the user, the amount, and the date and time of purchase.

[0828] "Purchase feedback" refers to advice and information provided to users regarding their purchasing behavior based on generative models.

[0829] This invention is a system that helps users set shopping goals and track their progress in real time to help them achieve them. Using a server and a device (smartphone), the system collects user purchasing data and provides feedback using a generative AI model.

[0830] The server performs the following processes: When a user registers, their purchasing goals and personal information are saved in a database. This will later become the basic data for building a generative model. For example, if a user enters "keep my monthly shopping budget under 30,000 yen," the server records this in the database.

[0831] Next, the purchase data entered on the terminal (purchase date and time, product name, price, etc.) is periodically sent to the server. For example, if a user enters data such as "Purchase a T-shirt for 2,500 yen," the server receives this and stores it in a database.

[0832] The server builds and updates a generative AI model based on the collected data. This model is built using TensorFlow and is updated each time new purchase data is entered. The model predicts the user's progress based on the latest data and provides specific feedback for achieving the next goal. For example, feedback such as "To achieve next month's budget, be careful not to exceed 10,000 yen remaining this month" may be generated.

[0833] The generated feedback is stored in a database and provided to the user. In addition to receiving feedback via smartphone, users also have access to a dashboard that allows them to visually check their purchasing goals and progress. The screen displays the remaining amount of a pre-set budget and a list of recently purchased items.

[0834] The generated feedback may also be provided in a form that can be viewed by parents, teachers, and other relevant parties. In this case, parents and teachers can view the feedback through a web interface and provide appropriate advice to the user.

[0835] For example, if a user sets a monthly shopping budget of ¥30,000, the app collects purchasing data, visually displays progress, and the generative AI model provides appropriate advice, such as, "Your remaining budget for this month is ¥10,000. We recommend you be careful with your next purchase."

[0836] An example of a prompt to be input to a generative AI model could be, "Based on the user's current purchasing data, estimate the next purchase budget and generate feedback."

[0837] This allows users to efficiently manage their purchasing goals, check their progress in real time, and support them in achieving their goals. Parents and teachers can also keep track of their children's learning progress and provide appropriate advice.

[0838] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0839] Step 1:

[0840] The user enters their shopping goals and personal information.

[0841] Input: Shopping goal set by the user (e.g., less than 30,000 yen per month) and personal information (e.g., name, age, budget)

[0842] Output: User profile information stored in the database

[0843] Specific operation: The user enters their goals and personal information into the smartphone application, which is then sent to the server and recorded in a database.

[0844] Step 2:

[0845] The terminal inputs the purchase data.

[0846] Input: Information about the product purchased by the user (e.g., T-shirt, 2,500 yen, purchase date and time, etc.)

[0847] Output: Purchasing data stored in a database

[0848] Specific operation: When a user enters information about a purchased item into the terminal, this information is sent to the server and stored in a database.

[0849] Step 3:

[0850] The server collects purchasing data and builds and updates the generative AI model.

[0851] Input: Purchasing data stored in the database

[0852] Output: An updated generative AI model

[0853] How it works: The server periodically collects purchasing data from a database and builds and updates a generative AI model using TensorFlow, etc. The accuracy of the model improves with each new piece of data.

[0854] Step 4:

[0855] The server generates feedback based on the generative AI model.

[0856] Input: Updated generative AI model and the user's latest purchase data

[0857] Output: The generated feedback message

[0858] Specific operation: The server uses the generated AI model to analyze the user's purchasing behavior and generate feedback such as, "To achieve next month's budget, you need to be careful not to exceed 10,000 yen remaining."

[0859] Step 5:

[0860] The server generates feedback, stores it in a database, and provides it to the terminal.

[0861] Input: The generated feedback message

[0862] Output: Feedback displayed on the user's smartphone

[0863] Specific operation: The server stores the generated feedback in a database and sends it to the user's device for display. For example, the user's application screen might display "This month's remaining budget is 10,000 yen."

[0864] Step 6:

[0865] The device displays visual feedback to the user.

[0866] Input: Feedback message sent by the server

[0867] Output: Feedback information displayed on the dashboard

[0868] What it does: The device displays the feedback it receives in real time in a dashboard format. By opening the app, users can check their current remaining budget, recent purchase history, and more at a glance.

[0869] Step 7:

[0870] Access an interface where parents and teachers can review feedback.

[0871] Input: Parent or teacher request

[0872] Output: Child's purchasing feedback displayed in the web interface

[0873] Specific operation: When a parent or teacher sends a request to check feedback information through a web interface, the server provides the feedback data, allowing the parent or teacher to check their child's purchasing status and goal achievement status.

[0874] These steps enable users to receive real-time support to efficiently achieve their purchasing goals, while parents and teachers can effectively track learning progress and provide appropriate advice.

[0875] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0876] The present invention can be implemented in the following manner.

[0877] System Overview

[0878] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback from a generative model. It also combines an emotion engine that recognizes the user's emotions to further increase children's motivation. The system aims to increase children's motivation for learning and activities, with the users (children, parents, and teachers) participating in their respective roles. The system primarily consists of a server, a terminal (tablet), and an emotion engine.

[0879] Server-side program

[0880] The server performs the following process:

[0881] 1. Managing your user profile

[0882] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[0883] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[0884] 2. Receiving progress data

[0885] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0886] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0887] 3. Building and updating the generative model

[0888] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[0889] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0890] 4. Generate feedback

[0891] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0892] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0893] 5. Storing and Providing Feedback

[0894] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0895] Example: Parents and teachers can view children's feedback through a web interface.

[0896] Terminal (tablet) side program

[0897] The terminal performs the following process:

[0898] 1. Data Entry

[0899] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0900] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0901] 2. Sending progress data

[0902] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0903] Example: When a child enters the test results, the device immediately sends the data to the server.

[0904] 3. Collaboration with generative models

[0905] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0906] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0907] 4. Viewing Feedback

[0908] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0909] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0910] 5. Providing information to parents and teachers

[0911] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0912] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0913] Emotion Engine

[0914] The emotion engine performs the following processing:

[0915] 1. User Emotion Recognition

[0916] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[0917] Example: Using a camera while a child is learning, the system can recognize emotional states such as "happy" or "tired" from their facial expressions.

[0918] 2. Sending Emotional Data

[0919] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[0920] Example: If the emotion engine recognizes the emotion "fun," it sends that data to the server, which then generates an encouraging message in the next feedback, saying, "You're doing your best, so keep it up next time!"

[0921] 3. Providing emotional information to parents and teachers

[0922] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[0923] Example: A parent learns through an emotion interface that their child is perceived as "tired" and advises them to moderate their learning for the day.

[0924] User (child, parent, teacher) usage

[0925] The user uses the system as follows:

[0926] 1. Use by Children

[0927] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback. At the same time, the emotion engine analyzes their facial expressions and voice to understand their emotional state.

[0928] Example: A child enters "I will complete all my math homework today" and sees the result in feedback. If the emotion engine recognizes that they are "tired," that data will also be reflected in the feedback.

[0929] 2. Use by parents and teachers

[0930] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and feedback provided by the generative model, as well as the emotional state recognized by the emotion engine.

[0931] Example: A parent may discover that their child has a strong aptitude for science and provide specific advice to further support them. Then, through the emotion engine, they may learn that their child is enjoying the activity and encourage them to continue learning at this rate.

[0932] The above is an embodiment of the present invention.

[0933] The processing flow will be explained below.

[0934] Server-side program processing steps

[0935] Step 1: Manage your user profile

[0936] When a user registers, the server stores the user's profile information (name, age, grade, goals) in a database.

[0937] Example: If a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this information in a database.

[0938] Step 2: Receiving progress data

[0939] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[0940] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[0941] Step 3: Building and updating the generative model

[0942] The server builds a generative model based on the collected data and updates it as new progress data is received.

[0943] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[0944] Step 4: Generate feedback

[0945] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[0946] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[0947] Step 5: Save and provide feedback

[0948] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[0949] Example: Parents and teachers can view children's feedback through a web interface.

[0950] Processing steps of the terminal (tablet) program

[0951] Step 1: Data entry

[0952] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[0953] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[0954] Step 2: Sending progress data

[0955] The device periodically sends progress data to the server, especially automatically when new data is entered.

[0956] Example: When a child enters the test results, the device immediately sends the data to the server.

[0957] Step 3: Working with the generative model

[0958] The device receives feedback on the latest generative model from the server and stores it in local storage.

[0959] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[0960] Step 4: View your feedback

[0961] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[0962] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[0963] Step 5: Inform parents and teachers

[0964] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[0965] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[0966] Emotion Engine Processing Steps

[0967] Step 1: Recognizing user emotions

[0968] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[0969] Example: Using a camera while a child is learning, the system can recognize emotional states such as "happy" or "tired" from their facial expressions.

[0970] Step 2: Sending emotion data

[0971] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[0972] Example: If the emotion engine recognizes the emotion "fun," it sends that data to the server, which then generates an encouraging message in the next feedback, saying, "You're doing your best, so keep it up next time!"

[0973] Step 3: Providing emotional information to parents and teachers

[0974] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[0975] Example: A parent learns through an emotion interface that their child is perceived as "tired" and advises them to moderate their learning for the day.

[0976] User (child, parent, teacher) processing steps

[0977] Step 1: Child access

[0978] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback. At the same time, the emotion engine analyzes their facial expressions and voice to understand their emotional state.

[0979] Example: A child enters "I will complete all my math homework today" and sees the result in feedback. If the emotion engine recognizes that they are "tired," that data will also be reflected in the feedback.

[0980] Step 2: Use by parents and teachers

[0981] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and feedback provided by the generative model, as well as the emotional state recognized by the emotion engine.

[0982] Example: A parent may discover that their child has a strong aptitude for science and provide specific advice to further support them. Then, through the emotion engine, they may learn that their child is enjoying the activity and encourage them to continue learning at this rate.

[0983] Example 2

[0984] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0985] In today's educational environment, it is important for children to set goals independently and manage their progress toward them in order to improve their learning outcomes. However, it is difficult for children to accurately grasp their own progress and decide on their next actions based on that. Furthermore, there are few ways for parents and teachers to grasp children's learning situation and emotional state in real time, making it difficult to provide appropriate feedback and support. Furthermore, a lack of feedback that takes emotional changes into account can lead to a decline in children's motivation. There is a need for a system that can solve these issues and effectively promote learning while maintaining children's motivation.

[0986] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0987] In this invention, the server includes: means for inputting and saving a child's goals and progress data; means for constructing and updating a generative model based on the collected data; means for generating feedback based on the constructed generative model; means for providing the generated feedback to parents and teachers; means for recognizing and digitizing the user's emotions; means for transmitting the recognized emotional data to the server; and means for providing an interface through which parents and teachers can check the feedback. This facilitates children's goal setting and progress management, and enables children to maintain their learning motivation through accurate feedback using the generative AI model. Furthermore, feedback that takes emotional data into account allows parents and teachers to grasp a child's learning situation and emotional state in real time and provide appropriate support.

[0988] "Goals" refer to the specific objectives of learning or activities that a child is trying to achieve.

[0989] "Progress data" refers to information that shows a child's achievement level and learning results toward the goals they have set.

[0990] "Generative model" refers to a machine learning model that is built and updated based on collected progress data.

[0991] "Feedback" refers to messages based on generative models that provide advice and evaluation of the user's progress and next goals.

[0992] An "emotion engine" is a system that analyzes information obtained from the device's sensors and camera, and recognizes and digitizes the user's emotional state.

[0993] "User" refers to all users of this system, including children, parents, and teachers.

[0994] "Interface" refers to the GUI (graphical user interface) or web interface that allows users to interact with the system.

[0995] MODE FOR CARRYING OUT THE INVENTION

[0996] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback from a generative model. It also combines an emotion engine that recognizes the user's emotions to further increase children's motivation. This system involves users (children, parents, and teachers) in their respective roles, aiming to increase children's motivation for learning and activities.

[0997] System Configuration

[0998] The system mainly consists of a server, a terminal (tablet), and an emotion engine. The specific roles of each component are as follows:

[0999] server

[1000] The server manages the child's goals and progress data, and generates and provides feedback using a generative AI model. The server manages user profiles, receives progress data, builds and updates the generative model, generates feedback, and stores and provides the feedback. A typical server machine is used as the hardware, and Python, MySQL, TensorFlow, and Flask are used as the software.

[1001] User profile management: The server stores profile information in a MySQL database when a new user registers.

[1002] Receiving progress data: The server periodically receives progress data sent from the device and stores it in a MySQL database.

[1003] Building and updating generative models: The server builds and updates generative AI models using TensorFlow based on the collected data.

[1004] Feedback generation: The server uses the generative AI model to generate feedback messages and save them in JSON format.

[1005] Storing and serving feedback: The server stores the generated feedback in a MySQL database and serves it in a web interface via Flask.

[1006] Device (tablet)

[1007] The terminal is a device that allows users to input goals and progress data and display feedback. Specifically, we will use a tablet terminal and develop applications for Android and iOS.

[1008] Data Entry: The device stores user-entered goals and daily progress information in local storage. For example, if a child enters "my goal for today is to complete all my math homework," that information is stored in the device's SQLite database.

[1009] Sending progress data: The device automatically sends progress data to the server at regular intervals. For example, when test results are entered, the device sends the data to the server.

[1010] Interaction with generative models: The device receives feedback from the server and stores it in local storage. For example, if a message is received saying, "Your next goal is to get 90 points in math," it will be stored on the device.

[1011] Displaying feedback: The device visually displays the received feedback to the user, for example, "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1012] Providing information to parents and teachers: The device will display appropriate feedback information when requested by a parent or teacher.

[1013] Emotion Engine

[1014] The emotion engine is responsible for recognizing and digitizing the user's emotions. Specifically, it uses the device's camera and microphone to analyze the user's facial expressions and tone of voice, and uses software modules (such as OpenCV and TensorFlow) to recognize the user's emotional state.

[1015] User emotion recognition: The emotion engine analyzes the user's facial expressions and tone of voice to recognize their current emotional state. For example, a camera can recognize a child's facial expression while they are studying and analyze their emotions, such as "happy" or "tired."

[1016] Emotion data transmission: The emotion engine transmits the recognized emotion data to the server via the device, and the server integrates this data into the generative AI model. For example, if the emotion "fun" is recognized, the data is transmitted to the server and reflected in the feedback.

[1017] Providing emotional information to parents and teachers: The emotional state recognized by the emotion engine is also displayed in an interface used by parents and teachers. Parents can see through the interface that their child is recognized as tired and can advise them to moderate their studying for the day.

[1018] Examples and prompts

[1019] Examples of use for children:

[1020] Daily goal achievement and progress are entered into the tablet, and the generated feedback is then confirmed. At the same time, the emotion engine analyzes facial expressions and voice to understand the child's emotional state. A child may enter "I will complete all my math homework today," and the result is confirmed in the feedback. If the emotion engine recognizes that the child is "tired," this data is also reflected in the feedback.

[1021] Examples of parent and teacher use cases:

[1022] Using a tablet or web interface, parents can view their child's progress data, feedback provided by the generative model, and the emotional state recognized by the emotion engine. The parent can see that their child has a strong aptitude for science and provide specific advice to further support them. The emotion engine can also tell them that their child is having fun and encourage them to continue learning at this pace.

[1023] Example prompt:

[1024] Child prompt: "What is your goal for today (e.g., complete all math assignments)?"

[1025] Child prompt: "Enter your test scores (e.g., Math: 90, English: 85)."

[1026] Parent prompt: "Do you want to check in on your child's progress? (Yes / No)"

[1027] Parent prompt: "Would you like to check in with your child's emotional state? (Yes / No)"

[1028] Teacher prompt: "Would you like to review student progress data? (Yes / No)"

[1029] Teacher prompt: "Would you like to review student feedback? (Yes / No)"

[1030] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1031] Server-side program

[1032] Step 1: Manage your user profile

[1033] Specific operation: The server receives profile information (name, age, grade, goals) submitted through the new user registration form and stores it in an SQL database (e.g., MySQL).

[1034] Input: New user registration information (e.g. name, age, grade, goal)

[1035] Data processing: Analyzes the received information and executes INSERT queries to the database to store the information.

[1036] Output: User information saved in the database

[1037] Step 1:

[1038] The server saves the profile information received from the new user registration form into a MySQL database using an INSERT query, which stores the user's basic information in the database.

[1039] Step 2: Receiving progress data

[1040] Specific operation: The server periodically receives progress data (e.g., test scores and task completion levels) sent from the device and stores it in an SQL database.

[1041] Input: Progress data sent from the device (e.g. Math: 90 points, English: 85 points, Science: 92 points)

[1042] Data processing: Parsing the received data and running INSERT or UPDATE queries in the database to store the information.

[1043] Output: Updated progress data saved to the database

[1044] Step 2:

[1045] The server receives the progress data sent from the device and saves it in a MySQL database using INSERT or UPDATE queries, which stores the progress data in the database.

[1046] Step 3: Building and updating the generative model

[1047] Specific operation: The server builds and updates a generative model using a Python machine learning library (e.g., TensorFlow) based on the collected progress data.

[1048] Input: Progress data stored in the database

[1049] Data processing: Preprocessing data, training machine learning models, and building and updating generative models.

[1050] Output: Latest generated model file

[1051] Step 3:

[1052] The server uses TensorFlow to build a generative AI model based on the collected progress data and updates the model based on new data, resulting in an up-to-date generative model.

[1053] Step 4: Generate feedback

[1054] Specific behavior: The server generates feedback messages based on the user's progress and goals using the latest generative model.

[1055] Input: The latest generative model and user progress data

[1056] Data processing: Use the model to make predictions and generate appropriate feedback messages.

[1057] Output: Feedback message (e.g., "You need to improve your English grade by 5 points to achieve your next goal")

[1058] Step 4:

[1059] The server analyzes the progress and goals based on the latest generative model and uses natural language processing to construct feedback messages, which are then provided to the user.

[1060] Step 5: Save and provide feedback

[1061] What it does: The server stores the generated feedback in a database and makes it available for parents and teachers to review via a web interface.

[1062] Input: The generated feedback message

[1063] Data processing: storing feedback messages in a database and preparing the data for display in the web interface.

[1064] Output: Feedback stored in the database and displayed in the web interface

[1065] Step 5:

[1066] The server stores the generated feedback in a MySQL database using INSERT queries and makes it available to parents and teachers through a web interface using Flask, allowing them to review the generated feedback.

[1067] Terminal (tablet) side program

[1068] Step 1: Data entry

[1069] What it does: The device stores the goals and daily progress information entered by the user in local storage (e.g., SQLite).

[1070] Input: User-supplied goals and progress data (e.g., today's goal is to complete all math assignments)

[1071] Data processing: Analyzes input information and saves it to local storage.

[1072] Output: Goal and progress data stored in local storage

[1073] Step 1:

[1074] The device receives the goal and progress data entered by the user and stores it in an SQLite database, which stores the user-entered data in local storage.

[1075] Step 2: Sending progress data

[1076] Specific operation: The device automatically sends progress data to the server at regular intervals.

[1077] Input: Progress data stored in local storage

[1078] Data processing: Format the data and send it to the server as an HTTP request.

[1079] Output: Progress data sent to the server

[1080] Step 2:

[1081] The device periodically sends the progress data stored in the local storage to the server, whereby the progress data is uploaded to the server.

[1082] Step 3: Working with the generative model

[1083] Specific operation: The device receives the latest feedback message from the server and stores it in local storage.

[1084] Input: Feedback message sent by the server

[1085] Data processing: Parse the feedback messages and save them to local storage.

[1086] Output: Feedback message saved to local storage

[1087] Step 3:

[1088] The terminal receives the latest feedback message sent by the server and stores it in local storage, thereby storing the feedback on the terminal.

[1089] Step 4: View your feedback

[1090] Specific behavior: The device visually displays the received feedback to the user.

[1091] Input: Feedback message stored in local storage

[1092] Data processing: Formatting feedback messages for visual display.

[1093] Output: Feedback message displayed on the screen

[1094] Step 4:

[1095] The device visually displays the feedback messages stored in the local storage in the form of a dashboard, allowing the user to review the feedback.

[1096] Step 5: Inform parents and teachers

[1097] Specific behavior: The device will display feedback information appropriately when a parent or teacher requests confirmation.

[1098] Input: Parent and teacher requests

[1099] Data processing: Retrieve feedback information from local storage and display it in an appropriate format.

[1100] Output: Feedback information displayed on the screen

[1101] Step 5:

[1102] Upon receiving a request from a parent or teacher, the device will display the feedback information in an appropriate format, which can then be viewed by the parent or teacher on their dashboard.

[1103] Emotion Engine

[1104] Step 1: Recognizing user emotions

[1105] Specific operation: The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone to recognize their current emotional state.

[1106] Input: User facial and voice data captured by camera and microphone

[1107] Data processing: Analyze facial and voice data to classify and recognize emotions.

[1108] Output: Perceived emotional state (e.g., happy, tired)

[1109] Step 1:

[1110] The emotion engine recognizes the user's current emotional state by capturing and analyzing the user's facial expressions and voice using a camera and microphone, and the user's emotions are then recognized as data.

[1111] Step 2: Sending emotion data

[1112] Specific operation: The emotion engine sends the recognized emotion data to the server via the device.

[1113] Input: Recognized emotion data

[1114] Data processing: Emotion data is formatted and sent to the server via an HTTP request.

[1115] Output: Emotion data sent to the server

[1116] Step 2:

[1117] The emotion engine sends the recognized emotion data to the server via the device, whereby the emotion data is uploaded to the server.

[1118] Step 3: Providing emotional information to parents and teachers

[1119] Specific operation: The emotional state recognized by the emotion engine is displayed on an interface used by parents and teachers.

[1120] Input: Emotion data sent from the server

[1121] Data processing: Formatting emotion data for display on the interface.

[1122] Output: Emotion information displayed on the interface

[1123] Step 3:

[1124] The emotional data recognized by the emotion engine is displayed on the interface for parents and teachers, allowing them to understand the child's mental state, and this allows parents and teachers to check emotional information in real time.

[1125] (Application example 2)

[1126] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1127] Conventional learning support systems only provide feedback based on a child's progress data and do not take into account the child's emotional state. As a result, children's motivation tends to decline, and there are issues with their ability to continue learning. There was also a need for a system that would allow parents and teachers to support children's emotional state as well as their academic progress.

[1128] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1129] In this invention, the server includes means for inputting and saving a child's goals and progress data, means for constructing and updating a generative model based on the collected data, means for generating feedback based on the constructed generative model, means for providing the generated feedback to parents and teachers, means for analyzing emotional states and incorporating them as data, and means for reflecting the emotional data in the feedback. This allows for the provision of feedback from both the perspectives of learning progress and emotional states, improving children's motivation and providing continuous learning support.

[1130] "Means for inputting and saving children's goals and progress data" refers to a function that allows children to input their learning goals and daily progress, and to electronically record and save that data.

[1131] "Means for building and updating generative models based on collected data" refers to a function that collects data on children's progress, creates a generative model based on this data to predict learning progress and outcomes, and keeps the model up to date every time the data is updated.

[1132] "Means for generating feedback based on the constructed generative model" is a function that generates advice and feedback regarding a child's learning progress and next goals based on information predicted and analyzed by the generative model.

[1133] "Means for providing generated feedback to parents and teachers" is a function for providing feedback on learning generated by the system so that it can be viewed by parents and teachers.

[1134] "Means for analyzing emotional state and capturing it as data" is a function that uses sensors such as a camera and microphone to analyze a child's current emotional state from their facial expressions and tone of voice, and obtains the results as data.

[1135] "Means of using emotional data to reflect in feedback" is a function that adjusts the feedback content of the generative model based on the acquired emotional data, and provides appropriate advice and support from an emotional perspective.

[1136] The "means for visually displaying" is a function for displaying the generated feedback and progress status to the user in a visually easy-to-understand format (graphs, messages, etc.).

[1137] "Means for providing an interface" refers to the function of providing a user interface that allows parents and teachers to access the system and check feedback and progress.

[1138] MODE FOR CARRYING OUT THE INVENTION

[1139] System Overview

[1140] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback using a generative model. The system aims to improve children's motivation for learning and activities, with users (children, parents, and teachers) participating in their respective roles. The system is primarily composed of a server, a device (smartphone), and an emotion engine.

[1141] Server-side program

[1142] The server does the following:

[1143] 1. Managing your user profile

[1144] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[1145] 2. Receiving progress data

[1146] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[1147] 3. Building and updating the generative model

[1148] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[1149] 4. Generate feedback

[1150] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals, and also incorporates emotional data into the feedback.

[1151] 5. Storing and Providing Feedback

[1152] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1153] Terminal (smartphone) side program

[1154] The terminal does the following:

[1155] 1. Data Entry

[1156] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1157] 2. Sending progress data

[1158] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1159] 3. Collaboration with generative models

[1160] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1161] 4. Viewing Feedback

[1162] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1163] 5. Providing information to parents and teachers

[1164] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1165] Emotion Engine

[1166] The emotion engine does the following:

[1167] 1. User Emotion Recognition

[1168] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[1169] 2. Sending Emotional Data

[1170] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[1171] 3. Providing emotional information to parents and teachers

[1172] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[1173] Examples and prompts

[1174] Examples:

[1175] Children take English tests at brick-and-mortar stores. Test results are entered using a smartphone, and emotions during learning are monitored via a camera. Parents can view test results and emotional data in real time using a dedicated app.

[1176] Prompt statement:

[1177] "With the real-time learning assistance assistant, you set your goals and manually input your test results. Then, it uses the camera to collect emotional data and send it to a server for feedback."

[1178] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1179] Step 1:

[1180] Enter your child's goals and progress data

[1181] The user (child) uses a smartphone application to input goals and progress data. For example, the user might input, "Today's goal is to complete all of my math homework." The input data is saved in the device's local storage. The input data includes goal information and progress information.

[1182] Step 2:

[1183] Sending progress data to the server

[1184] The terminal periodically sends the progress data entered by the user to the server. In particular, when new data is entered, it is automatically sent to the server. The server stores the received data in a database. The input data is progress information, and the output is progress data stored in the server-side database.

[1185] Step 3:

[1186] Building and updating generative models

[1187] The server builds and updates a generative model based on the collected data. For example, it creates a generative model that says, "Your math grade is 90, and your next goal should be 95." The input data is the saved progress data, and the output is the updated generative model.

[1188] Step 4:

[1189] Generate feedback

[1190] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals, for example, "Your next goal is to get a 95 in math." The input data are the generative model and progress data, and the output is the generated feedback message.

[1191] Step 5:

[1192] Storing and Providing Feedback

[1193] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review. For example, parents and teachers can review their child's feedback through the web interface. The input data are the generated feedback messages, and the output are the feedback messages stored in the database and those viewable via the web interface.

[1194] Step 6:

[1195] Acquiring emotion data

[1196] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone. For example, a child may use the camera while studying, and the engine can recognize emotional states such as "happy" or "tired" from their facial expressions. The input data is real-time video and audio from the camera and microphone, and the output is analyzed emotional data.

[1197] Step 7:

[1198] Sending emotion data to the server

[1199] The emotion engine sends the recognized emotion data to the server via the device. The server integrates this data into the generative model and reflects it in the feedback. For example, it generates an encouraging message such as "You're doing well, so keep it up next time!" The input data is emotion data, and the output is the integrated generative model and an updated feedback message.

[1200] Step 8:

[1201] View Feedback

[1202] The device visually displays the received feedback to the user, for example, "Your current science grade is 92. Your next goal is to get your English grade to 90." The input data is the generated feedback message, and the output is the feedback visually displayed to the user.

[1203] Step 9:

[1204] Providing information to parents and teachers

[1205] When a device receives a request from a parent or teacher to check, it displays the appropriate feedback and emotional information. For example, when a parent sends a request, the device displays the user's progress and feedback in a dashboard format. The input data is the request from the parent or teacher and the saved feedback and emotional information, and the output is the visually displayed information.

[1206] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1207] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1208] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1209] [Third embodiment]

[1210] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1211] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1212] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1213] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1214] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1215] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1216] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1217] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1218] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1219] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1220] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1221] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1222] The present invention can be implemented in the following manner.

[1223] System Overview

[1224] This invention provides a learning support system that allows children to visually grasp their goals and visualize their progress through feedback using a generative model. This system aims to improve children's motivation for learning and activities, with children, parents, and teachers each playing their respective roles. The system is primarily composed of a server and a terminal (tablet).

[1225] Server-side program

[1226] The server performs the following process:

[1227] 1. Managing your user profile

[1228] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[1229] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[1230] 2. Receiving progress data

[1231] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[1232] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[1233] 3. Building and updating the generative model

[1234] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[1235] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[1236] 4. Generate feedback

[1237] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[1238] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[1239] 5. Storing and Providing Feedback

[1240] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1241] Example: Parents and teachers can view children's feedback through a web interface.

[1242] Terminal (tablet) side program

[1243] The terminal performs the following process:

[1244] 1. Data Entry

[1245] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1246] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[1247] 2. Sending progress data

[1248] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1249] Example: When a child enters the test results, the device immediately sends the data to the server.

[1250] 3. Collaboration with generative models

[1251] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1252] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[1253] 4. Viewing Feedback

[1254] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1255] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1256] 5. Providing information to parents and teachers

[1257] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1258] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[1259] User (child, parent, teacher) usage

[1260] The user uses the system as follows:

[1261] 1. Use by Children

[1262] Children enter their daily goal achievements into a tablet and check their progress data.

[1263] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[1264] 2. Use by parents and teachers

[1265] Parents and teachers use a tablet or web interface to view a child's progress data and the feedback provided by the generative model.

[1266] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[1267] The above is an embodiment of the present invention.

[1268] The processing flow will be explained below.

[1269] Server-side program processing steps

[1270] Step 1: Manage your user profile

[1271] When a user registers, the server stores the user's profile information (name, age, grade, goals) in a database.

[1272] Example: If a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this information in a database.

[1273] Step 2: Receiving progress data

[1274] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[1275] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[1276] Step 3: Building and updating the generative model

[1277] The server builds a generative model based on the collected data and updates it as new progress data is received.

[1278] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[1279] Step 4: Generate feedback

[1280] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[1281] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[1282] Step 5: Save and provide feedback

[1283] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1284] Example: Parents and teachers can view children's feedback through a web interface.

[1285] Processing steps of the terminal (tablet) program

[1286] Step 1: Data entry

[1287] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1288] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[1289] Step 2: Sending progress data

[1290] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1291] Example: When a child enters the test results, the device immediately sends the data to the server.

[1292] Step 3: Working with the generative model

[1293] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1294] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[1295] Step 4: View your feedback

[1296] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1297] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1298] Step 5: Inform parents and teachers

[1299] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1300] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[1301] User (child, parent, teacher) usage processing steps

[1302] Step 1: Child access

[1303] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback.

[1304] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[1305] Step 2: Use by parents and teachers

[1306] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and the feedback provided by the generative model.

[1307] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[1308] Example 1

[1309] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1310] In today's educational environment, children lack the means to visually grasp their goals and effectively manage their progress. It is also difficult for parents and teachers to grasp children's learning progress in real time and provide appropriate feedback. This creates a problem of lowering children's motivation to learn.

[1311] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1312] In this invention, the server includes means for inputting and saving a child's goal and progress information, means for receiving and saving progress data sent from the terminal, means for building and updating a generative AI model based on the collected data, means for generating individual feedback messages based on the generative AI model, and means for providing the generated feedback to parents and teachers. This allows parents and teachers to visually grasp the child's goal achievement status, check the progress in real time, and provide appropriate feedback.

[1313] "Goals" are specific outcomes or targets for learning or activities that a child wants to achieve.

[1314] "Progress information" refers to the process and results of a child's learning and activities toward a goal.

[1315] A "storage means" is a method or device for permanently recording data and making it available for later reference.

[1316] "Means for receiving" refers to a method or device for receiving information or data sent from outside.

[1317] A "generative AI model" is an algorithm that learns from collected data and makes future predictions and analyses.

[1318] "Individual feedback messages" are messages that provide specific advice or next steps based on the user's progress and goals.

[1319] "Means for providing" refers to a method or device for delivering the generated information or message to users or related parties.

[1320] A "visual display means" is a method or apparatus for displaying data or information on a screen or device in a manner that is easy for a user to understand.

[1321] A "Web interface" is a collection of screens and input devices that allow users to access and operate a system via the Internet.

[1322] This invention provides a learning support system that allows children to visually grasp their goals and visualize their progress through feedback generated by a generative AI model. The system is primarily composed of a server and a terminal (tablet).

[1323] Server-side program

[1324] The server performs the following process.

[1325] 1. Managing your user profile

[1326] When a user registers, the server stores their profile information, such as their name, age, grade, and goals, in a database. This information serves as the basic data for building a generative model.

[1327] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[1328] 2. Receiving progress data

[1329] The server periodically receives progress data (e.g., test scores and assignment completion rates) sent from the device and stores it in a database.

[1330] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[1331] 3. Building and updating the generative model

[1332] The server builds a generative AI model based on the collected data, and when new progress data is received, it updates the generative model, enabling more accurate predictions.

[1333] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[1334] 4. Generate feedback

[1335] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[1336] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[1337] 5. Storing and Providing Feedback

[1338] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1339] Example: Parents and teachers can view children's feedback through a web interface.

[1340] Terminal (tablet) side program

[1341] The terminal performs the following process:

[1342] 1. Data Entry

[1343] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1344] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[1345] 2. Sending progress data

[1346] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1347] Example: When a child enters the test results, the device immediately sends the data to the server.

[1348] 3. Collaboration with generative models

[1349] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1350] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[1351] 4. Viewing Feedback

[1352] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1353] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1354] 5. Providing information to parents and teachers

[1355] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1356] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[1357] User (child, parent, teacher) usage

[1358] The user uses the system as follows:

[1359] 1. Use by Children

[1360] Children enter their daily goal achievements into a tablet and check their progress data.

[1361] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[1362] 2. Use by parents and teachers

[1363] Parents and teachers use a tablet or web interface to view a child's progress data and the feedback provided by the generative model.

[1364] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[1365] Specific examples

[1366] A user registers "Yamada Taro" in the system, stating that "my goal is to become a scientist." One week later, Yamada Taro enters "Test results: 90 points in math, 85 points in English, 92 points in science" into his device. The device immediately sends the data to the server. The server receives the new data and stores it in a database. The server updates the generative model, analysing that "to achieve the next goal, you need to improve your English grade by 5 points," and stores this feedback in the database. The device receives the feedback from the server and displays on the dashboard, "Your current science grade is 92 points. Your next goal is to improve your English grade to 90 points." Parents can view all the feedback using a web interface.

[1367] Prompt Sentence Examples

[1368] "Register a new user profile. Enter your name, age, grade, and goals."

[1369] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1370] Step 1:

[1371] Register a new user profile

[1372] Input: The user enters their name, age, grade, and goal.

[1373] Action: The user enters the required information into the web form (e.g., "Name: Yamada Taro, Age: 12, Grade: 1st year of junior high school, Goal: Scientist") and presses the submit button.

[1374] Data processing: The server receives the entered data, checks the format, and then saves it in the user table in the database.

[1375] Output: The profile information is saved and a success message is displayed to the user.

[1376] Step 2:

[1377] Entering and submitting progress data

[1378] Input: Children input their daily progress data, such as test results, into the tablet.

[1379] Action: The child enters their test results in the input field: "Math: 90 points, English: 85 points, Science: 92 points" and presses the save button.

[1380] Data processing: The device saves the input data in local storage and immediately sends it to the server.

[1381] Output: Progress data is sent to the server and saved to local storage.

[1382] Step 3:

[1383] Receiving and saving progress data

[1384] Input: Progress data sent from the device.

[1385] Operation: The server receives HTTP requests from the terminal, interprets the data, and inserts it into a progress table in the database.

[1386] Data processing: Convert the received progress data and add it to the database.

[1387] Output: The progress data is saved in the database and a success response is returned to the device.

[1388] Step 4:

[1389] Updating the generative model

[1390] Input: New progress data stored in the database.

[1391] How it works: The server starts the learning process for the generative AI model based on new progress data. It retrains the existing model with the latest data.

[1392] Data processing: The generative AI model learns from all collected data to improve prediction accuracy.

[1393] Output: The updated generative model is saved and used for the next feedback generation.

[1394] Step 5:

[1395] Generate feedback

[1396] Input: Updated generative model and new progress data.

[1397] How it works: The server uses the generative model to analyze the progress data and create personalized feedback messages.

[1398] Data processing: A generative AI model compares the user's progress with their goals and calculates the necessary advice (e.g., "You need to improve your English grade by 5 points to achieve your next goal").

[1399] Output: The generated feedback messages are stored in a database.

[1400] Step 6:

[1401] Receiving and viewing feedback

[1402] Input: The feedback message sent by the server.

[1403] Operation: The device receives the feedback sent from the server, stores it in local storage, and visually displays it to the user.

[1404] Data processing: Visualize the received feedback in the form of a dashboard on the device.

[1405] Output: A message like "Your current science grade is 92. Your next goal is to get your English grade to 90." will be displayed.

[1406] Step 7:

[1407] Providing information to parents and teachers

[1408] Input: A request for parents and teachers to review feedback.

[1409] How it works: A parent or teacher submits a request to review information via a web interface or tablet.

[1410] Data processing: The server receives the request, retrieves the relevant feedback information from the database, and displays it.

[1411] Output: Parents and teachers can view progress and feedback information in a dashboard format.

[1412] (Application example 1)

[1413] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1414] Conventional shopping support systems have difficulty monitoring users' purchasing behavior in real time and providing individualized feedback. This has resulted in insufficient support for users to achieve their shopping goals. It has also made it difficult for parents and teachers to effectively monitor their children's learning progress and provide appropriate advice. In light of these circumstances, there is a need for a more comprehensive system that can help users achieve both their purchasing and learning goals.

[1415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1416] In this invention, the server includes means for inputting and saving a child's goals and progress data, means for constructing and updating a generative model based on the collected data, means for generating feedback based on the constructed generative model, means for providing the generated feedback to parents and teachers, means for inputting and saving a user's purchasing goals and purchasing data, means for constructing and updating a generative model based on the collected purchasing data, means for generating purchasing feedback based on the constructed generative model, and means for providing the generated feedback to the user. This enables real-time support for users to achieve their shopping goals, and allows parents and teachers to properly understand their children's learning progress and provide appropriate advice.

[1417] "Progress data" is data that indicates the progress of the goals set by the user.

[1418] A "generative model" is a model built based on collected data to predict and analyze user behavior and progress.

[1419] "Feedback" refers to advice or information provided to users, parents, or teachers based on the generative model.

[1420] "Tools provided to parents and teachers" refers to functions that allow parents and teachers to check information about their children's learning progress and goal achievement.

[1421] "Purchase Goals" refers to specific shopping goals or plans set by a user.

[1422] "Purchase data" refers to data such as information about the products actually purchased by the user, the amount, and the date and time of purchase.

[1423] "Purchase feedback" refers to advice and information provided to users regarding their purchasing behavior based on generative models.

[1424] This invention is a system that helps users set shopping goals and track their progress in real time to help them achieve them. Using a server and a device (smartphone), the system collects user purchasing data and provides feedback using a generative AI model.

[1425] The server performs the following processes: When a user registers, their purchasing goals and personal information are saved in a database. This will later become the basic data for building a generative model. For example, if a user enters "keep my monthly shopping budget under 30,000 yen," the server records this in the database.

[1426] Next, the purchase data entered on the terminal (purchase date and time, product name, price, etc.) is periodically sent to the server. For example, if a user enters data such as "Purchase a T-shirt for 2,500 yen," the server receives this and stores it in a database.

[1427] The server builds and updates a generative AI model based on the collected data. This model is built using TensorFlow and is updated each time new purchase data is entered. The model predicts the user's progress based on the latest data and provides specific feedback for achieving the next goal. For example, feedback such as "To achieve next month's budget, be careful not to exceed 10,000 yen remaining this month" may be generated.

[1428] The generated feedback is stored in a database and provided to the user. In addition to receiving feedback via smartphone, users also have access to a dashboard that allows them to visually check their purchasing goals and progress. The screen displays the remaining amount of a pre-set budget and a list of recently purchased items.

[1429] The generated feedback may also be provided in a form that can be viewed by parents, teachers, and other relevant parties. In this case, parents and teachers can view the feedback through a web interface and provide appropriate advice to the user.

[1430] For example, if a user sets a monthly shopping budget of ¥30,000, the app collects purchasing data, visually displays progress, and the generative AI model provides appropriate advice, such as, "Your remaining budget for this month is ¥10,000. We recommend you be careful with your next purchase."

[1431] An example of a prompt to be input to a generative AI model could be, "Based on the user's current purchasing data, estimate the next purchase budget and generate feedback."

[1432] This allows users to efficiently manage their purchasing goals, check their progress in real time, and support them in achieving their goals. Parents and teachers can also keep track of their children's learning progress and provide appropriate advice.

[1433] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1434] Step 1:

[1435] The user enters their shopping goals and personal information.

[1436] Input: Shopping goal set by the user (e.g., less than 30,000 yen per month) and personal information (e.g., name, age, budget)

[1437] Output: User profile information stored in the database

[1438] Specific operation: The user enters their goals and personal information into the smartphone application, which is then sent to the server and recorded in a database.

[1439] Step 2:

[1440] The terminal inputs the purchase data.

[1441] Input: Information about the product purchased by the user (e.g., T-shirt, 2,500 yen, purchase date and time, etc.)

[1442] Output: Purchasing data stored in a database

[1443] Specific operation: When a user enters information about a purchased item into the terminal, this information is sent to the server and stored in a database.

[1444] Step 3:

[1445] The server collects purchasing data and builds and updates the generative AI model.

[1446] Input: Purchasing data stored in the database

[1447] Output: An updated generative AI model

[1448] How it works: The server periodically collects purchasing data from a database and builds and updates a generative AI model using TensorFlow, etc. The accuracy of the model improves with each new piece of data.

[1449] Step 4:

[1450] The server generates feedback based on the generative AI model.

[1451] Input: Updated generative AI model and the user's latest purchase data

[1452] Output: The generated feedback message

[1453] Specific operation: The server uses the generated AI model to analyze the user's purchasing behavior and generate feedback such as, "To achieve next month's budget, you need to be careful not to exceed 10,000 yen remaining."

[1454] Step 5:

[1455] The server generates feedback, stores it in a database, and provides it to the terminal.

[1456] Input: The generated feedback message

[1457] Output: Feedback displayed on the user's smartphone

[1458] Specific operation: The server stores the generated feedback in a database and sends it to the user's device for display. For example, the user's application screen might display "This month's remaining budget is 10,000 yen."

[1459] Step 6:

[1460] The device displays visual feedback to the user.

[1461] Input: Feedback message sent by the server

[1462] Output: Feedback information displayed on the dashboard

[1463] What it does: The device displays the feedback it receives in real time in a dashboard format. By opening the app, users can check their current remaining budget, recent purchase history, and more at a glance.

[1464] Step 7:

[1465] Access an interface where parents and teachers can review feedback.

[1466] Input: Parent or teacher request

[1467] Output: Child's purchasing feedback displayed in the web interface

[1468] Specific operation: When a parent or teacher sends a request to check feedback information through a web interface, the server provides the feedback data, allowing the parent or teacher to check their child's purchasing status and goal achievement status.

[1469] These steps enable users to receive real-time support to efficiently achieve their purchasing goals, while parents and teachers can effectively track learning progress and provide appropriate advice.

[1470] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1471] The present invention can be implemented in the following manner.

[1472] System Overview

[1473] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback from a generative model. It also combines an emotion engine that recognizes the user's emotions to further increase children's motivation. The system aims to increase children's motivation for learning and activities, with the users (children, parents, and teachers) participating in their respective roles. The system primarily consists of a server, a terminal (tablet), and an emotion engine.

[1474] Server-side program

[1475] The server performs the following process:

[1476] 1. Managing your user profile

[1477] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[1478] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[1479] 2. Receiving progress data

[1480] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[1481] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[1482] 3. Building and updating the generative model

[1483] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[1484] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[1485] 4. Generate feedback

[1486] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[1487] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[1488] 5. Storing and Providing Feedback

[1489] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1490] Example: Parents and teachers can view children's feedback through a web interface.

[1491] Terminal (tablet) side program

[1492] The terminal performs the following process:

[1493] 1. Data Entry

[1494] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1495] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[1496] 2. Sending progress data

[1497] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1498] Example: When a child enters the test results, the device immediately sends the data to the server.

[1499] 3. Collaboration with generative models

[1500] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1501] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[1502] 4. Viewing Feedback

[1503] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1504] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1505] 5. Providing information to parents and teachers

[1506] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1507] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[1508] Emotion Engine

[1509] The emotion engine performs the following processing:

[1510] 1. User Emotion Recognition

[1511] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[1512] Example: Using a camera while a child is learning, the system can recognize emotional states such as "happy" or "tired" from their facial expressions.

[1513] 2. Sending Emotional Data

[1514] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[1515] Example: If the emotion engine recognizes the emotion "fun," it sends that data to the server, which then generates an encouraging message in the next feedback, saying, "You're doing your best, so keep it up next time!"

[1516] 3. Providing emotional information to parents and teachers

[1517] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[1518] Example: A parent learns through an emotion interface that their child is perceived as "tired" and advises them to moderate their learning for the day.

[1519] User (child, parent, teacher) usage

[1520] The user uses the system as follows:

[1521] 1. Use by Children

[1522] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback. At the same time, the emotion engine analyzes their facial expressions and voice to understand their emotional state.

[1523] Example: A child enters "I will complete all my math homework today" and sees the result in feedback. If the emotion engine recognizes that they are "tired," that data will also be reflected in the feedback.

[1524] 2. Use by parents and teachers

[1525] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and feedback provided by the generative model, as well as the emotional state recognized by the emotion engine.

[1526] Example: A parent may discover that their child has a strong aptitude for science and provide specific advice to further support them. Then, through the emotion engine, they may learn that their child is enjoying the activity and encourage them to continue learning at this rate.

[1527] The above is an embodiment of the present invention.

[1528] The processing flow will be explained below.

[1529] Server-side program processing steps

[1530] Step 1: Manage your user profile

[1531] When a user registers, the server stores the user's profile information (name, age, grade, goals) in a database.

[1532] Example: If a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this information in a database.

[1533] Step 2: Receiving progress data

[1534] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[1535] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[1536] Step 3: Building and updating the generative model

[1537] The server builds a generative model based on the collected data and updates it as new progress data is received.

[1538] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[1539] Step 4: Generate feedback

[1540] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[1541] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[1542] Step 5: Save and provide feedback

[1543] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1544] Example: Parents and teachers can view children's feedback through a web interface.

[1545] Processing steps of the terminal (tablet) program

[1546] Step 1: Data entry

[1547] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1548] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[1549] Step 2: Sending progress data

[1550] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1551] Example: When a child enters the test results, the device immediately sends the data to the server.

[1552] Step 3: Working with the generative model

[1553] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1554] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[1555] Step 4: View your feedback

[1556] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1557] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1558] Step 5: Inform parents and teachers

[1559] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1560] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[1561] Emotion Engine Processing Steps

[1562] Step 1: Recognizing user emotions

[1563] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[1564] Example: Using a camera while a child is learning, the system can recognize emotional states such as "happy" or "tired" from their facial expressions.

[1565] Step 2: Sending emotion data

[1566] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[1567] Example: If the emotion engine recognizes the emotion "fun," it sends that data to the server, which then generates an encouraging message in the next feedback, saying, "You're doing your best, so keep it up next time!"

[1568] Step 3: Providing emotional information to parents and teachers

[1569] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[1570] Example: A parent learns through an emotion interface that their child is perceived as "tired" and advises them to moderate their learning for the day.

[1571] User (child, parent, teacher) processing steps

[1572] Step 1: Child access

[1573] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback. At the same time, the emotion engine analyzes their facial expressions and voice to understand their emotional state.

[1574] Example: A child enters "I will complete all my math homework today" and sees the result in feedback. If the emotion engine recognizes that they are "tired," that data will also be reflected in the feedback.

[1575] Step 2: Use by parents and teachers

[1576] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and feedback provided by the generative model, as well as the emotional state recognized by the emotion engine.

[1577] Example: A parent may discover that their child has a strong aptitude for science and provide specific advice to further support them. Then, through the emotion engine, they may learn that their child is enjoying the activity and encourage them to continue learning at this rate.

[1578] Example 2

[1579] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1580] In today's educational environment, it is important for children to set goals independently and manage their progress toward them in order to improve their learning outcomes. However, it is difficult for children to accurately grasp their own progress and decide on their next actions based on that. Furthermore, there are few ways for parents and teachers to grasp children's learning situation and emotional state in real time, making it difficult to provide appropriate feedback and support. Furthermore, a lack of feedback that takes emotional changes into account can lead to a decline in children's motivation. There is a need for a system that can solve these issues and effectively promote learning while maintaining children's motivation.

[1581] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1582] In this invention, the server includes: means for inputting and saving a child's goals and progress data; means for constructing and updating a generative model based on the collected data; means for generating feedback based on the constructed generative model; means for providing the generated feedback to parents and teachers; means for recognizing and digitizing the user's emotions; means for transmitting the recognized emotional data to the server; and means for providing an interface through which parents and teachers can check the feedback. This facilitates children's goal setting and progress management, and enables children to maintain their learning motivation through accurate feedback using the generative AI model. Furthermore, feedback that takes emotional data into account allows parents and teachers to grasp a child's learning situation and emotional state in real time and provide appropriate support.

[1583] "Goals" refer to the specific objectives of learning or activities that a child is trying to achieve.

[1584] "Progress data" refers to information that shows a child's achievement level and learning results toward the goals they have set.

[1585] "Generative model" refers to a machine learning model that is built and updated based on collected progress data.

[1586] "Feedback" refers to messages based on generative models that provide advice and evaluation of the user's progress and next goals.

[1587] An "emotion engine" is a system that analyzes information obtained from the device's sensors and camera, and recognizes and digitizes the user's emotional state.

[1588] "User" refers to all users of this system, including children, parents, and teachers.

[1589] "Interface" refers to the GUI (graphical user interface) or web interface that allows users to interact with the system.

[1590] MODE FOR CARRYING OUT THE INVENTION

[1591] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback from a generative model. It also combines an emotion engine that recognizes the user's emotions to further increase children's motivation. This system involves users (children, parents, and teachers) in their respective roles, aiming to increase children's motivation for learning and activities.

[1592] System Configuration

[1593] The system mainly consists of a server, a terminal (tablet), and an emotion engine. The specific roles of each component are as follows:

[1594] server

[1595] The server manages the child's goals and progress data, and generates and provides feedback using a generative AI model. The server manages user profiles, receives progress data, builds and updates the generative model, generates feedback, and stores and provides the feedback. A typical server machine is used as the hardware, and Python, MySQL, TensorFlow, and Flask are used as the software.

[1596] User profile management: The server stores profile information in a MySQL database when a new user registers.

[1597] Receiving progress data: The server periodically receives progress data sent from the device and stores it in a MySQL database.

[1598] Building and updating generative models: The server builds and updates generative AI models using TensorFlow based on the collected data.

[1599] Feedback generation: The server uses the generative AI model to generate feedback messages and save them in JSON format.

[1600] Storing and serving feedback: The server stores the generated feedback in a MySQL database and serves it in a web interface via Flask.

[1601] Device (tablet)

[1602] The terminal is a device that allows users to input goals and progress data and display feedback. Specifically, we will use a tablet terminal and develop applications for Android and iOS.

[1603] Data Entry: The device stores user-entered goals and daily progress information in local storage. For example, if a child enters "my goal for today is to complete all my math homework," that information is stored in the device's SQLite database.

[1604] Sending progress data: The device automatically sends progress data to the server at regular intervals. For example, when test results are entered, the device sends the data to the server.

[1605] Interaction with generative models: The device receives feedback from the server and stores it in local storage. For example, if a message is received saying, "Your next goal is to get 90 points in math," it will be stored on the device.

[1606] Displaying feedback: The device visually displays the received feedback to the user, for example, "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1607] Providing information to parents and teachers: The device will display appropriate feedback information when requested by a parent or teacher.

[1608] Emotion Engine

[1609] The emotion engine is responsible for recognizing and digitizing the user's emotions. Specifically, it uses the device's camera and microphone to analyze the user's facial expressions and tone of voice, and uses software modules (such as OpenCV and TensorFlow) to recognize the user's emotional state.

[1610] User emotion recognition: The emotion engine analyzes the user's facial expressions and tone of voice to recognize their current emotional state. For example, a camera can recognize a child's facial expression while they are studying and analyze their emotions, such as "happy" or "tired."

[1611] Emotion data transmission: The emotion engine transmits the recognized emotion data to the server via the device, and the server integrates this data into the generative AI model. For example, if the emotion "fun" is recognized, the data is transmitted to the server and reflected in the feedback.

[1612] Providing emotional information to parents and teachers: The emotional state recognized by the emotion engine is also displayed in an interface used by parents and teachers. Parents can see through the interface that their child is recognized as tired and can advise them to moderate their studying for the day.

[1613] Examples and prompts

[1614] Examples of use for children:

[1615] Daily goal achievement and progress are entered into the tablet, and the generated feedback is then confirmed. At the same time, the emotion engine analyzes facial expressions and voice to understand the child's emotional state. A child may enter "I will complete all my math homework today," and the result is confirmed in the feedback. If the emotion engine recognizes that the child is "tired," this data is also reflected in the feedback.

[1616] Examples of parent and teacher use cases:

[1617] Using a tablet or web interface, parents can view their child's progress data, feedback provided by the generative model, and the emotional state recognized by the emotion engine. The parent can see that their child has a strong aptitude for science and provide specific advice to further support them. The emotion engine can also tell them that their child is having fun and encourage them to continue learning at this pace.

[1618] Example prompt:

[1619] Child prompt: "What is your goal for today (e.g., complete all math assignments)?"

[1620] Child prompt: "Enter your test scores (e.g., Math: 90, English: 85)."

[1621] Parent prompt: "Do you want to check in on your child's progress? (Yes / No)"

[1622] Parent prompt: "Would you like to check in with your child's emotional state? (Yes / No)"

[1623] Teacher prompt: "Would you like to review student progress data? (Yes / No)"

[1624] Teacher prompt: "Would you like to review student feedback? (Yes / No)"

[1625] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1626] Server-side program

[1627] Step 1: Manage your user profile

[1628] Specific operation: The server receives profile information (name, age, grade, goals) submitted through the new user registration form and stores it in an SQL database (e.g., MySQL).

[1629] Input: New user registration information (e.g. name, age, grade, goal)

[1630] Data processing: Analyzes the received information and executes INSERT queries to the database to store the information.

[1631] Output: User information saved in the database

[1632] Step 1:

[1633] The server saves the profile information received from the new user registration form into a MySQL database using an INSERT query, which stores the user's basic information in the database.

[1634] Step 2: Receiving progress data

[1635] Specific operation: The server periodically receives progress data (e.g., test scores and task completion levels) sent from the device and stores it in an SQL database.

[1636] Input: Progress data sent from the device (e.g. Math: 90 points, English: 85 points, Science: 92 points)

[1637] Data processing: Parsing the received data and running INSERT or UPDATE queries in the database to store the information.

[1638] Output: Updated progress data saved to the database

[1639] Step 2:

[1640] The server receives the progress data sent from the device and saves it in a MySQL database using INSERT or UPDATE queries, which stores the progress data in the database.

[1641] Step 3: Building and updating the generative model

[1642] Specific operation: The server builds and updates a generative model using a Python machine learning library (e.g., TensorFlow) based on the collected progress data.

[1643] Input: Progress data stored in the database

[1644] Data processing: Preprocessing data, training machine learning models, and building and updating generative models.

[1645] Output: Latest generated model file

[1646] Step 3:

[1647] The server uses TensorFlow to build a generative AI model based on the collected progress data and updates the model based on new data, resulting in an up-to-date generative model.

[1648] Step 4: Generate feedback

[1649] Specific behavior: The server generates feedback messages based on the user's progress and goals using the latest generative model.

[1650] Input: The latest generative model and user progress data

[1651] Data processing: Use the model to make predictions and generate appropriate feedback messages.

[1652] Output: Feedback message (e.g., "You need to improve your English grade by 5 points to achieve your next goal")

[1653] Step 4:

[1654] The server analyzes the progress and goals based on the latest generative model and uses natural language processing to construct feedback messages, which are then provided to the user.

[1655] Step 5: Save and provide feedback

[1656] What it does: The server stores the generated feedback in a database and makes it available for parents and teachers to review via a web interface.

[1657] Input: The generated feedback message

[1658] Data processing: storing feedback messages in a database and preparing the data for display in the web interface.

[1659] Output: Feedback stored in the database and displayed in the web interface

[1660] Step 5:

[1661] The server stores the generated feedback in a MySQL database using INSERT queries and makes it available to parents and teachers through a web interface using Flask, allowing them to review the generated feedback.

[1662] Terminal (tablet) side program

[1663] Step 1: Data entry

[1664] What it does: The device stores the goals and daily progress information entered by the user in local storage (e.g., SQLite).

[1665] Input: User-supplied goals and progress data (e.g., today's goal is to complete all math assignments)

[1666] Data processing: Analyzes input information and saves it to local storage.

[1667] Output: Goal and progress data stored in local storage

[1668] Step 1:

[1669] The device receives the goal and progress data entered by the user and stores it in an SQLite database, which stores the user-entered data in local storage.

[1670] Step 2: Sending progress data

[1671] Specific operation: The device automatically sends progress data to the server at regular intervals.

[1672] Input: Progress data stored in local storage

[1673] Data processing: Format the data and send it to the server as an HTTP request.

[1674] Output: Progress data sent to the server

[1675] Step 2:

[1676] The device periodically sends the progress data stored in the local storage to the server, whereby the progress data is uploaded to the server.

[1677] Step 3: Working with the generative model

[1678] Specific operation: The device receives the latest feedback message from the server and stores it in local storage.

[1679] Input: Feedback message sent by the server

[1680] Data processing: Parse the feedback messages and save them to local storage.

[1681] Output: Feedback message saved to local storage

[1682] Step 3:

[1683] The terminal receives the latest feedback message sent by the server and stores it in local storage, thereby storing the feedback on the terminal.

[1684] Step 4: View your feedback

[1685] Specific behavior: The device visually displays the received feedback to the user.

[1686] Input: Feedback message stored in local storage

[1687] Data processing: Formatting feedback messages for visual display.

[1688] Output: Feedback message displayed on the screen

[1689] Step 4:

[1690] The device visually displays the feedback messages stored in the local storage in the form of a dashboard, allowing the user to review the feedback.

[1691] Step 5: Inform parents and teachers

[1692] Specific behavior: The device will display feedback information appropriately when a parent or teacher requests confirmation.

[1693] Input: Parent and teacher requests

[1694] Data processing: Retrieve feedback information from local storage and display it in an appropriate format.

[1695] Output: Feedback information displayed on the screen

[1696] Step 5:

[1697] Upon receiving a request from a parent or teacher, the device will display the feedback information in an appropriate format, which can then be viewed by the parent or teacher on their dashboard.

[1698] Emotion Engine

[1699] Step 1: Recognizing user emotions

[1700] Specific operation: The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone to recognize their current emotional state.

[1701] Input: User facial and voice data captured by camera and microphone

[1702] Data processing: Analyze facial and voice data to classify and recognize emotions.

[1703] Output: Perceived emotional state (e.g., happy, tired)

[1704] Step 1:

[1705] The emotion engine recognizes the user's current emotional state by capturing and analyzing the user's facial expressions and voice using a camera and microphone, and the user's emotions are then recognized as data.

[1706] Step 2: Sending emotion data

[1707] Specific operation: The emotion engine sends the recognized emotion data to the server via the device.

[1708] Input: Recognized emotion data

[1709] Data processing: Emotion data is formatted and sent to the server via an HTTP request.

[1710] Output: Emotion data sent to the server

[1711] Step 2:

[1712] The emotion engine sends the recognized emotion data to the server via the device, whereby the emotion data is uploaded to the server.

[1713] Step 3: Providing emotional information to parents and teachers

[1714] Specific operation: The emotional state recognized by the emotion engine is displayed on an interface used by parents and teachers.

[1715] Input: Emotion data sent from the server

[1716] Data processing: Formatting emotion data for display on the interface.

[1717] Output: Emotion information displayed on the interface

[1718] Step 3:

[1719] The emotional data recognized by the emotion engine is displayed on the interface for parents and teachers, allowing them to understand the child's mental state, and this allows parents and teachers to check emotional information in real time.

[1720] (Application example 2)

[1721] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1722] Conventional learning support systems only provide feedback based on a child's progress data and do not take into account the child's emotional state. As a result, children's motivation tends to decline, and there are issues with their ability to continue learning. There was also a need for a system that would allow parents and teachers to support children's emotional state as well as their academic progress.

[1723] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1724] In this invention, the server includes means for inputting and saving a child's goals and progress data, means for constructing and updating a generative model based on the collected data, means for generating feedback based on the constructed generative model, means for providing the generated feedback to parents and teachers, means for analyzing emotional states and incorporating them as data, and means for reflecting the emotional data in the feedback. This allows for the provision of feedback from both the perspectives of learning progress and emotional states, improving children's motivation and providing continuous learning support.

[1725] "Means for inputting and saving children's goals and progress data" refers to a function that allows children to input their learning goals and daily progress, and to electronically record and save that data.

[1726] "Means for building and updating generative models based on collected data" refers to a function that collects data on children's progress, creates a generative model based on this data to predict learning progress and outcomes, and keeps the model up to date every time the data is updated.

[1727] "Means for generating feedback based on the constructed generative model" is a function that generates advice and feedback regarding a child's learning progress and next goals based on information predicted and analyzed by the generative model.

[1728] "Means for providing generated feedback to parents and teachers" is a function for providing feedback on learning generated by the system so that it can be viewed by parents and teachers.

[1729] "Means for analyzing emotional state and capturing it as data" is a function that uses sensors such as a camera and microphone to analyze a child's current emotional state from their facial expressions and tone of voice, and obtains the results as data.

[1730] "Means of using emotional data to reflect in feedback" is a function that adjusts the feedback content of the generative model based on the acquired emotional data, and provides appropriate advice and support from an emotional perspective.

[1731] The "means for visually displaying" is a function for displaying the generated feedback and progress status to the user in a visually easy-to-understand format (graphs, messages, etc.).

[1732] "Means for providing an interface" refers to the function of providing a user interface that allows parents and teachers to access the system and check feedback and progress.

[1733] MODE FOR CARRYING OUT THE INVENTION

[1734] System Overview

[1735] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback using a generative model. The system aims to improve children's motivation for learning and activities, with users (children, parents, and teachers) participating in their respective roles. The system is primarily composed of a server, a device (smartphone), and an emotion engine.

[1736] Server-side program

[1737] The server does the following:

[1738] 1. Managing your user profile

[1739] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[1740] 2. Receiving progress data

[1741] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[1742] 3. Building and updating the generative model

[1743] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[1744] 4. Generate feedback

[1745] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals, and also incorporates emotional data into the feedback.

[1746] 5. Storing and Providing Feedback

[1747] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1748] Terminal (smartphone) side program

[1749] The terminal does the following:

[1750] 1. Data Entry

[1751] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1752] 2. Sending progress data

[1753] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1754] 3. Collaboration with generative models

[1755] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1756] 4. Viewing Feedback

[1757] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1758] 5. Providing information to parents and teachers

[1759] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1760] Emotion Engine

[1761] The emotion engine does the following:

[1762] 1. User Emotion Recognition

[1763] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[1764] 2. Sending Emotional Data

[1765] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[1766] 3. Providing emotional information to parents and teachers

[1767] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[1768] Examples and prompts

[1769] Examples:

[1770] Children take English tests at brick-and-mortar stores. Test results are entered using a smartphone, and emotions during learning are monitored via a camera. Parents can view test results and emotional data in real time using a dedicated app.

[1771] Prompt statement:

[1772] "With the real-time learning assistance assistant, you set your goals and manually input your test results. Then, it uses the camera to collect emotional data and send it to a server for feedback."

[1773] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1774] Step 1:

[1775] Enter your child's goals and progress data

[1776] The user (child) uses a smartphone application to input goals and progress data. For example, the user might input, "Today's goal is to complete all of my math homework." The input data is saved in the device's local storage. The input data includes goal information and progress information.

[1777] Step 2:

[1778] Sending progress data to the server

[1779] The terminal periodically sends the progress data entered by the user to the server. In particular, when new data is entered, it is automatically sent to the server. The server stores the received data in a database. The input data is progress information, and the output is progress data stored in the server-side database.

[1780] Step 3:

[1781] Building and updating generative models

[1782] The server builds and updates a generative model based on the collected data. For example, it creates a generative model that says, "Your math grade is 90, and your next goal should be 95." The input data is the saved progress data, and the output is the updated generative model.

[1783] Step 4:

[1784] Generate feedback

[1785] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals, for example, "Your next goal is to get a 95 in math." The input data are the generative model and progress data, and the output is the generated feedback message.

[1786] Step 5:

[1787] Storing and Providing Feedback

[1788] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review. For example, parents and teachers can review their child's feedback through the web interface. The input data are the generated feedback messages, and the output are the feedback messages stored in the database and those viewable via the web interface.

[1789] Step 6:

[1790] Acquiring emotion data

[1791] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone. For example, a child may use the camera while studying, and the engine can recognize emotional states such as "happy" or "tired" from their facial expressions. The input data is real-time video and audio from the camera and microphone, and the output is analyzed emotional data.

[1792] Step 7:

[1793] Sending emotion data to the server

[1794] The emotion engine sends the recognized emotion data to the server via the device. The server integrates this data into the generative model and reflects it in the feedback. For example, it generates an encouraging message such as "You're doing well, so keep it up next time!" The input data is emotion data, and the output is the integrated generative model and an updated feedback message.

[1795] Step 8:

[1796] View Feedback

[1797] The device visually displays the received feedback to the user, for example, "Your current science grade is 92. Your next goal is to get your English grade to 90." The input data is the generated feedback message, and the output is the feedback visually displayed to the user.

[1798] Step 9:

[1799] Providing information to parents and teachers

[1800] When a device receives a request from a parent or teacher to check, it displays the appropriate feedback and emotional information. For example, when a parent sends a request, the device displays the user's progress and feedback in a dashboard format. The input data is the request from the parent or teacher and the saved feedback and emotional information, and the output is the visually displayed information.

[1801] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1802] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1803] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1804] [Fourth embodiment]

[1805] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1806] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1807] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1808] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1809] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1810] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1811] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1812] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1813] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1814] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1815] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1816] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1817] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1818] The present invention can be implemented in the following manner.

[1819] System Overview

[1820] This invention provides a learning support system that allows children to visually grasp their goals and visualize their progress through feedback using a generative model. This system aims to improve children's motivation for learning and activities, with children, parents, and teachers each playing their respective roles. The system is primarily composed of a server and a terminal (tablet).

[1821] Server-side program

[1822] The server performs the following process:

[1823] 1. Managing your user profile

[1824] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[1825] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[1826] 2. Receiving progress data

[1827] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[1828] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[1829] 3. Building and updating the generative model

[1830] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[1831] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[1832] 4. Generate feedback

[1833] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[1834] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[1835] 5. Storing and Providing Feedback

[1836] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1837] Example: Parents and teachers can view children's feedback through a web interface.

[1838] Terminal (tablet) side program

[1839] The terminal performs the following process:

[1840] 1. Data Entry

[1841] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1842] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[1843] 2. Sending progress data

[1844] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1845] Example: When a child enters the test results, the device immediately sends the data to the server.

[1846] 3. Collaboration with generative models

[1847] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1848] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[1849] 4. Viewing Feedback

[1850] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1851] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1852] 5. Providing information to parents and teachers

[1853] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1854] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[1855] User (child, parent, teacher) usage

[1856] The user uses the system as follows:

[1857] 1. Use by Children

[1858] Children enter their daily goal achievements into a tablet and check their progress data.

[1859] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[1860] 2. Use by parents and teachers

[1861] Parents and teachers use a tablet or web interface to view a child's progress data and the feedback provided by the generative model.

[1862] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[1863] The above is an embodiment of the present invention.

[1864] The processing flow will be explained below.

[1865] Server-side program processing steps

[1866] Step 1: Manage your user profile

[1867] When a user registers, the server stores the user's profile information (name, age, grade, goals) in a database.

[1868] Example: If a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this information in a database.

[1869] Step 2: Receiving progress data

[1870] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[1871] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[1872] Step 3: Building and updating the generative model

[1873] The server builds a generative model based on the collected data and updates it as new progress data is received.

[1874] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[1875] Step 4: Generate feedback

[1876] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[1877] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[1878] Step 5: Save and provide feedback

[1879] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1880] Example: Parents and teachers can view children's feedback through a web interface.

[1881] Processing steps of the terminal (tablet) program

[1882] Step 1: Data entry

[1883] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1884] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[1885] Step 2: Sending progress data

[1886] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1887] Example: When a child enters the test results, the device immediately sends the data to the server.

[1888] Step 3: Working with the generative model

[1889] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1890] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[1891] Step 4: View your feedback

[1892] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1893] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1894] Step 5: Inform parents and teachers

[1895] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1896] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[1897] User (child, parent, teacher) usage processing steps

[1898] Step 1: Child access

[1899] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback.

[1900] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[1901] Step 2: Use by parents and teachers

[1902] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and the feedback provided by the generative model.

[1903] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[1904] Example 1

[1905] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1906] In today's educational environment, children lack the means to visually grasp their goals and effectively manage their progress. It is also difficult for parents and teachers to grasp children's learning progress in real time and provide appropriate feedback. This creates a problem of lowering children's motivation to learn.

[1907] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1908] In this invention, the server includes means for inputting and saving a child's goal and progress information, means for receiving and saving progress data sent from the terminal, means for building and updating a generative AI model based on the collected data, means for generating individual feedback messages based on the generative AI model, and means for providing the generated feedback to parents and teachers. This allows parents and teachers to visually grasp the child's goal achievement status, check the progress in real time, and provide appropriate feedback.

[1909] "Goals" are specific outcomes or targets for learning or activities that a child wants to achieve.

[1910] "Progress information" refers to the process and results of a child's learning and activities toward a goal.

[1911] A "storage means" is a method or device for permanently recording data and making it available for later reference.

[1912] "Means for receiving" refers to a method or device for receiving information or data sent from outside.

[1913] A "generative AI model" is an algorithm that learns from collected data and makes future predictions and analyses.

[1914] "Individual feedback messages" are messages that provide specific advice or next steps based on the user's progress and goals.

[1915] "Means for providing" refers to a method or device for delivering the generated information or message to users or related parties.

[1916] A "visual display means" is a method or apparatus for displaying data or information on a screen or device in a manner that is easy for a user to understand.

[1917] A "Web interface" is a collection of screens and input devices that allow users to access and operate a system via the Internet.

[1918] This invention provides a learning support system that allows children to visually grasp their goals and visualize their progress through feedback generated by a generative AI model. The system is primarily composed of a server and a terminal (tablet).

[1919] Server-side program

[1920] The server performs the following process.

[1921] 1. Managing your user profile

[1922] When a user registers, the server stores their profile information, such as their name, age, grade, and goals, in a database. This information serves as the basic data for building a generative model.

[1923] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[1924] 2. Receiving progress data

[1925] The server periodically receives progress data (e.g., test scores and assignment completion rates) sent from the device and stores it in a database.

[1926] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[1927] 3. Building and updating the generative model

[1928] The server builds a generative AI model based on the collected data, and when new progress data is received, it updates the generative model, enabling more accurate predictions.

[1929] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[1930] 4. Generate feedback

[1931] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[1932] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[1933] 5. Storing and Providing Feedback

[1934] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[1935] Example: Parents and teachers can view children's feedback through a web interface.

[1936] Terminal (tablet) side program

[1937] The terminal performs the following process:

[1938] 1. Data Entry

[1939] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[1940] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[1941] 2. Sending progress data

[1942] The device periodically sends progress data to the server, especially automatically when new data is entered.

[1943] Example: When a child enters the test results, the device immediately sends the data to the server.

[1944] 3. Collaboration with generative models

[1945] The device receives feedback on the latest generative model from the server and stores it in local storage.

[1946] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[1947] 4. Viewing Feedback

[1948] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[1949] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[1950] 5. Providing information to parents and teachers

[1951] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[1952] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[1953] User (child, parent, teacher) usage

[1954] The user uses the system as follows:

[1955] 1. Use by Children

[1956] Children enter their daily goal achievements into a tablet and check their progress data.

[1957] Example: A child enters "I will complete all my math assignments today," sees the feedback, and learns that their next goal is "I will improve my English grades."

[1958] 2. Use by parents and teachers

[1959] Parents and teachers use a tablet or web interface to view a child's progress data and the feedback provided by the generative model.

[1960] Example: A parent realizes that their child has a strong aptitude for science and provides specific advice to further support them.

[1961] Specific examples

[1962] A user registers "Yamada Taro" in the system, stating that "my goal is to become a scientist." One week later, Yamada Taro enters "Test results: 90 points in math, 85 points in English, 92 points in science" into his device. The device immediately sends the data to the server. The server receives the new data and stores it in a database. The server updates the generative model, analysing that "to achieve the next goal, you need to improve your English grade by 5 points," and stores this feedback in the database. The device receives the feedback from the server and displays on the dashboard, "Your current science grade is 92 points. Your next goal is to improve your English grade to 90 points." Parents can view all the feedback using a web interface.

[1963] Prompt Sentence Examples

[1964] "Register a new user profile. Enter your name, age, grade, and goals."

[1965] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1966] Step 1:

[1967] Register a new user profile

[1968] Input: The user enters their name, age, grade, and goal.

[1969] Action: The user enters the required information into the web form (e.g., "Name: Yamada Taro, Age: 12, Grade: 1st year of junior high school, Goal: Scientist") and presses the submit button.

[1970] Data processing: The server receives the entered data, checks the format, and then saves it in the user table in the database.

[1971] Output: The profile information is saved and a success message is displayed to the user.

[1972] Step 2:

[1973] Entering and submitting progress data

[1974] Input: Children input their daily progress data, such as test results, into the tablet.

[1975] Action: The child enters their test results in the input field: "Math: 90 points, English: 85 points, Science: 92 points" and presses the save button.

[1976] Data processing: The device saves the input data in local storage and immediately sends it to the server.

[1977] Output: Progress data is sent to the server and saved to local storage.

[1978] Step 3:

[1979] Receiving and saving progress data

[1980] Input: Progress data sent from the device.

[1981] Operation: The server receives HTTP requests from the terminal, interprets the data, and inserts it into a progress table in the database.

[1982] Data processing: Convert the received progress data and add it to the database.

[1983] Output: The progress data is saved in the database and a success response is returned to the device.

[1984] Step 4:

[1985] Updating the generative model

[1986] Input: New progress data stored in the database.

[1987] How it works: The server starts the learning process for the generative AI model based on new progress data. It retrains the existing model with the latest data.

[1988] Data processing: The generative AI model learns from all collected data to improve prediction accuracy.

[1989] Output: The updated generative model is saved and used for the next feedback generation.

[1990] Step 5:

[1991] Generate feedback

[1992] Input: Updated generative model and new progress data.

[1993] How it works: The server uses the generative model to analyze the progress data and create personalized feedback messages.

[1994] Data processing: A generative AI model compares the user's progress with their goals and calculates the necessary advice (e.g., "You need to improve your English grade by 5 points to achieve your next goal").

[1995] Output: The generated feedback messages are stored in a database.

[1996] Step 6:

[1997] Receiving and viewing feedback

[1998] Input: The feedback message sent by the server.

[1999] Operation: The device receives the feedback sent from the server, stores it in local storage, and visually displays it to the user.

[2000] Data processing: Visualize the received feedback in the form of a dashboard on the device.

[2001] Output: A message like "Your current science grade is 92. Your next goal is to get your English grade to 90." will be displayed.

[2002] Step 7:

[2003] Providing information to parents and teachers

[2004] Input: A request for parents and teachers to review feedback.

[2005] How it works: A parent or teacher submits a request to review information via a web interface or tablet.

[2006] Data processing: The server receives the request, retrieves the relevant feedback information from the database, and displays it.

[2007] Output: Parents and teachers can view progress and feedback information in a dashboard format.

[2008] (Application example 1)

[2009] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2010] Conventional shopping support systems have difficulty monitoring users' purchasing behavior in real time and providing individualized feedback. This has resulted in insufficient support for users to achieve their shopping goals. It has also made it difficult for parents and teachers to effectively monitor their children's learning progress and provide appropriate advice. In light of these circumstances, there is a need for a more comprehensive system that can help users achieve both their purchasing and learning goals.

[2011] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2012] In this invention, the server includes means for inputting and saving a child's goals and progress data, means for constructing and updating a generative model based on the collected data, means for generating feedback based on the constructed generative model, means for providing the generated feedback to parents and teachers, means for inputting and saving a user's purchasing goals and purchasing data, means for constructing and updating a generative model based on the collected purchasing data, means for generating purchasing feedback based on the constructed generative model, and means for providing the generated feedback to the user. This enables real-time support for users to achieve their shopping goals, and allows parents and teachers to properly understand their children's learning progress and provide appropriate advice.

[2013] "Progress data" is data that indicates the progress of the goals set by the user.

[2014] A "generative model" is a model built based on collected data to predict and analyze user behavior and progress.

[2015] "Feedback" refers to advice or information provided to users, parents, or teachers based on the generative model.

[2016] "Tools provided to parents and teachers" refers to functions that allow parents and teachers to check information about their children's learning progress and goal achievement.

[2017] "Purchase Goals" refers to specific shopping goals or plans set by a user.

[2018] "Purchase data" refers to data such as information about the products actually purchased by the user, the amount, and the date and time of purchase.

[2019] "Purchase feedback" refers to advice and information provided to users regarding their purchasing behavior based on generative models.

[2020] This invention is a system that helps users set shopping goals and track their progress in real time to help them achieve them. Using a server and a device (smartphone), the system collects user purchasing data and provides feedback using a generative AI model.

[2021] The server performs the following processes: When a user registers, their purchasing goals and personal information are saved in a database. This will later become the basic data for building a generative model. For example, if a user enters "keep my monthly shopping budget under 30,000 yen," the server records this in the database.

[2022] Next, the purchase data entered on the terminal (purchase date and time, product name, price, etc.) is periodically sent to the server. For example, if a user enters data such as "Purchase a T-shirt for 2,500 yen," the server receives this and stores it in a database.

[2023] The server builds and updates a generative AI model based on the collected data. This model is built using TensorFlow and is updated each time new purchase data is entered. The model predicts the user's progress based on the latest data and provides specific feedback for achieving the next goal. For example, feedback such as "To achieve next month's budget, be careful not to exceed 10,000 yen remaining this month" may be generated.

[2024] The generated feedback is stored in a database and provided to the user. In addition to receiving feedback via smartphone, users also have access to a dashboard that allows them to visually check their purchasing goals and progress. The screen displays the remaining amount of a pre-set budget and a list of recently purchased items.

[2025] The generated feedback may also be provided in a form that can be viewed by parents, teachers, and other relevant parties. In this case, parents and teachers can view the feedback through a web interface and provide appropriate advice to the user.

[2026] For example, if a user sets a monthly shopping budget of ¥30,000, the app collects purchasing data, visually displays progress, and the generative AI model provides appropriate advice, such as, "Your remaining budget for this month is ¥10,000. We recommend you be careful with your next purchase."

[2027] An example of a prompt to be input to a generative AI model could be, "Based on the user's current purchasing data, estimate the next purchase budget and generate feedback."

[2028] This allows users to efficiently manage their purchasing goals, check their progress in real time, and support them in achieving their goals. Parents and teachers can also keep track of their children's learning progress and provide appropriate advice.

[2029] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2030] Step 1:

[2031] The user enters their shopping goals and personal information.

[2032] Input: Shopping goal set by the user (e.g., less than 30,000 yen per month) and personal information (e.g., name, age, budget)

[2033] Output: User profile information stored in the database

[2034] Specific operation: The user enters their goals and personal information into the smartphone application, which is then sent to the server and recorded in a database.

[2035] Step 2:

[2036] The terminal inputs the purchase data.

[2037] Input: Information about the product purchased by the user (e.g., T-shirt, 2,500 yen, purchase date and time, etc.)

[2038] Output: Purchasing data stored in a database

[2039] Specific operation: When a user enters information about a purchased item into the terminal, this information is sent to the server and stored in a database.

[2040] Step 3:

[2041] The server collects purchasing data and builds and updates the generative AI model.

[2042] Input: Purchasing data stored in the database

[2043] Output: An updated generative AI model

[2044] How it works: The server periodically collects purchasing data from a database and builds and updates a generative AI model using TensorFlow, etc. The accuracy of the model improves with each new piece of data.

[2045] Step 4:

[2046] The server generates feedback based on the generative AI model.

[2047] Input: Updated generative AI model and the user's latest purchase data

[2048] Output: The generated feedback message

[2049] Specific operation: The server uses the generated AI model to analyze the user's purchasing behavior and generate feedback such as, "To achieve next month's budget, you need to be careful not to exceed 10,000 yen remaining."

[2050] Step 5:

[2051] The server generates feedback, stores it in a database, and provides it to the terminal.

[2052] Input: The generated feedback message

[2053] Output: Feedback displayed on the user's smartphone

[2054] Specific operation: The server stores the generated feedback in a database and sends it to the user's device for display. For example, the user's application screen might display "This month's remaining budget is 10,000 yen."

[2055] Step 6:

[2056] The device displays visual feedback to the user.

[2057] Input: Feedback message sent by the server

[2058] Output: Feedback information displayed on the dashboard

[2059] What it does: The device displays the feedback it receives in real time in a dashboard format. By opening the app, users can check their current remaining budget, recent purchase history, and more at a glance.

[2060] Step 7:

[2061] Access an interface where parents and teachers can review feedback.

[2062] Input: Parent or teacher request

[2063] Output: Child's purchasing feedback displayed in the web interface

[2064] Specific operation: When a parent or teacher sends a request to check feedback information through a web interface, the server provides the feedback data, allowing the parent or teacher to check their child's purchasing status and goal achievement status.

[2065] These steps enable users to receive real-time support to efficiently achieve their purchasing goals, while parents and teachers can effectively track learning progress and provide appropriate advice.

[2066] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2067] The present invention can be implemented in the following manner.

[2068] System Overview

[2069] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback from a generative model. It also combines an emotion engine that recognizes the user's emotions to further increase children's motivation. The system aims to increase children's motivation for learning and activities, with the users (children, parents, and teachers) participating in their respective roles. The system primarily consists of a server, a terminal (tablet), and an emotion engine.

[2070] Server-side program

[2071] The server performs the following process:

[2072] 1. Managing your user profile

[2073] When a user registers, the server saves their profile information (name, age, grade, goals) in a database. This profile information will later become the basic data for building a generative model.

[2074] Example: When a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this in the database.

[2075] 2. Receiving progress data

[2076] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[2077] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[2078] 3. Building and updating the generative model

[2079] The server builds a generative model based on the collected data, and as new progress data is received, it updates the generative model, enabling more accurate predictions.

[2080] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[2081] 4. Generate feedback

[2082] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[2083] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[2084] 5. Storing and Providing Feedback

[2085] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[2086] Example: Parents and teachers can view children's feedback through a web interface.

[2087] Terminal (tablet) side program

[2088] The terminal performs the following process:

[2089] 1. Data Entry

[2090] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[2091] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[2092] 2. Sending progress data

[2093] The device periodically sends progress data to the server, especially automatically when new data is entered.

[2094] Example: When a child enters the test results, the device immediately sends the data to the server.

[2095] 3. Collaboration with generative models

[2096] The device receives feedback on the latest generative model from the server and stores it in local storage.

[2097] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[2098] 4. Viewing Feedback

[2099] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[2100] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[2101] 5. Providing information to parents and teachers

[2102] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[2103] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[2104] Emotion Engine

[2105] The emotion engine performs the following processing:

[2106] 1. User Emotion Recognition

[2107] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[2108] Example: Using a camera while a child is learning, the system can recognize emotional states such as "happy" or "tired" from their facial expressions.

[2109] 2. Sending Emotional Data

[2110] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[2111] Example: If the emotion engine recognizes the emotion "fun," it sends that data to the server, which then generates an encouraging message in the next feedback, saying, "You're doing your best, so keep it up next time!"

[2112] 3. Providing emotional information to parents and teachers

[2113] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[2114] Example: A parent learns through an emotion interface that their child is perceived as "tired" and advises them to moderate their learning for the day.

[2115] User (child, parent, teacher) usage

[2116] The user uses the system as follows:

[2117] 1. Use by Children

[2118] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback. At the same time, the emotion engine analyzes their facial expressions and voice to understand their emotional state.

[2119] Example: A child enters "I will complete all my math homework today" and sees the result in feedback. If the emotion engine recognizes that they are "tired," that data will also be reflected in the feedback.

[2120] 2. Use by parents and teachers

[2121] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and feedback provided by the generative model, as well as the emotional state recognized by the emotion engine.

[2122] Example: A parent may discover that their child has a strong aptitude for science and provide specific advice to further support them. Then, through the emotion engine, they may learn that their child is enjoying the activity and encourage them to continue learning at this rate.

[2123] The above is an embodiment of the present invention.

[2124] The processing flow will be explained below.

[2125] Server-side program processing steps

[2126] Step 1: Manage your user profile

[2127] When a user registers, the server stores the user's profile information (name, age, grade, goals) in a database.

[2128] Example: If a user enters "Yamada Taro, 12 years old, first year junior high school student, goal is to become a scientist," the server records this information in a database.

[2129] Step 2: Receiving progress data

[2130] The server periodically receives progress data (such as test scores and task completion rates) sent from the terminal and stores it in a database.

[2131] Example: When a device sends data such as "Math: 90 points, English: 85 points, Science: 92 points" to a server, the server receives this data and stores it in a database.

[2132] Step 3: Building and updating the generative model

[2133] The server builds a generative model based on the collected data and updates it as new progress data is received.

[2134] Example: A generative model predicts the user's progress based on the latest data and provides analysis results such as "To achieve your next goal, you need to improve your English grades by 5 points."

[2135] Step 4: Generate feedback

[2136] The server uses the generative model to generate personalized feedback messages based on the user's current progress and goals.

[2137] Example: Generating feedback such as, "Your science grades are excellent. Your next goal is to get your English grade above 90."

[2138] Step 5: Save and provide feedback

[2139] The server stores the generated feedback in a database and provides it via a web interface for parents and teachers to review.

[2140] Example: Parents and teachers can view children's feedback through a web interface.

[2141] Processing steps of the terminal (tablet) program

[2142] Step 1: Data entry

[2143] The device receives the goals and daily progress information entered by the user and stores it in local storage.

[2144] Example: A child types, "My goal today is to complete all my math homework," and that is saved to the device's local storage.

[2145] Step 2: Sending progress data

[2146] The device periodically sends progress data to the server, especially automatically when new data is entered.

[2147] Example: When a child enters the test results, the device immediately sends the data to the server.

[2148] Step 3: Working with the generative model

[2149] The device receives feedback on the latest generative model from the server and stores it in local storage.

[2150] Example: When feedback is sent from the server, the device receives it and stores a message such as "Your next goal is to get 90 points in math."

[2151] Step 4: View your feedback

[2152] The device visually displays the received feedback to the user, showing their current progress and next goals in the form of a dashboard.

[2153] Example: Display the message on the device screen: "Your current science grade is 92. Your next goal is to get your English grade to 90."

[2154] Step 5: Inform parents and teachers

[2155] The device will display appropriate feedback information when a parent or teacher requests confirmation.

[2156] Example: Parents or teachers can submit requests, and the device will display the user's progress and feedback in a dashboard format.

[2157] Emotion Engine Processing Steps

[2158] Step 1: Recognizing user emotions

[2159] The emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, thereby recognizing the user's current emotional state.

[2160] Example: Using a camera while a child is learning, the system can recognize emotional states such as "happy" or "tired" from their facial expressions.

[2161] Step 2: Sending emotion data

[2162] The emotion engine sends the recognized emotion data to the server via the device, which then integrates this data into the generative model and uses it as feedback.

[2163] Example: If the emotion engine recognizes the emotion "fun," it sends that data to the server, which then generates an encouraging message in the next feedback, saying, "You're doing your best, so keep it up next time!"

[2164] Step 3: Providing emotional information to parents and teachers

[2165] The emotional state recognized by the emotion engine is also displayed on an interface used by parents and teachers, allowing them to understand their child's mental state and provide appropriate support.

[2166] Example: A parent learns through an emotion interface that their child is perceived as "tired" and advises them to moderate their learning for the day.

[2167] User (child, parent, teacher) processing steps

[2168] Step 1: Child access

[2169] The user (child) enters their daily goal achievement and progress into the tablet and checks the feedback. At the same time, the emotion engine analyzes their facial expressions and voice to understand their emotional state.

[2170] Example: A child enters "I will complete all my math homework today" and sees the result in feedback. If the emotion engine recognizes that they are "tired," that data will also be reflected in the feedback.

[2171] Step 2: Use by parents and teachers

[2172] Parents and teachers use a tablet or web interface to view the user's (child's) progress data and feedback provided by the generative model, as well as the emotional state recognized by the emotion engine.

[2173] Example: A parent may discover that their child has a strong aptitude for science and provide specific advice to further support them. Then, through the emotion engine, they may learn that their child is enjoying the activity and encourage them to continue learning at this rate.

[2174] Example 2

[2175] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2176] In today's educational environment, it is important for children to set goals independently and manage their progress toward them in order to improve their learning outcomes. However, it is difficult for children to accurately grasp their own progress and decide on their next actions based on that. Furthermore, there are few ways for parents and teachers to grasp children's learning situation and emotional state in real time, making it difficult to provide appropriate feedback and support. Furthermore, a lack of feedback that takes emotional changes into account can lead to a decline in children's motivation. There is a need for a system that can solve these issues and effectively promote learning while maintaining children's motivation.

[2177] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2178] In this invention, the server includes: means for inputting and saving a child's goals and progress data; means for constructing and updating a generative model based on the collected data; means for generating feedback based on the constructed generative model; means for providing the generated feedback to parents and teachers; means for recognizing and digitizing the user's emotions; means for transmitting the recognized emotional data to the server; and means for providing an interface through which parents and teachers can check the feedback. This facilitates children's goal setting and progress management, and enables children to maintain their learning motivation through accurate feedback using the generative AI model. Furthermore, feedback that takes emotional data into account allows parents and teachers to grasp a child's learning situation and emotional state in real time and provide appropriate support.

[2179] "Goals" refer to the specific objectives of learning or activities that a child is trying to achieve.

[2180] "Progress data" refers to information that shows a child's achievement level and learning results toward the goals they have set.

[2181] "Generative model" refers to a machine learning model that is built and updated based on collected progress data.

[2182] "Feedback" refers to messages based on generative models that provide advice and evaluation of the user's progress and next goals.

[2183] An "emotion engine" is a system that analyzes information obtained from the device's sensors and camera, and recognizes and digitizes the user's emotional state.

[2184] "User" refers to all users of this system, including children, parents, and teachers.

[2185] "Interface" refers to the GUI (graphical user interface) or web interface that allows users to interact with the system.

[2186] MODE FOR CARRYING OUT THE INVENTION

[2187] This invention is a learning support system that allows children to visually grasp their goals and visualize their progress through feedback from a generative model. It also combines an emotion engine that recognizes the user's emotions to further increase children's motivation. This system involves users (children, parents, and teachers) in their respective roles, aiming to increase children's motivation for learning and activities.

[2188] System Configuration

[2189] The system mainly consists of a server, a terminal (tablet), and an emotion engine. The specific roles of each component are as follows:

[2190] server

[2191] The server manages the child's goals and progress data, and generates and provides feedback using a generative AI model. The server manages user profiles, receives progress data, builds and updates the generative model, generates feedback, and stores and provides the feedback. A typical server machine is used as the hardware, and Python, MySQL, TensorFlow, and Flask are used as the software.

[2192] User profile management: The server stores profile information in a MySQL database when a new user registers.

[2193] Receiving progress data: The server periodically receives progress data sent from the device and stores it in a MySQL database.

[2194] Building and updating generative models: The server builds and updates generative AI models using TensorFlow based on the collected data.

[2195] Feedback generation: The server uses the generative AI model to generate feedback messages and save them in JSON format.

[2196] Storing and serving feedback: The server stores the generated feedback in a MySQL database and serves it in a web interface via Flask.

[2197] Device (tablet)

[2198] The terminal is a device that allows users to input goals and progress data and display feedback. Specifically, we will use a tablet terminal and develop applications for Android and iOS.

[2199] Data Entry: The device stores user-entered goals and daily progress information in local storage. For example, if a child enters "my goal for today is to complete all my math homework," that information is stored in the device's SQLite database.

[2200] Sending progress data: The device automatically sends progress data to the server at regular intervals. For example, when test results are entered, the device sends the data to the server.

[2201] Interaction with generative models: The device receives feedback from the server and stores it in local storage. For example, if a message is received saying, "Your next goal is to get 90 points in math," it will be stored on the device.

[2202] Displaying feedback: The device visually displays the received feedback to the user, for example, "Your current science grade is 92. Your next goal is to get your English grade to 90."

[2203] Providing information to parents and teachers: The device will display appropriate feedback information when requested by a parent or teacher.

[2204] Emotion Engine

[2205] The emotion engine is responsible for recognizing and digitizing the user's emotions. Specifically, it uses the device's camera and microphone to analyze the user's facial expressions and tone of voice, and uses software modules (...

Claims

1. A means to input and store your child's goals and progress data; A means to build and update generative models based on collected data; a means for generating feedback based on the constructed generative model; a means of providing generated feedback to parents and teachers; A system including:

2. 10. The system of claim 1, further comprising means for visually displaying the generated feedback.

3. 10. The system of claim 1, further comprising means for providing an interface through which a parent or teacher can view the feedback.

Images