system

Abstract

Provide a system. 【Solution means】 Means for collecting children's personality, interests, and past behavior data, Means for analyzing the collected data and selecting the most suitable activities for children, Means for proposing the selected activities to the user, Means for analyzing the feedback obtained after the implementation of the activities, Means for updating activity proposals based on the feedback, Means for proposing an individualized education plan, Means for further improving the proposed content based on the execution results of the plan, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, guardians with children aged 0 to 12 are required to accurately understand the individual personalities and interests of their children and choose the most suitable hobbies to develop their future potential. However, due to the diversification of education and the time constraints of guardians, this selection has become a very difficult task. In addition, it is not easy to provide feedback and re-evaluate when the hobby once selected is not suitable for the child. Thus, it is necessary to timely find hobbies suitable for the growth of children and continuously evaluate them, but the current methods have limitations.

Means for Solving the Problems

[0005] To address the aforementioned challenges, the present invention provides a system that includes means for collecting and analyzing data on a child's personality, interests, and past behavior. Specifically, it includes means for selecting the most suitable activity for the child based on this data and proposing it to the user. Furthermore, it includes means for analyzing feedback obtained after the activity has been completed and updating the activity suggestion as needed. This system incorporates the opinions of parents and children into the analysis, enabling the best choices to be made at all times. This provides a dashboard that allows for easy tracking of growth effects, enabling users to monitor their child's progress in real time.

[0006] "A child's personality" refers to the psychological characteristics and individuality that emerge through a child's behavior and reactions.

[0007] "Interest" refers to the degree of interest or concern a child shows towards a particular activity or subject.

[0008] "Behavioral data" refers to a collection of records and information about a child's past behavior, which is used to assess their personality and interests.

[0009] "Analysis methods" refer to the processes and functions used to analyze collected data and extract useful information from it.

[0010] "Activity selection" means choosing appropriate extracurricular activities and experiential learning opportunities for children based on the analyzed data.

[0011] "Feedback" refers to evaluations and opinions about the results and process of an activity, which provides guidance for improving and continuing that activity.

[0012] A "dashboard" refers to an interface that allows users to visually check their child's progress and the effectiveness of their activities.

[0013] "User" refers to the parents or children who use the system. [Brief explanation of the drawing]

[0014] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing apparatus and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing apparatus and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing apparatus and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing apparatus and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] To implement this invention, it is necessary to build a system that collects and analyzes diverse data on children and proposes optimal activities based on the results. The system mainly operates with three main elements: a server, a terminal, and a user.

[0036] The server first has the functionality to automatically collect data such as children's report cards, conversation history, artwork, and video data from schools, parents, and teachers. This data is stored in cloud storage. The server then analyzes the collected data using generative AI models and machine learning algorithms. This analysis utilizes natural language processing technology to identify children's personalities from teachers' comments and conversations, and image recognition technology to evaluate artwork and photographs and extract children's interests.

[0037] The device provides an interface for parents and children to access the system. It functions as a platform for users to input their individual preferences, such as activities they excel at or areas of interest. This allows the device to collect user feedback, send it to the server, and use it for further analysis.

[0038] Users receive optimal activities suggested by the server on their device and try out new lessons based on these suggestions. The server receives feedback from teachers and coaches after the activities, analyzes this feedback, and makes necessary improvements. Through a dashboard provided by the system, users can track how their child is growing and how their interests are changing. This dashboard allows users to visually check the progress and effectiveness of the activities, helping them understand their child's development in real time.

[0039] For example, if past data indicates a child's interest in art, and the parents also indicate they are considering music-related activities, the server analyzes this information and suggests art classes or music workshops. Furthermore, based on feedback from teachers after the activities, the suggestions are refined and used for the next stage. This ensures that the system consistently recommends extracurricular activities that best match the child's interests and personality.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server collects student report cards and teacher comments from school performance management systems and online platforms. Furthermore, it stores artwork, photos, and videos uploaded by parents and teachers in cloud storage.

[0043] Step 2:

[0044] The server analyzes the collected data using generative AI models and machine learning algorithms. Here, natural language processing techniques are used to analyze teacher comments and conversation history to identify children's personality traits and interests. Image recognition technology is also used to evaluate children's skills and passions from their artwork and photographs.

[0045] Step 3:

[0046] Through a terminal, users access the system and input their opinions and wishes regarding their child's future goals and current interests. This information is transmitted to the server in real time.

[0047] Step 4:

[0048] The server combines analysis results with user feedback to select the most suitable extracurricular activities for children. This process takes into account past behavioral data, interests, and preferences to present a variety of options.

[0049] Step 5:

[0050] The server generates a detailed suggestion report based on the selected activities and sends it to the terminal. The user can review this report on the terminal and try out the suggested activities.

[0051] Step 6:

[0052] After the activity is completed, teachers and coaches provide feedback to users via the device. This includes evaluations of the children's activities and suggestions for improvement.

[0053] Step 7:

[0054] The server analyzes the collected feedback to identify the next activities to pursue and the corresponding improvements. This allows users to continuously track their child's progress on a dashboard and adjust the plan as needed.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] In today's world, selecting and proposing the most suitable activities for each child, tailored to their individual personality and interests, is complex. Traditional methods make it difficult to systematically collect and analyze diverse data about children, resulting in the inability to provide individually optimized suggestions. Furthermore, it is difficult to appropriately evaluate the changes that occur after an activity and reflect them in future suggestions. This leads to challenges in adequately supporting children's growth and the deepening of their interests.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes a management device for collecting information, a storage device for storing the data collected by the management device, and an analysis device for analyzing the stored data using natural language processing and image recognition technology. This makes it possible to effectively analyze diverse data on children and accurately propose activities that are appropriate for each individual child. Furthermore, by using a generative AI model based on the analysis results, the effectiveness of the activities can be continuously monitored, and suggestions can be updated to reflect the feedback.

[0060] "Information" refers to all kinds of data necessary to provide individually optimized activity suggestions for children, including their personality, interests, academic performance, and activity history.

[0061] A "management device" is an electronic system or device used to collect and organize information about children from schools, parents, teachers, and other sources.

[0062] A "storage device" refers to a data storage or database system used to store information collected by a management device, and is a device that enables secure storage and rapid access to data.

[0063] An "analysis device" is a data processing device or software that analyzes stored data using natural language processing and image recognition technologies to derive activities suitable for children.

[0064] A "generative AI model" is a machine learning model used to suggest the most suitable activities for children based on analysis. It is an algorithm that generates recommended actions based on past data and newly entered information.

[0065] "Activities" refer to specific learning or experiences, such as classes or workshops, that aim to improve children's skills and interests through participation.

[0066] A "computational device" is a device or system that operates an AI model based on data obtained from an analysis device to provide optimal activity suggestions for children and their guardians.

[0067] A "display device" refers to a screen or application used by a server to visually present activity suggestions generated by a computing device.

[0068] A "renewal device" is a system that reviews the analysis and generation process based on feedback collected after an activity, in order to make the best possible suggestions, and incorporates these findings into the next proposal.

[0069] An "input device" refers to a device or interface that allows users to input their opinions and new requests regarding activities, thereby enabling individually customized suggestions.

[0070] A "control device" is a device or software that incorporates user opinions and requests obtained from input devices into the server's data analysis process, and ultimately reflects them in the proposed content.

[0071] A "visualization device" is a device that provides a graphical user interface for visually confirming the effects of growth and activities, making it easy for users to understand their progress.

[0072] The system of this invention mainly consists of three elements: a server, a terminal, and a user. The server plays a central role in collecting, analyzing, generating, and updating information. The information collected includes a child's personality, interests, academic performance, and past activity history. The information is provided by schools, parents, and teachers and collected through a management device. The data is securely stored in storage devices such as cloud storage systems (e.g., AWS® S3 or Google® Cloud Storage).

[0073] The server then uses an analysis device equipped with natural language processing and image recognition technologies to analyze the collected data in detail. Machine learning algorithms and generative AI models enable analysis that suggests activities best suited to the child. This process generates suggestions in a safe and secure environment. Example prompts can be used at this stage. For example, a sentence like, "For A, who is interested in painting, we recommend an art class," could be used.

[0074] The terminal provides an interface for users to access the system and review suggested activities. A web browser or dedicated application acts as a bridge for information between the user and the server. Users can input their thoughts and preferences through the terminal. This information is then reflected in the analysis process by the control unit.

[0075] Users can visually review the server-generated suggestions on a dashboard on their device. This allows them to track their child's progress and the effectiveness of their activities in real time. The server collects feedback after activities and uses an update device to revise and improve the suggestions, reflecting them in future suggestions. Therefore, the system can provide an environment that continuously supports children's growth and interests.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The server collects information from schools, parents, and teachers. Inputs include children's grades, conversation history, artwork, and video data. This information is collected through a management device, and the collected data is stored in cloud storage. The server periodically retrieves new data and stores it in its storage device based on an execution schedule.

[0079] Step 2:

[0080] The server analyzes the data stored in the storage device using an analysis device. The input includes diverse data collected in step 1. The server utilizes natural language processing techniques to extract the child's personality traits from the text data. It also analyzes visual data using image recognition techniques to identify interests and concerns. The output here is a summary of the child's personality profile and interests.

[0081] Step 3:

[0082] The server uses the analysis results and the generative AI model to generate activity suggestions. The input is the personality profile and interest summary obtained in step 2. The server inputs prompt sentences into the generative AI model to create specific activity suggestions for children. For example, the output might be, "For Person A, who is interested in music, we recommend a music lesson."

[0083] Step 4:

[0084] The terminal visually displays the server-generated suggestions through a user interface. The input is the activity suggestions generated in step 3. The terminal displays them clearly to the user and provides details about the activities. The user can review these suggestions and decide which activities to try.

[0085] Step 5:

[0086] Users provide feedback through their devices, sharing their experiences and impressions of the activities. This feedback is based on their experiences and achievements gained through the activities. This feedback is sent from the device to a server where it is organized.

[0087] Step 6:

[0088] The server re-analyzes the feedback information using an update device and incorporates it into the next suggestion. The input is the feedback data collected in step 5. The server analyzes this data and adjusts and modifies the activity suggestion as needed. This ensures that the child is always provided with the most up-to-date and optimal activities.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] Today's children have diverse interests and personalities, making it difficult to find the optimal learning plan and activities tailored to them. Furthermore, there is a lack of systems that effectively utilize feedback to appropriately track children's growth and provide appropriate suggestions. Therefore, there is a need to easily provide individualized educational plans and support effective learning.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for collecting data on a child's personality, interests, and past behavior; means for analyzing the collected data and selecting the most suitable activities for the child; and means for proposing an individualized educational plan. This makes it possible to provide individualized plans that are tailored to the diverse characteristics of children, and to further update the suggestions based on the results of implementation.

[0094] "A child's personality" refers to the distinctive temperament and behavioral patterns that each child possesses.

[0095] "Interest" refers to a child's interest in or willingness to engage with a particular activity or area of ​​study.

[0096] "Past behavioral data" refers to records of activities and learning experiences a child has had up to that point.

[0097] "Means of collection" refers to the technologies and methods used to gather necessary information, such as databases and sensors.

[0098] "Means of analysis" refer to algorithms and processes used to analyze collected data and derive meaningful conclusions.

[0099] "Means for selecting the optimal activity" refers to a method for determining the most appropriate activity for a child based on analyzed data.

[0100] "Means of proposal" refers to a system for notifying users of selected activities and encouraging their implementation.

[0101] "Methods for analyzing feedback" refer to techniques for analyzing evaluations and opinions obtained after an activity and using them to improve the system.

[0102] "Means of updating" refers to methods for continuously improving the system's proposals based on new information and feedback.

[0103] "Means of proposing individualized education plans" refers to a system that develops and provides educational content tailored to each individual child.

[0104] As a form of implementing the invention, this application example involves building a system that analyzes a child's personality, interests, and past behavioral data to suggest optimal activities. This system operates on the cloud using a smartphone.

[0105] The server first automatically collects all relevant information from schools and parents, such as children's report cards, conversation history, artwork, and video data. Next, the collected data is stored in cloud storage and analyzed using generative AI models and machine learning algorithms. This analysis uses natural language processing technology to identify children's personalities from teachers' comments and conversations, and image recognition technology to evaluate artwork and photos and extract children's interests.

[0106] The terminal functions as a platform that makes it easy for users to access the system, providing an interface for parents and children to input their personal opinions and wishes. This input data is then sent to the server to further improve the accuracy of the analysis.

[0107] Users can view the optimal activities suggested by the server on their devices, gaining new opportunities for learning and extracurricular activities. After the activity, feedback from teachers and coaches is sent back to the server, which then leads to new suggestions. This feedback process ensures that an optimized educational plan is always provided, creating a personalized learning experience.

[0108] For example, if a child is interested in science, participation in science experiment classes or science camps may be suggested. Similarly, if a child is interested in English, an online program where they can converse with a native English-speaking instructor may be recommended.

[0109] An example of a prompt for a generative AI model might be: "Analyze the child's learning history and interests to generate new activity suggestions. Current interests are {object of interest}, and past activity history is {history data}. Generate the optimal activity selection."

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The server automatically collects information such as children's report cards, conversation history, artwork, and video data from schools and parents. Input data is obtained from school databases and files provided by parents. This data is stored in cloud storage for later analysis. The output is structured data securely stored in the cloud.

[0113] Step 2:

[0114] The server analyzes data stored in cloud storage using machine learning algorithms and generative AI models. Here, natural language processing techniques are used to identify children's personalities from teacher comments and conversations, and image recognition techniques are used to evaluate artwork and photographs and extract children's interests. The input is the data collected in step 1, and the output is characteristic information about children's personalities and interests as a result of the analysis.

[0115] Step 3:

[0116] The device provides an interface that parents and children can access through an application to input their opinions on areas and activities that interest the child. This input data is sent to a server and used for further analysis. The input is the user's opinion, and the output is the user's feedback data prepared for analysis.

[0117] Step 4:

[0118] The server uses a generative AI model to select and suggest activities best suited to the child, based on the analyzed data and user input. The input here consists of previous analysis results and user feedback, while the output is a list of suggested activities. A prompt such as "Analyze the child's learning history and interests to generate new activity suggestions" is used for the generative AI model.

[0119] Step 5:

[0120] Users receive activities suggested by the server via their terminal, select one, and perform it. After participating in an activity, users input feedback on it within the application. This reports the activity results to the server. The input consists of the user's opinions and feedback after the activity, and the output is an updated activity log.

[0121] Step 6:

[0122] The server reanalyzes the feedback received after an activity and improves and updates the original activity proposal as needed. This continuously optimizes the individualized educational plan. The input is feedback data, and the output is an improved proposal for the next activity.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] To implement this invention, it is necessary to incorporate an emotion engine that recognizes the user's emotions, in addition to a data collection and analysis system for suggesting optimal activities to the user. The system consists of three elements: a server, a terminal, and a user, and each element works in cooperation with the others.

[0125] The server first collects the child's report card, teacher comments, past behavioral data, and photos and artwork provided by parents and teachers into cloud storage. This allows for the accumulation of data on the child's personality and interests. The server then analyzes this data using a generative AI model and machine learning algorithms to select the most suitable activities for the child. In this process, opinions from parents and children collected through the device are also taken into consideration.

[0126] The emotion engine analyzes the user's emotional data in real time. This engine recognizes emotions from the user's facial expressions, tone of voice, language choices, etc., while they are using the device, and sends this information to the server. The server then adjusts or improves the lesson suggestions, taking the emotional data into consideration.

[0127] Users view suggestions from the server on their devices and participate in the recommended activities. After the activity, teachers and coaches input feedback into the devices, and the server uses this feedback to assess the user's progress. The emotion engine also monitors the user's emotional changes during the activity, reporting to the server, for example, whether the user is enjoying themselves or experiencing difficulties. Based on this, the server evaluates the effectiveness of the activity and prepares to make further suggestions.

[0128] As a concrete example, suppose a child is working on a math lesson. In this case, the emotional engine senses the child's concentration from their facial expression and confirms that the activity has been appropriately selected. On the other hand, if fatigue or frustration is detected by the emotional engine, the server can suggest alternative activities or approaches. This ensures that the support best suited to the child's current state is always provided.

[0129] The following describes the processing flow.

[0130] Step 1:

[0131] The server automates data collection from schools and homes. Specifically, it stores report cards, teacher comments, past behavioral data, and artwork, photos, and videos uploaded by parents and teachers in cloud storage.

[0132] Step 2:

[0133] The server begins analyzing the collected data using generative AI models and machine learning algorithms. The analysis uses natural language processing to identify personality traits and interests from textual information, and image recognition technology to evaluate photos and artwork, thereby understanding the children's skills and areas of interest.

[0134] Step 3:

[0135] Through the device, users input information and opinions about their child's interests and future goals. The entered data is sent to a server and incorporated into the analysis.

[0136] Step 4:

[0137] The server selects the most suitable activity for the child based on analysis results and user feedback. It considers past behavioral data and interests, and generates suggestions after exploring a variety of options.

[0138] Step 5:

[0139] The emotion engine analyzes emotional data collected in real time while the user is operating their device. It determines the user's emotional state from facial expressions, tone of voice, language, etc., and sends the information to the server.

[0140] Step 6:

[0141] The server adjusts activity suggestions based on emotional data. It refines suggestions based on enjoyment and stress indicators, recommending more appropriate activities.

[0142] Step 7:

[0143] Users view activity suggestions provided by the server on their devices and participate in the recommended activities.

[0144] Step 8:

[0145] After a user completes an activity, teachers or coaches provide feedback via the device. This feedback, which specifies the user's achievements and areas for improvement during the activity, is sent to the server.

[0146] Step 9:

[0147] The server comprehensively analyzes feedback and emotional data to track the child's development. Based on this, it prepares a dashboard where users can see suggestions for the next steps.

[0148] (Example 2)

[0149] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0150] Activities and educational programs that children participate in should be tailored to their individual personalities and interests, but currently, there is a problem in accurately selecting such activities. Furthermore, there are insufficient means to properly utilize feedback after activities and incorporate it into future suggestions. Moreover, because it is not possible to track changes in users' emotions in real time and reflect them in activity suggestions, there is a need to respond to individual needs.

[0151] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0152] In this invention, the server includes means for collecting information about a child's personality, interests, and past behavior; means for analyzing the collected information using generative information processing technology and machine learning technology to select the most suitable activity for the child; and means for recognizing the user's emotions and analyzing the emotional data. This makes it possible to select the most suitable activity suggestion based on the individual characteristics of each child, and also enables improvement of the suggestion using feedback and emotional data.

[0153] "Personality" refers to psychological characteristics that describe an individual's behavior and way of thinking, and it is an element that constitutes a consistent pattern of an individual.

[0154] "Interest" is an emotional tendency that indicates the degree of interest or excitement an individual feels towards a particular activity or matter.

[0155] "Past behavior" refers to information that records a series of activities and choices an individual has made up to that point, and it is data that can be used to predict future behavior.

[0156] "Information processing technology" refers to technologies that use computer systems and algorithms to collect, analyze, and manage data.

[0157] "Machine learning technology" is a technique that learns patterns and rules from large amounts of data and uses that knowledge to make predictions and classifications about new data.

[0158] An "activity" is a set of actions or endeavors undertaken with a specific purpose in mind, and may include educational, cultural, or recreational elements.

[0159] A "user" is someone who uses the system and benefits from it, and in this context, it mainly refers to children and their guardians.

[0160] "Emotional data" refers to information that indicates the emotional state of a user, derived from their facial expressions, tone of voice, language choices, and other factors.

[0161] "Feedback" refers to evaluations and opinions received after an activity, and is an important source of information for improving future actions and choices.

[0162] A "suggestion" is an action or choice recommended to the user based on collected and analyzed information.

[0163] To implement this invention, multiple components are used in combination. The system consists of three elements: a server, a terminal, and a user, which work together in cooperation.

[0164] The server plays a central role in collecting and analyzing large amounts of data and generating activity suggestions. Specifically, the server continuously collects information about the child's personality, interests, and past behavior using cloud storage. Next, the server analyzes this information using generative AI models and machine learning techniques to select the most suitable activities for the child. User feedback transmitted from the device is also taken into consideration during this selection process.

[0165] The device is equipped with an emotion engine that analyzes the user's emotions in real time. This engine extracts emotional data from the user's actions through the device (e.g., facial recognition, voice tone analysis, etc.). The collected emotional data is sent to a server and used to improve the suitability of activity suggestions.

[0166] Through the provided interface, users review proposals from the server and participate in selected activities. At this time, users can input feedback on the activity's progress and results into their terminal. The server collects and analyzes this feedback to improve the accuracy of future activity proposals.

[0167] As a concrete example, let's consider a scenario where a user is working on a math lesson. The server recommends activities based on the user's past math performance and interests. The device uses an emotion engine to determine the user's level of concentration and reports to the server, for example, whether the user is "concentrating" or "feeling fatigued." Based on the feedback the user provides after the activity, the server evaluates the effectiveness of the activity and uses this information to make future recommendations.

[0168] Examples of prompts include, "How can I check if the child's attention span is sustained?" and "Please suggest alternative approaches to engage the child's interest." In this way, the system dynamically adjusts activities to provide the optimal educational experience.

[0169] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0170] Step 1:

[0171] The server collects data on children's personalities, interests, and past behaviors from cloud storage. This input data includes report cards, teacher comments, and past activity records. The server integrates this data to generate individual profiles, which provides a foundation for analyzing each child's unique characteristics.

[0172] Step 2:

[0173] The server analyzes the collected data using a generative AI model and machine learning techniques. This data processing executes a specific algorithm to select the optimal activity. The input is profile data, and the output is a list of recommended activities for each child. The server then prepares activity suggestions based on this.

[0174] Step 3:

[0175] The device collects the user's (child's) emotional data in real time. It uses a camera and microphone to record the user's facial expressions and voice tone as input. The device's emotion engine analyzes this data to detect changes in emotion. The collected emotional information is then sent to a server as output. This data is used to adjust activity suggestions.

[0176] Step 4:

[0177] Users receive and participate in suggested activities via their devices. They then review the activity information output from the server and proceed with the activity. Specific features include calendar notifications and interactive guidelines designed to pique their interest.

[0178] Step 5:

[0179] Users provide feedback via a device after completing an activity. This feedback includes their satisfaction level, the difficulty they experienced, and areas for improvement. The device sends this information to a server, where it is stored in a database. The server analyzes this information to evaluate the effectiveness of the activity.

[0180] Step 6:

[0181] The server updates its next activity suggestions based on feedback and sentiment data. In this process, the algorithm re-evaluates the input information and generates new activity proposals. The output is a recommended plan that will be incorporated into the next suggestion.

[0182] (Application Example 2)

[0183] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0184] This invention aims to solve the problem of suggesting optimal activities based on each child's individual personality and emotional state. Conventional educational support systems have difficulty providing individualized support that takes into account the user's real-time emotions, and tend to offer uniform activity suggestions. As a result, it has been difficult to draw out children's motivation and concentration, and individual learning effects have not been fully achieved.

[0185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0186] In this invention, the server includes a device for collecting data on the child's nature, interests, and past behavior; a device for analyzing the collected data and determining the optimal behavior for the child; and a device including an emotion recognition engine for analyzing the user's emotional state in real time. This enables the dynamic suggestion of optimal activities according to the child's emotions and state, and flexible educational support tailored to individual characteristics.

[0187] "A child's characteristics" refer to the personality and behavioral traits that each child possesses, and serve as the basis for judging the suitability of their education and activities.

[0188] "Interest" refers to the objects or fields that a child is particularly interested in and drawn to.

[0189] "Action data" refers to information that records a child's past activity history and behavioral patterns.

[0190] "Analysis" refers to the process of analyzing information based on collected data and deriving meaningful conclusions.

[0191] An "emotion recognition engine" refers to a device or software that analyzes a user's facial expressions, tone of voice, and manner of speaking in real time to identify their emotional state.

[0192] "Suggestion" refers to the act of showing users what is considered the most appropriate activity or behavior based on analysis results and emotional state.

[0193] "Evaluation information" refers to the feedback and results obtained after the implementation of a proposed activity, and is information used to improve subsequent proposals.

[0194] An "information display device" refers to a device or platform that allows for the visual confirmation of the progress and effectiveness of activities.

[0195] The system implementing this invention consists of three elements: a server, a terminal, and a user. The server utilizes cloud-based data storage to collect data on the child's characteristics, interests, and past behavior, and analyzes this data using generative AI models and machine learning algorithms. It also implements an emotion recognition engine to recognize the user's emotional state in real time.

[0196] The emotion recognition engine collects the user's facial expressions and tone of voice in real time through the camera and microphone installed in the device. This data is processed using libraries such as OpenCV and TENSORFLOW® to identify the user's current emotional state.

[0197] The user's terminal receives analysis results and suggestions from the server and presents them to the user via a display device. The user participates in the suggested activities and inputs feedback into the terminal. The server further analyzes the data based on this feedback and optimizes the next activity suggestion.

[0198] A concrete example is monitoring a user's behavior while they are working on a math problem. If the emotion recognition engine detects from the user's facial expressions that they are concentrating, an encouraging message such as "Keep up the good work!" will be displayed on the device. On the other hand, if fatigue is detected, a suggestion such as "Why don't you take a short break?" will be made.

[0199] To implement this system, you can use example prompts such as, "Please analyze whether the child is highly interested based on their facial expressions and voice data."

[0200] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0201] Step 1:

[0202] The server collects data on children's characteristics, interests, and past behavior into cloud-based data storage. Inputs include children's report cards, teacher comments, behavioral history, and photos and artwork provided by parents and teachers. Outputs generate detailed datasets related to each individual child. This data serves as the basis for subsequent analysis by generative AI models and machine learning algorithms.

[0203] Step 2:

[0204] The server uses a generative AI model to analyze the collected data and determine the optimal actions for the child. The input is the dataset collected in Step 1. The output is a list of suggested activities appropriate to the child's personality and interests. Data processing includes data preprocessing and feature extraction.

[0205] Step 3:

[0206] The user receives activity suggestions displayed through their device. The input is a list of suggestions from the server, which the device presents to the user in an easy-to-understand format. The output is a list of suggested activities displayed to the user.

[0207] Step 4:

[0208] The user performs an activity, and the emotion recognition engine monitors their emotional state in real time. The recognition engine receives facial expression data and audio data obtained from the device's camera and microphone as input. Data processing includes analysis using OpenCV and TensorFlow. The output is information about the detected emotional state.

[0209] Step 5:

[0210] The server adjusts activity suggestions based on emotional state information and analyzes feedback received from the terminal after the activity is completed. Inputs are emotional data from the emotion recognition engine and feedback from the terminal. Output is the adjusted activity suggestion for the next activity. Data processing based on the feedback analysis is performed and reflected in the suggestion.

[0211] Step 6:

[0212] The terminal presents the user with adjusted suggestions, providing them with a guide to the improved next activity. The input is the updated suggestions from the server, and the output is information suggesting the next activity the user should participate in.

[0213] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0214] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0215] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0216] [Second Embodiment]

[0217] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0218] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0219] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0220] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0221] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0222] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0223] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0224] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0225] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0226] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0227] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0228] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0229] To implement this invention, it is necessary to build a system that collects and analyzes diverse data on children and proposes optimal activities based on the results. The system mainly operates with three main elements: a server, a terminal, and a user.

[0230] The server first has the functionality to automatically collect data such as children's report cards, conversation history, artwork, and video data from schools, parents, and teachers. This data is stored in cloud storage. The server then analyzes the collected data using generative AI models and machine learning algorithms. This analysis utilizes natural language processing technology to identify children's personalities from teachers' comments and conversations, and image recognition technology to evaluate artwork and photographs and extract children's interests.

[0231] The device provides an interface for parents and children to access the system. It functions as a platform for users to input their individual preferences, such as activities they excel at or areas of interest. This allows the device to collect user feedback, send it to the server, and use it for further analysis.

[0232] Users receive optimal activities suggested by the server on their device and try out new lessons based on these suggestions. The server receives feedback from teachers and coaches after the activities, analyzes this feedback, and makes necessary improvements. Through a dashboard provided by the system, users can track how their child is growing and how their interests are changing. This dashboard allows users to visually check the progress and effectiveness of the activities, helping them understand their child's development in real time.

[0233] For example, if past data indicates a child's interest in art, and the parents also indicate they are considering music-related activities, the server analyzes this information and suggests art classes or music workshops. Furthermore, based on feedback from teachers after the activities, the suggestions are refined and used for the next stage. This ensures that the system consistently recommends extracurricular activities that best match the child's interests and personality.

[0234] The following describes the processing flow.

[0235] Step 1:

[0236] The server collects student report cards and teacher comments from school performance management systems and online platforms. Furthermore, it stores artwork, photos, and videos uploaded by parents and teachers in cloud storage.

[0237] Step 2:

[0238] The server analyzes the collected data using generative AI models and machine learning algorithms. Here, natural language processing techniques are used to analyze teacher comments and conversation history to identify children's personality traits and interests. Image recognition technology is also used to evaluate children's skills and passions from their artwork and photographs.

[0239] Step 3:

[0240] Through a terminal, users access the system and input their opinions and wishes regarding their child's future goals and current interests. This information is transmitted to the server in real time.

[0241] Step 4:

[0242] The server combines analysis results with user feedback to select the most suitable extracurricular activities for children. This process takes into account past behavioral data, interests, and preferences to present a variety of options.

[0243] Step 5:

[0244] The server generates a detailed suggestion report based on the selected activities and sends it to the terminal. The user can review this report on the terminal and try out the suggested activities.

[0245] Step 6:

[0246] After the activity is completed, teachers and coaches provide feedback to users via the device. This includes evaluations of the children's activities and suggestions for improvement.

[0247] Step 7:

[0248] The server analyzes the collected feedback to identify the next activities to pursue and the corresponding improvements. This allows users to continuously track their child's progress on a dashboard and adjust the plan as needed.

[0249] (Example 1)

[0250] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0251] In today's world, selecting and proposing the most suitable activities for each child, tailored to their individual personality and interests, is complex. Traditional methods make it difficult to systematically collect and analyze diverse data about children, resulting in the inability to provide individually optimized suggestions. Furthermore, it is difficult to appropriately evaluate the changes that occur after an activity and reflect them in future suggestions. This leads to challenges in adequately supporting children's growth and the deepening of their interests.

[0252] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0253] In this invention, the server includes a management device for collecting information, a storage device for storing the data collected by the management device, and an analysis device for analyzing the stored data using natural language processing and image recognition technology. This makes it possible to effectively analyze diverse data on children and accurately propose activities that are appropriate for each individual child. Furthermore, by using a generative AI model based on the analysis results, the effectiveness of the activities can be continuously monitored, and suggestions can be updated to reflect the feedback.

[0254] "Information" refers to all kinds of data necessary to provide individually optimized activity suggestions for children, including their personality, interests, academic performance, and activity history.

[0255] A "management device" is an electronic system or device used to collect and organize information about children from schools, parents, teachers, and other sources.

[0256] A "storage device" refers to a data storage or database system used to store information collected by a management device, and is a device that enables secure storage and rapid access to data.

[0257] An "analysis device" is a data processing device or software that analyzes stored data using natural language processing and image recognition technologies to derive activities suitable for children.

[0258] A "generative AI model" is a machine learning model used to suggest the most suitable activities for children based on analysis. It is an algorithm that generates recommended actions based on past data and newly entered information.

[0259] "Activities" refer to specific learning or experiences, such as classes or workshops, that aim to improve children's skills and interests through participation.

[0260] A "computational device" is a device or system that operates an AI model based on data obtained from an analysis device to provide optimal activity suggestions for children and their guardians.

[0261] A "display device" refers to a screen or application used by a server to visually present activity suggestions generated by a computing device.

[0262] A "renewal device" is a system that reviews the analysis and generation process based on feedback collected after an activity, in order to make the best possible suggestions, and incorporates these findings into the next proposal.

[0263] An "input device" refers to a device or interface that allows users to input their opinions and new requests regarding activities, thereby enabling individually customized suggestions.

[0264] A "control device" is a device or software that incorporates user opinions and requests obtained from input devices into the server's data analysis process, and ultimately reflects them in the proposed content.

[0265] A "visualization device" is a device that provides a graphical user interface for visually confirming the effects of growth and activities, making it easy for users to understand their progress.

[0266] The system of this invention mainly consists of three elements: a server, a terminal, and a user. The server plays a central role in collecting, analyzing, generating, and updating information. The information collected includes a child's personality, interests, academic performance, and past activity history. The information is provided by schools, parents, and teachers and collected through a management device. The data is securely stored in storage devices such as cloud storage systems (e.g., AWS S3 or Google Cloud Storage).

[0267] The server then uses an analysis device equipped with natural language processing and image recognition technologies to analyze the collected data in detail. Machine learning algorithms and generative AI models enable analysis that suggests activities best suited to the child. This process generates suggestions in a safe and secure environment. Example prompts can be used at this stage. For example, a sentence like, "For A, who is interested in painting, we recommend an art class," could be used.

[0268] The terminal provides an interface for users to access the system and review suggested activities. A web browser or dedicated application acts as a bridge for information between the user and the server. Users can input their thoughts and preferences through the terminal. This information is then reflected in the analysis process by the control unit.

[0269] Users can visually review the server-generated suggestions on a dashboard on their device. This allows them to track their child's progress and the effectiveness of their activities in real time. The server collects feedback after activities and uses an update device to revise and improve the suggestions, reflecting them in future suggestions. Therefore, the system can provide an environment that continuously supports children's growth and interests.

[0270] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0271] Step 1:

[0272] The server collects information from schools, parents, and teachers. Inputs include children's grades, conversation history, artwork, and video data. This information is collected through a management device, and the collected data is stored in cloud storage. The server periodically retrieves new data and stores it in its storage device based on an execution schedule.

[0273] Step 2:

[0274] The server analyzes the data stored in the storage device using an analysis device. The input includes diverse data collected in step 1. The server utilizes natural language processing techniques to extract the child's personality traits from the text data. It also analyzes visual data using image recognition techniques to identify interests and concerns. The output here is a summary of the child's personality profile and interests.

[0275] Step 3:

[0276] The server uses the analysis results and the generative AI model to generate activity suggestions. The input is the personality profile and interest summary obtained in step 2. The server inputs prompt sentences into the generative AI model to create specific activity suggestions for children. For example, the output might be, "For Person A, who is interested in music, we recommend a music lesson."

[0277] Step 4:

[0278] The terminal visually displays the suggestions generated by the server through the user interface. The input is the activity suggestions generated in Step 3. The terminal displays it clearly to the user and provides details of the activity content. The user can check these suggestions and decide which activity to try.

[0279] Step 5:

[0280] The user inputs feedback through the terminal and provides the results and feelings of the activity. The input is information based on the experiences and results obtained through the activity. These feedbacks are sent from the terminal to the server and organized by the server.

[0281] Step 6:

[0282] The server re-analyzes the feedback information using the update device and reflects it in the next suggestions. The input is the feedback data collected in Step 5. The server analyzes this data and adjusts and modifies the activity suggestions as necessary. This makes it possible to always provide the most suitable activities for children in the latest state.

[0283] (Application Example 1)

[0284] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0285] Modern children have diverse interests and personalities, and it is difficult to find the most suitable learning plans and activities according to them. Also, there is a lack of a system that effectively utilizes feedback to appropriately track children's growth and provide corresponding suggestions. Therefore, it is necessary to easily provide individualized education plans and support effective learning.

[0286] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following means.

[0287] In this invention, the server includes means for collecting data on children's personalities, interests, and past behavior, means for analyzing the collected data to select the most suitable activities for the children, and means for proposing an individualized education plan. As a result, it is possible to provide an individualized plan according to the diverse characteristics of children and further update the proposal based on the execution results.

[0288] "Children's personality" refers to the characteristic temperament and behavior patterns of individual children.

[0289] "Interest" refers to the interest and willingness of children to show interest and participate in specific activities or learning fields.

[0290] "Past behavior data" refers to records related to the activities and learning histories that children have experienced so far.

[0291] "Means for collecting" refers to the technology and methods for collecting necessary information using databases and sensors.

[0292] "Means for analyzing" refers to the algorithms and processes for analyzing the collected data to derive meaningful conclusions.

[0293] "Means for selecting the most suitable activities" refers to the method for determining the most appropriate activities for children based on the analyzed data.

[0294] "Means for proposing" refers to the mechanism for notifying the user of the selected activities and prompting implementation.

[0295] "Means for analyzing feedback" refers to the technology for analyzing the evaluations and opinions obtained after activities and utilizing them for system improvement.

[0296] "Means of updating" refers to methods for continuously improving the system's proposals based on new information and feedback.

[0297] "Means of proposing individualized education plans" refers to a system that develops and provides educational content tailored to each individual child.

[0298] As a form of implementing the invention, this application example involves building a system that analyzes a child's personality, interests, and past behavioral data to suggest optimal activities. This system operates on the cloud using a smartphone.

[0299] The server first automatically collects all relevant information from schools and parents, such as children's report cards, conversation history, artwork, and video data. Next, the collected data is stored in cloud storage and analyzed using generative AI models and machine learning algorithms. This analysis uses natural language processing technology to identify children's personalities from teachers' comments and conversations, and image recognition technology to evaluate artwork and photos and extract children's interests.

[0300] The terminal functions as a platform that makes it easy for users to access the system, providing an interface for parents and children to input their personal opinions and wishes. This input data is then sent to the server to further improve the accuracy of the analysis.

[0301] Users can view the optimal activities suggested by the server on their devices, gaining new opportunities for learning and extracurricular activities. After the activity, feedback from teachers and coaches is sent back to the server, which then leads to new suggestions. This feedback process ensures that an optimized educational plan is always provided, creating a personalized learning experience.

[0302] As a specific example, for instance, if a child is interested in science, participation in a science experiment classroom or a science camp is proposed. Also, if there is an interest in English, a program of having an online conversation with a native instructor may be recommended.

[0303] As an example of a prompt sentence for the generative AI model, "Analyze the learning history and interests of a child and generate new activity proposals. The current interest is {object of interest}, and the past activity history is {history data}. Please generate optimal activity options." can be considered.

[0304] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0305] Step 1:

[0306] The server automatically collects information such as a child's report card, conversation history, works, video data, etc. from schools and guardians. The input data is obtained from the school's database or files provided by the guardians. These data are stored in cloud storage in preparation for later analysis. The output is structured data safely stored on the cloud.

[0307] Step 2:

[0308] The server analyzes the data stored in cloud storage using machine learning algorithms or generative AI models. Here, natural language processing technology is used to identify a child's personality from teachers' comments and conversations, image recognition technology is used to evaluate works and photos, and the child's interests are extracted. The input is the data collected in Step 1, and the output is the characteristic information of the child's personality and interests as the analysis result.

[0309] Step 3:

[0310] The device provides an interface that parents and children can access through an application to input their opinions on areas and activities that interest the child. This input data is sent to a server and used for further analysis. The input is the user's opinion, and the output is the user's feedback data prepared for analysis.

[0311] Step 4:

[0312] The server uses a generative AI model to select and suggest activities best suited to the child, based on the analyzed data and user input. The input here consists of previous analysis results and user feedback, while the output is a list of suggested activities. A prompt such as "Analyze the child's learning history and interests to generate new activity suggestions" is used for the generative AI model.

[0313] Step 5:

[0314] Users receive activities suggested by the server via their terminal, select one, and perform it. After participating in an activity, users input feedback on it within the application. This reports the activity results to the server. The input consists of the user's opinions and feedback after the activity, and the output is an updated activity log.

[0315] Step 6:

[0316] The server reanalyzes the feedback received after an activity and improves and updates the original activity proposal as needed. This continuously optimizes the individualized educational plan. The input is feedback data, and the output is an improved proposal for the next activity.

[0317] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0318] To implement this invention, it is necessary to incorporate an emotion engine that recognizes the user's emotions, in addition to a data collection and analysis system for suggesting optimal activities to the user. The system consists of three elements: a server, a terminal, and a user, and each element works in cooperation with the others.

[0319] The server first collects the child's report card, teacher comments, past behavioral data, and photos and artwork provided by parents and teachers into cloud storage. This allows for the accumulation of data on the child's personality and interests. The server then analyzes this data using a generative AI model and machine learning algorithms to select the most suitable activities for the child. In this process, opinions from parents and children collected through the device are also taken into consideration.

[0320] The emotion engine analyzes the user's emotional data in real time. This engine recognizes emotions from the user's facial expressions, tone of voice, language choices, etc., while they are using the device, and sends this information to the server. The server then adjusts or improves the lesson suggestions, taking the emotional data into consideration.

[0321] Users view suggestions from the server on their devices and participate in the recommended activities. After the activity, teachers and coaches input feedback into the devices, and the server uses this feedback to assess the user's progress. The emotion engine also monitors the user's emotional changes during the activity, reporting to the server, for example, whether the user is enjoying themselves or experiencing difficulties. Based on this, the server evaluates the effectiveness of the activity and prepares to make further suggestions.

[0322] As a concrete example, suppose a child is working on a math lesson. In this case, the emotional engine senses the child's concentration from their facial expression and confirms that the activity has been appropriately selected. On the other hand, if fatigue or frustration is detected by the emotional engine, the server can suggest alternative activities or approaches. This ensures that the support best suited to the child's current state is always provided.

[0323] The following describes the processing flow.

[0324] Step 1:

[0325] The server automates data collection from schools and homes. Specifically, it stores report cards, teacher comments, past behavioral data, and artwork, photos, and videos uploaded by parents and teachers in cloud storage.

[0326] Step 2:

[0327] The server begins analyzing the collected data using generative AI models and machine learning algorithms. The analysis uses natural language processing to identify personality traits and interests from textual information, and image recognition technology to evaluate photos and artwork, thereby understanding the children's skills and areas of interest.

[0328] Step 3:

[0329] Through the device, users input information and opinions about their child's interests and future goals. The entered data is sent to a server and incorporated into the analysis.

[0330] Step 4:

[0331] The server selects the most suitable activity for the child based on analysis results and user feedback. It considers past behavioral data and interests, and generates suggestions after exploring a variety of options.

[0332] Step 5:

[0333] The emotion engine analyzes emotional data collected in real time while the user is operating their device. It determines the user's emotional state from facial expressions, tone of voice, language, etc., and sends the information to the server.

[0334] Step 6:

[0335] The server adjusts activity suggestions based on emotional data. It refines suggestions based on enjoyment and stress indicators, recommending more appropriate activities.

[0336] Step 7:

[0337] Users view activity suggestions provided by the server on their devices and participate in the recommended activities.

[0338] Step 8:

[0339] After a user completes an activity, teachers or coaches provide feedback via the device. This feedback, which specifies the user's achievements and areas for improvement during the activity, is sent to the server.

[0340] Step 9:

[0341] The server comprehensively analyzes feedback and emotional data to track the child's development. Based on this, it prepares a dashboard where users can see suggestions for the next steps.

[0342] (Example 2)

[0343] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0344] Activities and educational programs that children participate in should be tailored to their individual personalities and interests, but currently, there is a problem in accurately selecting such activities. Furthermore, there are insufficient means to properly utilize feedback after activities and incorporate it into future suggestions. Moreover, because it is not possible to track changes in users' emotions in real time and reflect them in activity suggestions, there is a need to respond to individual needs.

[0345] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0346] In this invention, the server includes means for collecting information about a child's personality, interests, and past behavior; means for analyzing the collected information using generative information processing technology and machine learning technology to select the most suitable activity for the child; and means for recognizing the user's emotions and analyzing the emotional data. This makes it possible to select the most suitable activity suggestion based on the individual characteristics of each child, and also enables improvement of the suggestion using feedback and emotional data.

[0347] "Personality" refers to psychological characteristics that describe an individual's behavior and way of thinking, and it is an element that constitutes a consistent pattern of an individual.

[0348] "Interest" is an emotional tendency that indicates the degree of interest or excitement an individual feels towards a particular activity or matter.

[0349] "Past behavior" refers to information that records a series of activities and choices an individual has made up to that point, and it is data that can be used to predict future behavior.

[0350] "Information processing technology" refers to technologies that use computer systems and algorithms to collect, analyze, and manage data.

[0351] "Machine learning technology" is a technique that learns patterns and rules from large amounts of data and uses that knowledge to make predictions and classifications about new data.

[0352] An "activity" is a set of actions or endeavors undertaken with a specific purpose in mind, and may include educational, cultural, or recreational elements.

[0353] A "user" is someone who uses the system and benefits from it, and in this context, it mainly refers to children and their guardians.

[0354] "Emotional data" refers to information that indicates the emotional state of a user, derived from their facial expressions, tone of voice, language choices, and other factors.

[0355] "Feedback" refers to evaluations and opinions received after an activity, and is an important source of information for improving future actions and choices.

[0356] A "suggestion" is an action or choice recommended to the user based on collected and analyzed information.

[0357] To implement this invention, multiple components are used in combination. The system consists of three elements: a server, a terminal, and a user, which work together in cooperation.

[0358] The server plays a central role in collecting and analyzing large amounts of data and generating activity suggestions. Specifically, the server continuously collects information about the child's personality, interests, and past behavior using cloud storage. Next, the server analyzes this information using generative AI models and machine learning techniques to select the most suitable activities for the child. User feedback transmitted from the device is also taken into consideration during this selection process.

[0359] The device is equipped with an emotion engine that analyzes the user's emotions in real time. This engine extracts emotional data from the user's actions through the device (e.g., facial recognition, voice tone analysis, etc.). The collected emotional data is sent to a server and used to improve the suitability of activity suggestions.

[0360] Through the provided interface, users review proposals from the server and participate in selected activities. At this time, users can input feedback on the activity's progress and results into their terminal. The server collects and analyzes this feedback to improve the accuracy of future activity proposals.

[0361] As a concrete example, let's consider a scenario where a user is working on a math lesson. The server recommends activities based on the user's past math performance and interests. The device uses an emotion engine to determine the user's level of concentration and reports to the server, for example, whether the user is "concentrating" or "feeling fatigued." Based on the feedback the user provides after the activity, the server evaluates the effectiveness of the activity and uses this information to make future recommendations.

[0362] Examples of prompts include, "How can I check if the child's attention span is sustained?" and "Please suggest alternative approaches to engage the child's interest." In this way, the system dynamically adjusts activities to provide the optimal educational experience.

[0363] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0364] Step 1:

[0365] The server collects data on children's personalities, interests, and past behaviors from cloud storage. This input data includes report cards, teacher comments, and past activity records. The server integrates this data to generate individual profiles, which provides a foundation for analyzing each child's unique characteristics.

[0366] Step 2:

[0367] The server analyzes the collected data using a generative AI model and machine learning techniques. This data processing executes a specific algorithm to select the optimal activity. The input is profile data, and the output is a list of recommended activities for each child. The server then prepares activity suggestions based on this.

[0368] Step 3:

[0369] The device collects the user's (child's) emotional data in real time. It uses a camera and microphone to record the user's facial expressions and voice tone as input. The device's emotion engine analyzes this data to detect changes in emotion. The collected emotional information is then sent to a server as output. This data is used to adjust activity suggestions.

[0370] Step 4:

[0371] Users receive and participate in suggested activities via their devices. They then review the activity information output from the server and proceed with the activity. Specific features include calendar notifications and interactive guidelines designed to pique their interest.

[0372] Step 5:

[0373] Users provide feedback via a device after completing an activity. This feedback includes their satisfaction level, the difficulty they experienced, and areas for improvement. The device sends this information to a server, where it is stored in a database. The server analyzes this information to evaluate the effectiveness of the activity.

[0374] Step 6:

[0375] The server updates its next activity suggestions based on feedback and sentiment data. In this process, the algorithm re-evaluates the input information and generates new activity proposals. The output is a recommended plan that will be incorporated into the next suggestion.

[0376] (Application Example 2)

[0377] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0378] This invention aims to solve the problem of suggesting optimal activities based on each child's individual personality and emotional state. Conventional educational support systems have difficulty providing individualized support that takes into account the user's real-time emotions, and tend to offer uniform activity suggestions. As a result, it has been difficult to draw out children's motivation and concentration, and individual learning effects have not been fully achieved.

[0379] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0380] In this invention, the server includes a device for collecting data on the child's nature, interests, and past behavior; a device for analyzing the collected data and determining the optimal behavior for the child; and a device including an emotion recognition engine for analyzing the user's emotional state in real time. This enables the dynamic suggestion of optimal activities according to the child's emotions and state, and flexible educational support tailored to individual characteristics.

[0381] "A child's characteristics" refer to the personality and behavioral traits that each child possesses, and serve as the basis for judging the suitability of their education and activities.

[0382] "Interest" refers to the objects or fields that a child is particularly interested in and drawn to.

[0383] "Action data" refers to information that records a child's past activity history and behavioral patterns.

[0384] "Analysis" refers to the process of analyzing information based on collected data and deriving meaningful conclusions.

[0385] An "emotion recognition engine" refers to a device or software that analyzes a user's facial expressions, tone of voice, and manner of speaking in real time to identify their emotional state.

[0386] "Suggestion" refers to the act of showing users what is considered the most appropriate activity or behavior based on analysis results and emotional state.

[0387] "Evaluation information" refers to the feedback and results obtained after the implementation of a proposed activity, and is information used to improve subsequent proposals.

[0388] An "information display device" refers to a device or platform that allows for the visual confirmation of the progress and effectiveness of activities.

[0389] The system implementing this invention consists of three elements: a server, a terminal, and a user. The server utilizes cloud-based data storage to collect data on the child's characteristics, interests, and past behavior, and analyzes this data using generative AI models and machine learning algorithms. It also implements an emotion recognition engine to recognize the user's emotional state in real time.

[0390] The emotion recognition engine collects the user's facial expressions and tone of voice in real time through the camera and microphone built into the device. This data is processed using libraries such as OpenCV and TensorFlow to identify the user's current emotional state.

[0391] The user's terminal receives analysis results and suggestions from the server and presents them to the user via a display device. The user participates in the suggested activities and inputs feedback into the terminal. The server further analyzes the data based on this feedback and optimizes the next activity suggestion.

[0392] A concrete example is monitoring a user's behavior while they are working on a math problem. If the emotion recognition engine detects from the user's facial expressions that they are concentrating, an encouraging message such as "Keep up the good work!" will be displayed on the device. On the other hand, if fatigue is detected, a suggestion such as "Why don't you take a short break?" will be made.

[0393] To implement this system, you can use example prompts such as, "Please analyze whether the child is highly interested based on their facial expressions and voice data."

[0394] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0395] Step 1:

[0396] The server collects data on children's characteristics, interests, and past behavior into cloud-based data storage. Inputs include children's report cards, teacher comments, behavioral history, and photos and artwork provided by parents and teachers. Outputs generate detailed datasets related to each individual child. This data serves as the basis for subsequent analysis by generative AI models and machine learning algorithms.

[0397] Step 2:

[0398] The server uses a generative AI model to analyze the collected data and determine the optimal actions for the child. The input is the dataset collected in Step 1. The output is a list of suggested activities appropriate to the child's personality and interests. Data processing includes data preprocessing and feature extraction.

[0399] Step 3:

[0400] The user receives activity suggestions displayed through their device. The input is a list of suggestions from the server, which the device presents to the user in an easy-to-understand format. The output is a list of suggested activities displayed to the user.

[0401] Step 4:

[0402] The user performs an activity, and the emotion recognition engine monitors their emotional state in real time. The recognition engine receives facial expression data and audio data obtained from the device's camera and microphone as input. Data processing includes analysis using OpenCV and TensorFlow. The output is information about the detected emotional state.

[0403] Step 5:

[0404] The server adjusts activity suggestions based on emotional state information and analyzes feedback received from the terminal after the activity is completed. Inputs are emotional data from the emotion recognition engine and feedback from the terminal. Output is the adjusted activity suggestion for the next activity. Data processing based on the feedback analysis is performed and reflected in the suggestion.

[0405] Step 6:

[0406] The terminal presents the user with adjusted suggestions, providing them with a guide to the improved next activity. The input is the updated suggestions from the server, and the output is information suggesting the next activity the user should participate in.

[0407] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0408] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0409] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0410] [Third Embodiment]

[0411] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0412] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0413] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0414] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0415] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0416] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0417] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0418] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0419] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0420] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0421] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0422] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0423] To implement this invention, it is necessary to build a system that collects and analyzes diverse data on children and proposes optimal activities based on the results. The system mainly operates with three main elements: a server, a terminal, and a user.

[0424] The server first has the functionality to automatically collect data such as children's report cards, conversation history, artwork, and video data from schools, parents, and teachers. This data is stored in cloud storage. The server then analyzes the collected data using generative AI models and machine learning algorithms. This analysis utilizes natural language processing technology to identify children's personalities from teachers' comments and conversations, and image recognition technology to evaluate artwork and photographs and extract children's interests.

[0425] The device provides an interface for parents and children to access the system. It functions as a platform for users to input their individual preferences, such as activities they excel at or areas of interest. This allows the device to collect user feedback, send it to the server, and use it for further analysis.

[0426] Users receive optimal activities suggested by the server on their device and try out new lessons based on these suggestions. The server receives feedback from teachers and coaches after the activities, analyzes this feedback, and makes necessary improvements. Through a dashboard provided by the system, users can track how their child is growing and how their interests are changing. This dashboard allows users to visually check the progress and effectiveness of the activities, helping them understand their child's development in real time.

[0427] For example, if past data indicates a child's interest in art, and the parents also indicate they are considering music-related activities, the server analyzes this information and suggests art classes or music workshops. Furthermore, based on feedback from teachers after the activities, the suggestions are refined and used for the next stage. This ensures that the system consistently recommends extracurricular activities that best match the child's interests and personality.

[0428] The following describes the processing flow.

[0429] Step 1:

[0430] The server collects student report cards and teacher comments from school performance management systems and online platforms. Furthermore, it stores artwork, photos, and videos uploaded by parents and teachers in cloud storage.

[0431] Step 2:

[0432] The server analyzes the collected data using generative AI models and machine learning algorithms. Here, natural language processing techniques are used to analyze teacher comments and conversation history to identify children's personality traits and interests. Image recognition technology is also used to evaluate children's skills and passions from their artwork and photographs.

[0433] Step 3:

[0434] Through a terminal, users access the system and input their opinions and wishes regarding their child's future goals and current interests. This information is transmitted to the server in real time.

[0435] Step 4:

[0436] The server combines analysis results with user feedback to select the most suitable extracurricular activities for children. This process takes into account past behavioral data, interests, and preferences to present a variety of options.

[0437] Step 5:

[0438] The server generates a detailed suggestion report based on the selected activities and sends it to the terminal. The user can review this report on the terminal and try out the suggested activities.

[0439] Step 6:

[0440] After the activity is completed, teachers and coaches provide feedback to users via the device. This includes evaluations of the children's activities and suggestions for improvement.

[0441] Step 7:

[0442] The server analyzes the collected feedback to identify the next activities to pursue and the corresponding improvements. This allows users to continuously track their child's progress on a dashboard and adjust the plan as needed.

[0443] (Example 1)

[0444] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0445] In today's world, selecting and proposing the most suitable activities for each child, tailored to their individual personality and interests, is complex. Traditional methods make it difficult to systematically collect and analyze diverse data about children, resulting in the inability to provide individually optimized suggestions. Furthermore, it is difficult to appropriately evaluate the changes that occur after an activity and reflect them in future suggestions. This leads to challenges in adequately supporting children's growth and the deepening of their interests.

[0446] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0447] In this invention, the server includes a management device for collecting information, a storage device for storing the data collected by the management device, and an analysis device for analyzing the stored data using natural language processing and image recognition technology. This makes it possible to effectively analyze diverse data on children and accurately propose activities that are appropriate for each individual child. Furthermore, by using a generative AI model based on the analysis results, the effectiveness of the activities can be continuously monitored, and suggestions can be updated to reflect the feedback.

[0448] "Information" refers to all kinds of data necessary to provide individually optimized activity suggestions for children, including their personality, interests, academic performance, and activity history.

[0449] A "management device" is an electronic system or device used to collect and organize information about children from schools, parents, teachers, and other sources.

[0450] A "storage device" refers to a data storage or database system used to store information collected by a management device, and is a device that enables secure storage and rapid access to data.

[0451] An "analysis device" is a data processing device or software that analyzes stored data using natural language processing and image recognition technologies to derive activities suitable for children.

[0452] A "generative AI model" is a machine learning model used to suggest the most suitable activities for children based on analysis. It is an algorithm that generates recommended actions based on past data and newly entered information.

[0453] "Activities" refer to specific learning or experiences, such as classes or workshops, that aim to improve children's skills and interests through participation.

[0454] A "computational device" is a device or system that operates an AI model based on data obtained from an analysis device to provide optimal activity suggestions for children and their guardians.

[0455] A "display device" refers to a screen or application used by a server to visually present activity suggestions generated by a computing device.

[0456] A "renewal device" is a system that reviews the analysis and generation process based on feedback collected after an activity, in order to make the best possible suggestions, and incorporates these findings into the next proposal.

[0457] An "input device" refers to a device or interface that allows users to input their opinions and new requests regarding activities, thereby enabling individually customized suggestions.

[0458] A "control device" is a device or software that incorporates user opinions and requests obtained from input devices into the server's data analysis process, and ultimately reflects them in the proposed content.

[0459] A "visualization device" is a device that provides a graphical user interface for visually confirming the effects of growth and activities, making it easy for users to understand their progress.

[0460] The system of this invention mainly consists of three elements: a server, a terminal, and a user. The server plays a central role in collecting, analyzing, generating, and updating information. The information collected includes a child's personality, interests, academic performance, and past activity history. The information is provided by schools, parents, and teachers and collected through a management device. The data is securely stored in storage devices such as cloud storage systems (e.g., AWS S3 or Google Cloud Storage).

[0461] The server then uses an analysis device equipped with natural language processing and image recognition technologies to analyze the collected data in detail. Machine learning algorithms and generative AI models enable analysis that suggests activities best suited to the child. This process generates suggestions in a safe and secure environment. Example prompts can be used at this stage. For example, a sentence like, "For A, who is interested in painting, we recommend an art class," could be used.

[0462] The terminal provides an interface for users to access the system and review suggested activities. A web browser or dedicated application acts as a bridge for information between the user and the server. Users can input their thoughts and preferences through the terminal. This information is then reflected in the analysis process by the control unit.

[0463] Users can visually review the server-generated suggestions on a dashboard on their device. This allows them to track their child's progress and the effectiveness of their activities in real time. The server collects feedback after activities and uses an update device to revise and improve the suggestions, reflecting them in future suggestions. Therefore, the system can provide an environment that continuously supports children's growth and interests.

[0464] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0465] Step 1:

[0466] The server collects information from schools, parents, and teachers. Inputs include children's grades, conversation history, artwork, and video data. This information is collected through a management device, and the collected data is stored in cloud storage. The server periodically retrieves new data and stores it in its storage device based on an execution schedule.

[0467] Step 2:

[0468] The server analyzes the data stored in the storage device using an analysis device. The input includes diverse data collected in step 1. The server utilizes natural language processing techniques to extract the child's personality traits from the text data. It also analyzes visual data using image recognition techniques to identify interests and concerns. The output here is a summary of the child's personality profile and interests.

[0469] Step 3:

[0470] The server uses the analysis results and the generative AI model to generate activity suggestions. The input is the personality profile and interest summary obtained in step 2. The server inputs prompt sentences into the generative AI model to create specific activity suggestions for children. For example, the output might be, "For Person A, who is interested in music, we recommend a music lesson."

[0471] Step 4:

[0472] The terminal visually displays the server-generated suggestions through a user interface. The input is the activity suggestions generated in step 3. The terminal displays them clearly to the user and provides details about the activities. The user can review these suggestions and decide which activities to try.

[0473] Step 5:

[0474] Users provide feedback through their devices, sharing their experiences and impressions of the activities. This feedback is based on their experiences and achievements gained through the activities. This feedback is sent from the device to a server where it is organized.

[0475] Step 6:

[0476] The server re-analyzes the feedback information using an update device and incorporates it into the next suggestion. The input is the feedback data collected in step 5. The server analyzes this data and adjusts and modifies the activity suggestion as needed. This ensures that the child is always provided with the most up-to-date and optimal activities.

[0477] (Application Example 1)

[0478] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0479] Today's children have diverse interests and personalities, making it difficult to find the optimal learning plan and activities tailored to them. Furthermore, there is a lack of systems that effectively utilize feedback to appropriately track children's growth and provide appropriate suggestions. Therefore, there is a need to easily provide individualized educational plans and support effective learning.

[0480] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0481] In this invention, the server includes means for collecting data on a child's personality, interests, and past behavior; means for analyzing the collected data and selecting the most suitable activities for the child; and means for proposing an individualized educational plan. This makes it possible to provide individualized plans that are tailored to the diverse characteristics of children, and to further update the suggestions based on the results of implementation.

[0482] "A child's personality" refers to the distinctive temperament and behavioral patterns that each child possesses.

[0483] "Interest" refers to a child's interest in or willingness to engage with a particular activity or area of ​​study.

[0484] "Past behavioral data" refers to records of activities and learning experiences a child has had up to that point.

[0485] "Means of collection" refers to the technologies and methods used to gather necessary information, such as databases and sensors.

[0486] "Means of analysis" refer to algorithms and processes used to analyze collected data and derive meaningful conclusions.

[0487] "Means for selecting the optimal activity" refers to a method for determining the most appropriate activity for a child based on analyzed data.

[0488] "Means of proposal" refers to a system for notifying users of selected activities and encouraging their implementation.

[0489] "Methods for analyzing feedback" refer to techniques for analyzing evaluations and opinions obtained after an activity and using them to improve the system.

[0490] "Means of updating" refers to methods for continuously improving the system's proposals based on new information and feedback.

[0491] "Means of proposing individualized education plans" refers to a system that develops and provides educational content tailored to each individual child.

[0492] As a form of implementing the invention, this application example involves building a system that analyzes a child's personality, interests, and past behavioral data to suggest optimal activities. This system operates on the cloud using a smartphone.

[0493] The server first automatically collects all relevant information from schools and parents, such as children's report cards, conversation history, artwork, and video data. Next, the collected data is stored in cloud storage and analyzed using generative AI models and machine learning algorithms. This analysis uses natural language processing technology to identify children's personalities from teachers' comments and conversations, and image recognition technology to evaluate artwork and photos and extract children's interests.

[0494] The terminal functions as a platform that makes it easy for users to access the system, providing an interface for parents and children to input their personal opinions and wishes. This input data is then sent to the server to further improve the accuracy of the analysis.

[0495] Users can view the optimal activities suggested by the server on their devices, gaining new opportunities for learning and extracurricular activities. After the activity, feedback from teachers and coaches is sent back to the server, which then leads to new suggestions. This feedback process ensures that an optimized educational plan is always provided, creating a personalized learning experience.

[0496] For example, if a child is interested in science, participation in science experiment classes or science camps may be suggested. Similarly, if a child is interested in English, an online program where they can converse with a native English-speaking instructor may be recommended.

[0497] An example of a prompt for a generative AI model might be: "Analyze the child's learning history and interests to generate new activity suggestions. Current interests are {object of interest}, and past activity history is {history data}. Generate the optimal activity selection."

[0498] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0499] Step 1:

[0500] The server automatically collects information such as children's report cards, conversation history, artwork, and video data from schools and parents. Input data is obtained from school databases and files provided by parents. This data is stored in cloud storage for later analysis. The output is structured data securely stored in the cloud.

[0501] Step 2:

[0502] The server analyzes data stored in cloud storage using machine learning algorithms and generative AI models. Here, natural language processing techniques are used to identify children's personalities from teacher comments and conversations, and image recognition techniques are used to evaluate artwork and photographs and extract children's interests. The input is the data collected in step 1, and the output is characteristic information about children's personalities and interests as a result of the analysis.

[0503] Step 3:

[0504] The device provides an interface that parents and children can access through an application to input their opinions on areas and activities that interest the child. This input data is sent to a server and used for further analysis. The input is the user's opinion, and the output is the user's feedback data prepared for analysis.

[0505] Step 4:

[0506] The server uses a generative AI model to select and suggest activities best suited to the child, based on the analyzed data and user input. The input here consists of previous analysis results and user feedback, while the output is a list of suggested activities. A prompt such as "Analyze the child's learning history and interests to generate new activity suggestions" is used for the generative AI model.

[0507] Step 5:

[0508] Users receive activities suggested by the server via their terminal, select one, and perform it. After participating in an activity, users input feedback on it within the application. This reports the activity results to the server. The input consists of the user's opinions and feedback after the activity, and the output is an updated activity log.

[0509] Step 6:

[0510] The server reanalyzes the feedback received after an activity and improves and updates the original activity proposal as needed. This continuously optimizes the individualized educational plan. The input is feedback data, and the output is an improved proposal for the next activity.

[0511] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0512] To implement this invention, it is necessary to incorporate an emotion engine that recognizes the user's emotions, in addition to a data collection and analysis system for suggesting optimal activities to the user. The system consists of three elements: a server, a terminal, and a user, and each element works in cooperation with the others.

[0513] The server first collects the child's report card, teacher comments, past behavioral data, and photos and artwork provided by parents and teachers into cloud storage. This allows for the accumulation of data on the child's personality and interests. The server then analyzes this data using a generative AI model and machine learning algorithms to select the most suitable activities for the child. In this process, opinions from parents and children collected through the device are also taken into consideration.

[0514] The emotion engine analyzes the user's emotional data in real time. This engine recognizes emotions from the user's facial expressions, tone of voice, language choices, etc., while they are using the device, and sends this information to the server. The server then adjusts or improves the lesson suggestions, taking the emotional data into consideration.

[0515] Users view suggestions from the server on their devices and participate in the recommended activities. After the activity, teachers and coaches input feedback into the devices, and the server uses this feedback to assess the user's progress. The emotion engine also monitors the user's emotional changes during the activity, reporting to the server, for example, whether the user is enjoying themselves or experiencing difficulties. Based on this, the server evaluates the effectiveness of the activity and prepares to make further suggestions.

[0516] As a concrete example, suppose a child is working on a math lesson. In this case, the emotional engine senses the child's concentration from their facial expression and confirms that the activity has been appropriately selected. On the other hand, if fatigue or frustration is detected by the emotional engine, the server can suggest alternative activities or approaches. This ensures that the support best suited to the child's current state is always provided.

[0517] The following describes the processing flow.

[0518] Step 1:

[0519] The server automates data collection from schools and homes. Specifically, it stores report cards, teacher comments, past behavioral data, and artwork, photos, and videos uploaded by parents and teachers in cloud storage.

[0520] Step 2:

[0521] The server begins analyzing the collected data using generative AI models and machine learning algorithms. The analysis uses natural language processing to identify personality traits and interests from textual information, and image recognition technology to evaluate photos and artwork, thereby understanding the children's skills and areas of interest.

[0522] Step 3:

[0523] Through the device, users input information and opinions about their child's interests and future goals. The entered data is sent to a server and incorporated into the analysis.

[0524] Step 4:

[0525] The server selects the most suitable activity for the child based on analysis results and user feedback. It considers past behavioral data and interests, and generates suggestions after exploring a variety of options.

[0526] Step 5:

[0527] The emotion engine analyzes emotional data collected in real time while the user is operating their device. It determines the user's emotional state from facial expressions, tone of voice, language, etc., and sends the information to the server.

[0528] Step 6:

[0529] The server adjusts activity suggestions based on emotional data. It refines suggestions based on enjoyment and stress indicators, recommending more appropriate activities.

[0530] Step 7:

[0531] Users view activity suggestions provided by the server on their devices and participate in the recommended activities.

[0532] Step 8:

[0533] After a user completes an activity, teachers or coaches provide feedback via the device. This feedback, which specifies the user's achievements and areas for improvement during the activity, is sent to the server.

[0534] Step 9:

[0535] The server comprehensively analyzes feedback and emotional data to track the child's development. Based on this, it prepares a dashboard where users can see suggestions for the next steps.

[0536] (Example 2)

[0537] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0538] Activities and educational programs that children participate in should be tailored to their individual personalities and interests, but currently, there is a problem in accurately selecting such activities. Furthermore, there are insufficient means to properly utilize feedback after activities and incorporate it into future suggestions. Moreover, because it is not possible to track changes in users' emotions in real time and reflect them in activity suggestions, there is a need to respond to individual needs.

[0539] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0540] In this invention, the server includes means for collecting information about a child's personality, interests, and past behavior; means for analyzing the collected information using generative information processing technology and machine learning technology to select the most suitable activity for the child; and means for recognizing the user's emotions and analyzing the emotional data. This makes it possible to select the most suitable activity suggestion based on the individual characteristics of each child, and also enables improvement of the suggestion using feedback and emotional data.

[0541] "Personality" refers to psychological characteristics that describe an individual's behavior and way of thinking, and it is an element that constitutes a consistent pattern of an individual.

[0542] "Interest" is an emotional tendency that indicates the degree of interest or excitement an individual feels towards a particular activity or matter.

[0543] "Past behavior" refers to information that records a series of activities and choices an individual has made up to that point, and it is data that can be used to predict future behavior.

[0544] "Information processing technology" refers to technologies that use computer systems and algorithms to collect, analyze, and manage data.

[0545] "Machine learning technology" is a technique that learns patterns and rules from large amounts of data and uses that knowledge to make predictions and classifications about new data.

[0546] An "activity" is a set of actions or endeavors undertaken with a specific purpose in mind, and may include educational, cultural, or recreational elements.

[0547] A "user" is someone who uses the system and benefits from it, and in this context, it mainly refers to children and their guardians.

[0548] "Emotional data" refers to information that indicates the emotional state of a user, derived from their facial expressions, tone of voice, language choices, and other factors.

[0549] "Feedback" refers to evaluations and opinions received after an activity, and is an important source of information for improving future actions and choices.

[0550] A "suggestion" is an action or choice recommended to the user based on collected and analyzed information.

[0551] To implement this invention, multiple components are used in combination. The system consists of three elements: a server, a terminal, and a user, which work together in cooperation.

[0552] The server plays a central role in collecting and analyzing large amounts of data and generating activity suggestions. Specifically, the server continuously collects information about the child's personality, interests, and past behavior using cloud storage. Next, the server analyzes this information using generative AI models and machine learning techniques to select the most suitable activities for the child. User feedback transmitted from the device is also taken into consideration during this selection process.

[0553] The device is equipped with an emotion engine that analyzes the user's emotions in real time. This engine extracts emotional data from the user's actions through the device (e.g., facial recognition, voice tone analysis, etc.). The collected emotional data is sent to a server and used to improve the suitability of activity suggestions.

[0554] Through the provided interface, users review proposals from the server and participate in selected activities. At this time, users can input feedback on the activity's progress and results into their terminal. The server collects and analyzes this feedback to improve the accuracy of future activity proposals.

[0555] As a concrete example, let's consider a scenario where a user is working on a math lesson. The server recommends activities based on the user's past math performance and interests. The device uses an emotion engine to determine the user's level of concentration and reports to the server, for example, whether the user is "concentrating" or "feeling fatigued." Based on the feedback the user provides after the activity, the server evaluates the effectiveness of the activity and uses this information to make future recommendations.

[0556] Examples of prompts include, "How can I check if the child's attention span is sustained?" and "Please suggest alternative approaches to engage the child's interest." In this way, the system dynamically adjusts activities to provide the optimal educational experience.

[0557] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0558] Step 1:

[0559] The server collects data on children's personalities, interests, and past behaviors from cloud storage. This input data includes report cards, teacher comments, and past activity records. The server integrates this data to generate individual profiles, which provides a foundation for analyzing each child's unique characteristics.

[0560] Step 2:

[0561] The server analyzes the collected data using a generative AI model and machine learning techniques. This data processing executes a specific algorithm to select the optimal activity. The input is profile data, and the output is a list of recommended activities for each child. The server then prepares activity suggestions based on this.

[0562] Step 3:

[0563] The device collects the user's (child's) emotional data in real time. It uses a camera and microphone to record the user's facial expressions and voice tone as input. The device's emotion engine analyzes this data to detect changes in emotion. The collected emotional information is then sent to a server as output. This data is used to adjust activity suggestions.

[0564] Step 4:

[0565] Users receive and participate in suggested activities via their devices. They then review the activity information output from the server and proceed with the activity. Specific features include calendar notifications and interactive guidelines designed to pique their interest.

[0566] Step 5:

[0567] Users provide feedback via a device after completing an activity. This feedback includes their satisfaction level, the difficulty they experienced, and areas for improvement. The device sends this information to a server, where it is stored in a database. The server analyzes this information to evaluate the effectiveness of the activity.

[0568] Step 6:

[0569] The server updates its next activity suggestions based on feedback and sentiment data. In this process, the algorithm re-evaluates the input information and generates new activity proposals. The output is a recommended plan that will be incorporated into the next suggestion.

[0570] (Application Example 2)

[0571] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0572] This invention aims to solve the problem of suggesting optimal activities based on each child's individual personality and emotional state. Conventional educational support systems have difficulty providing individualized support that takes into account the user's real-time emotions, and tend to offer uniform activity suggestions. As a result, it has been difficult to draw out children's motivation and concentration, and individual learning effects have not been fully achieved.

[0573] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0574] In this invention, the server includes a device for collecting data on the child's nature, interests, and past behavior; a device for analyzing the collected data and determining the optimal behavior for the child; and a device including an emotion recognition engine for analyzing the user's emotional state in real time. This enables the dynamic suggestion of optimal activities according to the child's emotions and state, and flexible educational support tailored to individual characteristics.

[0575] "A child's characteristics" refer to the personality and behavioral traits that each child possesses, and serve as the basis for judging the suitability of their education and activities.

[0576] "Interest" refers to the objects or fields that a child is particularly interested in and drawn to.

[0577] "Action data" refers to information that records a child's past activity history and behavioral patterns.

[0578] "Analysis" refers to the process of analyzing information based on collected data and deriving meaningful conclusions.

[0579] An "emotion recognition engine" refers to a device or software that analyzes a user's facial expressions, tone of voice, and manner of speaking in real time to identify their emotional state.

[0580] "Suggestion" refers to the act of showing users what is considered the most appropriate activity or behavior based on analysis results and emotional state.

[0581] "Evaluation information" refers to the feedback and results obtained after the implementation of a proposed activity, and is information used to improve subsequent proposals.

[0582] An "information display device" refers to a device or platform that allows for the visual confirmation of the progress and effectiveness of activities.

[0583] The system implementing this invention consists of three elements: a server, a terminal, and a user. The server utilizes cloud-based data storage to collect data on the child's characteristics, interests, and past behavior, and analyzes this data using generative AI models and machine learning algorithms. It also implements an emotion recognition engine to recognize the user's emotional state in real time.

[0584] The emotion recognition engine collects the user's facial expressions and tone of voice in real time through the camera and microphone built into the device. This data is processed using libraries such as OpenCV and TensorFlow to identify the user's current emotional state.

[0585] The user's terminal receives analysis results and suggestions from the server and presents them to the user via a display device. The user participates in the suggested activities and inputs feedback into the terminal. The server further analyzes the data based on this feedback and optimizes the next activity suggestion.

[0586] A concrete example is monitoring a user's behavior while they are working on a math problem. If the emotion recognition engine detects from the user's facial expressions that they are concentrating, an encouraging message such as "Keep up the good work!" will be displayed on the device. On the other hand, if fatigue is detected, a suggestion such as "Why don't you take a short break?" will be made.

[0587] To implement this system, you can use example prompts such as, "Please analyze whether the child is highly interested based on their facial expressions and voice data."

[0588] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0589] Step 1:

[0590] The server collects data on children's characteristics, interests, and past behavior into cloud-based data storage. Inputs include children's report cards, teacher comments, behavioral history, and photos and artwork provided by parents and teachers. Outputs generate detailed datasets related to each individual child. This data serves as the basis for subsequent analysis by generative AI models and machine learning algorithms.

[0591] Step 2:

[0592] The server uses a generative AI model to analyze the collected data and determine the optimal actions for the child. The input is the dataset collected in Step 1. The output is a list of suggested activities appropriate to the child's personality and interests. Data processing includes data preprocessing and feature extraction.

[0593] Step 3:

[0594] The user receives activity suggestions displayed through their device. The input is a list of suggestions from the server, which the device presents to the user in an easy-to-understand format. The output is a list of suggested activities displayed to the user.

[0595] Step 4:

[0596] The user performs an activity, and the emotion recognition engine monitors their emotional state in real time. The recognition engine receives facial expression data and audio data obtained from the device's camera and microphone as input. Data processing includes analysis using OpenCV and TensorFlow. The output is information about the detected emotional state.

[0597] Step 5:

[0598] The server adjusts activity suggestions based on emotional state information and analyzes feedback received from the terminal after the activity is completed. Inputs are emotional data from the emotion recognition engine and feedback from the terminal. Output is the adjusted activity suggestion for the next activity. Data processing based on the feedback analysis is performed and reflected in the suggestion.

[0599] Step 6:

[0600] The terminal presents the user with adjusted suggestions, providing them with a guide to the improved next activity. The input is the updated suggestions from the server, and the output is information suggesting the next activity the user should participate in.

[0601] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0602] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0603] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0604] [Fourth Embodiment]

[0605] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0606] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0607] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0608] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0609] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0610] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0611] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0612] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0613] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0614] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0615] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0616] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0617] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0618] To implement this invention, it is necessary to build a system that collects and analyzes diverse data on children and proposes optimal activities based on the results. The system mainly operates with three main elements: a server, a terminal, and a user.

[0619] The server first has the functionality to automatically collect data such as children's report cards, conversation history, artwork, and video data from schools, parents, and teachers. This data is stored in cloud storage. The server then analyzes the collected data using generative AI models and machine learning algorithms. This analysis utilizes natural language processing technology to identify children's personalities from teachers' comments and conversations, and image recognition technology to evaluate artwork and photographs and extract children's interests.

[0620] The device provides an interface for parents and children to access the system. It functions as a platform for users to input their individual preferences, such as activities they excel at or areas of interest. This allows the device to collect user feedback, send it to the server, and use it for further analysis.

[0621] Users receive optimal activities suggested by the server on their device and try out new lessons based on these suggestions. The server receives feedback from teachers and coaches after the activities, analyzes this feedback, and makes necessary improvements. Through a dashboard provided by the system, users can track how their child is growing and how their interests are changing. This dashboard allows users to visually check the progress and effectiveness of the activities, helping them understand their child's development in real time.

[0622] For example, if past data indicates a child's interest in art, and the parents also indicate they are considering music-related activities, the server analyzes this information and suggests art classes or music workshops. Furthermore, based on feedback from teachers after the activities, the suggestions are refined and used for the next stage. This ensures that the system consistently recommends extracurricular activities that best match the child's interests and personality.

[0623] The following describes the processing flow.

[0624] Step 1:

[0625] The server collects student report cards and teacher comments from school performance management systems and online platforms. Furthermore, it stores artwork, photos, and videos uploaded by parents and teachers in cloud storage.

[0626] Step 2:

[0627] The server analyzes the collected data using generative AI models and machine learning algorithms. Here, natural language processing techniques are used to analyze teacher comments and conversation history to identify children's personality traits and interests. Image recognition technology is also used to evaluate children's skills and passions from their artwork and photographs.

[0628] Step 3:

[0629] Through a terminal, users access the system and input their opinions and wishes regarding their child's future goals and current interests. This information is transmitted to the server in real time.

[0630] Step 4:

[0631] The server combines analysis results with user feedback to select the most suitable extracurricular activities for children. This process takes into account past behavioral data, interests, and preferences to present a variety of options.

[0632] Step 5:

[0633] The server generates a detailed suggestion report based on the selected activities and sends it to the terminal. The user can review this report on the terminal and try out the suggested activities.

[0634] Step 6:

[0635] After the activity is completed, teachers and coaches provide feedback to users via the device. This includes evaluations of the children's activities and suggestions for improvement.

[0636] Step 7:

[0637] The server analyzes the collected feedback to identify the next activities to pursue and the corresponding improvements. This allows users to continuously track their child's progress on a dashboard and adjust the plan as needed.

[0638] (Example 1)

[0639] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0640] In today's world, selecting and proposing the most suitable activities for each child, tailored to their individual personality and interests, is complex. Traditional methods make it difficult to systematically collect and analyze diverse data about children, resulting in the inability to provide individually optimized suggestions. Furthermore, it is difficult to appropriately evaluate the changes that occur after an activity and reflect them in future suggestions. This leads to challenges in adequately supporting children's growth and the deepening of their interests.

[0641] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0642] In this invention, the server includes a management device for collecting information, a storage device for storing the data collected by the management device, and an analysis device for analyzing the stored data using natural language processing and image recognition technology. This makes it possible to effectively analyze diverse data on children and accurately propose activities that are appropriate for each individual child. Furthermore, by using a generative AI model based on the analysis results, the effectiveness of the activities can be continuously monitored, and suggestions can be updated to reflect the feedback.

[0643] "Information" refers to all kinds of data necessary to provide individually optimized activity suggestions for children, including their personality, interests, academic performance, and activity history.

[0644] A "management device" is an electronic system or device used to collect and organize information about children from schools, parents, teachers, and other sources.

[0645] A "storage device" refers to a data storage or database system used to store information collected by a management device, and is a device that enables secure storage and rapid access to data.

[0646] An "analysis device" is a data processing device or software that analyzes stored data using natural language processing and image recognition technologies to derive activities suitable for children.

[0647] A "generative AI model" is a machine learning model used to suggest the most suitable activities for children based on analysis. It is an algorithm that generates recommended actions based on past data and newly entered information.

[0648] "Activities" refer to specific learning or experiences, such as classes or workshops, that aim to improve children's skills and interests through participation.

[0649] A "computational device" is a device or system that operates an AI model based on data obtained from an analysis device to provide optimal activity suggestions for children and their guardians.

[0650] A "display device" refers to a screen or application used by a server to visually present activity suggestions generated by a computing device.

[0651] A "renewal device" is a system that reviews the analysis and generation process based on feedback collected after an activity, in order to make the best possible suggestions, and incorporates these findings into the next proposal.

[0652] An "input device" refers to a device or interface that allows users to input their opinions and new requests regarding activities, thereby enabling individually customized suggestions.

[0653] A "control device" is a device or software that incorporates user opinions and requests obtained from input devices into the server's data analysis process, and ultimately reflects them in the proposed content.

[0654] A "visualization device" is a device that provides a graphical user interface for visually confirming the effects of growth and activities, making it easy for users to understand their progress.

[0655] The system of this invention mainly consists of three elements: a server, a terminal, and a user. The server plays a central role in collecting, analyzing, generating, and updating information. The information collected includes a child's personality, interests, academic performance, and past activity history. The information is provided by schools, parents, and teachers and collected through a management device. The data is securely stored in storage devices such as cloud storage systems (e.g., AWS S3 or Google Cloud Storage).

[0656] The server then uses an analysis device equipped with natural language processing and image recognition technologies to analyze the collected data in detail. Machine learning algorithms and generative AI models enable analysis that suggests activities best suited to the child. This process generates suggestions in a safe and secure environment. Example prompts can be used at this stage. For example, a sentence like, "For A, who is interested in painting, we recommend an art class," could be used.

[0657] The terminal provides an interface for users to access the system and review suggested activities. A web browser or dedicated application acts as a bridge for information between the user and the server. Users can input their thoughts and preferences through the terminal. This information is then reflected in the analysis process by the control unit.

[0658] Users can visually review the server-generated suggestions on a dashboard on their device. This allows them to track their child's progress and the effectiveness of their activities in real time. The server collects feedback after activities and uses an update device to revise and improve the suggestions, reflecting them in future suggestions. Therefore, the system can provide an environment that continuously supports children's growth and interests.

[0659] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0660] Step 1:

[0661] The server collects information from schools, parents, and teachers. Inputs include children's grades, conversation history, artwork, and video data. This information is collected through a management device, and the collected data is stored in cloud storage. The server periodically retrieves new data and stores it in its storage device based on an execution schedule.

[0662] Step 2:

[0663] The server analyzes the data stored in the storage device using an analysis device. The input includes diverse data collected in step 1. The server utilizes natural language processing techniques to extract the child's personality traits from the text data. It also analyzes visual data using image recognition techniques to identify interests and concerns. The output here is a summary of the child's personality profile and interests.

[0664] Step 3:

[0665] The server uses the analysis results and the generative AI model to generate activity suggestions. The input is the personality profile and interest summary obtained in step 2. The server inputs prompt sentences into the generative AI model to create specific activity suggestions for children. For example, the output might be, "For Person A, who is interested in music, we recommend a music lesson."

[0666] Step 4:

[0667] The terminal visually displays the server-generated suggestions through a user interface. The input is the activity suggestions generated in step 3. The terminal displays them clearly to the user and provides details about the activities. The user can review these suggestions and decide which activities to try.

[0668] Step 5:

[0669] Users provide feedback through their devices, sharing their experiences and impressions of the activities. This feedback is based on their experiences and achievements gained through the activities. This feedback is sent from the device to a server where it is organized.

[0670] Step 6:

[0671] The server re-analyzes the feedback information using an update device and incorporates it into the next suggestion. The input is the feedback data collected in step 5. The server analyzes this data and adjusts and modifies the activity suggestion as needed. This ensures that the child is always provided with the most up-to-date and optimal activities.

[0672] (Application Example 1)

[0673] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0674] Today's children have diverse interests and personalities, making it difficult to find the optimal learning plan and activities tailored to them. Furthermore, there is a lack of systems that effectively utilize feedback to appropriately track children's growth and provide appropriate suggestions. Therefore, there is a need to easily provide individualized educational plans and support effective learning.

[0675] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0676] In this invention, the server includes means for collecting data on a child's personality, interests, and past behavior; means for analyzing the collected data and selecting the most suitable activities for the child; and means for proposing an individualized educational plan. This makes it possible to provide individualized plans that are tailored to the diverse characteristics of children, and to further update the suggestions based on the results of implementation.

[0677] "A child's personality" refers to the distinctive temperament and behavioral patterns that each child possesses.

[0678] "Interest" refers to a child's interest in or willingness to engage with a particular activity or area of ​​study.

[0679] "Past behavioral data" refers to records of activities and learning experiences a child has had up to that point.

[0680] "Means of collection" refers to the technologies and methods used to gather necessary information, such as databases and sensors.

[0681] "Means of analysis" refer to algorithms and processes used to analyze collected data and derive meaningful conclusions.

[0682] "Means for selecting the optimal activity" refers to a method for determining the most appropriate activity for a child based on analyzed data.

[0683] "Means of proposal" refers to a system for notifying users of selected activities and encouraging their implementation.

[0684] "Methods for analyzing feedback" refer to techniques for analyzing evaluations and opinions obtained after an activity and using them to improve the system.

[0685] "Means of updating" refers to methods for continuously improving the system's proposals based on new information and feedback.

[0686] "Means of proposing individualized education plans" refers to a system that develops and provides educational content tailored to each individual child.

[0687] As a form of implementing the invention, this application example involves building a system that analyzes a child's personality, interests, and past behavioral data to suggest optimal activities. This system operates on the cloud using a smartphone.

[0688] The server first automatically collects all relevant information from schools and parents, such as children's report cards, conversation history, artwork, and video data. Next, the collected data is stored in cloud storage and analyzed using generative AI models and machine learning algorithms. This analysis uses natural language processing technology to identify children's personalities from teachers' comments and conversations, and image recognition technology to evaluate artwork and photos and extract children's interests.

[0689] The terminal functions as a platform that makes it easy for users to access the system, providing an interface for parents and children to input their personal opinions and wishes. This input data is then sent to the server to further improve the accuracy of the analysis.

[0690] Users can view the optimal activities suggested by the server on their devices, gaining new opportunities for learning and extracurricular activities. After the activity, feedback from teachers and coaches is sent back to the server, which then leads to new suggestions. This feedback process ensures that an optimized educational plan is always provided, creating a personalized learning experience.

[0691] For example, if a child is interested in science, participation in science experiment classes or science camps may be suggested. Similarly, if a child is interested in English, an online program where they can converse with a native English-speaking instructor may be recommended.

[0692] An example of a prompt for a generative AI model might be: "Analyze the child's learning history and interests to generate new activity suggestions. Current interests are {object of interest}, and past activity history is {history data}. Generate the optimal activity selection."

[0693] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0694] Step 1:

[0695] The server automatically collects information such as children's report cards, conversation history, artwork, and video data from schools and parents. Input data is obtained from school databases and files provided by parents. This data is stored in cloud storage for later analysis. The output is structured data securely stored in the cloud.

[0696] Step 2:

[0697] The server analyzes data stored in cloud storage using machine learning algorithms and generative AI models. Here, natural language processing techniques are used to identify children's personalities from teacher comments and conversations, and image recognition techniques are used to evaluate artwork and photographs and extract children's interests. The input is the data collected in step 1, and the output is characteristic information about children's personalities and interests as a result of the analysis.

[0698] Step 3:

[0699] The device provides an interface that parents and children can access through an application to input their opinions on areas and activities that interest the child. This input data is sent to a server and used for further analysis. The input is the user's opinion, and the output is the user's feedback data prepared for analysis.

[0700] Step 4:

[0701] The server uses a generative AI model to select and suggest activities best suited to the child, based on the analyzed data and user input. The input here consists of previous analysis results and user feedback, while the output is a list of suggested activities. A prompt such as "Analyze the child's learning history and interests to generate new activity suggestions" is used for the generative AI model.

[0702] Step 5:

[0703] Users receive activities suggested by the server via their terminal, select one, and perform it. After participating in an activity, users input feedback on it within the application. This reports the activity results to the server. The input consists of the user's opinions and feedback after the activity, and the output is an updated activity log.

[0704] Step 6:

[0705] The server reanalyzes the feedback received after an activity and improves and updates the original activity proposal as needed. This continuously optimizes the individualized educational plan. The input is feedback data, and the output is an improved proposal for the next activity.

[0706] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0707] To implement this invention, it is necessary to incorporate an emotion engine that recognizes the user's emotions, in addition to a data collection and analysis system for suggesting optimal activities to the user. The system consists of three elements: a server, a terminal, and a user, and each element works in cooperation with the others.

[0708] The server first collects the child's report card, teacher comments, past behavioral data, and photos and artwork provided by parents and teachers into cloud storage. This allows for the accumulation of data on the child's personality and interests. The server then analyzes this data using a generative AI model and machine learning algorithms to select the most suitable activities for the child. In this process, opinions from parents and children collected through the device are also taken into consideration.

[0709] The emotion engine analyzes the user's emotional data in real time. This engine recognizes emotions from the user's facial expressions, tone of voice, language choices, etc., while they are using the device, and sends this information to the server. The server then adjusts or improves the lesson suggestions, taking the emotional data into consideration.

[0710] Users view suggestions from the server on their devices and participate in the recommended activities. After the activity, teachers and coaches input feedback into the devices, and the server uses this feedback to assess the user's progress. The emotion engine also monitors the user's emotional changes during the activity, reporting to the server, for example, whether the user is enjoying themselves or experiencing difficulties. Based on this, the server evaluates the effectiveness of the activity and prepares to make further suggestions.

[0711] As a concrete example, suppose a child is working on a math lesson. In this case, the emotional engine senses the child's concentration from their facial expression and confirms that the activity has been appropriately selected. On the other hand, if fatigue or frustration is detected by the emotional engine, the server can suggest alternative activities or approaches. This ensures that the support best suited to the child's current state is always provided.

[0712] The following describes the processing flow.

[0713] Step 1:

[0714] The server automates data collection from schools and homes. Specifically, it stores report cards, teacher comments, past behavioral data, and artwork, photos, and videos uploaded by parents and teachers in cloud storage.

[0715] Step 2:

[0716] The server begins analyzing the collected data using generative AI models and machine learning algorithms. The analysis uses natural language processing to identify personality traits and interests from textual information, and image recognition technology to evaluate photos and artwork, thereby understanding the children's skills and areas of interest.

[0717] Step 3:

[0718] Through the device, users input information and opinions about their child's interests and future goals. The entered data is sent to a server and incorporated into the analysis.

[0719] Step 4:

[0720] The server selects the most suitable activity for the child based on analysis results and user feedback. It considers past behavioral data and interests, and generates suggestions after exploring a variety of options.

[0721] Step 5:

[0722] The emotion engine analyzes emotional data collected in real time while the user is operating their device. It determines the user's emotional state from facial expressions, tone of voice, language, etc., and sends the information to the server.

[0723] Step 6:

[0724] The server adjusts activity suggestions based on emotional data. It refines suggestions based on enjoyment and stress indicators, recommending more appropriate activities.

[0725] Step 7:

[0726] Users view activity suggestions provided by the server on their devices and participate in the recommended activities.

[0727] Step 8:

[0728] After a user completes an activity, teachers or coaches provide feedback via the device. This feedback, which specifies the user's achievements and areas for improvement during the activity, is sent to the server.

[0729] Step 9:

[0730] The server comprehensively analyzes feedback and emotional data to track the child's development. Based on this, it prepares a dashboard where users can see suggestions for the next steps.

[0731] (Example 2)

[0732] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0733] Activities and educational programs that children participate in should be tailored to their individual personalities and interests, but currently, there is a problem in accurately selecting such activities. Furthermore, there are insufficient means to properly utilize feedback after activities and incorporate it into future suggestions. Moreover, because it is not possible to track changes in users' emotions in real time and reflect them in activity suggestions, there is a need to respond to individual needs.

[0734] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0735] In this invention, the server includes means for collecting information about a child's personality, interests, and past behavior; means for analyzing the collected information using generative information processing technology and machine learning technology to select the most suitable activity for the child; and means for recognizing the user's emotions and analyzing the emotional data. This makes it possible to select the most suitable activity suggestion based on the individual characteristics of each child, and also enables improvement of the suggestion using feedback and emotional data.

[0736] "Personality" refers to psychological characteristics that describe an individual's behavior and way of thinking, and it is an element that constitutes a consistent pattern of an individual.

[0737] "Interest" is an emotional tendency that indicates the degree of interest or excitement an individual feels towards a particular activity or matter.

[0738] "Past behavior" refers to information that records a series of activities and choices an individual has made up to that point, and it is data that can be used to predict future behavior.

[0739] "Information processing technology" refers to technologies that use computer systems and algorithms to collect, analyze, and manage data.

[0740] "Machine learning technology" is a technique that learns patterns and rules from large amounts of data and uses that knowledge to make predictions and classifications about new data.

[0741] An "activity" is a set of actions or endeavors undertaken with a specific purpose in mind, and may include educational, cultural, or recreational elements.

[0742] A "user" is someone who uses the system and benefits from it, and in this context, it mainly refers to children and their guardians.

[0743] "Emotional data" refers to information that indicates the emotional state of a user, derived from their facial expressions, tone of voice, language choices, and other factors.

[0744] "Feedback" refers to evaluations and opinions received after an activity, and is an important source of information for improving future actions and choices.

[0745] A "suggestion" is an action or choice recommended to the user based on collected and analyzed information.

[0746] To implement this invention, multiple components are used in combination. The system consists of three elements: a server, a terminal, and a user, which work together in cooperation.

[0747] The server plays a central role in collecting and analyzing large amounts of data and generating activity suggestions. Specifically, the server continuously collects information about the child's personality, interests, and past behavior using cloud storage. Next, the server analyzes this information using generative AI models and machine learning techniques to select the most suitable activities for the child. User feedback transmitted from the device is also taken into consideration during this selection process.

[0748] The device is equipped with an emotion engine that analyzes the user's emotions in real time. This engine extracts emotional data from the user's actions through the device (e.g., facial recognition, voice tone analysis, etc.). The collected emotional data is sent to a server and used to improve the suitability of activity suggestions.

[0749] Through the provided interface, users review proposals from the server and participate in selected activities. At this time, users can input feedback on the activity's progress and results into their terminal. The server collects and analyzes this feedback to improve the accuracy of future activity proposals.

[0750] As a concrete example, let's consider a scenario where a user is working on a math lesson. The server recommends activities based on the user's past math performance and interests. The device uses an emotion engine to determine the user's level of concentration and reports to the server, for example, whether the user is "concentrating" or "feeling fatigued." Based on the feedback the user provides after the activity, the server evaluates the effectiveness of the activity and uses this information to make future recommendations.

[0751] Examples of prompts include, "How can I check if the child's attention span is sustained?" and "Please suggest alternative approaches to engage the child's interest." In this way, the system dynamically adjusts activities to provide the optimal educational experience.

[0752] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0753] Step 1:

[0754] The server collects data on children's personalities, interests, and past behaviors from cloud storage. This input data includes report cards, teacher comments, and past activity records. The server integrates this data to generate individual profiles, which provides a foundation for analyzing each child's unique characteristics.

[0755] Step 2:

[0756] The server analyzes the collected data using a generative AI model and machine learning techniques. This data processing executes a specific algorithm to select the optimal activity. The input is profile data, and the output is a list of recommended activities for each child. The server then prepares activity suggestions based on this.

[0757] Step 3:

[0758] The device collects the user's (child's) emotional data in real time. It uses a camera and microphone to record the user's facial expressions and voice tone as input. The device's emotion engine analyzes this data to detect changes in emotion. The collected emotional information is then sent to a server as output. This data is used to adjust activity suggestions.

[0759] Step 4:

[0760] Users receive and participate in suggested activities via their devices. They then review the activity information output from the server and proceed with the activity. Specific features include calendar notifications and interactive guidelines designed to pique their interest.

[0761] Step 5:

[0762] Users provide feedback via a device after completing an activity. This feedback includes their satisfaction level, the difficulty they experienced, and areas for improvement. The device sends this information to a server, where it is stored in a database. The server analyzes this information to evaluate the effectiveness of the activity.

[0763] Step 6:

[0764] The server updates its next activity suggestions based on feedback and sentiment data. In this process, the algorithm re-evaluates the input information and generates new activity proposals. The output is a recommended plan that will be incorporated into the next suggestion.

[0765] (Application Example 2)

[0766] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0767] This invention aims to solve the problem of suggesting optimal activities based on each child's individual personality and emotional state. Conventional educational support systems have difficulty providing individualized support that takes into account the user's real-time emotions, and tend to offer uniform activity suggestions. As a result, it has been difficult to draw out children's motivation and concentration, and individual learning effects have not been fully achieved.

[0768] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0769] In this invention, the server includes a device for collecting data on the child's nature, interests, and past behavior; a device for analyzing the collected data and determining the optimal behavior for the child; and a device including an emotion recognition engine for analyzing the user's emotional state in real time. This enables the dynamic suggestion of optimal activities according to the child's emotions and state, and flexible educational support tailored to individual characteristics.

[0770] "A child's characteristics" refer to the personality and behavioral traits that each child possesses, and serve as the basis for judging the suitability of their education and activities.

[0771] "Interest" refers to the objects or fields that a child is particularly interested in and drawn to.

[0772] "Action data" refers to information that records a child's past activity history and behavioral patterns.

[0773] "Analysis" refers to the process of analyzing information based on collected data and deriving meaningful conclusions.

[0774] An "emotion recognition engine" refers to a device or software that analyzes a user's facial expressions, tone of voice, and manner of speaking in real time to identify their emotional state.

[0775] "Suggestion" refers to the act of showing users what is considered the most appropriate activity or behavior based on analysis results and emotional state.

[0776] "Evaluation information" refers to the feedback and results obtained after the implementation of a proposed activity, and is information used to improve subsequent proposals.

[0777] An "information display device" refers to a device or platform that allows for the visual confirmation of the progress and effectiveness of activities.

[0778] The system implementing this invention consists of three elements: a server, a terminal, and a user. The server utilizes cloud-based data storage to collect data on the child's characteristics, interests, and past behavior, and analyzes this data using generative AI models and machine learning algorithms. It also implements an emotion recognition engine to recognize the user's emotional state in real time.

[0779] The emotion recognition engine collects the user's facial expressions and tone of voice in real time through the camera and microphone built into the device. This data is processed using libraries such as OpenCV and TensorFlow to identify the user's current emotional state.

[0780] The user's terminal receives analysis results and suggestions from the server and presents them to the user via a display device. The user participates in the suggested activities and inputs feedback into the terminal. The server further analyzes the data based on this feedback and optimizes the next activity suggestion.

[0781] A concrete example is monitoring a user's behavior while they are working on a math problem. If the emotion recognition engine detects from the user's facial expressions that they are concentrating, an encouraging message such as "Keep up the good work!" will be displayed on the device. On the other hand, if fatigue is detected, a suggestion such as "Why don't you take a short break?" will be made.

[0782] To implement this system, you can use example prompts such as, "Please analyze whether the child is highly interested based on their facial expressions and voice data."

[0783] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0784] Step 1:

[0785] The server collects data on children's characteristics, interests, and past behavior into cloud-based data storage. Inputs include children's report cards, teacher comments, behavioral history, and photos and artwork provided by parents and teachers. Outputs generate detailed datasets related to each individual child. This data serves as the basis for subsequent analysis by generative AI models and machine learning algorithms.

[0786] Step 2:

[0787] The server uses a generative AI model to analyze the collected data and determine the optimal actions for the child. The input is the dataset collected in Step 1. The output is a list of suggested activities appropriate to the child's personality and interests. Data processing includes data preprocessing and feature extraction.

[0788] Step 3:

[0789] The user receives activity suggestions displayed through their device. The input is a list of suggestions from the server, which the device presents to the user in an easy-to-understand format. The output is a list of suggested activities displayed to the user.

[0790] Step 4:

[0791] The user performs an activity, and the emotion recognition engine monitors their emotional state in real time. The recognition engine receives facial expression data and audio data obtained from the device's camera and microphone as input. Data processing includes analysis using OpenCV and TensorFlow. The output is information about the detected emotional state.

[0792] Step 5:

[0793] The server adjusts activity suggestions based on emotional state information and analyzes feedback received from the terminal after the activity is completed. Inputs are emotional data from the emotion recognition engine and feedback from the terminal. Output is the adjusted activity suggestion for the next activity. Data processing based on the feedback analysis is performed and reflected in the suggestion.

[0794] Step 6:

[0795] The terminal presents the user with adjusted suggestions, providing them with a guide to the improved next activity. The input is the updated suggestions from the server, and the output is information suggesting the next activity the user should participate in.

[0796] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0797] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0798] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0799] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0800] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0801] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0802] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0803] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0804] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0805] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0806] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0807] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0808] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0809] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0810] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0811] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0812] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0813] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0814] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0815] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0816] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0817] The following is further disclosed regarding the embodiments described above.

[0818] (Claim 1)

[0819] A means of collecting data on a child's personality, interests, and past behavior,

[0820] A means for analyzing the collected data and selecting the most suitable activity for the child,

[0821] A means of proposing the selected activities to the user,

[0822] A means for analyzing the feedback obtained after the implementation of the aforementioned activities,

[0823] A means of updating the activity proposal based on the aforementioned feedback,

[0824] A system that includes this.

[0825] (Claim 2)

[0826] A means of inputting the opinions of parents and children,

[0827] A means for reflecting the input opinions in the analysis,

[0828] The system according to claim 1, including the following:

[0829] (Claim 3)

[0830] A means of providing a dashboard for tracking the growth effects of activities,

[0831] The aforementioned dashboard provides a means for users to check their growth status,

[0832] The system according to claim 1, including the following:

[0833] "Example 1"

[0834] (Claim 1)

[0835] A management device for collecting information,

[0836] A storage device for storing data collected by the aforementioned management device,

[0837] An analysis device for analyzing the stored data using natural language processing and image recognition technology,

[0838] A computing device that proposes activities using a generative AI model based on the analysis results,

[0839] A display device for visualizing the proposals made by the aforementioned computing device,

[0840] A device for updating and re-analyzing feedback after the activity,

[0841] A system that includes this.

[0842] (Claim 2)

[0843] An input device for entering opinions,

[0844] A control device for reflecting the input opinions in data analysis,

[0845] The system according to claim 1, including the following:

[0846] (Claim 3)

[0847] A means for providing a visualization device to visually confirm the impact of growth,

[0848] Means for displaying information on the aforementioned visualization device,

[0849] The system according to claim 1, including the following:

[0850] "Application Example 1"

[0851] (Claim 1)

[0852] A means of collecting data on a child's personality, interests, and past behavior,

[0853] A means for analyzing the collected data and selecting the most suitable activity for the child,

[0854] A means of proposing the selected activities to the user,

[0855] A means for analyzing the feedback obtained after the implementation of the aforementioned activities,

[0856] A means of updating the activity proposal based on the aforementioned feedback,

[0857] Means of proposing individualized education plans,

[0858] Means for further improving the proposed plan based on the results of its implementation,

[0859] A system that includes this.

[0860] (Claim 2)

[0861] A means of inputting the opinions of parents and children,

[0862] A means for reflecting the input opinions in the analysis,

[0863] A means of making further proposals based on the results of the implementation of the education plan,

[0864] The system according to claim 1, including the following:

[0865] (Claim 3)

[0866] A means of providing a dashboard for tracking the growth effects of activities,

[0867] The aforementioned dashboard provides a means for users to check their growth status,

[0868] A way to review the educational plan proposed through the dashboard,

[0869] The system according to claim 1, including the following:

[0870] "Example 2 of combining an emotion engine"

[0871] (Claim 1)

[0872] Means of collecting information about a child's personality, interests, and past behavior,

[0873] The collected information is analyzed using generative information processing technology and machine learning technology, and a means is used to select the most suitable activity for the child.

[0874] A means of proposing the selected activities to the user,

[0875] A means of recognizing user emotions and analyzing that emotional data,

[0876] A means for analyzing evaluation information obtained after the implementation of the selected activities,

[0877] A means for updating activity suggestions based on the aforementioned evaluation information and sentiment data,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] A means of collecting and inputting the opinions of parents and children,

[0881] A means for reflecting the input opinions in the analysis,

[0882] The system according to claim 1, including the following:

[0883] (Claim 3)

[0884] A means for providing an information display device for tracking the growth effect of an activity,

[0885] The information display device provides a means for the user to check growth information,

[0886] The system according to claim 1, including the following:

[0887] "Application example 2 when combining with an emotional engine"

[0888] (Claim 1)

[0889] A device that collects data on a child's characteristics, interests, and past behaviors,

[0890] A device that analyzes the collected data and determines the optimal action for the child,

[0891] A device that proposes the aforementioned determined action to the user,

[0892] A device for analyzing evaluation information obtained after the aforementioned action,

[0893] A device that updates action proposals based on the aforementioned evaluation information,

[0894] A device including an emotion recognition engine for analyzing the user's emotional state in real time,

[0895] A device that adjusts action suggestions considering the emotion data obtained from the emotion recognition engine,

[0896] A system that includes this.

[0897] (Claim 2)

[0898] A device for inputting the opinions of parents and children,

[0899] A device that incorporates the input opinions into the analysis,

[0900] This includes a device that recognizes the emotional state of a user from their facial expressions and tone of voice collected through a smart device.

[0901] The system according to claim 1.

[0902] (Claim 3)

[0903] A device that provides an information display device for tracking the growth effect of behavior,

[0904] A device on which the user can check the growth status on the aforementioned information display device,

[0905] Includes a device that notifies whether the user is interested in and engaged with the task.

[0906] The system according to claim 1. [Explanation of Symbols]

[0907] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting data on a child's personality, interests, and past behavior, A means for analyzing the collected data and selecting the most suitable activity for the child, A means of proposing the selected activities to the user, A means for analyzing the feedback obtained after the implementation of the aforementioned activities, A means of updating the activity proposal based on the aforementioned feedback, Means of proposing individualized education plans, Means for further improving the proposed plan based on the results of its implementation, A system that includes this.

2. A means of inputting the opinions of parents and children, A means for reflecting the input opinions in the analysis, A means of making further proposals based on the results of the implementation of the education plan, The system according to claim 1, including the following:

3. A means of providing a dashboard for tracking the growth effects of activities, The aforementioned dashboard provides a means for users to check their growth status, A way to review the educational plan proposed through the dashboard, The system according to claim 1, including the following:

Images