system

Abstract

We provide the system. [Solution] A means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment, The aforementioned collected data is analyzed using artificial intelligence to evaluate the developmental status of children, Based on the results obtained from the aforementioned analysis means, a dialogue means for providing educational dialogue to children, A feedback generation means for providing feedback to parents by analyzing the results obtained from the aforementioned analysis means and the emotional state of the child, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional methods for evaluating children's development have problems such as children being nervous at the time of diagnosis and unable to demonstrate their normal abilities, it taking a long time for guardians to obtain appropriate information, and it being difficult to eliminate the anxiety caused by the lack of unified opinions among experts. Furthermore, there is a lack of means to accurately grasp long-term growth trends, and there is also a problem that early detection and appropriate support plans cannot be established.

Means for Solving the Problems

[0005] This invention provides means for collecting data obtained from a device that monitors a child's biological information and behavior in a home environment, and means for analyzing this data using artificial intelligence to evaluate the child's developmental state. Furthermore, it solves the aforementioned problems by providing dialogue means for providing educational dialogue to the child based on the analysis results, and feedback generation means for providing feedback to the parents by analyzing the analysis results and emotional state. In addition, it provides means for predicting the child's developmental patterns over the long term based on the accumulated data, making it possible to provide support with a view to future growth.

[0006] "Children's biometric information" refers to information that indicates a child's physical condition, such as heart rate, body temperature, and activity level.

[0007] "Behavior" refers to the everyday actions and activities of children.

[0008] "Home environment" refers to the environment within the house where a child typically lives their daily life.

[0009] "Monitoring devices" refer to equipment such as cameras and wearable devices used to record a child's biometric information and behavior.

[0010] "Means for collecting data" refers to methods or devices for storing or transferring information obtained from monitoring devices.

[0011] Artificial intelligence is a technology that enables computers to imitate human intellectual behavior.

[0012] "Analysis means" refers to an algorithm or device for evaluating a child's developmental status based on data.

[0013] "Educational dialogue" is a form of communication designed to promote children's education and development.

[0014] A "means of interaction" refers to a program or device used for interacting with children.

[0015] The "feedback generation means" is a method or device for providing advice and information to the guardians based on the analysis results.

[0016] The "development pattern" indicates the trends and characteristics in the growth and development of children.

[0017] The "prediction means" is an algorithm or device for estimating future development based on past data.

Brief Description of the Drawings

[0018] [Figure 1] It is a conceptual diagram showing an example of the configuration of the data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of the data processing device and the smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of the data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of the data processing device and the smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of the data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of the data processing device and the headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of the data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of the data processing device and the robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] The following system configuration is conceivable as an embodiment of this invention. The system monitors and evaluates the developmental status of a child and operates using various devices installed in the home environment.

[0040] First, the user installs a remote camera and wearable device in their home. These devices collect the child's biometric information (heart rate, body temperature, etc.) and behavioral information (playing patterns, walking patterns, etc.) in real time.

[0041] Next, this data is sent to the server via the terminal. The server preprocesses the received data, removing noise and standardizing it. Then, the server analyzes the data using artificial intelligence-based analytical tools to assess the child's developmental status. Assessment items include language development, motor skills, and social skills.

[0042] The analysis results are not simply left as they are; the device provides interactive dialogue tailored to the child. For example, based on the evaluation score generated on the server, an interactive AI character engages in educational dialogue with the child through play. This aligns with the objective of improving the child's developmental indicators.

[0043] Furthermore, by analyzing the collected facial expression data, the server can understand the child's emotional state in real time. Based on this, it can generate feedback, such as suggesting relaxing activities if the child is feeling anxious, and notify the parent user.

[0044] Furthermore, this system has a function to predict future development. Based on data accumulated over a long period, the server uses time-series analysis techniques to predict future growth patterns and provides parents with guidance plans and support measures for the future.

[0045] This series of processes allows parents to accurately understand how their child's development is progressing compared to others and how they should provide support. This invention is not merely a developmental assessment tool, but is also extremely useful for providing educational support optimized for each individual child. For example, if the server determines that a child is behind in language development, it can provide the child with an interactive game that allows for speech training via the device, thereby promoting learning in a natural way.

[0046] The following describes the processing flow.

[0047] Step 1:

[0048] The user installs a remote camera and wearable device in their home environment. These devices capture the child's biometric and behavioral information and transmit the data to a server via the device or home network.

[0049] Step 2:

[0050] The server preprocesses the received data. Specifically, it splits video data into individual frames, removes noise from audio data, and standardizes biometric information obtained from wearable devices.

[0051] Step 3:

[0052] The server uses pre-processed data to perform multimodal analysis utilizing artificial intelligence. Image analysis is used to evaluate children's motor skills, speech recognition is used to analyze language development, and the various indicators are integrated.

[0053] Step 4:

[0054] The device uses analysis results from the server to provide children with conversations and games through an interactive AI character. For example, if language training is needed, it will conduct language practice through interactive games.

[0055] Step 5:

[0056] The server uses facial recognition technology to analyze the child's emotions from video data and evaluates their state in real time. If the stress level is high, it prepares to notify the user of that information as feedback.

[0057] Step 6:

[0058] Based on the aggregated analysis data, the server generates expert-reviewed feedback and sends it to the user as a report via email or a dedicated app.

[0059] Step 7:

[0060] The server analyzes long-term accumulated data to predict future developmental patterns. Using time-series models, it can, for example, propose a learning plan for the next semester.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] Supporting children's health and learning during their developmental stages is challenging because it relies on limited resources and environments within the home, making timely and appropriate responses difficult. Furthermore, the lack of systematic tools for accurately understanding a child's condition and enabling parents to appropriately utilize that information is a barrier to effective childcare support.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment, means for analysis using a generative AI model, and means for dialogue that provides educational dialogue. This enables real-time evaluation of the child's developmental status and provides appropriate educational support and feedback to parents based on that evaluation.

[0066] "Biometric information" refers to data that indicates a child's physical condition, such as heart rate and body temperature.

[0067] "Behavior" refers to patterns of a child's movements and activities, including how they play and walk.

[0068] "Means of collection" refers to devices that use sensors, cameras, etc., to acquire children's biometric information and behavior.

[0069] "Analysis methods using generative AI models" refer to the process of using artificial intelligence technology to process collected data and evaluate the developmental status of children.

[0070] "Dialogue means" refers to the functions of a system that allows for direct interaction with children, provided based on analysis.

[0071] A "feedback generation method" is a mechanism that generates information to convey analysis results and the child's emotional state to the parents.

[0072] "Emotional state analysis methods" refer to the process of inferring a child's emotions from their facial expressions and behavior, and then determining the appropriate response.

[0073] A "predictive tool" is an analytical technique used to predict future growth patterns based on accumulated data.

[0074] One possible embodiment of this invention is a system combining a data collection device, a server, and a terminal installed in a home environment. First, the user starts by installing a remote camera and a wearable device in their home. These devices are means of collecting the child's daily activities and biometric information, recording heart rate, body temperature, movement patterns, etc., in real time.

[0075] The collected data is transmitted to the server via the terminal. Data communication is secure through an encrypted protocol using Wi-Fi or Bluetooth. The server preprocesses the received data and analyzes it using a generative AI model. This generative AI model uses the collected data to evaluate the child's development, for example, assessing motor skills and social skills.

[0076] Furthermore, based on the analysis results, the device uses an interactive AI character to generate educational conversations with the child. These conversations are customized according to the child's developmental stage; for example, if it is determined that the child's language development is delayed, it will provide a game that allows for speech training.

[0077] Furthermore, the server uses facial recognition technology to analyze the child's emotional state in real time and provides relaxation techniques and notifications to parents as needed. In the long term, it predicts the child's growth patterns based on accumulated data through time-series analysis and proposes an instructional plan to parents based on the results.

[0078] As a concrete example, here is an example of a prompt message:

[0079] "Please tell me about recommended activities for assessing and improving children's current motor skills."

[0080] The entire system provides parents with information to better support their children's growth and development, and enables them to address individual educational needs.

[0081] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0082] Step 1:

[0083] The user installs a remote camera and wearable device in their home. The inputs to this process are the device's installation location and environmental settings. The device then begins collecting biometric and behavioral information about the child, including heart rate, body temperature, and movement patterns. The output is a real-time updated data stream.

[0084] Step 2:

[0085] The device receives collected data and transmits it to the server via Wi-Fi or Bluetooth. The input consists of biometric and behavioral data acquired from sensors. During this transmission process, the data is securely sent using encryption technology. The output is a verified data packet sent to the server.

[0086] Step 3:

[0087] The server preprocesses the received data. The input is raw data sent from the terminal. This step involves denoising the data, adjusting timestamps, and standardizing the data. The output is preprocessed data in a format suitable for analysis.

[0088] Step 4:

[0089] The server analyzes the preprocessed data using a generative AI model. The input is the standardized data output in step 3. The analysis is performed by applying machine learning algorithms to evaluate developmental and emotional states. The output is the evaluation results of the child's motor skills, language development, emotional state, etc.

[0090] Step 5:

[0091] The device controls an interactive AI character based on evaluation results from the server and generates educational dialogues. The input is the analysis results. In this step, the dialogue content is programmed according to the child's developmental stage and displayed as a game or educational activity. The output is the interactive experience provided on the screen.

[0092] Step 6:

[0093] The server analyzes facial expression data in real time and provides feedback to the parent user about the child's emotional state. The input is facial expression data from a remote camera. The server analyzes this data and identifies emotions such as anxiety and joy. The output is a suggestion of relaxation activities and a notification to the parent based on this analysis.

[0094] Step 7:

[0095] The server analyzes accumulated long-term data and uses time-series data to predict growth patterns. The input is historical evaluation data. Time-series analysis techniques are used to predict future development. The output includes predicted growth trends and suggested guidance plans for parents.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] In modern society, accurately understanding a child's developmental stage and providing optimal educational support is a crucial challenge. However, there is a lack of adequate systems for effectively monitoring a child's biometric information and behavior in real time within the home environment and providing appropriate educational guidance based on that data. Furthermore, current systems make it difficult for parents and guardians to accurately understand information about their child's developmental process and take appropriate action. Therefore, innovative solutions are needed to support children's development more efficiently and effectively.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment; analysis means using machine learning to analyze the collected data and evaluate the child's developmental status; and interactive dialogue means for providing educational dialogue to the child based on the results obtained from the analysis means. This enables effective real-time support for a child's development within the home, allowing parents and guardians to take appropriate and prompt action.

[0101] "Children's biometric information" refers to data indicating a child's health status, such as heart rate and body temperature.

[0102] "Devices for monitoring behavior in a home environment" refer to remote cameras and sensor devices installed in the home to capture a child's movements and behavioral patterns.

[0103] "Means of data collection" refers to the functions and processes for collecting children's biometric and behavioral data.

[0104] "Machine learning-based analysis methods" refer to algorithms and models used to evaluate a child's developmental status using collected data.

[0105] "Interactive dialogue means" refers to methods and systems that enable educational and interactive communication with children based on analysis results.

[0106] "Information provision methods" refer to the methods and mechanisms for reporting and notifying parents or guardians of the results obtained through analysis.

[0107] "Automated robot-based behavioral support methods" refer to support methods using robots that operate within the home for the purpose of promoting children's learning and growth.

[0108] The system for implementing this invention has a configuration that effectively supports child development in a home environment. The user installs remote cameras and wearable sensors in the home to collect the child's biometric information and behavioral data. This allows the user to monitor the child's health status and daily activities in real time.

[0109] The collected data is sent to the server via the terminal. The server processes the received data using machine learning, performing noise reduction and data standardization before evaluating the child's developmental status. This analysis utilizes AI models implemented in Python or other appropriate programming languages. The AI ​​models analyze the data based on multiple evaluation criteria, including language development, motor skills, and social skills.

[0110] Analysis results from the server enable educational interactions using robots within the home through interactive dialogue mechanisms. Based on instructions from the server, the robot engages in interactive educational activities with children. These activities include, for example, language training games to increase vocabulary and role-playing to improve social skills.

[0111] Furthermore, the server provides parents and guardians with the results obtained from the analysis. This information is presented as feedback on the child's development through smart glasses or other display devices. This allows parents and guardians to understand their child's current condition and make informed decisions about their next actions.

[0112] As a concrete example, if the server determines that a child is experiencing a delay in language development, the robot will perform an activity to increase vocabulary through simple questions such as, "Which do you prefer? Cats or dogs?" Another example of a prompt when using this invention is to input to the generating AI model, "Design an interactive language training game for a child. The game should be designed so that parents can monitor it through smart glasses, and the child should be able to learn while having fun."

[0113] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0114] Step 1:

[0115] Users install remote cameras and wearable sensors in their homes to collect biometric and behavioral data about their children. These devices acquire biometric data such as heart rate and body temperature, as well as behavioral data such as walking patterns and playtime. The acquired data is transmitted to a device in real time.

[0116] Step 2:

[0117] The terminal transfers data received from the user to the server. The server removes noise from the data and normalizes it as needed. This process prepares the data for AI analysis. The input is raw data such as heart rate and behavioral patterns, and the output is standardized data necessary for the analysis process.

[0118] Step 3:

[0119] The server inputs pre-processed data into an AI analysis engine to evaluate the child's developmental stage. The AI ​​model uses generative AI technology to analyze multiple data points (language development, motor skills, social skills). The evaluation results are output as a numerical evaluation score.

[0120] Step 4:

[0121] The server sends instructions to the robot through interactive dialogue based on the evaluation score obtained. The robot provides children with educational dialogues and activities according to the evaluation score. For example, it conducts activities to improve vocabulary through language training games.

[0122] Step 5:

[0123] The server generates feedback for parents or guardians based on the analysis results and the child's emotional state. This feedback is sent via the device to smart glasses or another display device. The feedback includes information about the child's developmental environment and the support they need. The inputs are the evaluation score and emotional state, and the output is the feedback message.

[0124] Step 6:

[0125] Users utilize feedback provided by the server to plan the next steps in supporting their child's development at home. Based on this feedback, they can adjust educational plans and curricula as needed.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] The following configuration is conceivable as an embodiment of this invention. The system is designed to provide more optimized feedback by evaluating the child's developmental stage and also taking into account the emotional state of the parent, who is the user.

[0128] Users place remote cameras and wearable devices in their home environment, and these devices are used to automatically collect biometric information and behavioral data of their children. These devices transmit data to a server via Wi-Fi or Bluetooth.

[0129] The server preprocesses received biometric and behavioral data, and then uses artificial intelligence to analyze it. This allows for the evaluation of various developmental indicators, such as motor skills and language development. In addition, an emotion recognition algorithm is used to analyze the child's emotional state from the collected data, determining whether they are relaxed or stressed.

[0130] Based on the analysis results, the device displays customized interactive educational content for the child through an interactive AI character. For example, if the server determines that the child needs to improve their social skills, the AI ​​character will suggest a social game.

[0131] Furthermore, the system incorporates an emotion engine that analyzes the user's (parent's) emotional state. To do this, the server performs emotion analysis using the user's voice data and facial expression data acquired from the camera. The emotion engine determines the parent's stress level and level of interest, and adjusts the content and timing of feedback accordingly.

[0132] Finally, the server generates expert-reviewed feedback from the analyzed data and emotional state information, which is then delivered to the user via email or app notification. For example, the server could include advice such as suggesting that parents engage in relaxing activities with their children at the end of the day to reduce stress.

[0133] In this way, by considering not only the child's biometric information and behavioral data but also the parent's emotional state, the system consistently provides effective childcare support. For example, to improve delays in a child's motor skills indicated by the analysis results, the emotional engine can utilize methods to attract the parent's attention, thereby supporting parents in actively participating in their child's training.

[0134] The following describes the processing flow.

[0135] Step 1:

[0136] Users install remote cameras and wearable devices in their home environment to continuously record their child's biometric information and behavior. Additionally, if parents use a smartphone or PC, they can configure it to capture audio and facial expressions via the camera.

[0137] Step 2:

[0138] The recorded biometric and behavioral data is sent to the device, which temporarily stores this data. The data is then configured to be uploaded to a server via Wi-Fi or Bluetooth.

[0139] Step 3:

[0140] The server preprocesses the received data. This processing includes noise reduction and frame splitting of video data, clearance of audio data, and standardization of biometric data from wearable devices.

[0141] Step 4:

[0142] The server uses artificial intelligence to analyze child development indicators based on pre-processed data. For example, it analyzes and scores a child's motor movements from image data and evaluates language development from audio data.

[0143] Step 5:

[0144] Simultaneously, the server uses an emotion engine to analyze the parent's voice and facial expression data. The emotion engine extracts voice tone and facial expression characteristics and evaluates the parent's emotional state in real time.

[0145] Step 6:

[0146] Based on the analysis results, the device uses an interactive AI character to provide educational content suitable for children. The character engages in conversations and games designed to improve the skills children need, based on the analyzed developmental indicators.

[0147] Step 7:

[0148] The server integrates the data obtained through analysis and generates expert-reviewed feedback. This feedback is customized to take into account the parent's emotional state, and may include suggestions for relaxation methods if the parent is feeling stressed, for example.

[0149] Step 8:

[0150] Ultimately, the server provides feedback to the user via email or app notification. At this time, it will also suggest the next steps and recommended actions to receive effective support.

[0151] Step 9:

[0152] In the long term, the server will analyze the accumulated data, update models to predict future development, and present these predictions to parents, enabling planned childcare support.

[0153] (Example 2)

[0154] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0155] In modern childcare, there is a need to accurately understand a child's developmental and emotional state and provide appropriate feedback to parents. However, conventional technology has not adequately achieved the ability to comprehensively analyze a child's physical and emotional data and provide optimal feedback that also takes into account the parent's emotional state. Therefore, a more intelligent and comprehensive support system is needed to promote a child's development.

[0156] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0157] In this invention, the server includes means for monitoring a child's biometric information and behavior and transmitting data using communication technology, means for preprocessing and analyzing the data and evaluating developmental indicators, and means for evaluating the user's emotions. This makes it possible to provide parents with customized feedback tailored to the individual developmental needs of their children, thereby intelligently and effectively providing childcare support.

[0158] "Children's biometric information" refers to data that represents a child's physical condition, including heart rate, body temperature, and respiratory rate.

[0159] "Behavioral data" refers to information that records a child's daily activities, describing things like distance traveled, playtime, and types of activities.

[0160] "Communication technology" refers to technologies for sending and receiving data between devices, and includes wireless communication methods such as Wi-Fi and Bluetooth.

[0161] An "external processing device" is a computer system that receives data and performs analysis processing, and includes equipment such as servers.

[0162] "Preprocessing" refers to the process of preparing data to make it easier to analyze, and includes tasks such as organizing, cleansing, and imputing missing values.

[0163] "Artificial intelligence" refers to the technology of computers imitating human intellectual behavior, and involves methods such as machine learning and data analysis.

[0164] "Developmental indicators" are standards used to evaluate a child's growth and development, and refer to things like motor skills and language abilities.

[0165] "Emotional state" refers to an individual's emotional reactions and circumstances, including stress, joy, and relaxation.

[0166] "Emotional analysis methods" refer to methods of analyzing an individual's emotional state based on data, and utilize techniques such as voice analysis and facial expression analysis.

[0167] "Interactive educational information" refers to content that aims to achieve educational effects through interaction, and includes games and conversational simulations.

[0168] "Customized feedback" refers to feedback that is tailored to an individual's characteristics and needs.

[0169] This system aims to assess a child's developmental stage and provide appropriate feedback to parents. The system is structured as follows:

[0170] Users install remote cameras and wearable devices in their home environment. These devices incorporate wireless communication technologies such as Bluetooth and Wi-Fi to automatically collect biometric and behavioral data of children. Specific data includes heart rate, body temperature, distance traveled, and playtime.

[0171] Data is transmitted to the server in real time. The server is equipped with a high-performance processor and artificial intelligence model, and performs preprocessing on the received data. Preprocessing includes data cleansing and imputation of missing values. Subsequently, machine learning algorithms are used to evaluate developmental indicators such as children's motor skills and language abilities, and emotion recognition algorithms are used to analyze their emotional state.

[0172] The server further analyzes the parents' voice data and camera footage, and uses an emotion engine to determine stress levels and levels of interest. Based on these analysis results, it generates educational content tailored to the parents' emotions and the child's development.

[0173] Based on the analysis results, the device presents educational content to the child through an interactive AI character. For example, if it determines that the child's motor skills are somewhat lacking, it will introduce engaging fitness games to encourage activity.

[0174] Finally, the server generates expert-reviewed feedback. This feedback is delivered via email or app notifications on the parent's smartphone or computer. A specific example is "suggestions for relaxing evening activities to do with children, aimed at reducing parental stress."

[0175] An example of a prompt message would be: "Generate an appropriate encouraging message based on the latest data on the child's athletic ability. Include techniques to capture the parent's attention."

[0176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0177] Step 1:

[0178] The user installs a remote camera and wearable devices in their home. These devices collect the child's heart rate, body temperature, and activity level in real time. The input consists of biometric and behavioral data obtained from the child. This data is transmitted to a server using wireless communication technology. The output is the raw data sent to the server.

[0179] Step 2:

[0180] The server preprocesses the received data. Specifically, it cleanses the data, detects and imputes outliers and missing values. The input is the biometric and behavioral data transmitted in step 1. The output is a preprocessed, consistent dataset. This processing ensures data quality that allows for accurate subsequent analysis.

[0181] Step 3:

[0182] The server performs AI analysis using pre-processed data. It uses a generative AI model to perform data calculations to evaluate developmental indicators such as motor skills and language abilities. It also analyzes children's emotional states using an emotion recognition algorithm. The input is a pre-processed dataset. The output is the evaluation of developmental indicators and emotional states as analysis results. At this stage, specific emotion models and machine learning algorithms are used.

[0183] Step 4:

[0184] The server evaluates the parent's emotional state. It takes in parental voice data and camera footage as input and uses an emotion engine to analyze stress levels and interest levels. The output is evaluation data regarding the user's (parent's) emotional state. This evaluation allows for the customization of feedback based on the parent's situation.

[0185] Step 5:

[0186] The device generates and presents educational content through an interactive AI character based on analysis results obtained from the server. The input consists of the child's developmental indicators and emotional state, obtained from the server. The output is child-specific educational content, presented in a game or dialogue format.

[0187] Step 6:

[0188] The server generates feedback based on the analysis results and the parents' emotional state. It designs specific suggestions and activity plans and notifies the parents. The inputs are child development data and parent emotional analysis data. The output is a feedback message sent to the parents' devices, which includes expert-reviewed advice. This feedback serves to engage the parents and support their parenting.

[0189] (Application Example 2)

[0190] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0191] In modern family environments, there is a lack of childcare support systems that effectively support children's development while reducing parental stress. In particular, there is a need for technology that analyzes children's biometric information and behavior in real time, and provides appropriate feedback while also considering the parents' emotional state.

[0192] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0193] In this invention, the server includes a device for collecting a child's biometric information and behavioral data, a device for analyzing the collected data using machine learning technology, a device for generating educational dialogues based on the analysis results, an emotion analysis device for analyzing the parent's voice data and facial expression data, an emotion engine for adjusting the content of feedback according to the parent's emotional state, and an interaction generation device for providing interactive educational support. This makes it possible to provide childcare support that promotes the child's development while taking into account the parent's emotional state.

[0194] "Children's biometric information" refers to physical information measured by heart rate, body temperature, and motion sensors.

[0195] "Behavioral data" refers to data that records a child's daily movements and activity patterns.

[0196] A "device" is hardware used to collect biometric information and behavioral data.

[0197] "Machine learning techniques" is a general term for algorithms used to analyze large amounts of data and find patterns and trends.

[0198] "Analysis equipment" refers to hardware and software used to evaluate a child's developmental status using collected data.

[0199] "Educational dialogue" is interactive communication designed to promote a child's development.

[0200] An "emotion analysis device" is hardware and software that uses voice data and facial expression data to determine a parent's emotional state.

[0201] The "emotional engine" is a function that adjusts the content and timing of feedback based on the parent's emotional state.

[0202] An "interaction generation device" is a device that generates content and activities to support education through dialogue with children.

[0203] This invention provides a childcare support system that effectively assists child development in the home environment while also being considerate of parents. The specific system configuration includes the following elements:

[0204] The server collects data using a device equipped with multiple sensors to gather children's biometric and behavioral data. This device includes cameras and wearable sensors, and transmits the data to the server via Wi-Fi or Bluetooth.

[0205] The server analyzes the collected data using machine learning techniques such as Python and TENSORFLOW®. This analysis evaluates developmental indicators such as children's motor skills and language development, and also determines the children's emotional state using an emotion analysis algorithm.

[0206] Based on the analysis results, the device provides dialogue to the child through an interaction generator. The interaction generator dynamically generates educational content using a generation AI model and presents it to the child visually and audibly.

[0207] Furthermore, the server uses an emotion analysis device to analyze voice and facial expression data in order to understand the parent's emotional state. The analysis results are processed by an emotion engine, and feedback is adjusted according to the parent's stress level and level of interest.

[0208] The user (parent) receives final feedback via app notifications or email, which includes specific activity suggestions. For example, when suggesting a quiz game or physical activity for parents and children to play together, a prompt message such as "Please suggest activities to reduce parental stress" is sent to the generating AI model.

[0209] This system allows users to monitor their children's growth while fostering better parent-child relationships.

[0210] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0211] Step 1:

[0212] The server collects real-time biometric and behavioral data of children from sensors placed throughout the home. Inputs are biometric data (heart rate, body temperature, etc.) and behavioral data from the sensors, and output is the aggregation and temporary storage of this data. Specifically, data is transferred from the sensors to the server using Wi-Fi or Bluetooth.

[0213] Step 2:

[0214] The server analyzes the collected data using machine learning techniques. The input consists of biometric and behavioral data aggregated in Step 1. The data is analyzed using Python and TensorFlow, generating output that evaluates children's motor skills, language development, and emotional states. Specifically, it performs data cleansing and normalization, then inputs the data into the model to recognize patterns.

[0215] Step 3:

[0216] The server designs interactions using a dialogue generation device based on the analysis results. The input is the analysis results obtained in step 2, and the generating AI model is used to generate educational content and activity content. The output is dialogue prompts and game content optimized for children. Specifically, after the dialogue content is determined, information to be presented to the child is created using speech synthesis.

[0217] Step 4:

[0218] The device provides the child with content received from the interaction generator. The input consists of dialogue prompts and game content generated in step 3, which are presented to the child as audio and video. The output allows observation of the child's response. In terms of specific actions, interaction is carried out using the screen and speaker.

[0219] Step 5:

[0220] The server collects parent voice and facial expression data and performs emotion analysis. The input is parent voice and facial expression data, and the output is data indicating the parent's emotional state (e.g., stress level). Specifically, it uses a microphone and camera to acquire detailed audio and video data and uses an analysis algorithm to estimate emotions.

[0221] Step 6:

[0222] The server uses an emotion engine to adjust the content of the feedback. The input is the parent's emotional state obtained in step 5 and the analysis results from step 2, and based on this, it generates optimal feedback for the parent. The output is activity suggestions and advice provided to the parent. Specifically, a prompt "Please suggest relaxing activities" is sent to the generating AI model, and the model's response is incorporated as feedback.

[0223] Step 7:

[0224] Users receive feedback and suggestions from the server via their devices and incorporate them into their daily lives. The input is the feedback generated in step 6, and the output is the addition of new activities and challenges to the parent-child relationship. Specifically, users receive notifications using their smartphones or tablets and perform the suggested activities together.

[0225] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0226] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0227] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0228] [Second Embodiment]

[0229] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0230] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0231] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0232] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0233] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0234] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0235] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0236] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0237] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0238] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0239] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0240] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0241] The following system configuration is conceivable as an embodiment of this invention. The system monitors and evaluates the developmental status of a child and operates using various devices installed in the home environment.

[0242] First, the user installs a remote camera and wearable device in their home. These devices collect the child's biometric information (heart rate, body temperature, etc.) and behavioral information (playing patterns, walking patterns, etc.) in real time.

[0243] Next, this data is sent to the server via the terminal. The server preprocesses the received data, removing noise and standardizing it. Then, the server analyzes the data using artificial intelligence-based analytical tools to assess the child's developmental status. Assessment items include language development, motor skills, and social skills.

[0244] The analysis results are not simply left as they are; the device provides interactive dialogue tailored to the child. For example, based on the evaluation score generated on the server, an interactive AI character engages in educational dialogue with the child through play. This aligns with the objective of improving the child's developmental indicators.

[0245] Furthermore, by analyzing the collected facial expression data, the server can understand the child's emotional state in real time. Based on this, it can generate feedback, such as suggesting relaxing activities if the child is feeling anxious, and notify the parent user.

[0246] Furthermore, this system has a function to predict future development. Based on data accumulated over a long period, the server uses time-series analysis techniques to predict future growth patterns and provides parents with guidance plans and support measures for the future.

[0247] This series of processes allows parents to accurately understand how their child's development is progressing compared to others and how they should provide support. This invention is not merely a developmental assessment tool, but is also extremely useful for providing educational support optimized for each individual child. For example, if the server determines that a child is behind in language development, it can provide the child with an interactive game that allows for speech training via the device, thereby promoting learning in a natural way.

[0248] The following describes the processing flow.

[0249] Step 1:

[0250] The user installs a remote camera and wearable device in their home environment. These devices capture the child's biometric and behavioral information and transmit the data to a server via the device or home network.

[0251] Step 2:

[0252] The server preprocesses the received data. Specifically, it splits video data into individual frames, removes noise from audio data, and standardizes biometric information obtained from wearable devices.

[0253] Step 3:

[0254] The server uses pre-processed data to perform multimodal analysis utilizing artificial intelligence. Image analysis is used to evaluate children's motor skills, speech recognition is used to analyze language development, and the various indicators are integrated.

[0255] Step 4:

[0256] The device uses analysis results from the server to provide children with conversations and games through an interactive AI character. For example, if language training is needed, it will conduct language practice through interactive games.

[0257] Step 5:

[0258] The server uses facial recognition technology to analyze the child's emotions from video data and evaluates their state in real time. If the stress level is high, it prepares to notify the user of that information as feedback.

[0259] Step 6:

[0260] Based on the aggregated analysis data, the server generates expert-reviewed feedback and sends it to the user as a report via email or a dedicated app.

[0261] Step 7:

[0262] The server analyzes long-term accumulated data to predict future developmental patterns. Using time-series models, it can, for example, propose a learning plan for the next semester.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] Supporting children's health and learning during their developmental stages is challenging because it relies on limited resources and environments within the home, making timely and appropriate responses difficult. Furthermore, the lack of systematic tools for accurately understanding a child's condition and enabling parents to appropriately utilize that information is a barrier to effective childcare support.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment, means for analysis using a generative AI model, and means for dialogue that provides educational dialogue. This enables real-time evaluation of the child's developmental status and provides appropriate educational support and feedback to parents based on that evaluation.

[0268] "Biometric information" refers to data that indicates a child's physical condition, such as heart rate and body temperature.

[0269] "Behavior" refers to patterns of a child's movements and activities, including how they play and walk.

[0270] "Means of collection" refers to devices that use sensors, cameras, etc., to acquire children's biometric information and behavior.

[0271] "Analysis methods using generative AI models" refer to the process of using artificial intelligence technology to process collected data and evaluate the developmental status of children.

[0272] "Dialogue means" refers to the functions of a system that allows for direct interaction with children, provided based on analysis.

[0273] A "feedback generation method" is a mechanism that generates information to convey analysis results and the child's emotional state to the parents.

[0274] "Emotional state analysis methods" refer to the process of inferring a child's emotions from their facial expressions and behavior, and then determining the appropriate response.

[0275] A "predictive tool" is an analytical technique used to predict future growth patterns based on accumulated data.

[0276] One possible embodiment of this invention is a system combining a data collection device, a server, and a terminal installed in a home environment. First, the user starts by installing a remote camera and a wearable device in their home. These devices are means of collecting the child's daily activities and biometric information, recording heart rate, body temperature, movement patterns, etc., in real time.

[0277] The collected data is transmitted to the server via the terminal. Data communication is secure through an encrypted protocol using Wi-Fi or Bluetooth. The server preprocesses the received data and analyzes it using a generative AI model. This generative AI model uses the collected data to evaluate the child's development, for example, assessing motor skills and social skills.

[0278] Furthermore, based on the analysis results, the device uses an interactive AI character to generate educational conversations with the child. These conversations are customized according to the child's developmental stage; for example, if it is determined that the child's language development is delayed, it will provide a game that allows for speech training.

[0279] Furthermore, the server uses facial recognition technology to analyze the child's emotional state in real time and perform relaxation techniques or notify the parents as needed. In the long term, it predicts the child's growth pattern based on the accumulated data through time series analysis and proposes a guidance plan based on the results to the parents.

[0280] As a specific example, examples of prompt sentences are shown below:

[0281] "Please tell me the activities recommended for the current evaluation and improvement of the child's motor ability."

[0282] The entire system enables the parents to provide information for better supporting the growth and development of the child and to respond according to individual educational needs.

[0283] The flow of the specific process in Example 1 will be described using FIG. 11.

[0284] Step 1:

[0285] The user installs a remote camera and a wearable device at home. The input of this process is the installation location and environmental settings of the device. Thereby, the device begins to collect the child's biological information and behavioral information including heart rate, body temperature, and movement patterns. The output is a data stream that is updated in real time.

[0286] Step 2:

[0287] The terminal receives the collected data and transmits it to the server via Wi-Fi or Bluetooth. The input is the data of the biological information and behavioral information obtained from the sensor. In this transmission process, the data performs an operation of being safely sent using encryption technology. The output is a verified data packet sent to the server.

[0288] Step 3:

[0289] The server preprocesses the received data. The input is raw data sent from the terminal. This step involves denoising the data, adjusting timestamps, and standardizing the data. The output is preprocessed data in a format suitable for analysis.

[0290] Step 4:

[0291] The server analyzes the preprocessed data using a generative AI model. The input is the standardized data output in step 3. The analysis is performed by applying machine learning algorithms to evaluate developmental and emotional states. The output is the evaluation results of the child's motor skills, language development, emotional state, etc.

[0292] Step 5:

[0293] The device controls an interactive AI character based on evaluation results from the server and generates educational dialogues. The input is the analysis results. In this step, the dialogue content is programmed according to the child's developmental stage and displayed as a game or educational activity. The output is the interactive experience provided on the screen.

[0294] Step 6:

[0295] The server analyzes facial expression data in real time and provides feedback to the parent user about the child's emotional state. The input is facial expression data from a remote camera. The server analyzes this data and identifies emotions such as anxiety and joy. The output is a suggestion of relaxation activities and a notification to the parent based on this analysis.

[0296] Step 7:

[0297] The server analyzes accumulated long-term data and uses time-series data to predict growth patterns. The input is historical evaluation data. Time-series analysis techniques are used to predict future development. The output includes predicted growth trends and suggested guidance plans for parents.

[0298] (Application Example 1)

[0299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0300] In modern society, accurately understanding a child's developmental stage and providing optimal educational support is a crucial challenge. However, there is a lack of adequate systems for effectively monitoring a child's biometric information and behavior in real time within the home environment and providing appropriate educational guidance based on that data. Furthermore, current systems make it difficult for parents and guardians to accurately understand information about their child's developmental process and take appropriate action. Therefore, innovative solutions are needed to support children's development more efficiently and effectively.

[0301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0302] In this invention, the server includes means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment; analysis means using machine learning to analyze the collected data and evaluate the child's developmental status; and interactive dialogue means for providing educational dialogue to the child based on the results obtained from the analysis means. This enables effective real-time support for a child's development within the home, allowing parents and guardians to take appropriate and prompt action.

[0303] "Children's biometric information" refers to data indicating a child's health status, such as heart rate and body temperature.

[0304] "Devices for monitoring behavior in a home environment" refer to remote cameras and sensor devices installed in the home to capture a child's movements and behavioral patterns.

[0305] "Means of data collection" refers to the functions and processes for collecting children's biometric and behavioral data.

[0306] The "analysis means using machine learning" is an algorithm or model used to evaluate a child's development status using the collected data.

[0307] The "interactive interaction means" refers to a method or system that realizes educational and interactive communication with children based on the analysis results.

[0308] The "information providing means" is a method or mechanism for reporting and notifying the results obtained by analysis to parents or guardians.

[0309] The "behavior support means using an automated robot" is a support method using a robot that operates in the home for the purpose of promoting a child's learning and growth.

[0310] The system for implementing this invention has a configuration for effectively supporting a child's development in a home environment. The user installs a remote camera and wearable sensors in the home to collect the child's biological information and behavioral data. Thereby, the user can monitor the child's health status and daily activities in real time.

[0311] The collected data is transmitted to the server via the terminal. The server processes the received data by machine learning, performs noise removal and data standardization, and then evaluates the child's development status. For this analysis, an AI model implemented in Python or other appropriate programming languages is utilized. The AI model analyzes the data based on multiple evaluation criteria such as language development, motor ability, and social skills.

[0312] The analysis results from the server are realized through the interactive interaction means, and educational interactions using a robot are realized in the home. The robot conducts interactive educational activities with the child based on instructions from the server. This activity includes, for example, language training games to increase vocabulary and role-playing to improve social skills.

[0313] Furthermore, the server provides parents and guardians with the results obtained from the analysis. This information is presented as feedback on the child's development through smart glasses or other display devices. This allows parents and guardians to understand their child's current condition and make informed decisions about their next actions.

[0314] As a concrete example, if the server determines that a child is experiencing a delay in language development, the robot will perform an activity to increase vocabulary through simple questions such as, "Which do you prefer? Cats or dogs?" Another example of a prompt when using this invention is to input to the generating AI model, "Design an interactive language training game for a child. The game should be designed so that parents can monitor it through smart glasses, and the child should be able to learn while having fun."

[0315] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0316] Step 1:

[0317] Users install remote cameras and wearable sensors in their homes to collect biometric and behavioral data about their children. These devices acquire biometric data such as heart rate and body temperature, as well as behavioral data such as walking patterns and playtime. The acquired data is transmitted to a device in real time.

[0318] Step 2:

[0319] The terminal transfers data received from the user to the server. The server removes noise from the data and normalizes it as needed. This process prepares the data for AI analysis. The input is raw data such as heart rate and behavioral patterns, and the output is standardized data necessary for the analysis process.

[0320] Step 3:

[0321] The server inputs pre-processed data into an AI analysis engine to evaluate the child's developmental stage. The AI ​​model uses generative AI technology to analyze multiple data points (language development, motor skills, social skills). The evaluation results are output as a numerical evaluation score.

[0322] Step 4:

[0323] The server sends instructions to the robot through interactive dialogue based on the evaluation score obtained. The robot provides children with educational dialogues and activities according to the evaluation score. For example, it conducts activities to improve vocabulary through language training games.

[0324] Step 5:

[0325] The server generates feedback for parents or guardians based on the analysis results and the child's emotional state. This feedback is sent via the device to smart glasses or another display device. The feedback includes information about the child's developmental environment and the support they need. The inputs are the evaluation score and emotional state, and the output is the feedback message.

[0326] Step 6:

[0327] Users utilize feedback provided by the server to plan the next steps in supporting their child's development at home. Based on this feedback, they can adjust educational plans and curricula as needed.

[0328] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0329] The following configuration is conceivable as an embodiment of this invention. The system is designed to provide more optimized feedback by evaluating the child's developmental stage and also taking into account the emotional state of the parent, who is the user.

[0330] Users place remote cameras and wearable devices in their home environment, and these devices are used to automatically collect biometric information and behavioral data of their children. These devices transmit data to a server via Wi-Fi or Bluetooth.

[0331] The server preprocesses received biometric and behavioral data, and then uses artificial intelligence to analyze it. This allows for the evaluation of various developmental indicators, such as motor skills and language development. In addition, an emotion recognition algorithm is used to analyze the child's emotional state from the collected data, determining whether they are relaxed or stressed.

[0332] Based on the analysis results, the device displays customized interactive educational content for the child through an interactive AI character. For example, if the server determines that the child needs to improve their social skills, the AI ​​character will suggest a social game.

[0333] Furthermore, the system incorporates an emotion engine that analyzes the user's (parent's) emotional state. To do this, the server performs emotion analysis using the user's voice data and facial expression data acquired from the camera. The emotion engine determines the parent's stress level and level of interest, and adjusts the content and timing of feedback accordingly.

[0334] Finally, the server generates expert-reviewed feedback from the analyzed data and emotional state information, which is then delivered to the user via email or app notification. For example, the server could include advice such as suggesting that parents engage in relaxing activities with their children at the end of the day to reduce stress.

[0335] In this way, by considering not only the child's biometric information and behavioral data but also the parent's emotional state, the system consistently provides effective childcare support. For example, to improve delays in a child's motor skills indicated by the analysis results, the emotional engine can utilize methods to attract the parent's attention, thereby supporting parents in actively participating in their child's training.

[0336] The following describes the processing flow.

[0337] Step 1:

[0338] Users install remote cameras and wearable devices in their home environment to continuously record their child's biometric information and behavior. Additionally, if parents use a smartphone or PC, they can configure it to capture audio and facial expressions via the camera.

[0339] Step 2:

[0340] The recorded biometric and behavioral data is sent to the device, which temporarily stores this data. The data is then configured to be uploaded to a server via Wi-Fi or Bluetooth.

[0341] Step 3:

[0342] The server preprocesses the received data. This processing includes noise reduction and frame splitting of video data, clearance of audio data, and standardization of biometric data from wearable devices.

[0343] Step 4:

[0344] The server uses artificial intelligence to analyze child development indicators based on pre-processed data. For example, it analyzes and scores a child's motor movements from image data and evaluates language development from audio data.

[0345] Step 5:

[0346] Simultaneously, the server uses an emotion engine to analyze the parent's voice and facial expression data. The emotion engine extracts voice tone and facial expression characteristics and evaluates the parent's emotional state in real time.

[0347] Step 6:

[0348] Based on the analysis results, the device uses an interactive AI character to provide educational content suitable for children. The character engages in conversations and games designed to improve the skills children need, based on the analyzed developmental indicators.

[0349] Step 7:

[0350] The server integrates the data obtained through analysis and generates expert-reviewed feedback. This feedback is customized to take into account the parent's emotional state, and may include suggestions for relaxation methods if the parent is feeling stressed, for example.

[0351] Step 8:

[0352] Ultimately, the server provides feedback to the user via email or app notification. At this time, it will also suggest the next steps and recommended actions to receive effective support.

[0353] Step 9:

[0354] In the long term, the server will analyze the accumulated data, update models to predict future development, and present these predictions to parents, enabling planned childcare support.

[0355] (Example 2)

[0356] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0357] In modern childcare, there is a need to accurately understand a child's developmental and emotional state and provide appropriate feedback to parents. However, conventional technology has not adequately achieved the ability to comprehensively analyze a child's physical and emotional data and provide optimal feedback that also takes into account the parent's emotional state. Therefore, a more intelligent and comprehensive support system is needed to promote a child's development.

[0358] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0359] In this invention, the server includes means for monitoring a child's biometric information and behavior and transmitting data using communication technology, means for preprocessing and analyzing the data and evaluating developmental indicators, and means for evaluating the user's emotions. This makes it possible to provide parents with customized feedback tailored to the individual developmental needs of their children, thereby intelligently and effectively providing childcare support.

[0360] "Children's biometric information" refers to data that represents a child's physical condition, including heart rate, body temperature, and respiratory rate.

[0361] "Behavioral data" refers to information that records a child's daily activities, describing things like distance traveled, playtime, and types of activities.

[0362] "Communication technology" refers to technologies for sending and receiving data between devices, and includes wireless communication methods such as Wi-Fi and Bluetooth.

[0363] An "external processing device" is a computer system that receives data and performs analysis processing, and includes equipment such as servers.

[0364] "Preprocessing" refers to the process of preparing data to make it easier to analyze, and includes tasks such as organizing, cleansing, and imputing missing values.

[0365] "Artificial intelligence" refers to the technology of computers imitating human intellectual behavior, and involves methods such as machine learning and data analysis.

[0366] "Developmental indicators" are standards used to evaluate a child's growth and development, and refer to things like motor skills and language abilities.

[0367] "Emotional state" refers to an individual's emotional reactions and circumstances, including stress, joy, and relaxation.

[0368] "Emotional analysis methods" refer to methods of analyzing an individual's emotional state based on data, and utilize techniques such as voice analysis and facial expression analysis.

[0369] "Interactive educational information" refers to content that aims to achieve educational effects through interaction, and includes games and conversational simulations.

[0370] "Customized feedback" refers to feedback that is tailored to an individual's characteristics and needs.

[0371] This system aims to assess a child's developmental stage and provide appropriate feedback to parents. The system is structured as follows:

[0372] Users install remote cameras and wearable devices in their home environment. These devices incorporate wireless communication technologies such as Bluetooth and Wi-Fi to automatically collect biometric and behavioral data of children. Specific data includes heart rate, body temperature, distance traveled, and playtime.

[0373] Data is transmitted to the server in real time. The server is equipped with a high-performance processor and artificial intelligence model, and performs preprocessing on the received data. Preprocessing includes data cleansing and imputation of missing values. Subsequently, machine learning algorithms are used to evaluate developmental indicators such as children's motor skills and language abilities, and emotion recognition algorithms are used to analyze their emotional state.

[0374] The server further analyzes the parents' voice data and camera footage, and uses an emotion engine to determine stress levels and levels of interest. Based on these analysis results, it generates educational content tailored to the parents' emotions and the child's development.

[0375] Based on the analysis results, the device presents educational content to the child through an interactive AI character. For example, if it determines that the child's motor skills are somewhat lacking, it will introduce engaging fitness games to encourage activity.

[0376] Finally, the server generates expert-reviewed feedback. This feedback is delivered via email or app notifications on the parent's smartphone or computer. A specific example is "suggestions for relaxing evening activities to do with children, aimed at reducing parental stress."

[0377] An example of a prompt message would be: "Generate an appropriate encouraging message based on the latest data on the child's athletic ability. Include techniques to capture the parent's attention."

[0378] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0379] Step 1:

[0380] The user installs a remote camera and wearable devices in their home. These devices collect the child's heart rate, body temperature, and activity level in real time. The input consists of biometric and behavioral data obtained from the child. This data is transmitted to a server using wireless communication technology. The output is the raw data sent to the server.

[0381] Step 2:

[0382] The server preprocesses the received data. Specifically, it cleanses the data, detects and imputes outliers and missing values. The input is the biometric and behavioral data transmitted in step 1. The output is a preprocessed, consistent dataset. This processing ensures data quality that allows for accurate subsequent analysis.

[0383] Step 3:

[0384] The server performs AI analysis using pre-processed data. It uses a generative AI model to perform data calculations to evaluate developmental indicators such as motor skills and language abilities. It also analyzes children's emotional states using an emotion recognition algorithm. The input is a pre-processed dataset. The output is the evaluation of developmental indicators and emotional states as analysis results. At this stage, specific emotion models and machine learning algorithms are used.

[0385] Step 4:

[0386] The server evaluates the parent's emotional state. It takes in parental voice data and camera footage as input and uses an emotion engine to analyze stress levels and interest levels. The output is evaluation data regarding the user's (parent's) emotional state. This evaluation allows for the customization of feedback based on the parent's situation.

[0387] Step 5:

[0388] The device generates and presents educational content through an interactive AI character based on analysis results obtained from the server. The input consists of the child's developmental indicators and emotional state, obtained from the server. The output is child-specific educational content, presented in a game or dialogue format.

[0389] Step 6:

[0390] The server generates feedback based on the analysis results and the parents' emotional state. It designs specific suggestions and activity plans and notifies the parents. The inputs are child development data and parent emotional analysis data. The output is a feedback message sent to the parents' devices, which includes expert-reviewed advice. This feedback serves to engage the parents and support their parenting.

[0391] (Application Example 2)

[0392] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0393] In modern family environments, there is a lack of childcare support systems that effectively support children's development while reducing parental stress. In particular, there is a need for technology that analyzes children's biometric information and behavior in real time, and provides appropriate feedback while also considering the parents' emotional state.

[0394] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0395] In this invention, the server includes a device for collecting a child's biometric information and behavioral data, a device for analyzing the collected data using machine learning technology, a device for generating educational dialogues based on the analysis results, an emotion analysis device for analyzing the parent's voice data and facial expression data, an emotion engine for adjusting the content of feedback according to the parent's emotional state, and an interaction generation device for providing interactive educational support. This makes it possible to provide childcare support that promotes the child's development while taking into account the parent's emotional state.

[0396] "Children's biometric information" refers to physical information measured by heart rate, body temperature, and motion sensors.

[0397] "Behavioral data" refers to data that records a child's daily movements and activity patterns.

[0398] A "device" is hardware used to collect biometric information and behavioral data.

[0399] "Machine learning techniques" is a general term for algorithms used to analyze large amounts of data and find patterns and trends.

[0400] "Analysis equipment" refers to hardware and software used to evaluate a child's developmental status using collected data.

[0401] "Educational dialogue" is interactive communication designed to promote a child's development.

[0402] An "emotion analysis device" is hardware and software that uses voice data and facial expression data to determine a parent's emotional state.

[0403] The "emotional engine" is a function that adjusts the content and timing of feedback based on the parent's emotional state.

[0404] An "interaction generation device" is a device that generates content and activities to support education through dialogue with children.

[0405] This invention provides a childcare support system that effectively assists child development in the home environment while also being considerate of parents. The specific system configuration includes the following elements:

[0406] The server collects data using a device equipped with multiple sensors to gather children's biometric and behavioral data. This device includes cameras and wearable sensors, and transmits the data to the server via Wi-Fi or Bluetooth.

[0407] The server analyzes the collected data using machine learning techniques such as Python and TensorFlow. This analysis evaluates developmental indicators such as children's motor skills and language development, and also determines the children's emotional state using an emotion analysis algorithm.

[0408] Based on the analysis results, the device provides dialogue to the child through an interaction generator. The interaction generator dynamically generates educational content using a generation AI model and presents it to the child visually and audibly.

[0409] Furthermore, the server uses an emotion analysis device to analyze voice and facial expression data in order to understand the parent's emotional state. The analysis results are processed by an emotion engine, and feedback is adjusted according to the parent's stress level and level of interest.

[0410] The user (parent) receives final feedback via app notifications or email, which includes specific activity suggestions. For example, when suggesting a quiz game or physical activity for parents and children to play together, a prompt message such as "Please suggest activities to reduce parental stress" is sent to the generating AI model.

[0411] This system allows users to monitor their children's growth while fostering better parent-child relationships.

[0412] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0413] Step 1:

[0414] The server collects real-time biometric and behavioral data of children from sensors placed throughout the home. Inputs are biometric data (heart rate, body temperature, etc.) and behavioral data from the sensors, and output is the aggregation and temporary storage of this data. Specifically, data is transferred from the sensors to the server using Wi-Fi or Bluetooth.

[0415] Step 2:

[0416] The server analyzes the collected data using machine learning techniques. The input consists of biometric and behavioral data aggregated in Step 1. The data is analyzed using Python and TensorFlow, generating output that evaluates children's motor skills, language development, and emotional states. Specifically, it performs data cleansing and normalization, then inputs the data into the model to recognize patterns.

[0417] Step 3:

[0418] The server designs interactions using a dialogue generation device based on the analysis results. The input is the analysis results obtained in step 2, and the generating AI model is used to generate educational content and activity content. The output is dialogue prompts and game content optimized for children. Specifically, after the dialogue content is determined, information to be presented to the child is created using speech synthesis.

[0419] Step 4:

[0420] The device provides the child with content received from the interaction generator. The input consists of dialogue prompts and game content generated in step 3, which are presented to the child as audio and video. The output allows observation of the child's response. In terms of specific actions, interaction is carried out using the screen and speaker.

[0421] Step 5:

[0422] The server collects parent voice and facial expression data and performs emotion analysis. The input is parent voice and facial expression data, and the output is data indicating the parent's emotional state (e.g., stress level). Specifically, it uses a microphone and camera to acquire detailed audio and video data and uses an analysis algorithm to estimate emotions.

[0423] Step 6:

[0424] The server uses an emotion engine to adjust the content of the feedback. The input is the parent's emotional state obtained in step 5 and the analysis results from step 2, and based on this, it generates optimal feedback for the parent. The output is activity suggestions and advice provided to the parent. Specifically, a prompt "Please suggest relaxing activities" is sent to the generating AI model, and the model's response is incorporated as feedback.

[0425] Step 7:

[0426] Users receive feedback and suggestions from the server via their devices and incorporate them into their daily lives. The input is the feedback generated in step 6, and the output is the addition of new activities and challenges to the parent-child relationship. Specifically, users receive notifications using their smartphones or tablets and perform the suggested activities together.

[0427] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0428] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0429] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0430] [Third Embodiment]

[0431] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0432] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0433] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0434] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0435] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0436] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0437] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0438] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0439] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0440] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0441] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0442] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0443] The following system configuration is conceivable as an embodiment of this invention. The system monitors and evaluates the developmental status of a child and operates using various devices installed in the home environment.

[0444] First, the user installs a remote camera and wearable device in their home. These devices collect the child's biometric information (heart rate, body temperature, etc.) and behavioral information (playing patterns, walking patterns, etc.) in real time.

[0445] Next, this data is sent to the server via the terminal. The server preprocesses the received data, removing noise and standardizing it. Then, the server analyzes the data using artificial intelligence-based analytical tools to assess the child's developmental status. Assessment items include language development, motor skills, and social skills.

[0446] The analysis results are not simply left as they are; the device provides interactive dialogue tailored to the child. For example, based on the evaluation score generated on the server, an interactive AI character engages in educational dialogue with the child through play. This aligns with the objective of improving the child's developmental indicators.

[0447] Furthermore, by analyzing the collected facial expression data, the server can understand the child's emotional state in real time. Based on this, it can generate feedback, such as suggesting relaxing activities if the child is feeling anxious, and notify the parent user.

[0448] Furthermore, this system has a function to predict future development. Based on data accumulated over a long period, the server uses time-series analysis techniques to predict future growth patterns and provides parents with guidance plans and support measures for the future.

[0449] This series of processes allows parents to accurately understand how their child's development is progressing compared to others and how they should provide support. This invention is not merely a developmental assessment tool, but is also extremely useful for providing educational support optimized for each individual child. For example, if the server determines that a child is behind in language development, it can provide the child with an interactive game that allows for speech training via the device, thereby promoting learning in a natural way.

[0450] The following describes the processing flow.

[0451] Step 1:

[0452] The user installs a remote camera and wearable device in their home environment. These devices capture the child's biometric and behavioral information and transmit the data to a server via the device or home network.

[0453] Step 2:

[0454] The server preprocesses the received data. Specifically, it splits video data into individual frames, removes noise from audio data, and standardizes biometric information obtained from wearable devices.

[0455] Step 3:

[0456] The server uses pre-processed data to perform multimodal analysis utilizing artificial intelligence. Image analysis is used to evaluate children's motor skills, speech recognition is used to analyze language development, and the various indicators are integrated.

[0457] Step 4:

[0458] The device uses analysis results from the server to provide children with conversations and games through an interactive AI character. For example, if language training is needed, it will conduct language practice through interactive games.

[0459] Step 5:

[0460] The server uses facial recognition technology to analyze the child's emotions from video data and evaluates their state in real time. If the stress level is high, it prepares to notify the user of that information as feedback.

[0461] Step 6:

[0462] Based on the aggregated analysis data, the server generates expert-reviewed feedback and sends it to the user as a report via email or a dedicated app.

[0463] Step 7:

[0464] The server analyzes long-term accumulated data to predict future developmental patterns. Using time-series models, it can, for example, propose a learning plan for the next semester.

[0465] (Example 1)

[0466] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0467] Supporting children's health and learning during their developmental stages is challenging because it relies on limited resources and environments within the home, making timely and appropriate responses difficult. Furthermore, the lack of systematic tools for accurately understanding a child's condition and enabling parents to appropriately utilize that information is a barrier to effective childcare support.

[0468] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0469] In this invention, the server includes means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment, means for analysis using a generative AI model, and means for dialogue that provides educational dialogue. This enables real-time evaluation of the child's developmental status and provides appropriate educational support and feedback to parents based on that evaluation.

[0470] "Biometric information" refers to data that indicates a child's physical condition, such as heart rate and body temperature.

[0471] "Behavior" refers to patterns of a child's movements and activities, including how they play and walk.

[0472] "Means of collection" refers to devices that use sensors, cameras, etc., to acquire children's biometric information and behavior.

[0473] "Analysis methods using generative AI models" refer to the process of using artificial intelligence technology to process collected data and evaluate the developmental status of children.

[0474] "Dialogue means" refers to the functions of a system that allows for direct interaction with children, provided based on analysis.

[0475] A "feedback generation method" is a mechanism that generates information to convey analysis results and the child's emotional state to the parents.

[0476] "Emotional state analysis methods" refer to the process of inferring a child's emotions from their facial expressions and behavior, and then determining the appropriate response.

[0477] A "predictive tool" is an analytical technique used to predict future growth patterns based on accumulated data.

[0478] One possible embodiment of this invention is a system combining a data collection device, a server, and a terminal installed in a home environment. First, the user starts by installing a remote camera and a wearable device in their home. These devices are means of collecting the child's daily activities and biometric information, recording heart rate, body temperature, movement patterns, etc., in real time.

[0479] The collected data is transmitted to the server via the terminal. Data communication is secure through an encrypted protocol using Wi-Fi or Bluetooth. The server preprocesses the received data and analyzes it using a generative AI model. This generative AI model uses the collected data to evaluate the child's development, for example, assessing motor skills and social skills.

[0480] Furthermore, based on the analysis results, the device uses an interactive AI character to generate educational conversations with the child. These conversations are customized according to the child's developmental stage; for example, if it is determined that the child's language development is delayed, it will provide a game that allows for speech training.

[0481] Furthermore, the server uses facial recognition technology to analyze the child's emotional state in real time and provides relaxation techniques and notifications to parents as needed. In the long term, it predicts the child's growth patterns based on accumulated data through time-series analysis and proposes an instructional plan to parents based on the results.

[0482] As a concrete example, here is an example of a prompt message:

[0483] "Please tell me about recommended activities for assessing and improving children's current motor skills."

[0484] The entire system provides parents with information to better support their children's growth and development, and enables them to address individual educational needs.

[0485] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0486] Step 1:

[0487] The user installs a remote camera and wearable device in their home. The inputs to this process are the device's installation location and environmental settings. The device then begins collecting biometric and behavioral information about the child, including heart rate, body temperature, and movement patterns. The output is a real-time updated data stream.

[0488] Step 2:

[0489] The device receives collected data and transmits it to the server via Wi-Fi or Bluetooth. The input consists of biometric and behavioral data acquired from sensors. During this transmission process, the data is securely sent using encryption technology. The output is a verified data packet sent to the server.

[0490] Step 3:

[0491] The server preprocesses the received data. The input is raw data sent from the terminal. This step involves denoising the data, adjusting timestamps, and standardizing the data. The output is preprocessed data in a format suitable for analysis.

[0492] Step 4:

[0493] The server analyzes the preprocessed data using a generative AI model. The input is the standardized data output in step 3. The analysis is performed by applying machine learning algorithms to evaluate developmental and emotional states. The output is the evaluation results of the child's motor skills, language development, emotional state, etc.

[0494] Step 5:

[0495] The device controls an interactive AI character based on evaluation results from the server and generates educational dialogues. The input is the analysis results. In this step, the dialogue content is programmed according to the child's developmental stage and displayed as a game or educational activity. The output is the interactive experience provided on the screen.

[0496] Step 6:

[0497] The server analyzes facial expression data in real time and provides feedback to the parent user about the child's emotional state. The input is facial expression data from a remote camera. The server analyzes this data and identifies emotions such as anxiety and joy. The output is a suggestion of relaxation activities and a notification to the parent based on this analysis.

[0498] Step 7:

[0499] The server analyzes accumulated long-term data and uses time-series data to predict growth patterns. The input is historical evaluation data. Time-series analysis techniques are used to predict future development. The output includes predicted growth trends and suggested guidance plans for parents.

[0500] (Application Example 1)

[0501] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0502] In modern society, accurately understanding a child's developmental stage and providing optimal educational support is a crucial challenge. However, there is a lack of adequate systems for effectively monitoring a child's biometric information and behavior in real time within the home environment and providing appropriate educational guidance based on that data. Furthermore, current systems make it difficult for parents and guardians to accurately understand information about their child's developmental process and take appropriate action. Therefore, innovative solutions are needed to support children's development more efficiently and effectively.

[0503] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0504] In this invention, the server includes means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment; analysis means using machine learning to analyze the collected data and evaluate the child's developmental status; and interactive dialogue means for providing educational dialogue to the child based on the results obtained from the analysis means. This enables effective real-time support for a child's development within the home, allowing parents and guardians to take appropriate and prompt action.

[0505] "Children's biometric information" refers to data indicating a child's health status, such as heart rate and body temperature.

[0506] "Devices for monitoring behavior in a home environment" refer to remote cameras and sensor devices installed in the home to capture a child's movements and behavioral patterns.

[0507] "Means of data collection" refers to the functions and processes for collecting children's biometric and behavioral data.

[0508] "Machine learning-based analysis methods" refer to algorithms and models used to evaluate a child's developmental status using collected data.

[0509] "Interactive dialogue means" refers to methods and systems that enable educational and interactive communication with children based on analysis results.

[0510] "Information provision methods" refer to the methods and mechanisms for reporting and notifying parents or guardians of the results obtained through analysis.

[0511] "Automated robot-based behavioral support methods" refer to support methods using robots that operate within the home for the purpose of promoting children's learning and growth.

[0512] The system for implementing this invention has a configuration that effectively supports child development in a home environment. The user installs remote cameras and wearable sensors in the home to collect the child's biometric information and behavioral data. This allows the user to monitor the child's health status and daily activities in real time.

[0513] The collected data is sent to the server via the terminal. The server processes the received data using machine learning, performing noise reduction and data standardization before evaluating the child's developmental status. This analysis utilizes AI models implemented in Python or other appropriate programming languages. The AI ​​models analyze the data based on multiple evaluation criteria, including language development, motor skills, and social skills.

[0514] Analysis results from the server enable educational interactions using robots within the home through interactive dialogue mechanisms. Based on instructions from the server, the robot engages in interactive educational activities with children. These activities include, for example, language training games to increase vocabulary and role-playing to improve social skills.

[0515] Furthermore, the server provides parents and guardians with the results obtained from the analysis. This information is presented as feedback on the child's development through smart glasses or other display devices. This allows parents and guardians to understand their child's current condition and make informed decisions about their next actions.

[0516] As a concrete example, if the server determines that a child is experiencing a delay in language development, the robot will perform an activity to increase vocabulary through simple questions such as, "Which do you prefer? Cats or dogs?" Another example of a prompt when using this invention is to input to the generating AI model, "Design an interactive language training game for a child. The game should be designed so that parents can monitor it through smart glasses, and the child should be able to learn while having fun."

[0517] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0518] Step 1:

[0519] Users install remote cameras and wearable sensors in their homes to collect biometric and behavioral data about their children. These devices acquire biometric data such as heart rate and body temperature, as well as behavioral data such as walking patterns and playtime. The acquired data is transmitted to a device in real time.

[0520] Step 2:

[0521] The terminal transfers data received from the user to the server. The server removes noise from the data and normalizes it as needed. This process prepares the data for AI analysis. The input is raw data such as heart rate and behavioral patterns, and the output is standardized data necessary for the analysis process.

[0522] Step 3:

[0523] The server inputs pre-processed data into an AI analysis engine to evaluate the child's developmental stage. The AI ​​model uses generative AI technology to analyze multiple data points (language development, motor skills, social skills). The evaluation results are output as a numerical evaluation score.

[0524] Step 4:

[0525] The server sends instructions to the robot through interactive dialogue based on the evaluation score obtained. The robot provides children with educational dialogues and activities according to the evaluation score. For example, it conducts activities to improve vocabulary through language training games.

[0526] Step 5:

[0527] The server generates feedback for parents or guardians based on the analysis results and the child's emotional state. This feedback is sent via the device to smart glasses or another display device. The feedback includes information about the child's developmental environment and the support they need. The inputs are the evaluation score and emotional state, and the output is the feedback message.

[0528] Step 6:

[0529] Users utilize feedback provided by the server to plan the next steps in supporting their child's development at home. Based on this feedback, they can adjust educational plans and curricula as needed.

[0530] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0531] The following configuration is conceivable as an embodiment of this invention. The system is designed to provide more optimized feedback by evaluating the child's developmental stage and also taking into account the emotional state of the parent, who is the user.

[0532] Users place remote cameras and wearable devices in their home environment, and these devices are used to automatically collect biometric information and behavioral data of their children. These devices transmit data to a server via Wi-Fi or Bluetooth.

[0533] The server preprocesses received biometric and behavioral data, and then uses artificial intelligence to analyze it. This allows for the evaluation of various developmental indicators, such as motor skills and language development. In addition, an emotion recognition algorithm is used to analyze the child's emotional state from the collected data, determining whether they are relaxed or stressed.

[0534] Based on the analysis results, the device displays customized interactive educational content for the child through an interactive AI character. For example, if the server determines that the child needs to improve their social skills, the AI ​​character will suggest a social game.

[0535] Furthermore, the system incorporates an emotion engine that analyzes the user's (parent's) emotional state. To do this, the server performs emotion analysis using the user's voice data and facial expression data acquired from the camera. The emotion engine determines the parent's stress level and level of interest, and adjusts the content and timing of feedback accordingly.

[0536] Finally, the server generates expert-reviewed feedback from the analyzed data and emotional state information, which is then delivered to the user via email or app notification. For example, the server could include advice such as suggesting that parents engage in relaxing activities with their children at the end of the day to reduce stress.

[0537] In this way, by considering not only the child's biometric information and behavioral data but also the parent's emotional state, the system consistently provides effective childcare support. For example, to improve delays in a child's motor skills indicated by the analysis results, the emotional engine can utilize methods to attract the parent's attention, thereby supporting parents in actively participating in their child's training.

[0538] The following describes the processing flow.

[0539] Step 1:

[0540] Users install remote cameras and wearable devices in their home environment to continuously record their child's biometric information and behavior. Additionally, if parents use a smartphone or PC, they can configure it to capture audio and facial expressions via the camera.

[0541] Step 2:

[0542] The recorded biometric and behavioral data is sent to the device, which temporarily stores this data. The data is then configured to be uploaded to a server via Wi-Fi or Bluetooth.

[0543] Step 3:

[0544] The server preprocesses the received data. This processing includes noise reduction and frame splitting of video data, clearance of audio data, and standardization of biometric data from wearable devices.

[0545] Step 4:

[0546] The server uses artificial intelligence to analyze child development indicators based on pre-processed data. For example, it analyzes and scores a child's motor movements from image data and evaluates language development from audio data.

[0547] Step 5:

[0548] Simultaneously, the server uses an emotion engine to analyze the parent's voice and facial expression data. The emotion engine extracts voice tone and facial expression characteristics and evaluates the parent's emotional state in real time.

[0549] Step 6:

[0550] Based on the analysis results, the device uses an interactive AI character to provide educational content suitable for children. The character engages in conversations and games designed to improve the skills children need, based on the analyzed developmental indicators.

[0551] Step 7:

[0552] The server integrates the data obtained through analysis and generates expert-reviewed feedback. This feedback is customized to take into account the parent's emotional state, and may include suggestions for relaxation methods if the parent is feeling stressed, for example.

[0553] Step 8:

[0554] Ultimately, the server provides feedback to the user via email or app notification. At this time, it will also suggest the next steps and recommended actions to receive effective support.

[0555] Step 9:

[0556] In the long term, the server will analyze the accumulated data, update models to predict future development, and present these predictions to parents, enabling planned childcare support.

[0557] (Example 2)

[0558] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0559] In modern childcare, there is a need to accurately understand a child's developmental and emotional state and provide appropriate feedback to parents. However, conventional technology has not adequately achieved the ability to comprehensively analyze a child's physical and emotional data and provide optimal feedback that also takes into account the parent's emotional state. Therefore, a more intelligent and comprehensive support system is needed to promote a child's development.

[0560] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0561] In this invention, the server includes means for monitoring a child's biometric information and behavior and transmitting data using communication technology, means for preprocessing and analyzing the data and evaluating developmental indicators, and means for evaluating the user's emotions. This makes it possible to provide parents with customized feedback tailored to the individual developmental needs of their children, thereby intelligently and effectively providing childcare support.

[0562] "Children's biometric information" refers to data that represents a child's physical condition, including heart rate, body temperature, and respiratory rate.

[0563] "Behavioral data" refers to information that records a child's daily activities, describing things like distance traveled, playtime, and types of activities.

[0564] "Communication technology" refers to technologies for sending and receiving data between devices, and includes wireless communication methods such as Wi-Fi and Bluetooth.

[0565] An "external processing device" is a computer system that receives data and performs analysis processing, and includes equipment such as servers.

[0566] "Preprocessing" refers to the process of preparing data to make it easier to analyze, and includes tasks such as organizing, cleansing, and imputing missing values.

[0567] "Artificial intelligence" refers to the technology of computers imitating human intellectual behavior, and involves methods such as machine learning and data analysis.

[0568] "Developmental indicators" are standards used to evaluate a child's growth and development, and refer to things like motor skills and language abilities.

[0569] "Emotional state" refers to an individual's emotional reactions and circumstances, including stress, joy, and relaxation.

[0570] "Emotional analysis methods" refer to methods of analyzing an individual's emotional state based on data, and utilize techniques such as voice analysis and facial expression analysis.

[0571] "Interactive educational information" refers to content that aims to achieve educational effects through interaction, and includes games and conversational simulations.

[0572] "Customized feedback" refers to feedback that is tailored to an individual's characteristics and needs.

[0573] This system aims to assess a child's developmental stage and provide appropriate feedback to parents. The system is structured as follows:

[0574] Users install remote cameras and wearable devices in their home environment. These devices incorporate wireless communication technologies such as Bluetooth and Wi-Fi to automatically collect biometric and behavioral data of children. Specific data includes heart rate, body temperature, distance traveled, and playtime.

[0575] Data is transmitted to the server in real time. The server is equipped with a high-performance processor and artificial intelligence model, and performs preprocessing on the received data. Preprocessing includes data cleansing and imputation of missing values. Subsequently, machine learning algorithms are used to evaluate developmental indicators such as children's motor skills and language abilities, and emotion recognition algorithms are used to analyze their emotional state.

[0576] The server further analyzes the parents' voice data and camera footage, and uses an emotion engine to determine stress levels and levels of interest. Based on these analysis results, it generates educational content tailored to the parents' emotions and the child's development.

[0577] Based on the analysis results, the device presents educational content to the child through an interactive AI character. For example, if it determines that the child's motor skills are somewhat lacking, it will introduce engaging fitness games to encourage activity.

[0578] Finally, the server generates expert-reviewed feedback. This feedback is delivered via email or app notifications on the parent's smartphone or computer. A specific example is "suggestions for relaxing evening activities to do with children, aimed at reducing parental stress."

[0579] An example of a prompt message would be: "Generate an appropriate encouraging message based on the latest data on the child's athletic ability. Include techniques to capture the parent's attention."

[0580] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0581] Step 1:

[0582] The user installs a remote camera and wearable devices in their home. These devices collect the child's heart rate, body temperature, and activity level in real time. The input consists of biometric and behavioral data obtained from the child. This data is transmitted to a server using wireless communication technology. The output is the raw data sent to the server.

[0583] Step 2:

[0584] The server preprocesses the received data. Specifically, it cleanses the data, detects and imputes outliers and missing values. The input is the biometric and behavioral data transmitted in step 1. The output is a preprocessed, consistent dataset. This processing ensures data quality that allows for accurate subsequent analysis.

[0585] Step 3:

[0586] The server performs AI analysis using pre-processed data. It uses a generative AI model to perform data calculations to evaluate developmental indicators such as motor skills and language abilities. It also analyzes children's emotional states using an emotion recognition algorithm. The input is a pre-processed dataset. The output is the evaluation of developmental indicators and emotional states as analysis results. At this stage, specific emotion models and machine learning algorithms are used.

[0587] Step 4:

[0588] The server evaluates the parent's emotional state. It takes in parental voice data and camera footage as input and uses an emotion engine to analyze stress levels and interest levels. The output is evaluation data regarding the user's (parent's) emotional state. This evaluation allows for the customization of feedback based on the parent's situation.

[0589] Step 5:

[0590] The device generates and presents educational content through an interactive AI character based on analysis results obtained from the server. The input consists of the child's developmental indicators and emotional state, obtained from the server. The output is child-specific educational content, presented in a game or dialogue format.

[0591] Step 6:

[0592] The server generates feedback based on the analysis results and the parents' emotional state. It designs specific suggestions and activity plans and notifies the parents. The inputs are child development data and parent emotional analysis data. The output is a feedback message sent to the parents' devices, which includes expert-reviewed advice. This feedback serves to engage the parents and support their parenting.

[0593] (Application Example 2)

[0594] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0595] In modern family environments, there is a lack of childcare support systems that effectively support children's development while reducing parental stress. In particular, there is a need for technology that analyzes children's biometric information and behavior in real time, and provides appropriate feedback while also considering the parents' emotional state.

[0596] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0597] In this invention, the server includes a device for collecting a child's biometric information and behavioral data, a device for analyzing the collected data using machine learning technology, a device for generating educational dialogues based on the analysis results, an emotion analysis device for analyzing the parent's voice data and facial expression data, an emotion engine for adjusting the content of feedback according to the parent's emotional state, and an interaction generation device for providing interactive educational support. This makes it possible to provide childcare support that promotes the child's development while taking into account the parent's emotional state.

[0598] "Children's biometric information" refers to physical information measured by heart rate, body temperature, and motion sensors.

[0599] "Behavioral data" refers to data that records a child's daily movements and activity patterns.

[0600] A "device" is hardware used to collect biometric information and behavioral data.

[0601] "Machine learning techniques" is a general term for algorithms used to analyze large amounts of data and find patterns and trends.

[0602] "Analysis equipment" refers to hardware and software used to evaluate a child's developmental status using collected data.

[0603] "Educational dialogue" is interactive communication designed to promote a child's development.

[0604] An "emotion analysis device" is hardware and software that uses voice data and facial expression data to determine a parent's emotional state.

[0605] The "emotional engine" is a function that adjusts the content and timing of feedback based on the parent's emotional state.

[0606] An "interaction generation device" is a device that generates content and activities to support education through dialogue with children.

[0607] This invention provides a childcare support system that effectively assists child development in the home environment while also being considerate of parents. The specific system configuration includes the following elements:

[0608] The server collects data using a device equipped with multiple sensors to gather children's biometric and behavioral data. This device includes cameras and wearable sensors, and transmits the data to the server via Wi-Fi or Bluetooth.

[0609] The server analyzes the collected data using machine learning techniques such as Python and TensorFlow. This analysis evaluates developmental indicators such as children's motor skills and language development, and also determines the children's emotional state using an emotion analysis algorithm.

[0610] Based on the analysis results, the device provides dialogue to the child through an interaction generator. The interaction generator dynamically generates educational content using a generation AI model and presents it to the child visually and audibly.

[0611] Furthermore, the server uses an emotion analysis device to analyze voice and facial expression data in order to understand the parent's emotional state. The analysis results are processed by an emotion engine, and feedback is adjusted according to the parent's stress level and level of interest.

[0612] The user (parent) receives final feedback via app notifications or email, which includes specific activity suggestions. For example, when suggesting a quiz game or physical activity for parents and children to play together, a prompt message such as "Please suggest activities to reduce parental stress" is sent to the generating AI model.

[0613] This system allows users to monitor their children's growth while fostering better parent-child relationships.

[0614] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0615] Step 1:

[0616] The server collects real-time biometric and behavioral data of children from sensors placed throughout the home. Inputs are biometric data (heart rate, body temperature, etc.) and behavioral data from the sensors, and output is the aggregation and temporary storage of this data. Specifically, data is transferred from the sensors to the server using Wi-Fi or Bluetooth.

[0617] Step 2:

[0618] The server analyzes the collected data using machine learning techniques. The input consists of biometric and behavioral data aggregated in Step 1. The data is analyzed using Python and TensorFlow, generating output that evaluates children's motor skills, language development, and emotional states. Specifically, it performs data cleansing and normalization, then inputs the data into the model to recognize patterns.

[0619] Step 3:

[0620] The server designs interactions using a dialogue generation device based on the analysis results. The input is the analysis results obtained in step 2, and the generating AI model is used to generate educational content and activity content. The output is dialogue prompts and game content optimized for children. Specifically, after the dialogue content is determined, information to be presented to the child is created using speech synthesis.

[0621] Step 4:

[0622] The device provides the child with content received from the interaction generator. The input consists of dialogue prompts and game content generated in step 3, which are presented to the child as audio and video. The output allows observation of the child's response. In terms of specific actions, interaction is carried out using the screen and speaker.

[0623] Step 5:

[0624] The server collects parent voice and facial expression data and performs emotion analysis. The input is parent voice and facial expression data, and the output is data indicating the parent's emotional state (e.g., stress level). Specifically, it uses a microphone and camera to acquire detailed audio and video data and uses an analysis algorithm to estimate emotions.

[0625] Step 6:

[0626] The server uses an emotion engine to adjust the content of the feedback. The input is the parent's emotional state obtained in step 5 and the analysis results from step 2, and based on this, it generates optimal feedback for the parent. The output is activity suggestions and advice provided to the parent. Specifically, a prompt "Please suggest relaxing activities" is sent to the generating AI model, and the model's response is incorporated as feedback.

[0627] Step 7:

[0628] Users receive feedback and suggestions from the server via their devices and incorporate them into their daily lives. The input is the feedback generated in step 6, and the output is the addition of new activities and challenges to the parent-child relationship. Specifically, users receive notifications using their smartphones or tablets and perform the suggested activities together.

[0629] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0630] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0631] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0632] [Fourth Embodiment]

[0633] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0634] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0635] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0636] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0637] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0638] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0639] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0640] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0641] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0642] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0643] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0644] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0645] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0646] The following system configuration is conceivable as an embodiment of this invention. The system monitors and evaluates the developmental status of a child and operates using various devices installed in the home environment.

[0647] First, the user installs a remote camera and wearable device in their home. These devices collect the child's biometric information (heart rate, body temperature, etc.) and behavioral information (playing patterns, walking patterns, etc.) in real time.

[0648] Next, this data is sent to the server via the terminal. The server preprocesses the received data, removing noise and standardizing it. Then, the server analyzes the data using artificial intelligence-based analytical tools to assess the child's developmental status. Assessment items include language development, motor skills, and social skills.

[0649] The analysis results are not simply left as they are; the device provides interactive dialogue tailored to the child. For example, based on the evaluation score generated on the server, an interactive AI character engages in educational dialogue with the child through play. This aligns with the objective of improving the child's developmental indicators.

[0650] Furthermore, by analyzing the collected facial expression data, the server can understand the child's emotional state in real time. Based on this, it can generate feedback, such as suggesting relaxing activities if the child is feeling anxious, and notify the parent user.

[0651] Furthermore, this system has a function to predict future development. Based on data accumulated over a long period, the server uses time-series analysis techniques to predict future growth patterns and provides parents with guidance plans and support measures for the future.

[0652] This series of processes allows parents to accurately understand how their child's development is progressing compared to others and how they should provide support. This invention is not merely a developmental assessment tool, but is also extremely useful for providing educational support optimized for each individual child. For example, if the server determines that a child is behind in language development, it can provide the child with an interactive game that allows for speech training via the device, thereby promoting learning in a natural way.

[0653] The following describes the processing flow.

[0654] Step 1:

[0655] The user installs a remote camera and wearable device in their home environment. These devices capture the child's biometric and behavioral information and transmit the data to a server via the device or home network.

[0656] Step 2:

[0657] The server preprocesses the received data. Specifically, it splits video data into individual frames, removes noise from audio data, and standardizes biometric information obtained from wearable devices.

[0658] Step 3:

[0659] The server uses pre-processed data to perform multimodal analysis utilizing artificial intelligence. Image analysis is used to evaluate children's motor skills, speech recognition is used to analyze language development, and the various indicators are integrated.

[0660] Step 4:

[0661] The device uses analysis results from the server to provide children with conversations and games through an interactive AI character. For example, if language training is needed, it will conduct language practice through interactive games.

[0662] Step 5:

[0663] The server uses facial recognition technology to analyze the child's emotions from video data and evaluates their state in real time. If the stress level is high, it prepares to notify the user of that information as feedback.

[0664] Step 6:

[0665] Based on the aggregated analysis data, the server generates expert-reviewed feedback and sends it to the user as a report via email or a dedicated app.

[0666] Step 7:

[0667] The server analyzes long-term accumulated data to predict future developmental patterns. Using time-series models, it can, for example, propose a learning plan for the next semester.

[0668] (Example 1)

[0669] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0670] Supporting children's health and learning during their developmental stages is challenging because it relies on limited resources and environments within the home, making timely and appropriate responses difficult. Furthermore, the lack of systematic tools for accurately understanding a child's condition and enabling parents to appropriately utilize that information is a barrier to effective childcare support.

[0671] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0672] In this invention, the server includes means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment, means for analysis using a generative AI model, and means for dialogue that provides educational dialogue. This enables real-time evaluation of the child's developmental status and provides appropriate educational support and feedback to parents based on that evaluation.

[0673] "Biometric information" refers to data that indicates a child's physical condition, such as heart rate and body temperature.

[0674] "Behavior" refers to patterns of a child's movements and activities, including how they play and walk.

[0675] "Means of collection" refers to devices that use sensors, cameras, etc., to acquire children's biometric information and behavior.

[0676] "Analysis methods using generative AI models" refer to the process of using artificial intelligence technology to process collected data and evaluate the developmental status of children.

[0677] "Dialogue means" refers to the functions of a system that allows for direct interaction with children, provided based on analysis.

[0678] A "feedback generation method" is a mechanism that generates information to convey analysis results and the child's emotional state to the parents.

[0679] "Emotional state analysis methods" refer to the process of inferring a child's emotions from their facial expressions and behavior, and then determining the appropriate response.

[0680] A "predictive tool" is an analytical technique used to predict future growth patterns based on accumulated data.

[0681] One possible embodiment of this invention is a system combining a data collection device, a server, and a terminal installed in a home environment. First, the user starts by installing a remote camera and a wearable device in their home. These devices are means of collecting the child's daily activities and biometric information, recording heart rate, body temperature, movement patterns, etc., in real time.

[0682] The collected data is transmitted to the server via the terminal. Data communication is secure through an encrypted protocol using Wi-Fi or Bluetooth. The server preprocesses the received data and analyzes it using a generative AI model. This generative AI model uses the collected data to evaluate the child's development, for example, assessing motor skills and social skills.

[0683] Furthermore, based on the analysis results, the device uses an interactive AI character to generate educational conversations with the child. These conversations are customized according to the child's developmental stage; for example, if it is determined that the child's language development is delayed, it will provide a game that allows for speech training.

[0684] Furthermore, the server uses facial recognition technology to analyze the child's emotional state in real time and provides relaxation techniques and notifications to parents as needed. In the long term, it predicts the child's growth patterns based on accumulated data through time-series analysis and proposes an instructional plan to parents based on the results.

[0685] As a concrete example, here is an example of a prompt message:

[0686] "Please tell me about recommended activities for assessing and improving children's current motor skills."

[0687] The entire system provides parents with information to better support their children's growth and development, and enables them to address individual educational needs.

[0688] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0689] Step 1:

[0690] The user installs a remote camera and wearable device in their home. The inputs to this process are the device's installation location and environmental settings. The device then begins collecting biometric and behavioral information about the child, including heart rate, body temperature, and movement patterns. The output is a real-time updated data stream.

[0691] Step 2:

[0692] The device receives collected data and transmits it to the server via Wi-Fi or Bluetooth. The input consists of biometric and behavioral data acquired from sensors. During this transmission process, the data is securely sent using encryption technology. The output is a verified data packet sent to the server.

[0693] Step 3:

[0694] The server preprocesses the received data. The input is raw data sent from the terminal. This step involves denoising the data, adjusting timestamps, and standardizing the data. The output is preprocessed data in a format suitable for analysis.

[0695] Step 4:

[0696] The server analyzes the preprocessed data using a generative AI model. The input is the standardized data output in step 3. The analysis is performed by applying machine learning algorithms to evaluate developmental and emotional states. The output is the evaluation results of the child's motor skills, language development, emotional state, etc.

[0697] Step 5:

[0698] The device controls an interactive AI character based on evaluation results from the server and generates educational dialogues. The input is the analysis results. In this step, the dialogue content is programmed according to the child's developmental stage and displayed as a game or educational activity. The output is the interactive experience provided on the screen.

[0699] Step 6:

[0700] The server analyzes facial expression data in real time and provides feedback to the parent user about the child's emotional state. The input is facial expression data from a remote camera. The server analyzes this data and identifies emotions such as anxiety and joy. The output is a suggestion of relaxation activities and a notification to the parent based on this analysis.

[0701] Step 7:

[0702] The server analyzes accumulated long-term data and uses time-series data to predict growth patterns. The input is historical evaluation data. Time-series analysis techniques are used to predict future development. The output includes predicted growth trends and suggested guidance plans for parents.

[0703] (Application Example 1)

[0704] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0705] In modern society, accurately understanding a child's developmental stage and providing optimal educational support is a crucial challenge. However, there is a lack of adequate systems for effectively monitoring a child's biometric information and behavior in real time within the home environment and providing appropriate educational guidance based on that data. Furthermore, current systems make it difficult for parents and guardians to accurately understand information about their child's developmental process and take appropriate action. Therefore, innovative solutions are needed to support children's development more efficiently and effectively.

[0706] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0707] In this invention, the server includes means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment; analysis means using machine learning to analyze the collected data and evaluate the child's developmental status; and interactive dialogue means for providing educational dialogue to the child based on the results obtained from the analysis means. This enables effective real-time support for a child's development within the home, allowing parents and guardians to take appropriate and prompt action.

[0708] "Children's biometric information" refers to data indicating a child's health status, such as heart rate and body temperature.

[0709] "Devices for monitoring behavior in a home environment" refer to remote cameras and sensor devices installed in the home to capture a child's movements and behavioral patterns.

[0710] "Means of data collection" refers to the functions and processes for collecting children's biometric and behavioral data.

[0711] "Machine learning-based analysis methods" refer to algorithms and models used to evaluate a child's developmental status using collected data.

[0712] "Interactive dialogue means" refers to methods and systems that enable educational and interactive communication with children based on analysis results.

[0713] "Information provision methods" refer to the methods and mechanisms for reporting and notifying parents or guardians of the results obtained through analysis.

[0714] "Automated robot-based behavioral support methods" refer to support methods using robots that operate within the home for the purpose of promoting children's learning and growth.

[0715] The system for implementing this invention has a configuration that effectively supports child development in a home environment. The user installs remote cameras and wearable sensors in the home to collect the child's biometric information and behavioral data. This allows the user to monitor the child's health status and daily activities in real time.

[0716] The collected data is sent to the server via the terminal. The server processes the received data using machine learning, performing noise reduction and data standardization before evaluating the child's developmental status. This analysis utilizes AI models implemented in Python or other appropriate programming languages. The AI ​​models analyze the data based on multiple evaluation criteria, including language development, motor skills, and social skills.

[0717] Analysis results from the server enable educational interactions using robots within the home through interactive dialogue mechanisms. Based on instructions from the server, the robot engages in interactive educational activities with children. These activities include, for example, language training games to increase vocabulary and role-playing to improve social skills.

[0718] Furthermore, the server provides parents and guardians with the results obtained from the analysis. This information is presented as feedback on the child's development through smart glasses or other display devices. This allows parents and guardians to understand their child's current condition and make informed decisions about their next actions.

[0719] As a concrete example, if the server determines that a child is experiencing a delay in language development, the robot will perform an activity to increase vocabulary through simple questions such as, "Which do you prefer? Cats or dogs?" Another example of a prompt when using this invention is to input to the generating AI model, "Design an interactive language training game for a child. The game should be designed so that parents can monitor it through smart glasses, and the child should be able to learn while having fun."

[0720] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0721] Step 1:

[0722] Users install remote cameras and wearable sensors in their homes to collect biometric and behavioral data about their children. These devices acquire biometric data such as heart rate and body temperature, as well as behavioral data such as walking patterns and playtime. The acquired data is transmitted to a device in real time.

[0723] Step 2:

[0724] The terminal transfers data received from the user to the server. The server removes noise from the data and normalizes it as needed. This process prepares the data for AI analysis. The input is raw data such as heart rate and behavioral patterns, and the output is standardized data necessary for the analysis process.

[0725] Step 3:

[0726] The server inputs pre-processed data into an AI analysis engine to evaluate the child's developmental stage. The AI ​​model uses generative AI technology to analyze multiple data points (language development, motor skills, social skills). The evaluation results are output as a numerical evaluation score.

[0727] Step 4:

[0728] The server sends instructions to the robot through interactive dialogue based on the evaluation score obtained. The robot provides children with educational dialogues and activities according to the evaluation score. For example, it conducts activities to improve vocabulary through language training games.

[0729] Step 5:

[0730] The server generates feedback for parents or guardians based on the analysis results and the child's emotional state. This feedback is sent via the device to smart glasses or another display device. The feedback includes information about the child's developmental environment and the support they need. The inputs are the evaluation score and emotional state, and the output is the feedback message.

[0731] Step 6:

[0732] Users utilize feedback provided by the server to plan the next steps in supporting their child's development at home. Based on this feedback, they can adjust educational plans and curricula as needed.

[0733] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0734] The following configuration is conceivable as an embodiment of this invention. The system is designed to provide more optimized feedback by evaluating the child's developmental stage and also taking into account the emotional state of the parent, who is the user.

[0735] Users place remote cameras and wearable devices in their home environment, and these devices are used to automatically collect biometric information and behavioral data of their children. These devices transmit data to a server via Wi-Fi or Bluetooth.

[0736] The server preprocesses received biometric and behavioral data, and then uses artificial intelligence to analyze it. This allows for the evaluation of various developmental indicators, such as motor skills and language development. In addition, an emotion recognition algorithm is used to analyze the child's emotional state from the collected data, determining whether they are relaxed or stressed.

[0737] Based on the analysis results, the device displays customized interactive educational content for the child through an interactive AI character. For example, if the server determines that the child needs to improve their social skills, the AI ​​character will suggest a social game.

[0738] Furthermore, the system incorporates an emotion engine that analyzes the user's (parent's) emotional state. To do this, the server performs emotion analysis using the user's voice data and facial expression data acquired from the camera. The emotion engine determines the parent's stress level and level of interest, and adjusts the content and timing of feedback accordingly.

[0739] Finally, the server generates expert-reviewed feedback from the analyzed data and emotional state information, which is then delivered to the user via email or app notification. For example, the server could include advice such as suggesting that parents engage in relaxing activities with their children at the end of the day to reduce stress.

[0740] In this way, by considering not only the child's biometric information and behavioral data but also the parent's emotional state, the system consistently provides effective childcare support. For example, to improve delays in a child's motor skills indicated by the analysis results, the emotional engine can utilize methods to attract the parent's attention, thereby supporting parents in actively participating in their child's training.

[0741] The following describes the processing flow.

[0742] Step 1:

[0743] Users install remote cameras and wearable devices in their home environment to continuously record their child's biometric information and behavior. Additionally, if parents use a smartphone or PC, they can configure it to capture audio and facial expressions via the camera.

[0744] Step 2:

[0745] The recorded biometric and behavioral data is sent to the device, which temporarily stores this data. The data is then configured to be uploaded to a server via Wi-Fi or Bluetooth.

[0746] Step 3:

[0747] The server preprocesses the received data. This processing includes noise reduction and frame splitting of video data, clearance of audio data, and standardization of biometric data from wearable devices.

[0748] Step 4:

[0749] The server uses artificial intelligence to analyze child development indicators based on pre-processed data. For example, it analyzes and scores a child's motor movements from image data and evaluates language development from audio data.

[0750] Step 5:

[0751] Simultaneously, the server uses an emotion engine to analyze the parent's voice and facial expression data. The emotion engine extracts voice tone and facial expression characteristics and evaluates the parent's emotional state in real time.

[0752] Step 6:

[0753] Based on the analysis results, the device uses an interactive AI character to provide educational content suitable for children. The character engages in conversations and games designed to improve the skills children need, based on the analyzed developmental indicators.

[0754] Step 7:

[0755] The server integrates the data obtained through analysis and generates expert-reviewed feedback. This feedback is customized to take into account the parent's emotional state, and may include suggestions for relaxation methods if the parent is feeling stressed, for example.

[0756] Step 8:

[0757] Ultimately, the server provides feedback to the user via email or app notification. At this time, it will also suggest the next steps and recommended actions to receive effective support.

[0758] Step 9:

[0759] In the long term, the server will analyze the accumulated data, update models to predict future development, and present these predictions to parents, enabling planned childcare support.

[0760] (Example 2)

[0761] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0762] In modern childcare, there is a need to accurately understand a child's developmental and emotional state and provide appropriate feedback to parents. However, conventional technology has not adequately achieved the ability to comprehensively analyze a child's physical and emotional data and provide optimal feedback that also takes into account the parent's emotional state. Therefore, a more intelligent and comprehensive support system is needed to promote a child's development.

[0763] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0764] In this invention, the server includes means for monitoring a child's biometric information and behavior and transmitting data using communication technology, means for preprocessing and analyzing the data and evaluating developmental indicators, and means for evaluating the user's emotions. This makes it possible to provide parents with customized feedback tailored to the individual developmental needs of their children, thereby intelligently and effectively providing childcare support.

[0765] "Children's biometric information" refers to data that represents a child's physical condition, including heart rate, body temperature, and respiratory rate.

[0766] "Behavioral data" refers to information that records a child's daily activities, describing things like distance traveled, playtime, and types of activities.

[0767] "Communication technology" refers to technologies for sending and receiving data between devices, and includes wireless communication methods such as Wi-Fi and Bluetooth.

[0768] An "external processing device" is a computer system that receives data and performs analysis processing, and includes equipment such as servers.

[0769] "Preprocessing" refers to the process of preparing data to make it easier to analyze, and includes tasks such as organizing, cleansing, and imputing missing values.

[0770] "Artificial intelligence" refers to the technology of computers imitating human intellectual behavior, and involves methods such as machine learning and data analysis.

[0771] "Developmental indicators" are standards used to evaluate a child's growth and development, and refer to things like motor skills and language abilities.

[0772] "Emotional state" refers to an individual's emotional reactions and circumstances, including stress, joy, and relaxation.

[0773] "Emotional analysis methods" refer to methods of analyzing an individual's emotional state based on data, and utilize techniques such as voice analysis and facial expression analysis.

[0774] "Interactive educational information" refers to content that aims to achieve educational effects through interaction, and includes games and conversational simulations.

[0775] "Customized feedback" refers to feedback that is tailored to an individual's characteristics and needs.

[0776] This system aims to assess a child's developmental stage and provide appropriate feedback to parents. The system is structured as follows:

[0777] Users install remote cameras and wearable devices in their home environment. These devices incorporate wireless communication technologies such as Bluetooth and Wi-Fi to automatically collect biometric and behavioral data of children. Specific data includes heart rate, body temperature, distance traveled, and playtime.

[0778] Data is transmitted to the server in real time. The server is equipped with a high-performance processor and artificial intelligence model, and performs preprocessing on the received data. Preprocessing includes data cleansing and imputation of missing values. Subsequently, machine learning algorithms are used to evaluate developmental indicators such as children's motor skills and language abilities, and emotion recognition algorithms are used to analyze their emotional state.

[0779] The server further analyzes the parents' voice data and camera footage, and uses an emotion engine to determine stress levels and levels of interest. Based on these analysis results, it generates educational content tailored to the parents' emotions and the child's development.

[0780] Based on the analysis results, the device presents educational content to the child through an interactive AI character. For example, if it determines that the child's motor skills are somewhat lacking, it will introduce engaging fitness games to encourage activity.

[0781] Finally, the server generates expert-reviewed feedback. This feedback is delivered via email or app notifications on the parent's smartphone or computer. A specific example is "suggestions for relaxing evening activities to do with children, aimed at reducing parental stress."

[0782] An example of a prompt message would be: "Generate an appropriate encouraging message based on the latest data on the child's athletic ability. Include techniques to capture the parent's attention."

[0783] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0784] Step 1:

[0785] The user installs a remote camera and wearable devices in their home. These devices collect the child's heart rate, body temperature, and activity level in real time. The input consists of biometric and behavioral data obtained from the child. This data is transmitted to a server using wireless communication technology. The output is the raw data sent to the server.

[0786] Step 2:

[0787] The server preprocesses the received data. Specifically, it cleanses the data, detects and imputes outliers and missing values. The input is the biometric and behavioral data transmitted in step 1. The output is a preprocessed, consistent dataset. This processing ensures data quality that allows for accurate subsequent analysis.

[0788] Step 3:

[0789] The server performs AI analysis using pre-processed data. It uses a generative AI model to perform data calculations to evaluate developmental indicators such as motor skills and language abilities. It also analyzes children's emotional states using an emotion recognition algorithm. The input is a pre-processed dataset. The output is the evaluation of developmental indicators and emotional states as analysis results. At this stage, specific emotion models and machine learning algorithms are used.

[0790] Step 4:

[0791] The server evaluates the parent's emotional state. It takes in parental voice data and camera footage as input and uses an emotion engine to analyze stress levels and interest levels. The output is evaluation data regarding the user's (parent's) emotional state. This evaluation allows for the customization of feedback based on the parent's situation.

[0792] Step 5:

[0793] The device generates and presents educational content through an interactive AI character based on analysis results obtained from the server. The input consists of the child's developmental indicators and emotional state, obtained from the server. The output is child-specific educational content, presented in a game or dialogue format.

[0794] Step 6:

[0795] The server generates feedback based on the analysis results and the parents' emotional state. It designs specific suggestions and activity plans and notifies the parents. The inputs are child development data and parent emotional analysis data. The output is a feedback message sent to the parents' devices, which includes expert-reviewed advice. This feedback serves to engage the parents and support their parenting.

[0796] (Application Example 2)

[0797] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0798] In modern family environments, there is a lack of childcare support systems that effectively support children's development while reducing parental stress. In particular, there is a need for technology that analyzes children's biometric information and behavior in real time, and provides appropriate feedback while also considering the parents' emotional state.

[0799] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0800] In this invention, the server includes a device for collecting a child's biometric information and behavioral data, a device for analyzing the collected data using machine learning technology, a device for generating educational dialogues based on the analysis results, an emotion analysis device for analyzing the parent's voice data and facial expression data, an emotion engine for adjusting the content of feedback according to the parent's emotional state, and an interaction generation device for providing interactive educational support. This makes it possible to provide childcare support that promotes the child's development while taking into account the parent's emotional state.

[0801] "Children's biometric information" refers to physical information measured by heart rate, body temperature, and motion sensors.

[0802] "Behavioral data" refers to data that records a child's daily movements and activity patterns.

[0803] A "device" is hardware used to collect biometric information and behavioral data.

[0804] "Machine learning techniques" is a general term for algorithms used to analyze large amounts of data and find patterns and trends.

[0805] "Analysis equipment" refers to hardware and software used to evaluate a child's developmental status using collected data.

[0806] "Educational dialogue" is interactive communication designed to promote a child's development.

[0807] An "emotion analysis device" is hardware and software that uses voice data and facial expression data to determine a parent's emotional state.

[0808] The "emotional engine" is a function that adjusts the content and timing of feedback based on the parent's emotional state.

[0809] An "interaction generation device" is a device that generates content and activities to support education through dialogue with children.

[0810] This invention provides a childcare support system that effectively assists child development in the home environment while also being considerate of parents. The specific system configuration includes the following elements:

[0811] The server collects data using a device equipped with multiple sensors to gather children's biometric and behavioral data. This device includes cameras and wearable sensors, and transmits the data to the server via Wi-Fi or Bluetooth.

[0812] The server analyzes the collected data using machine learning techniques such as Python and TensorFlow. This analysis evaluates developmental indicators such as children's motor skills and language development, and also determines the children's emotional state using an emotion analysis algorithm.

[0813] Based on the analysis results, the device provides dialogue to the child through an interaction generator. The interaction generator dynamically generates educational content using a generation AI model and presents it to the child visually and audibly.

[0814] Furthermore, the server uses an emotion analysis device to analyze voice and facial expression data in order to understand the parent's emotional state. The analysis results are processed by an emotion engine, and feedback is adjusted according to the parent's stress level and level of interest.

[0815] The user (parent) receives final feedback via app notifications or email, which includes specific activity suggestions. For example, when suggesting a quiz game or physical activity for parents and children to play together, a prompt message such as "Please suggest activities to reduce parental stress" is sent to the generating AI model.

[0816] This system allows users to monitor their children's growth while fostering better parent-child relationships.

[0817] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0818] Step 1:

[0819] The server collects real-time biometric and behavioral data of children from sensors placed throughout the home. Inputs are biometric data (heart rate, body temperature, etc.) and behavioral data from the sensors, and output is the aggregation and temporary storage of this data. Specifically, data is transferred from the sensors to the server using Wi-Fi or Bluetooth.

[0820] Step 2:

[0821] The server analyzes the collected data using machine learning techniques. The input consists of biometric and behavioral data aggregated in Step 1. The data is analyzed using Python and TensorFlow, generating output that evaluates children's motor skills, language development, and emotional states. Specifically, it performs data cleansing and normalization, then inputs the data into the model to recognize patterns.

[0822] Step 3:

[0823] The server designs interactions using a dialogue generation device based on the analysis results. The input is the analysis results obtained in step 2, and the generating AI model is used to generate educational content and activity content. The output is dialogue prompts and game content optimized for children. Specifically, after the dialogue content is determined, information to be presented to the child is created using speech synthesis.

[0824] Step 4:

[0825] The device provides the child with content received from the interaction generator. The input consists of dialogue prompts and game content generated in step 3, which are presented to the child as audio and video. The output allows observation of the child's response. In terms of specific actions, interaction is carried out using the screen and speaker.

[0826] Step 5:

[0827] The server collects parent voice and facial expression data and performs emotion analysis. The input is parent voice and facial expression data, and the output is data indicating the parent's emotional state (e.g., stress level). Specifically, it uses a microphone and camera to acquire detailed audio and video data and uses an analysis algorithm to estimate emotions.

[0828] Step 6:

[0829] The server uses an emotion engine to adjust the content of the feedback. The input is the parent's emotional state obtained in step 5 and the analysis results from step 2, and based on this, it generates optimal feedback for the parent. The output is activity suggestions and advice provided to the parent. Specifically, a prompt "Please suggest relaxing activities" is sent to the generating AI model, and the model's response is incorporated as feedback.

[0830] Step 7:

[0831] Users receive feedback and suggestions from the server via their devices and incorporate them into their daily lives. The input is the feedback generated in step 6, and the output is the addition of new activities and challenges to the parent-child relationship. Specifically, users receive notifications using their smartphones or tablets and perform the suggested activities together.

[0832] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0833] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0834] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0835] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0836] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0837] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0838] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0839] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0840] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0841] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0842] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0843] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0844] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0845] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0846] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0847] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0848] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0849] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0850] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0851] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0852] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0853] The following is further disclosed regarding the embodiments described above.

[0854] (Claim 1)

[0855] A means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment,

[0856] The aforementioned collected data is analyzed using artificial intelligence to evaluate the developmental status of children,

[0857] Based on the results obtained from the aforementioned analysis means, a dialogue means for providing educational dialogue to children,

[0858] A feedback generation means for providing feedback to parents by analyzing the results obtained from the aforementioned analysis means and the emotional state of the child,

[0859] A system that includes this.

[0860] (Claim 2)

[0861] The system according to claim 1, further comprising a predictive means for predicting a child's developmental pattern over the long term based on accumulated data.

[0862] (Claim 3)

[0863] The system according to claim 1, wherein the dialogue means customizes the content of the dialogue according to the child's developmental indicators.

[0864] "Example 1"

[0865] (Claim 1)

[0866] A means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment,

[0867] Analysis means using a generative AI model for analyzing the collected data,

[0868] Based on the evaluation results obtained from the aforementioned analysis means, a dialogue means for providing educational dialogue is provided.

[0869] A feedback generation means that analyzes the aforementioned evaluation results and the child's emotional state and provides feedback to the parent,

[0870] An emotional state analysis method that analyzes a child's emotional state in real time and proposes appropriate activities,

[0871] A system that includes this.

[0872] (Claim 2)

[0873] The system according to claim 1, further comprising a prediction means for predicting a child's growth pattern over the long term based on accumulated data.

[0874] (Claim 3)

[0875] The system according to claim 1, wherein the dialogue means adjusts the content of the dialogue according to the child's developmental indicators.

[0876] "Application Example 1"

[0877] (Claim 1)

[0878] A means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment,

[0879] The aforementioned collected data is analyzed using machine learning to evaluate the developmental status of children,

[0880] Based on the results obtained from the aforementioned analysis means, an interactive dialogue means for providing educational dialogue to children is provided,

[0881] An information provision means for generating feedback to guardians by analyzing the results obtained from the aforementioned analysis means and the emotional state of the child,

[0882] A behavioral support system using automated robots to support the development of children in the home,

[0883] A system that includes this.

[0884] (Claim 2)

[0885] The system according to claim 1, further comprising analytical means for predicting a child's growth pattern over the long term based on integrated data.

[0886] (Claim 3)

[0887] The system according to claim 1, wherein the interactive dialogue means adjusts the content of the dialogue according to the child's developmental indicators.

[0888] "Example 2 of combining an emotion engine"

[0889] (Claim 1)

[0890] A means for collecting data from a device for monitoring a child's biometric information and behavior in a home environment, and transmitting that data to an external processing device using communication technology,

[0891] The collected data is preprocessed, and an analysis method using artificial intelligence is used to evaluate developmental indicators and emotional states.

[0892] An emotion analysis means for analyzing user voice and video data and evaluating their emotional state,

[0893] Based on the results obtained from the analysis means and emotion analysis means, a dialogue generation means for generating interactive educational information tailored to a child,

[0894] Based on the above results, a feedback generation means for providing optimal feedback to parents,

[0895] A system that includes this.

[0896] (Claim 2)

[0897] The system according to claim 1, further comprising prediction and promotion means for predicting a child's developmental patterns over the long term, taking into account accumulated analysis results and user emotional information, and for implementing methods to attract parental attention.

[0898] (Claim 3)

[0899] The system according to claim 1, wherein the dialogue generation means customizes the dialogue content according to the analyzed developmental indicators and emotional state, and proposes activities to reduce parental stress.

[0900] "Application example 2 when combining with an emotional engine"

[0901] (Claim 1)

[0902] A device for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment,

[0903] The aforementioned collected data is analyzed using machine learning technology to evaluate the developmental status of children, and

[0904] Based on the results obtained from the aforementioned analysis device, a dialogue generation device for providing educational dialogues to children is provided.

[0905] A feedback generation device for providing feedback to parents by analyzing the results obtained from the aforementioned analysis device and the emotional state of the child,

[0906] An emotion analysis device for analyzing the emotional state of parents using voice data and facial expression data,

[0907] Based on the emotional state of the parent obtained from the aforementioned emotion analysis device, an emotion engine is provided to adjust the content and timing of educational feedback.

[0908] An interaction generation device to support children's education while facilitating interactive exchange,

[0909] A system that includes this.

[0910] (Claim 2)

[0911] The system according to claim 1, further comprising a predictive engine for predicting a child's developmental patterns over the long term based on accumulated data and generating appropriate activity suggestions for parents.

[0912] (Claim 3)

[0913] The system according to claim 1, wherein the dialogue generation device generates dialogue content corresponding to the child's developmental indicators and also generates dialogue prompts that take into account the parent's emotional state. [Explanation of Symbols]

[0914] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting data obtained from a device for monitoring a child's biometric information and behavior in a home environment, The aforementioned collected data is analyzed using artificial intelligence to evaluate the developmental status of children, Based on the results obtained from the aforementioned analysis means, a dialogue means for providing educational dialogue to children, A feedback generation means for providing feedback to parents by analyzing the results obtained from the aforementioned analysis means and the emotional state of the child, A system that includes this.

2. The system according to claim 1, further comprising a prediction means for predicting a child's developmental pattern over the long term based on accumulated data.

3. The system according to claim 1, wherein the dialogue means customizes the content of the dialogue according to the child's developmental indicators.

Images