System

Abstract

A system is provided.SOLUTION: A system, comprising: means for obtaining a conversation with a child; means for using a generative artificial intelligence model to analyze the obtained conversation data and infer a mental state of the child; and means for notifying a parent of the mental state of the child.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's society, where dual-income households are on the rise, children are spending more time alone. As a result, parents have fewer opportunities to monitor their children's growth and mental state, increasing the likelihood that children will feel lonely or stressed. Furthermore, there are limited means to continuously monitor what children are doing and their mental state during the time they spend alone, and to ensure their safety. There is a need for a system that can solve these problems and ensure children's growth and safety while allowing parents to keep track of their children's situation with peace of mind. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for acquiring a conversation with a child, a means for analyzing the acquired conversation data and using a generative artificial intelligence model to infer the child's mental state, and a means for notifying a parent of the child's mental state. The generative artificial intelligence model may further include a means for generating learning content related to a subject and presenting it to the child. The system may also include a means for monitoring the child's behavior and detecting abnormalities, and a means for immediately notifying the parent if an abnormality is detected. The above provides a system that ensures the growth and safety of children during their time alone and allows parents to better understand their children's situation.

[0006] "Children" refers to preschool and school-age individuals, an age group that is increasingly spending time alone.

[0007] "Means for acquiring conversations" refers to devices or programs that use microphones, text input devices, etc. to collect children's speech in real time.

[0008] A "generative artificial intelligence model" refers to a machine learning algorithm or program designed to use natural language processing technology to analyze input conversational data and infer emotions and mental states.

[0009] "Mental state" refers to a child's emotional and psychological well-being, including feelings of loneliness, stress, and anxiety.

[0010] "Means of parental notification" refers to devices or programs that communicate information about a child's mental health to parents using email, messaging apps, telephone, etc.

[0011] "Learning content" refers to information provided to children to gain knowledge through learning activities, such as educationally relevant questions and reading materials.

[0012] "Means for monitoring behavior" refers to devices or programs that use cameras, sensors, microphones, etc. to record and monitor children's behavior in real time.

[0013] "Means for detecting anomalies" refers to algorithms or programs that analyze collected data and detect anomalies when there is a deviation from the normal range of behavior.

[0014] "Database" refers to a digital recording system for storing and managing children's conversation data, behavioral data, analysis results, etc.

[0015] "Accumulated Data" refers to information about a child's past actions and statements that is continuously collected and stored by the system. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention addresses the issue of children spending more time alone as the number of dual-income households increases, by providing a system that utilizes generative AI to promote children's development and monitor safety within the home.

[0038] System configuration and operation

[0039] Server Operation

[0040] The server hosts the generative AI model and database management system and performs the following operations:

[0041] 1. Load the generative AI model:

[0042] The server loads the generative AI model into memory at startup.

[0043] 2. Data Receipt and Storage:

[0044] The system receives the child's conversation data sent from the device and stores it in a database.

[0045] 3. Data Analysis:

[0046] The received conversation data is input into a generative AI model to analyze the child's mental state.

[0047] 4. Notification function:

[0048] Based on the analysis results, if an abnormal mental state is detected, a notification will be sent to the parents.

[0049] 5. Learning content generation:

[0050] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[0051] 6. Behavioral monitoring and anomaly detection:

[0052] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is notified.

[0053] Device behavior

[0054] The device mediates interactions with the child and performs the following actions:

[0055] 1. Conversation Acquisition:

[0056] The child's speech is captured using a microphone or text input device.

[0057] 2. Data transmission:

[0058] The acquired conversation data is sent to the server.

[0059] 3. Learning Support:

[0060] The learning content received from the server is presented to the child and they are prompted to respond.

[0061] 4. Condition Monitoring:

[0062] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[0063] Usage example

[0064] 1. Example conversation with a child:

[0065] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[0066] The terminal transmits this message to the server.

[0067] The server stores the received data and analyzes the content of the statements using a generative AI model.

[0068] The server identifies the mental state "lonely" and sends a notification to the parent that "child feels lonely."

[0069] 2. Learning support examples:

[0070] The user (child) tells the device, "I want to study math now."

[0071] The terminal sends this request to the server.

[0072] The server uses a generative AI model to generate appropriate math problems and sends them to the device.

[0073] The device presents questions to the child and prompts them to answer.

[0074] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[0075] 3. Condition monitoring example:

[0076] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[0077] The device transmits behavioral data to the server in real time.

[0078] The server analyzes the received behavioral data and notifies the parent if any abnormalities are detected.

[0079] summary

[0080] In this way, the system of this invention utilizes generative AI models to support children's development and monitor their safety, providing an environment where parents in dual-income households can feel secure knowing what is happening with their children. The specific implementation is programmed, interacting with children in real time, analyzing their mental state and behavior based on accumulated data, and providing necessary notifications and learning content. This makes children's alone time safer and more meaningful.

[0081] The processing flow will be explained below.

[0082] Specific explanation of system program processing

[0083] Retrieving conversations and sending data

[0084] Step 1:

[0085] The device receives the child's speech via a microphone or text input device. For example, the device receives a speech such as, "I was lonely at school today because my friends were not there."

[0086] Step 2:

[0087] The device sends the acquired speech data to the server, along with the speech content and a timestamp.

[0088] Step 3:

[0089] The server stores the data received from the device in a database, including the content of the message and a timestamp.

[0090] Step 4:

[0091] The server inputs the received data into a generative AI model to analyze the child's mental state. In this case, the emotion of "lonely" is identified.

[0092] Step 5:

[0093] The server stores the analysis results in a database and sends a notification to the parents if the child's mental state is determined to be "lonely."

[0094] Learning support and dialogue

[0095] Step 1:

[0096] The user (child) tells the device, "I want to study math now."

[0097] Step 2:

[0098] The device sends this request to the server, along with the request content and a timestamp.

[0099] Step 3:

[0100] The server stores the received requests in a database and inputs them into a generative AI model.

[0101] Step 4:

[0102] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[0103] Step 5:

[0104] The server sends the generated math problem to the terminal.

[0105] Step 6:

[0106] The device presents the received question to the child and waits for the child's answer.

[0107] Step 7:

[0108] The user (child) enters the answer into the terminal, and the terminal sends the answer to the server.

[0109] Step 8:

[0110] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[0111] Condition monitoring and anomaly detection

[0112] Step 1:

[0113] The device monitors children's behavior in real time through cameras and sensors.

[0114] Step 2:

[0115] The device sends the monitoring data, including the activity and a timestamp, to the server.

[0116] Step 3:

[0117] The server stores the received behavioral data in a database.

[0118] Step 4:

[0119] The server analyzes the accumulated data using a generative AI model and compares it with the normal range of behavior.

[0120] Step 5:

[0121] If the server detects an anomaly, it will immediately send a notification to the parent, for example if the device is stationary for an abnormally long period of time or if it is behaving suspiciously.

[0122] This allows the system to continuously monitor a child's mental state and behavior through daily interactions with the child and support their learning, and to support the child's growth and safety by promptly notifying parents if any abnormalities or dangers occur.

[0123] Example 1

[0124] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0125] With the increase in dual-income households, children spend more time alone, creating the issue of the need to properly monitor their growth and safety. Furthermore, there is a lack of effective ways to monitor children's mental health and academic progress. This creates a need for a system that allows parents to monitor their children's condition while at work and respond quickly if there is an abnormality.

[0126] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0127] In this invention, the server includes means for acquiring conversations with children, means for transmitting the acquired conversation data to the server, means for storing the received conversation data in a database, means for analyzing the child's mental state using a generative AI model based on the stored conversation data, and means for notifying parents if an abnormal mental state is detected based on the analysis results. This enables parents in dual-income households to understand their children's mental state and behavior in real time even while at work, and to provide appropriate learning support and safety checks.

[0128] "Children" refers to minors who are eligible to use the System.

[0129] "Conversation data" refers to a recording of a child's speech in the form of text or audio data.

[0130] "Server" refers to the central computing unit that hosts the generative AI model and receives, stores, and analyzes data.

[0131] A "generative AI model" refers to a program model that uses artificial intelligence technology to perform natural language processing and data analysis.

[0132] "Database" refers to an information management system for storing received conversation data and behavioral data.

[0133] "Mental state" refers to the emotional and psychological state that can be inferred from a child's words and behavior.

[0134] "Notification" refers to an alert or informational message sent from the server to the parent.

[0135] "Learning content" refers to the learning questions and materials generated by the generative AI model in response to the child's requests.

[0136] "Behavioral data" refers to data such as a child's movements and location collected using cameras and sensors.

[0137] "Abnormal" refers to a child's behavior or mental state that is different from normal and indicates an unnatural or dangerous condition.

[0138] The present invention is a system that supports the safety and development of children by addressing the increasing amount of time they spend alone due to the increase in dual-income households. This system includes a device that interacts with children and a server that analyzes data and sends notifications.

[0139] Hardware and software used

[0140] Terminals consist of interaction devices equipped with input devices such as microphones, cameras, sensors, and touchscreens. Examples include general-purpose tablets, smart speakers, and surveillance cameras.

[0141] The server is a central computing unit that hosts the generative AI model and receives, stores, and analyzes data. Software used includes deep learning frameworks (e.g., TensorFlow, PyTorch), database management systems (e.g., MySQL, PostgreSQL), and various notification APIs (e.g., Twilio, SendGrid).

[0142] Conversation capture and analysis

[0143] The device uses a microphone and text input device to capture the child's speech. For example, if a child says, "I missed my friends at school today," the device uses voice recognition software to convert this speech into text data. The converted text data is then securely transmitted to a server via the HTTPS protocol.

[0144] The server stores the received text data in a database such as MySQL. It then analyzes the data using a generative AI model (e.g., BERT, GPT-3). At this time, the prompt "Please analyze the mental state when the user tells a sad story" is input to the model.

[0145] Mental state detection and notification

[0146] The server determines the child's mental state based on the analysis results. For example, if the emotion "lonely" is detected, a notification is sent to the parent using Twilio or SendGrid. The notification includes a message saying "Your child is feeling lonely."

[0147] Creating and presenting learning content

[0148] When a user (child) says to the device, "I want to study math now," the device sends this request to the server.

[0149] The server uses a generative AI model to generate appropriate math problems. The prompt "Generate simple math problems for elementary school students" is input to the model. The generated math problems are sent back to the device, which presents them to the child. The child enters the answers, and the answer data is sent back to the server.

[0150] Behavioral data monitoring and analysis

[0151] The device continuously monitors the child's behavior using cameras and sensors, for example, recording the child's behavior while they are in their room, and transmitting the monitoring data to a server in real time.

[0152] The server analyzes the received behavioral data and notifies the parent if abnormal behavior is detected. For example, if a child remains motionless for a long time, a notification stating "Child has not moved for a long time" is sent to the parent.

[0153] In this way, the system of the present invention utilizes a generative AI model to analyze a child's mental state and behavior, and provides parents with the necessary notifications and learning support, thereby providing an environment where parents in dual-income households can feel at ease knowing what is happening with their children.

[0154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0155] Step 1: Obtaining conversation data

[0156] The device uses a microphone and text input device to capture what the child says. For example, if a child says, "I missed my friends at school today," the device uses voice recognition software to convert this speech into text data.

[0157] (Input) Child's speech (audio or text)

[0158] (Output) Text data (in the case of audio, data converted to text)

[0159] Step 2: Sending conversation data to the server

[0160] The device sends the retrieved text data to the server via an HTTP POST request, with the data being transferred securely using SSL encryption.

[0161] (Input) Text data

[0162] (Output) None (Transfer to server)

[0163] Step 3: Save your data

[0164] The server stores the received text data in MySQL (or other database management systems), including information such as what the child said, when they said it, and the speaker's ID.

[0165] (Input) Text data (transferred data)

[0166] (Output) Save data to database (persistent storage)

[0167] Step 4: Analyzing the conversation data

[0168] The server inputs the saved text data into a generative AI model to analyze the child's mental state. Specifically, the model is given a prompt sentence: "Please analyze the child's mental state when the user tells a sad story." This analysis uses the natural language processing capabilities of BERT and GPT-3.

[0169] (Input) Text data (saved conversation data), prompt sentence

[0170] (Output) Analysis results (mental state)

[0171] Step 5: Mental status notification

[0172] The server evaluates the child's mental state based on the analysis results, and if an abnormal state is detected, it immediately notifies the parent. For example, if the emotion "lonely" is detected, a notification stating "your child is feeling lonely" is sent to the parent using Twilio or SendGrid.

[0173] (Input) Analysis results (mental state)

[0174] (Output) Notification message (notification to parent)

[0175] Step 6: Request learning content

[0176] The user (child) requests learning content by telling the device, "I want to study math now."

[0177] (Input) Request statement ("I want to study math now.")

[0178] (Output) Request data (transferred to server)

[0179] Step 7: Generate learning content

[0180] The server receives the learning request and generates appropriate math problems using a generative AI model. For example, it generates problems by inputting a prompt sentence such as "Generate simple math problems for elementary school students."

[0181] (Input) Request data, prompt

[0182] (Output) Learning content (math problems)

[0183] Step 8: Present learning content and submit answers

[0184] The device receives a mathematical problem from the server and presents it to the child, prompting the child to answer it. The child enters the answer into the device, which then transmits the answer to the server.

[0185] (Input) Learning content (math problems)

[0186] (Output) Answer data (transferred to server)

[0187] Step 9: Monitor behavioral data

[0188] The device uses cameras and sensors to continuously monitor a child's behavior, for example by recording their movements and location in real time while they are in their room.

[0189] (Input) Behavioral data (data from cameras and sensors)

[0190] (Output) Behavioral data (transferred to server)

[0191] Step 10: Analyze behavioral data and detect anomalies

[0192] The server analyzes the received behavioral data and notifies parents if abnormal behavior is detected. For example, if a child stays in the same place for a long time, the server sends a message saying, "Your child has not moved for a long time."

[0193] (Input) Behavioral data (data monitored in real time)

[0194] (Output) Notification message (notification to parent)

[0195] (Application example 1)

[0196] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0197] With the increase in dual-income households, children are spending more time alone, making it necessary to provide appropriate supervision and learning support to ensure their growth and safety. It is also important to understand children's mental state and behavior in real time and detect abnormalities early. The present invention aims to provide a system that solves these problems.

[0198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0199] In this invention, the server includes means for acquiring conversation data with a child, means for analyzing the acquired conversation data and using a generative AI model to infer the child's mental state, means for notifying a parent of the child's mental state, means for generating appropriate learning content and presenting it to the child, means for monitoring the child's behavior and detecting abnormalities, means for immediately notifying the parent when an abnormality is detected, and means for generating appropriate prompt sentences and inputting them into the generative AI model. This makes it possible to provide necessary learning support while monitoring the child's mental state and behavior in real time at home, and to respond quickly when an abnormality is detected.

[0200] "Conversation with children" refers to communication within the home that includes words and sentences spoken by children.

[0201] "Means of acquisition" refers to the functionality for collecting data using a microphone, text input device, or other sensor device.

[0202] A "generative AI model" is an artificial intelligence algorithm or software that generates new information or content from original data through natural language processing and data analysis.

[0203] "Mental state" refers to a child's psychological health and emotional changes, and is an indicator for assessing that state.

[0204] "Means of notifying parents" refers to the means of communication to promptly inform parents of analysis results or abnormal situations, including email, SMS, in-app notifications, etc.

[0205] "Learning content" refers to questions, reading materials, and other content designed to support a child's education and intellectual development.

[0206] "Presentation means" is a function for showing the generated learning content to children or conveying it to them by voice.

[0207] "Behavioral monitoring measures" are features that use cameras and sensors to continuously observe a child's physical movements and location.

[0208] "Means for detecting anomalies" refers to the ability to identify unusual behavior or situations and issue a warning when danger or problems occur.

[0209] A "prompt" is an instruction or question used to direct a generative AI model to perform a specific analysis or generation.

[0210] "Smart device" means a device with internet connectivity and advanced computing capabilities, including smartphones and tablets.

[0211] The present invention provides a system that uses a generative AI model to support the development and safety of children in dual-income households. This system is composed of a server and a terminal, and is realized using the following means.

[0212] Server Operation

[0213] The server hosts the generative AI model and database management system and performs the following operations:

[0214] 1. Load the generative AI model:

[0215] The server loads the generative AI model into memory at startup, allowing it to analyze conversational and behavioral data in real time.

[0216] 2. Data Receipt and Storage:

[0217] Conversation data and behavioral data with the child sent from the device is received and stored in a database.

[0218] 3. Data Analysis:

[0219] The received conversation data is input into a generative AI model to analyze the child's mental state, and based on this analysis, a notification is sent to the parent.

[0220] 4. Learning content generation:

[0221] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[0222] 5. Behavioral monitoring and anomaly detection:

[0223] The received behavioral data is analyzed, and if any abnormalities are detected, a notification is sent to the parent.

[0224] Device behavior

[0225] The device mediates interactions with the child and performs the following actions:

[0226] 1. Conversation Acquisition:

[0227] The child's speech is captured using a microphone or text input device.

[0228] 2. Data transmission:

[0229] The acquired conversation data is sent to the server.

[0230] 3. Learning Support:

[0231] The learning content received from the server is presented to the child and they are prompted to respond.

[0232] 4. Condition Monitoring:

[0233] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[0234] Specific use cases

[0235] 1. Example conversation with a child:

[0236] The child says to the device, "I was lonely at school today because my friends weren't there."

[0237] The terminal transmits this message to the server.

[0238] The server stores the received data and analyzes the content of the statements using a generative AI model.

[0239] The server identifies the mental state "lonely" and sends a notification to the parent that "child feels lonely."

[0240] 2. Learning support examples:

[0241] The child tells the device, "I want to study math now."

[0242] The terminal sends this request to the server.

[0243] The server uses a generative AI model to generate appropriate math problems and sends them to the device.

[0244] The device presents questions to the child and prompts them to answer.

[0245] The child enters the answer into the terminal, which then sends the answer to the server.

[0246] 3. Condition monitoring example:

[0247] Cameras and sensors monitor children's behavior while they are in their room.

[0248] The device transmits behavioral data to the server in real time.

[0249] The server analyzes the received behavioral data and notifies the parent if any abnormalities are detected.

[0250] Prompt Sentence Examples

[0251] "Analyze your child's mental state when they say, 'I missed my friends at school today.'"

[0252] "Generate math problems suitable for a child who said they want to start studying math now."

[0253] "Determine whether your child's behavior in the room is abnormal."

[0254] As described above, the system of the present invention utilizes a generative AI model to support children's growth and monitor their safety, thereby providing an environment where parents in dual-income households can keep track of their children's situations with peace of mind.

[0255] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0256] Step 1:

[0257] Loading a generative AI model

[0258] When the server starts up, it loads the generative AI model into memory. In this process, the model file is read from disk and expanded into memory. The input is the path to the model file, and the output is the loaded generative AI model. This allows subsequent data analysis processing to be performed.

[0259] Step 2:

[0260] Conversation Acquisition

[0261] The device captures conversations with the child through a microphone and text input device. In this process, voice data and text data are input and the captured conversation data is output. This allows the necessary conversation content to be sent to the server.

[0262] Step 3:

[0263] Sending conversation data

[0264] The terminal transmits the acquired conversation data to the server. In this process, the conversation data is input and the data transmitted to the server is output, allowing the server to analyze the conversation data.

[0265] Step 4:

[0266] Conversation data analysis

[0267] The server inputs the transmitted conversation data into a generative AI model for analysis. In this process, the conversation data is input and an analysis result such as "child's mental state" is output. Specifically, the generative AI model infers emotions and mental state from the content of the conversation and generates the result.

[0268] Step 5:

[0269] Parental Notification

[0270] Based on the analysis results, the server will send a notification to the parents if it detects an abnormal mental state, such as loneliness, in the process. This process inputs the analysis results and outputs a notification message. The notification will be sent via email or in-app notification.

[0271] Step 6:

[0272] Generating learning content

[0273] The server generates math problems and storytelling content in response to the child's request. This process inputs a request from the child and outputs generated learning content. Using a generative AI model, appropriate learning content is generated based on the prompt sentence.

[0274] Step 7:

[0275] Presentation of learning content

[0276] The device presents the learning content received from the server to the child and prompts them to respond. In this process, the learning content is input and the results presented to the child are output. Specifically, the learning content is presented using a screen display or a voice read-out function.

[0277] Step 8:

[0278] Answers to learning content

[0279] The user (child) inputs the answer into the terminal. In this process, the child's answer is input and the answer sent to the terminal is output.

[0280] Step 9:

[0281] Sending answer data

[0282] The terminal transmits the child's answer to the server. In this process, answer data is input and the data transmitted to the server is output.

[0283] Step 10:

[0284] Analysis of answer data

[0285] The server analyzes the submitted answer data and determines whether it is correct. In this process, the answer data is input and the analysis results are output. A generative AI model is used to evaluate the correctness of the answer and generate feedback.

[0286] Step 11:

[0287] Behavioral monitoring

[0288] The device continuously monitors the child's behavior using cameras and sensors, and the process inputs behavioral data and outputs real-time monitored data.

[0289] Step 12:

[0290] Sending behavioral data

[0291] The terminal transmits the monitored behavioral data in real time to the server, where the behavioral data is input and the data transmitted to the server is output.

[0292] Step 13:

[0293] Analysis of behavioral data

[0294] The server analyzes the received behavioral data and detects anomalies. In this process, behavioral data is input and anomaly detection results are output.

[0295] Step 14:

[0296] Parent notification when an abnormality is detected

[0297] The server will immediately notify the parent if an anomaly is detected. This process inputs the anomaly detection result and outputs a notification message. Notifications can be sent via email or in-app notifications.

[0298] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0299] This invention addresses the issue of children spending more time alone as the number of dual-income households increases, by providing a system that utilizes generative AI and an emotion engine to promote children's development and monitor safety within the home.

[0300] System configuration and operation

[0301] Server Operation

[0302] The server hosts the generative AI model, emotion engine, and database management system, and performs the following operations:

[0303] 1. Load the generative AI model:

[0304] The server loads the generative AI model and emotion engine into memory at startup.

[0305] 2. Data Receipt and Storage:

[0306] The system receives the child's conversation data sent from the device and stores it in a database.

[0307] 3. Emotion analysis:

[0308] The received conversation data is input into an emotion engine to recognize the child's emotions. The emotion engine identifies emotions using, for example, text analysis or voice analysis.

[0309] 4. Data Analysis:

[0310] Emotional data analyzed by the emotion engine is input into a generative AI model to more accurately predict a child's mental state.

[0311] 5. Notification function:

[0312] Based on the analysis results, if an abnormal mental state is detected, a notification will be sent to the parents.

[0313] 6. Learning content generation:

[0314] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[0315] 7. Behavioral monitoring and anomaly detection:

[0316] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is notified.

[0317] Device behavior

[0318] The device mediates interactions with the child and performs the following actions:

[0319] 1. Conversation Acquisition:

[0320] The child's speech is captured using a microphone or text input device.

[0321] 2. Data transmission:

[0322] The acquired conversation data is sent to the server.

[0323] 3. Learning Support:

[0324] The learning content received from the server is presented to the child and they are prompted to respond.

[0325] 4. Condition Monitoring:

[0326] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[0327] Usage example

[0328] 1. Example conversation with a child:

[0329] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[0330] The terminal transmits this message to the server.

[0331] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[0332] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[0333] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[0334] 2. Learning support examples:

[0335] The user (child) tells the device, "I want to study math now."

[0336] The terminal sends this request to the server.

[0337] The server stores the received requests in a database and inputs them into a generative AI model.

[0338] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[0339] The server sends the generated mathematical problem to the terminal.

[0340] The terminal presents the received question to the child and waits for the child's answer.

[0341] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[0342] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[0343] 3. Condition monitoring example:

[0344] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[0345] The device transmits behavioral data to the server in real time.

[0346] The server stores the received behavioral data in a database.

[0347] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[0348] The server immediately sends a notification to the parent if it detects an anomaly, such as an abnormally long period of inactivity or suspicious behavior.

[0349] This system utilizes a generative AI model and emotion engine to more accurately predict and monitor a child's mental state and behavior, and provides necessary information to parents, supporting their child's development and safety. The specific implementation is programmed, and it is possible to interact with children in real time.

[0350] The processing flow will be explained below.

[0351] A detailed explanation of the system's program processing (including the emotion engine)

[0352] Retrieving conversations and sending data

[0353] Step 1:

[0354] The device receives the child's speech via a microphone or text input device. For example, the device receives a speech such as, "I was lonely at school today because my friends were not there."

[0355] Step 2:

[0356] The device sends the acquired speech data to the server, along with the speech content and a timestamp.

[0357] Step 3:

[0358] The server stores the data received from the device in a database, including the content of the message and a timestamp.

[0359] Step 4:

[0360] The server inputs the received speech data into an emotion engine to analyze the child's emotions. The emotion engine recognizes emotions such as "lonely," "happy," and "sad" based on the characteristics of the text and voice.

[0361] Step 5:

[0362] The server inputs the emotion analysis data from the emotion engine into the generative AI model to more accurately predict the child's mental state. In this case, the mental state of "lonely" is identified.

[0363] Step 6:

[0364] The server stores the analysis results in a database, and if the child's mental state is determined to be "lonely," it sends a notification to the parents that "the child is feeling lonely."

[0365] Learning support and dialogue

[0366] Step 1:

[0367] The user (child) tells the device, "I want to study math now."

[0368] Step 2:

[0369] The device sends this request to the server, along with the request content and a timestamp.

[0370] Step 3:

[0371] The server stores the received requests in a database and inputs them into a generative AI model.

[0372] Step 4:

[0373] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[0374] Step 5:

[0375] The server sends the generated math problem to the terminal.

[0376] Step 6:

[0377] The device presents the received question to the child and waits for the child's answer.

[0378] Step 7:

[0379] The user (child) enters the answer into the terminal, and the terminal sends the answer to the server.

[0380] Step 8:

[0381] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it sends the feedback "That's correct!"

[0382] Condition monitoring and anomaly detection

[0383] Step 1:

[0384] The device monitors children's behavior in real time through cameras and sensors.

[0385] Step 2:

[0386] The device sends the monitoring data to the server, which includes the activity details and a timestamp.

[0387] Step 3:

[0388] The server stores the received behavioral data in a database.

[0389] Step 4:

[0390] The server analyzes the accumulated data and real-time data using a generative AI model and emotion engine, and compares it with the normal range of behavior.

[0391] Step 5:

[0392] If the server detects an anomaly, it will immediately send a notification to the parent, for example if the device is stationary for an abnormally long period of time or if it is behaving suspiciously.

[0393] Emotion analysis using an emotion engine

[0394] Step 1:

[0395] The device captures the child's speech through a microphone and text input device.

[0396] Step 2:

[0397] The terminal transmits the acquired speech data to the server.

[0398] Step 3:

[0399] The server inputs the received utterance data into the emotion engine.

[0400] Step 4:

[0401] The emotion engine analyzes the content of the speech and identifies emotion labels, such as "lonely," "happy," and "sad."

[0402] Step 5:

[0403] The server inputs the emotion labels obtained from the emotion engine into a generative AI model to more accurately predict the child's mental state.

[0404] Step 6:

[0405] The server sends notifications to parents based on information obtained from the emotion engine and generative AI model, and also generates and presents learning content and reading content as needed.

[0406] In this way, this system, which combines an emotion engine, can more accurately grasp a child's mental state and provide the necessary support and notifications, allowing them to spend more secure and meaningful time together.

[0407] Example 2

[0408] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0409] With the increase in dual-income households, there is a problem that children spend more time alone. In such situations, it is difficult for parents to grasp changes in their children's mental state and behavior in real time, raising concerns about their children's growth and safety. Furthermore, there is often a lack of appropriate support when children study alone. This invention solves these problems and provides a system that supports children's growth and safety.

[0410] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring a conversation with a child, a means for analyzing the acquired conversation data and using an emotion engine to recognize the child's emotions, a means for using a generative AI model to infer the child's mental state based on the recognized emotion data, and a means for notifying the parent of the child's mental state. This makes it possible to grasp the child's emotions and mental state in real time and to immediately notify the parent if an abnormality is detected. In addition, the generative AI model generates learning content related to a subject and presents it to the child, thereby realizing learning support. This makes it possible to comprehensively support the child's growth and safety.

[0411] "Means for acquiring a conversation with a child" refers to an input device such as a microphone or text input device for acquiring a child's speech as voice or text.

[0412] "Emotion engine that analyzes acquired conversation data and recognizes children's emotions" refers to a software component that uses text analysis and voice analysis of input conversation data to recognize children's emotions.

[0413] "A generative artificial intelligence model that predicts a child's mental state based on recognized emotional data" refers to a machine learning model that uses emotion recognition results as input to predict a child's mental state and psychological tendencies.

[0414] "Means of notifying parents of their child's mental state" refers to communication functions that send notifications to parents via email, SMS, or a dedicated app if an abnormal mental state is detected.

[0415] "Means for generating learning content related to a subject and presenting it to a child" refers to a function for generating learning questions and content using a generative AI model based on the child's requests and learning situation, and presenting it to the child.

[0416] "Means for monitoring children's behavior and detecting abnormalities" refers to a system that uses cameras and sensors to collect children's behavioral data in real time, analyzes it using machine learning models, and detects abnormalities.

[0417] "Means for immediately notifying parents if abnormalities are detected" refers to a communication function for quickly sending a notification to parents when abnormal behavior is detected.

[0418] This invention addresses the issue of children spending more time alone as the number of dual-income households increases. It provides a system that utilizes a generative AI model and an emotion engine to promote children's development and monitor safety within the home.

[0419] System Configuration

[0420] Server Operation

[0421] The server hosts the generative AI model, emotion engine, and database management system, and implements the following key functions:

[0422] 1. Load the generative AI model:

[0423] The server loads the generative AI model and emotion engine into memory at startup, enabling rapid data processing and analysis using deep learning libraries such as TensorFlow and PyTorch.

[0424] 2. Data Receipt and Storage:

[0425] The server receives the child's conversation data sent from the device and stores it in a database using HTTP requests and WebSockets. MySQL or PostgreSQL is used as the database management system.

[0426] 3. Emotion analysis:

[0427] The server inputs the received conversation data into an emotion engine to recognize the child's emotions. The emotion engine uses a natural language processing library (e.g., NLTK or spaCy) to analyze the text and identify emotions.

[0428] 4. Data Analysis:

[0429] The server inputs the emotional data analyzed by the emotion engine into a generative AI model to predict the child's mental state. The generative AI model then comprehensively analyzes past and real-time data to predict the child's mental state.

[0430] 5. Notification function:

[0431] Based on the analysis results, the server will send a notification to the parents if an abnormal mental state is detected. Notifications can be sent via email, SMS, or a dedicated app. For example, Amazon SES (Simple Email Service) can be used.

[0432] 6. Learning content generation:

[0433] The server generates learning content based on the child's request and sends it to the device. For example, math problems or reading content may be generated. For example, a question such as "Please solve the following equation: What is 5 + 7?" may be generated.

[0434] 7. Behavioral monitoring and anomaly detection:

[0435] The server analyzes the behavioral data received from the device and notifies the parent if an abnormality is detected. The behavioral data includes data obtained from cameras and motion sensors.

[0436] Device behavior

[0437] The device mediates interactions with the child and implements the following functions in cooperation with the server:

[0438] 1. Conversation Acquisition:

[0439] The device uses a microphone and text input device to capture the child's speech, which is then converted into text data using voice recognition software (e.g., Google Speech-to-Text API).

[0440] 2. Data transmission:

[0441] The device sends the acquired conversation data to the server via HTTP requests or WebSockets. The data is packaged in JSON format and sent to a specific endpoint.

[0442] 3. Learning Support:

[0443] The device receives learning content from the server and presents it to the child, prompting them to answer. For example, generated math problems are displayed on the device screen, allowing the child to enter their answers.

[0444] 4. Condition Monitoring:

[0445] The device uses cameras and sensors to monitor the child's behavior and notifies the server if it detects anything unusual, such as if the child remains motionless for a long period of time or if it detects suspicious movements.

[0446] Usage example

[0447] 1. Example conversation with a child:

[0448] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[0449] The terminal transmits this message to the server.

[0450] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[0451] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[0452] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[0453] 2. Learning support examples:

[0454] The user (child) tells the device, "I want to study math now."

[0455] The terminal sends this request to the server.

[0456] The server stores the received requests in a database and inputs them into a generative AI model.

[0457] The server uses a generative AI model to generate appropriate math problems, such as "Please solve the following equation: What is 5 + 7?"

[0458] The server sends the generated mathematical problem to the terminal.

[0459] The terminal presents the received question to the child and waits for the child's answer.

[0460] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[0461] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[0462] 3. Condition monitoring example:

[0463] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[0464] The device transmits behavioral data to the server in real time.

[0465] The server stores the received behavioral data in a database.

[0466] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[0467] The server will immediately send a notification to its parent if it detects an anomaly, for example if it is quiet for an abnormally long period of time.

[0468] Example prompt

[0469] 1. "Explain what to do when your child tells you they're lonely and don't have any friends."

[0470] 2. "Explain how the system responds to a child's request to learn math."

[0471] This allows the system to utilize generative AI models and emotion engines to more accurately predict and monitor children's mental states and behaviors, providing parents with the information they need to support their children's growth and safety.

[0472] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0473] Step 1:

[0474] The device uses a microphone and a text input device to capture what the child says. For example, the child might say, "I was lonely at school today because my friends weren't there."

[0475] Input: Child's voice or text

[0476] Output: Conversation data in text format

[0477] Step 2:

[0478] The device sends the acquired conversation data to the server using HTTP requests or WebSockets. The data is packaged in JSON format and sent to a specific endpoint (e.g., / api / conversations).

[0479] Input: Text-based conversation data

[0480] Output: JSON formatted conversation data

[0481] Step 3:

[0482] The server receives the conversation data sent from the terminal and stores it in a database management system (e.g., MySQL, PostgreSQL).

[0483] Input: JSON format conversation data

[0484] Output: Conversation data stored in a database

[0485] Step 4:

[0486] The server retrieves conversation data from the database and inputs it into an emotion engine (e.g., NLTK, spaCy) to recognize the child's emotions. The emotion engine performs text analysis to identify emotions such as "lonely."

[0487] Input: Conversation data stored in the database

[0488] Output: Emotion data identified by the emotion engine

[0489] Step 5:

[0490] The server inputs the emotion data analyzed by the emotion engine into a generative AI model to infer the child's mental state. The generative AI model performs analysis using deep learning libraries (e.g., TensorFlow, PyTorch).

[0491] Input: Emotion data

[0492] Output: Inferred mental state data

[0493] Step 6:

[0494] The server then uses the estimated mental state data to notify parents if an abnormal state is detected. Notifications are sent via email, SMS, or a dedicated app (e.g., Amazon SES).

[0495] Input: Inferred mental state data

[0496] Output: Notification sent to parent

[0497] Step 7:

[0498] The server uses a generative AI model to generate learning content based on the child's requests, such as generating a math problem like "What is 5 + 7?"

[0499] Input: Child request data

[0500] Output: Generated learning content

[0501] Step 8:

[0502] The server sends the generated learning content to the device, which then presents it to the child and prompts them to respond.

[0503] Input: Generated learning content

[0504] Output: Learning content sent to the device

[0505] Step 9:

[0506] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[0507] Input: Child's answer data

[0508] Output: Answer data sent to the server

[0509] Step 10:

[0510] The server stores the received answer data in a database, analyzes it using a generative AI model, creates feedback, and sends it to the device. For example, if the answer is correct, it displays "That's correct!"

[0511] Input: Child's answer data

[0512] Output: Feedback message

[0513] (Application example 2)

[0514] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0515] With the increase in dual-income households, children are spending more time alone. Under these circumstances, it is difficult to constantly monitor their children's mental state and safety, and there is a particular problem of delayed detection of dangers or abnormal behavior that may occur while parents are away. Furthermore, while it is important to make effective use of the time children spend alone and provide them with learning support, providing appropriate timing and content for such support can be difficult.

[0516] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0517] In this invention, the server includes means for acquiring a voice or text conversation with a child, means for analyzing the acquired conversation data and using an emotion engine to identify the child's emotions, means for inputting the emotion data identified by the emotion engine into a generative AI model to infer the child's mental state, means for notifying the parent of the child's mental state, means for monitoring the child's behavior and detecting abnormalities, and means for immediately notifying the parent if an abnormality is detected. This makes it possible to support the child's safety and mental growth by analyzing the child's mental state and behavior in real time and notifying the parent.

[0518] "Means for capturing voice or text conversations" refers to technology for capturing voice or text spoken by a child using a microphone, text input device, or the like.

[0519] An "emotion engine" refers to software or algorithms that use text and voice analysis to recognize and identify a child's emotions.

[0520] A "generative AI model" is an artificial intelligence model that generates new information and learning content based on acquired data.

[0521] "Means for inferring mental state" refers to a technology that uses emotional data identified by the emotion engine to infer a child's current mental state.

[0522] "Means for notifying parents" refers to technology that uses communication methods such as email and mobile notifications to inform parents of their child's mental state or abnormalities in real time.

[0523] "Means for monitoring behavior and detecting abnormalities" refers to technology that uses cameras and sensors to monitor children's movements and detect abnormal behavior.

[0524] "Means for generating learning content and presenting it to children" refers to technology that uses a generative AI model to generate learning content related to a subject and presents it to children at an appropriate time.

[0525] "Means for analyzing in real time and providing feedback" refers to technology that instantly analyzes acquired voice and behavioral data and provides feedback to the child based on the results.

[0526] This invention addresses the issue of children spending more time alone as the number of dual-income households increases. It provides a system that utilizes a generative AI model and an emotion engine to promote children's development and monitor safety within the home. This system is realized primarily by combining a server and a terminal.

[0527] Server Operation

[0528] 1. Load the generative AI model and emotion engine:

[0529] When the server starts up, it loads the generative AI model and emotion engine into memory. The AI ​​model and emotion engine are software that play a central role in the analysis and generation processes.

[0530] 2. Data Receipt and Storage:

[0531] The system receives the child's conversation and behavior data sent from the device and stores it in a database for later analysis.

[0532] 3. Emotion analysis:

[0533] The received conversation data is input into an emotion engine, which uses text and voice analysis to identify the child's emotions.

[0534] 4. Data Analysis:

[0535] The emotional data analyzed by the emotion engine is input into a generative AI model to predict the child's mental state. The generative AI model then uses the received data to more accurately predict the child's mental state.

[0536] 5. Notification function:

[0537] If an abnormal mental state is detected based on the analysis results, a notification will be sent to the parents via email or mobile notification.

[0538] 6. Learning content generation:

[0539] Based on the child's request, appropriate learning content is generated using a generative AI model and sent to the device.

[0540] 7. Behavioral Monitoring and Anomaly Detection:

[0541] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is immediately notified.

[0542] Device behavior

[0543] 1. Conversation Acquisition:

[0544] The system captures the child's speech using a microphone or text input device, and the captured data is sent to a server in real time.

[0545] 2. Data transmission:

[0546] The acquired conversation data and behavioral data are sent to the server.

[0547] 3. Learning Support:

[0548] The learning content received from the server is presented to the child, encouraging the child to study.

[0549] 4. Condition Monitoring:

[0550] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[0551] Specific examples

[0552] 1. Example conversation with a child:

[0553] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[0554] The terminal transmits this message to the server.

[0555] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[0556] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[0557] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[0558] 2. Learning support examples:

[0559] The user (child) tells the device, "I want to study math now."

[0560] The terminal sends this request to the server.

[0561] The server stores the received requests in a database and inputs them into a generative AI model.

[0562] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[0563] The server sends the generated questions to the terminal.

[0564] The terminal presents the received question to the child and waits for the child's answer.

[0565] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[0566] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[0567] 3. Condition monitoring example:

[0568] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[0569] The device transmits behavioral data to the server in real time.

[0570] The server stores the received behavioral data in a database.

[0571] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[0572] The server sends an immediate notification to the parent if it detects an anomaly, such as an abnormally long period of inactivity or suspicious behavior.

[0573] Prompt Sentence Examples

[0574] Emotion Analysis Prompt: "Analyze this audio data to identify the emotion."

[0575] Learning content generation prompt: "Generate one basic math problem, such as an addition or subtraction problem."

[0576] This system uses a generative AI model and an emotion engine to analyze a child's mental state and behavior in real time, and provides parents with the information they need to support their child's growth and safety.

[0577] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0578] Step 1:

[0579] Starting the server and loading the model

[0580] When the server starts up, it loads the generative AI model and emotion engine into memory. This step makes the necessary software components available. The input is a server configuration file containing the generative AI model and emotion engine, and the output is the model and engine loaded into the server memory. Specifically, the server executes the model loading routine and allocates memory space.

[0581] Step 2:

[0582] Retrieving and sending conversation data

[0583] The device captures voice and text conversations from the user (child). As input, voice and text data are obtained using a microphone and text input device, and this is sent to the server. The output is raw data sent to the server. Specifically, the device captures data through a voice recording or text input interface, and sends the data to the server using a communication protocol.

[0584] Step 3:

[0585] Sentiment analysis of conversation data

[0586] The server inputs the received conversation data into an emotion engine to identify the child's emotions. The input is voice or text data sent from the device, which is analyzed by the emotion engine. The output is the identified emotion data. Specifically, the server runs the emotion engine's analysis algorithm and classifies emotions using text analysis and voice analysis technology.

[0587] Step 4:

[0588] Mental state inference from emotional data

[0589] The server inputs the emotion data identified by the emotion engine into the generative AI model to infer the child's mental state. The input is emotion data from the emotion engine, which is adapted and processed by the generative AI model. The output is the child's inferred mental state. Specifically, the server executes the generative AI model's data processing algorithm to infer the child's mental state based on the emotion data.

[0590] Step 5:

[0591] Mental status notification

[0592] The server sends a notification to the parent if an abnormality is detected based on the child's mental state. The input is the mental state data inferred by the generative AI model, and the information is conveyed to the parent via the notification system. The output is a notification message sent to the parent's device. Specifically, the server invokes the notification function to send an email or mobile notification.

[0593] Step 6:

[0594] Receiving a learning content generation request

[0595] When a user (child) inputs a learning request into the device, the device sends this request to the server. The input is voice or text data based on the user's learning request, which is sent to the server. The output is the request data sent to the server. Specifically, the device captures the request data using a voice or text input interface and sends it to the server using a communication protocol.

[0596] Step 7:

[0597] Generation and presentation of learning content

[0598] The server uses the generative AI model to generate appropriate learning content based on the received request and sends it to the device. The input is the learning request data from the user and the generative AI model, and learning content is created based on this. The output is the data that sends the generated learning content to the device. Specifically, the server uses the prompt sentences from the generative AI model to generate math problems and reading comprehension content and sends it to the device.

[0599] Step 8:

[0600] Implementing learning support

[0601] The device presents the learning content received from the server to the user (child) and allows them to work on it. The input is the learning content sent from the server, which is displayed on the user interface. The output is the answer data entered by the user. Specifically, the device displays the learning content using a display and speech synthesis function, and obtains the user's answers from the input device.

[0602] Step 9:

[0603] Monitoring and sending behavioral data

[0604] The device monitors the user's (child's) behavior using cameras and sensors and sends the data to a server. The input is the behavioral data acquired from the cameras and sensors, which is sent to the server. The output is the behavioral data sent to the server. Specifically, the device acquires video and sensor information in real time and uses a communication protocol to send it to the server.

[0605] Step 10:

[0606] Anomaly detection in behavioral data

[0607] The server analyzes the received behavioral data and immediately sends a notification to the parent if an abnormality is detected. The input is the behavioral data sent from the device, and an algorithm is applied to detect abnormalities based on this. The output is a notification message that is sent to the parent if an abnormality is detected. Specifically, the server executes the behavioral analysis algorithm, and if an abnormality is discovered, it notifies the parent via the notification system.

[0608] Through these steps, this system utilizes a generative AI model and emotion engine to analyze children's mental state and behavior in real time and provide parents with the necessary information to support their children's growth and safety.

[0609] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0610] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0611] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0612] [Second embodiment]

[0613] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0614] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0615] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0616] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0617] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0619] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0620] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0621] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0622] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0623] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0624] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0625] This invention addresses the issue of children spending more time alone as the number of dual-income households increases, by providing a system that utilizes generative AI to promote children's development and monitor safety within the home.

[0626] System configuration and operation

[0627] Server Operation

[0628] The server hosts the generative AI model and database management system and performs the following operations:

[0629] 1. Load the generative AI model:

[0630] The server loads the generative AI model into memory at startup.

[0631] 2. Data Receipt and Storage:

[0632] The system receives the child's conversation data sent from the device and stores it in a database.

[0633] 3. Data Analysis:

[0634] The received conversation data is input into a generative AI model to analyze the child's mental state.

[0635] 4. Notification function:

[0636] Based on the analysis results, if an abnormal mental state is detected, a notification will be sent to the parents.

[0637] 5. Learning content generation:

[0638] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[0639] 6. Behavioral monitoring and anomaly detection:

[0640] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is notified.

[0641] Device behavior

[0642] The device mediates interactions with the child and performs the following actions:

[0643] 1. Conversation Acquisition:

[0644] The child's speech is captured using a microphone or text input device.

[0645] 2. Data transmission:

[0646] The acquired conversation data is sent to the server.

[0647] 3. Learning Support:

[0648] The learning content received from the server is presented to the child and they are prompted to respond.

[0649] 4. Condition Monitoring:

[0650] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[0651] Usage example

[0652] 1. Example conversation with a child:

[0653] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[0654] The terminal transmits this message to the server.

[0655] The server stores the received data and analyzes the content of the statements using a generative AI model.

[0656] The server identifies the mental state "lonely" and sends a notification to the parent that "child feels lonely."

[0657] 2. Learning support examples:

[0658] The user (child) tells the device, "I want to study math now."

[0659] The terminal sends this request to the server.

[0660] The server uses a generative AI model to generate appropriate math problems and sends them to the device.

[0661] The device presents questions to the child and prompts them to answer.

[0662] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[0663] 3. Condition monitoring example:

[0664] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[0665] The device transmits behavioral data to the server in real time.

[0666] The server analyzes the received behavioral data and notifies the parent if any abnormalities are detected.

[0667] summary

[0668] In this way, the system of this invention utilizes generative AI models to support children's development and monitor their safety, providing an environment where parents in dual-income households can feel secure knowing what is happening with their children. The specific implementation is programmed, interacting with children in real time, analyzing their mental state and behavior based on accumulated data, and providing necessary notifications and learning content. This makes children's alone time safer and more meaningful.

[0669] The processing flow will be explained below.

[0670] Specific explanation of system program processing

[0671] Retrieving conversations and sending data

[0672] Step 1:

[0673] The device receives the child's speech via a microphone or text input device. For example, the device receives a speech such as, "I was lonely at school today because my friends were not there."

[0674] Step 2:

[0675] The device sends the acquired speech data to the server, along with the speech content and a timestamp.

[0676] Step 3:

[0677] The server stores the data received from the device in a database, including the content of the message and a timestamp.

[0678] Step 4:

[0679] The server inputs the received data into a generative AI model to analyze the child's mental state. In this case, the emotion of "lonely" is identified.

[0680] Step 5:

[0681] The server stores the analysis results in a database and sends a notification to the parents if the child's mental state is determined to be "lonely."

[0682] Learning support and dialogue

[0683] Step 1:

[0684] The user (child) tells the device, "I want to study math now."

[0685] Step 2:

[0686] The device sends this request to the server, along with the request content and a timestamp.

[0687] Step 3:

[0688] The server stores the received requests in a database and inputs them into a generative AI model.

[0689] Step 4:

[0690] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[0691] Step 5:

[0692] The server sends the generated math problem to the terminal.

[0693] Step 6:

[0694] The device presents the received question to the child and waits for the child's answer.

[0695] Step 7:

[0696] The user (child) enters the answer into the terminal, and the terminal sends the answer to the server.

[0697] Step 8:

[0698] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[0699] Condition monitoring and anomaly detection

[0700] Step 1:

[0701] The device monitors children's behavior in real time through cameras and sensors.

[0702] Step 2:

[0703] The device sends the monitoring data, including the activity and a timestamp, to the server.

[0704] Step 3:

[0705] The server stores the received behavioral data in a database.

[0706] Step 4:

[0707] The server analyzes the accumulated data using a generative AI model and compares it with the normal range of behavior.

[0708] Step 5:

[0709] If the server detects an anomaly, it will immediately send a notification to the parent, for example if the device is stationary for an abnormally long period of time or if it is behaving suspiciously.

[0710] This allows the system to continuously monitor a child's mental state and behavior through daily interactions with the child and support their learning, and to support the child's growth and safety by promptly notifying parents if any abnormalities or dangers occur.

[0711] Example 1

[0712] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0713] With the increase in dual-income households, children spend more time alone, creating the need to properly monitor their growth and safety. There is also a lack of effective ways to monitor children's mental health and academic progress. This creates a need for a system that allows parents to monitor their children's condition while they are at work and respond quickly if something unusual occurs.

[0714] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0715] In this invention, the server includes means for acquiring conversations with children, means for transmitting the acquired conversation data to the server, means for storing the received conversation data in a database, means for analyzing the child's mental state using a generative AI model based on the stored conversation data, and means for notifying parents if an abnormal mental state is detected based on the analysis results. This enables parents in dual-income households to understand their children's mental state and behavior in real time even while at work, and to provide appropriate learning support and safety checks.

[0716] "Children" refers to minors who are eligible to use the System.

[0717] "Conversation data" refers to a recording of a child's speech in the form of text or audio data.

[0718] "Server" refers to the central computing unit that hosts the generative AI model and receives, stores, and analyzes data.

[0719] A "generative AI model" refers to a program model that uses artificial intelligence technology to perform natural language processing and data analysis.

[0720] "Database" refers to an information management system for storing received conversation data and behavioral data.

[0721] "Mental state" refers to the emotional and psychological state that can be inferred from a child's words and behavior.

[0722] "Notification" refers to an alert or informational message sent from the server to the parent.

[0723] "Learning content" refers to the learning questions and materials generated by the generative AI model in response to the child's requests.

[0724] "Behavioral data" refers to data such as a child's movements and location collected using cameras and sensors.

[0725] "Abnormal" refers to a child's behavior or mental state that is different from normal and indicates an unnatural or dangerous condition.

[0726] The present invention is a system that supports the safety and development of children by addressing the increasing amount of time they spend alone due to the increase in dual-income households. This system includes a device that interacts with children and a server that analyzes data and sends notifications.

[0727] Hardware and software used

[0728] Terminals consist of interaction devices equipped with input devices such as microphones, cameras, sensors, and touchscreens. Examples include general-purpose tablets, smart speakers, and surveillance cameras.

[0729] The server is a central computing unit that hosts the generative AI model and receives, stores, and analyzes data. Software used includes deep learning frameworks (e.g., TensorFlow, PyTorch), database management systems (e.g., MySQL, PostgreSQL), and various notification APIs (e.g., Twilio, SendGrid).

[0730] Conversation capture and analysis

[0731] The device uses a microphone and text input device to capture the child's speech. For example, if a child says, "I missed my friends at school today," the device uses voice recognition software to convert this speech into text data. The converted text data is then securely transmitted to a server via the HTTPS protocol.

[0732] The server stores the received text data in a database such as MySQL. It then analyzes the data using a generative AI model (e.g., BERT, GPT-3). At this time, the prompt "Please analyze the mental state when the user tells a sad story" is input to the model.

[0733] Mental state detection and notification

[0734] The server determines the child's mental state based on the analysis results. For example, if the emotion "lonely" is detected, a notification is sent to the parent using Twilio or SendGrid. The notification includes a message saying "Your child is feeling lonely."

[0735] Creating and presenting learning content

[0736] When a user (child) says to the device, "I want to study math now," the device sends this request to the server.

[0737] The server uses a generative AI model to generate appropriate math problems. The prompt "Generate simple math problems for elementary school students" is input to the model. The generated math problems are sent back to the device, which presents them to the child. The child enters the answers, and the answer data is sent back to the server.

[0738] Behavioral data monitoring and analysis

[0739] The device continuously monitors the child's behavior using cameras and sensors, for example, recording the child's behavior while they are in their room, and transmitting the monitoring data to a server in real time.

[0740] The server analyzes the received behavioral data and notifies the parent if abnormal behavior is detected. For example, if a child remains motionless for a long time, a notification stating "Child has not moved for a long time" is sent to the parent.

[0741] In this way, the system of the present invention utilizes a generative AI model to analyze a child's mental state and behavior, and provides parents with the necessary notifications and learning support, thereby providing an environment where parents in dual-income households can feel at ease knowing what is happening with their children.

[0742] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0743] Step 1: Obtaining conversation data

[0744] The device uses a microphone and text input device to capture what the child says. For example, if a child says, "I missed my friends at school today," the device uses voice recognition software to convert this speech into text data.

[0745] (Input) Child's speech (audio or text)

[0746] (Output) Text data (in the case of audio, data converted to text)

[0747] Step 2: Sending conversation data to the server

[0748] The device sends the retrieved text data to the server via an HTTP POST request, with the data being transferred securely using SSL encryption.

[0749] (Input) Text data

[0750] (Output) None (Transfer to server)

[0751] Step 3: Save your data

[0752] The server stores the received text data in MySQL (or other database management systems), including information such as what the child said, when they said it, and the speaker's ID.

[0753] (Input) Text data (transferred data)

[0754] (Output) Save data to database (persistent storage)

[0755] Step 4: Analyzing the conversation data

[0756] The server inputs the saved text data into a generative AI model to analyze the child's mental state. Specifically, the model is given a prompt sentence: "Please analyze the child's mental state when the user tells a sad story." This analysis uses the natural language processing capabilities of BERT and GPT-3.

[0757] (Input) Text data (saved conversation data), prompt sentence

[0758] (Output) Analysis results (mental state)

[0759] Step 5: Mental status notification

[0760] The server evaluates the child's mental state based on the analysis results, and if an abnormal state is detected, it immediately notifies the parent. For example, if the emotion "lonely" is detected, a notification stating "your child is feeling lonely" is sent to the parent using Twilio or SendGrid.

[0761] (Input) Analysis results (mental state)

[0762] (Output) Notification message (notification to parent)

[0763] Step 6: Request learning content

[0764] The user (child) requests learning content by telling the device, "I want to study math now."

[0765] (Input) Request statement ("I want to study math now.")

[0766] (Output) Request data (transferred to server)

[0767] Step 7: Generate learning content

[0768] The server receives the learning request and generates appropriate math problems using a generative AI model. For example, it generates problems by inputting a prompt sentence such as "Generate simple math problems for elementary school students."

[0769] (Input) Request data, prompt

[0770] (Output) Learning content (math problems)

[0771] Step 8: Present learning content and submit answers

[0772] The device presents the child with a mathematical problem received from the server and prompts the child to answer it. The child enters the answer into the device, which then transmits the answer to the server.

[0773] (Input) Learning content (math problems)

[0774] (Output) Answer data (transferred to server)

[0775] Step 9: Monitor behavioral data

[0776] The device uses cameras and sensors to continuously monitor a child's behavior, for example by recording their movements and location in real time while they are in their room.

[0777] (Input) Behavioral data (data from cameras and sensors)

[0778] (Output) Behavioral data (transferred to server)

[0779] Step 10: Analyze behavioral data and detect anomalies

[0780] The server analyzes the received behavioral data and notifies parents if abnormal behavior is detected. For example, if a child stays in the same place for a long time, the server sends a message saying, "Your child has not moved for a long time."

[0781] (Input) Behavioral data (data monitored in real time)

[0782] (Output) Notification message (notification to parent)

[0783] (Application example 1)

[0784] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0785] With the increase in dual-income households, children are spending more time alone, making it necessary to provide appropriate supervision and learning support to ensure their growth and safety. It is also important to understand children's mental state and behavior in real time and detect abnormalities early. The present invention aims to provide a system that solves these problems.

[0786] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0787] In this invention, the server includes means for acquiring conversation data with a child, means for analyzing the acquired conversation data and using a generative AI model to infer the child's mental state, means for notifying a parent of the child's mental state, means for generating appropriate learning content and presenting it to the child, means for monitoring the child's behavior and detecting abnormalities, means for immediately notifying the parent when an abnormality is detected, and means for generating appropriate prompt sentences and inputting them into the generative AI model. This makes it possible to provide necessary learning support while monitoring the child's mental state and behavior in real time at home, and to respond quickly when an abnormality is detected.

[0788] "Conversation with children" refers to communication within the home that includes words and sentences spoken by children.

[0789] "Means of acquisition" refers to the functionality for collecting data using a microphone, text input device, or other sensor device.

[0790] A "generative AI model" is an artificial intelligence algorithm or software that generates new information or content from original data through natural language processing and data analysis.

[0791] "Mental state" refers to a child's psychological health and emotional changes, and is an indicator for assessing that state.

[0792] "Means of notifying parents" refers to the means of communication to promptly inform parents of analysis results or abnormal situations, including email, SMS, in-app notifications, etc.

[0793] "Learning content" refers to questions, reading materials, and other content designed to support a child's education and intellectual development.

[0794] "Presentation means" is a function for showing the generated learning content to children or conveying it to them by voice.

[0795] "Behavioral monitoring measures" are features that use cameras and sensors to continuously observe a child's physical movements and location.

[0796] "Means for detecting anomalies" refers to the ability to identify unusual behavior or situations and issue a warning when danger or problems occur.

[0797] A "prompt" is an instruction or question used to direct a generative AI model to perform a specific analysis or generation.

[0798] "Smart device" means a device with internet connectivity and advanced computing capabilities, including smartphones and tablets.

[0799] The present invention provides a system that uses a generative AI model to support the development and safety of children in dual-income households. This system is composed of a server and a terminal, and is realized using the following means.

[0800] Server Operation

[0801] The server hosts the generative AI model and database management system and performs the following operations:

[0802] 1. Load the generative AI model:

[0803] The server loads the generative AI model into memory at startup, allowing it to analyze conversational and behavioral data in real time.

[0804] 2. Data Receipt and Storage:

[0805] Conversation data and behavioral data with the child sent from the device is received and stored in a database.

[0806] 3. Data Analysis:

[0807] The received conversation data is input into a generative AI model to analyze the child's mental state, and based on this analysis, a notification is sent to the parent.

[0808] 4. Learning content generation:

[0809] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[0810] 5. Behavioral monitoring and anomaly detection:

[0811] The received behavioral data is analyzed, and if any abnormalities are detected, a notification is sent to the parent.

[0812] Device behavior

[0813] The device mediates interactions with the child and performs the following actions:

[0814] 1. Conversation Acquisition:

[0815] The child's speech is captured using a microphone or text input device.

[0816] 2. Data transmission:

[0817] The acquired conversation data is sent to the server.

[0818] 3. Learning Support:

[0819] The learning content received from the server is presented to the child and they are prompted to respond.

[0820] 4. Condition Monitoring:

[0821] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[0822] Specific use cases

[0823] 1. Example conversation with a child:

[0824] The child says to the device, "I was lonely at school today because my friends weren't there."

[0825] The terminal transmits this message to the server.

[0826] The server stores the received data and analyzes the content of the statements using a generative AI model.

[0827] The server identifies the mental state "lonely" and sends a notification to the parent that "child feels lonely."

[0828] 2. Learning support examples:

[0829] The child tells the device, "I want to study math now."

[0830] The terminal sends this request to the server.

[0831] The server uses a generative AI model to generate appropriate math problems and sends them to the device.

[0832] The device presents questions to the child and prompts them to answer.

[0833] The child enters the answer into the terminal, which then sends the answer to the server.

[0834] 3. Condition monitoring example:

[0835] Cameras and sensors monitor children's behavior while they are in their room.

[0836] The device transmits behavioral data to the server in real time.

[0837] The server analyzes the received behavioral data and notifies the parent if any abnormalities are detected.

[0838] Prompt Sentence Examples

[0839] "Analyze your child's mental state when they say, 'I missed my friends at school today.'"

[0840] "Generate math problems suitable for a child who said they want to start studying math now."

[0841] "Determine whether your child's behavior in the room is abnormal."

[0842] As described above, the system of the present invention utilizes a generative AI model to support children's growth and monitor their safety, thereby providing an environment where parents in dual-income households can keep track of their children's situations with peace of mind.

[0843] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0844] Step 1:

[0845] Loading a generative AI model

[0846] When the server starts up, it loads the generative AI model into memory. In this process, the model file is read from disk and expanded into memory. The input is the path to the model file, and the output is the loaded generative AI model. This allows subsequent data analysis processing to be performed.

[0847] Step 2:

[0848] Conversation Acquisition

[0849] The device captures conversations with the child through a microphone and text input device. In this process, voice data and text data are input and the captured conversation data is output. This allows the necessary conversation content to be sent to the server.

[0850] Step 3:

[0851] Sending conversation data

[0852] The terminal transmits the acquired conversation data to the server. In this process, the conversation data is input and the data transmitted to the server is output, allowing the server to analyze the conversation data.

[0853] Step 4:

[0854] Conversation data analysis

[0855] The server inputs the transmitted conversation data into a generative AI model for analysis. In this process, the conversation data is input and an analysis result such as "child's mental state" is output. Specifically, the generative AI model infers emotions and mental state from the content of the conversation and generates the result.

[0856] Step 5:

[0857] Parental Notification

[0858] Based on the analysis results, the server will send a notification to the parents if it detects an abnormal mental state, such as loneliness, in the process. This process inputs the analysis results and outputs a notification message. The notification will be sent via email or in-app notification.

[0859] Step 6:

[0860] Generating learning content

[0861] The server generates math problems and storytelling content in response to the child's request. This process takes a child's request as input and outputs generated learning content. Using a generative AI model, appropriate learning content is generated based on the prompt.

[0862] Step 7:

[0863] Presentation of learning content

[0864] The device presents the learning content received from the server to the child and prompts them to respond. In this process, the learning content is input and the results presented to the child are output. Specifically, the learning content is presented using a screen display or a voice read-out function.

[0865] Step 8:

[0866] Answers to learning content

[0867] The user (child) inputs the answer into the terminal. In this process, the child's answer is input and the answer sent to the terminal is output.

[0868] Step 9:

[0869] Sending answer data

[0870] The terminal transmits the child's answer to the server. In this process, answer data is input and the data transmitted to the server is output.

[0871] Step 10:

[0872] Analysis of answer data

[0873] The server analyzes the submitted answer data and determines whether it is correct. In this process, the answer data is input and the analysis results are output. A generative AI model is used to evaluate the correctness of the answer and generate feedback.

[0874] Step 11:

[0875] Behavioral monitoring

[0876] The device continuously monitors the child's behavior using cameras and sensors, and the process inputs behavioral data and outputs real-time monitored data.

[0877] Step 12:

[0878] Sending behavioral data

[0879] The terminal transmits the monitored behavioral data in real time to the server, where the behavioral data is input and the data transmitted to the server is output.

[0880] Step 13:

[0881] Analysis of behavioral data

[0882] The server analyzes the received behavioral data and detects anomalies. In this process, behavioral data is input and anomaly detection results are output.

[0883] Step 14:

[0884] Parent notification when an abnormality is detected

[0885] The server will immediately notify the parent if an anomaly is detected. This process inputs the anomaly detection result and outputs a notification message. Notifications can be sent via email or in-app notifications.

[0886] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0887] This invention addresses the issue of children spending more time alone as the number of dual-income households increases, by providing a system that utilizes generative AI and an emotion engine to promote children's development and monitor safety within the home.

[0888] System configuration and operation

[0889] Server Operation

[0890] The server hosts the generative AI model, emotion engine, and database management system, and performs the following operations:

[0891] 1. Load the generative AI model:

[0892] The server loads the generative AI model and emotion engine into memory at startup.

[0893] 2. Data Receipt and Storage:

[0894] The system receives the child's conversation data sent from the device and stores it in a database.

[0895] 3. Emotion analysis:

[0896] The received conversation data is input into an emotion engine to recognize the child's emotions. The emotion engine identifies emotions using, for example, text analysis or voice analysis.

[0897] 4. Data Analysis:

[0898] Emotional data analyzed by the emotion engine is input into a generative AI model to more accurately predict a child's mental state.

[0899] 5. Notification function:

[0900] Based on the analysis results, if an abnormal mental state is detected, a notification will be sent to the parents.

[0901] 6. Learning content generation:

[0902] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[0903] 7. Behavioral monitoring and anomaly detection:

[0904] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is notified.

[0905] Device behavior

[0906] The device mediates interactions with the child and performs the following actions:

[0907] 1. Conversation Acquisition:

[0908] The child's speech is captured using a microphone or text input device.

[0909] 2. Data transmission:

[0910] The acquired conversation data is sent to the server.

[0911] 3. Learning Support:

[0912] The learning content received from the server is presented to the child and they are prompted to respond.

[0913] 4. Condition Monitoring:

[0914] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[0915] Usage example

[0916] 1. Example conversation with a child:

[0917] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[0918] The terminal transmits this message to the server.

[0919] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[0920] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[0921] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[0922] 2. Learning support examples:

[0923] The user (child) tells the device, "I want to study math now."

[0924] The terminal sends this request to the server.

[0925] The server stores the received requests in a database and inputs them into a generative AI model.

[0926] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[0927] The server sends the generated mathematical problem to the terminal.

[0928] The terminal presents the received question to the child and waits for the child's answer.

[0929] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[0930] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[0931] 3. Condition monitoring example:

[0932] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[0933] The device transmits behavioral data to the server in real time.

[0934] The server stores the received behavioral data in a database.

[0935] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[0936] The server immediately sends a notification to the parent if it detects an anomaly, such as an abnormally long period of inactivity or suspicious behavior.

[0937] This system utilizes a generative AI model and emotion engine to more accurately predict and monitor a child's mental state and behavior, and provides necessary information to parents, supporting their child's development and safety. The specific implementation is programmed, and it is possible to interact with children in real time.

[0938] The processing flow will be explained below.

[0939] A detailed explanation of the system's program processing (including the emotion engine)

[0940] Retrieving conversations and sending data

[0941] Step 1:

[0942] The device receives the child's speech via a microphone or text input device. For example, the device receives a speech such as, "I was lonely at school today because my friends were not there."

[0943] Step 2:

[0944] The device sends the acquired speech data to the server, along with the speech content and a timestamp.

[0945] Step 3:

[0946] The server stores the data received from the device in a database, including the content of the message and a timestamp.

[0947] Step 4:

[0948] The server inputs the received speech data into an emotion engine to analyze the child's emotions. The emotion engine recognizes emotions such as "lonely," "happy," and "sad" based on the characteristics of the text and voice.

[0949] Step 5:

[0950] The server inputs the emotion analysis data from the emotion engine into the generative AI model to more accurately predict the child's mental state. In this case, the mental state of "lonely" is identified.

[0951] Step 6:

[0952] The server stores the analysis results in a database, and if the child's mental state is determined to be "lonely," it sends a notification to the parents that "the child is feeling lonely."

[0953] Learning support and dialogue

[0954] Step 1:

[0955] The user (child) tells the device, "I want to study math now."

[0956] Step 2:

[0957] The device sends this request to the server, along with the request content and a timestamp.

[0958] Step 3:

[0959] The server stores the received requests in a database and inputs them into a generative AI model.

[0960] Step 4:

[0961] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[0962] Step 5:

[0963] The server sends the generated math problem to the terminal.

[0964] Step 6:

[0965] The device presents the received question to the child and waits for the child's answer.

[0966] Step 7:

[0967] The user (child) enters the answer into the terminal, and the terminal sends the answer to the server.

[0968] Step 8:

[0969] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it sends the feedback "That's correct!"

[0970] Condition monitoring and anomaly detection

[0971] Step 1:

[0972] The device monitors children's behavior in real time through cameras and sensors.

[0973] Step 2:

[0974] The device sends the monitoring data to the server, which includes the activity details and a timestamp.

[0975] Step 3:

[0976] The server stores the received behavioral data in a database.

[0977] Step 4:

[0978] The server analyzes the accumulated data and real-time data using a generative AI model and emotion engine, and compares it with the normal range of behavior.

[0979] Step 5:

[0980] If the server detects an anomaly, it will immediately send a notification to the parent, for example if the device is stationary for an abnormally long period of time or if it is behaving suspiciously.

[0981] Emotion analysis using an emotion engine

[0982] Step 1:

[0983] The device captures the child's speech through a microphone and text input device.

[0984] Step 2:

[0985] The terminal transmits the acquired speech data to the server.

[0986] Step 3:

[0987] The server inputs the received utterance data into the emotion engine.

[0988] Step 4:

[0989] The emotion engine analyzes the content of the speech and identifies emotion labels, such as "lonely," "happy," and "sad."

[0990] Step 5:

[0991] The server inputs the emotion labels obtained from the emotion engine into a generative AI model to more accurately predict the child's mental state.

[0992] Step 6:

[0993] The server sends notifications to parents based on information obtained from the emotion engine and generative AI model, and also generates and presents learning content and reading content as needed.

[0994] In this way, this system, which combines an emotion engine, can more accurately grasp a child's mental state and provide the necessary support and notifications, allowing them to spend more secure and meaningful time together.

[0995] Example 2

[0996] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0997] With the increase in dual-income households, there is a problem that children spend more time alone. In such situations, it is difficult for parents to grasp changes in their children's mental state and behavior in real time, raising concerns about their children's growth and safety. Furthermore, there is often a lack of appropriate support when children study alone. This invention solves these problems and provides a system that supports children's growth and safety.

[0998] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring a conversation with a child, a means for analyzing the acquired conversation data and using an emotion engine to recognize the child's emotions, a means for using a generative AI model to infer the child's mental state based on the recognized emotion data, and a means for notifying the parent of the child's mental state. This makes it possible to grasp the child's emotions and mental state in real time and to immediately notify the parent if an abnormality is detected. In addition, the generative AI model generates learning content related to a subject and presents it to the child, thereby realizing learning support. This makes it possible to comprehensively support the child's growth and safety.

[0999] "Means for acquiring a conversation with a child" refers to an input device such as a microphone or text input device for acquiring a child's speech as voice or text.

[1000] "Emotion engine that analyzes acquired conversation data and recognizes children's emotions" refers to a software component that uses text analysis and voice analysis of input conversation data to recognize children's emotions.

[1001] "A generative artificial intelligence model that predicts a child's mental state based on recognized emotional data" refers to a machine learning model that uses emotion recognition results as input to predict a child's mental state and psychological tendencies.

[1002] "Means of notifying parents of their child's mental state" refers to communication functions that send notifications to parents via email, SMS, or a dedicated app if an abnormal mental state is detected.

[1003] "Means for generating learning content related to a subject and presenting it to a child" refers to a function for generating learning questions and content using a generative AI model based on the child's requests and learning situation, and presenting it to the child.

[1004] "Means for monitoring children's behavior and detecting abnormalities" refers to a system that uses cameras and sensors to collect children's behavioral data in real time, analyzes it using machine learning models, and detects abnormalities.

[1005] "Means for immediately notifying parents if abnormalities are detected" refers to a communication function for quickly sending a notification to parents when abnormal behavior is detected.

[1006] This invention addresses the issue of children spending more time alone as the number of dual-income households increases. It provides a system that utilizes a generative AI model and an emotion engine to promote children's development and monitor safety within the home.

[1007] System Configuration

[1008] Server Operation

[1009] The server hosts the generative AI model, emotion engine, and database management system, and implements the following key functions:

[1010] 1. Load the generative AI model:

[1011] The server loads the generative AI model and emotion engine into memory at startup, enabling rapid data processing and analysis using deep learning libraries such as TensorFlow and PyTorch.

[1012] 2. Data Receipt and Storage:

[1013] The server receives the child's conversation data sent from the device and stores it in a database using HTTP requests and WebSockets. MySQL or PostgreSQL is used as the database management system.

[1014] 3. Emotion analysis:

[1015] The server inputs the received conversation data into an emotion engine to recognize the child's emotions. The emotion engine uses a natural language processing library (e.g., NLTK or spaCy) to analyze the text and identify emotions.

[1016] 4. Data Analysis:

[1017] The server inputs the emotional data analyzed by the emotion engine into a generative AI model to predict the child's mental state. The generative AI model then comprehensively analyzes past and real-time data to predict the child's mental state.

[1018] 5. Notification function:

[1019] Based on the analysis results, the server will notify parents if an abnormal mental state is detected. Notifications can be sent via email, SMS, or a dedicated app. For example, Amazon SES (Simple Email Service) can be used.

[1020] 6. Learning content generation:

[1021] The server generates learning content based on the child's request and sends it to the device. For example, math problems or reading content are generated. For example, a question such as "Please solve the following calculation: What is 5 + 7?" is generated.

[1022] 7. Behavioral monitoring and anomaly detection:

[1023] The server analyzes the behavioral data received from the device and notifies the parent if an abnormality is detected. The behavioral data includes data obtained from cameras and motion sensors.

[1024] Device behavior

[1025] The device mediates interactions with the child and implements the following functions in cooperation with the server:

[1026] 1. Conversation Acquisition:

[1027] The device uses a microphone and text input device to capture the child's speech, which is then converted into text data using voice recognition software (e.g., Google Speech-to-Text API).

[1028] 2. Data transmission:

[1029] The device sends the acquired conversation data to the server via HTTP requests or WebSockets. The data is packaged in JSON format and sent to a specific endpoint.

[1030] 3. Learning Support:

[1031] The device receives learning content from the server and presents it to the child, prompting them to answer. For example, generated math problems are displayed on the device screen, allowing the child to enter their answers.

[1032] 4. Condition Monitoring:

[1033] The device uses cameras and sensors to monitor the child's behavior and notifies the server if it detects anything unusual, such as if the child remains motionless for a long period of time or if it detects suspicious movements.

[1034] Usage example

[1035] 1. Example conversation with a child:

[1036] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[1037] The terminal transmits this message to the server.

[1038] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[1039] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[1040] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[1041] 2. Learning support examples:

[1042] The user (child) tells the device, "I want to study math now."

[1043] The terminal sends this request to the server.

[1044] The server stores the received requests in a database and inputs them into a generative AI model.

[1045] The server uses a generative AI model to generate appropriate math problems, such as "Please solve the following equation: What is 5 + 7?"

[1046] The server sends the generated mathematical problem to the terminal.

[1047] The terminal presents the received question to the child and waits for the child's answer.

[1048] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1049] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[1050] 3. Condition monitoring example:

[1051] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[1052] The device transmits behavioral data to the server in real time.

[1053] The server stores the received behavioral data in a database.

[1054] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[1055] The server will immediately send a notification to its parent if it detects an anomaly, for example if it is quiet for an abnormally long period of time.

[1056] Example prompt

[1057] 1. "Explain what to do when your child tells you they're lonely and don't have any friends."

[1058] 2. "Explain how the system responds to a child's request to learn math."

[1059] This allows the system to utilize generative AI models and emotion engines to more accurately predict and monitor children's mental states and behaviors, providing parents with the information they need to support their children's growth and safety.

[1060] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1061] Step 1:

[1062] The device uses a microphone and a text input device to capture what the child says. For example, the child might say, "I was lonely at school today because my friends weren't there."

[1063] Input: Child's voice or text

[1064] Output: Conversation data in text format

[1065] Step 2:

[1066] The device sends the acquired conversation data to the server using HTTP requests or WebSockets. The data is packaged in JSON format and sent to a specific endpoint (e.g., / api / conversations).

[1067] Input: Text-based conversation data

[1068] Output: JSON formatted conversation data

[1069] Step 3:

[1070] The server receives the conversation data sent from the terminal and stores it in a database management system (e.g., MySQL, PostgreSQL).

[1071] Input: JSON format conversation data

[1072] Output: Conversation data stored in a database

[1073] Step 4:

[1074] The server retrieves conversation data from the database and inputs it into an emotion engine (e.g., NLTK, spaCy) to recognize the child's emotions. The emotion engine performs text analysis to identify emotions such as "lonely."

[1075] Input: Conversation data stored in the database

[1076] Output: Emotion data identified by the emotion engine

[1077] Step 5:

[1078] The server inputs the emotion data analyzed by the emotion engine into a generative AI model to infer the child's mental state. The generative AI model performs analysis using deep learning libraries (e.g., TensorFlow, PyTorch).

[1079] Input: Emotion data

[1080] Output: Inferred mental state data

[1081] Step 6:

[1082] The server then uses the estimated mental state data to notify parents if an abnormal state is detected. Notifications are sent via email, SMS, or a dedicated app (e.g., Amazon SES).

[1083] Input: Inferred mental state data

[1084] Output: Notification sent to parent

[1085] Step 7:

[1086] The server uses a generative AI model to generate learning content based on the child's requests, such as generating a math problem like "What is 5 + 7?"

[1087] Input: Child request data

[1088] Output: Generated learning content

[1089] Step 8:

[1090] The server sends the generated learning content to the device, which then presents it to the child and prompts them to respond.

[1091] Input: Generated learning content

[1092] Output: Learning content sent to the device

[1093] Step 9:

[1094] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1095] Input: Child's answer data

[1096] Output: Answer data sent to the server

[1097] Step 10:

[1098] The server stores the received answer data in a database, analyzes it using a generative AI model, creates feedback, and sends it to the device. For example, if the answer is correct, it displays "That's correct!"

[1099] Input: Child's answer data

[1100] Output: Feedback message

[1101] (Application example 2)

[1102] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1103] With the increase in dual-income households, children are spending more time alone. Under these circumstances, it is difficult to constantly monitor their children's mental state and safety, and there is a particular problem of delayed detection of dangers or abnormal behavior that may occur while parents are away. Furthermore, while it is important to make effective use of the time children spend alone and provide them with learning support, providing appropriate timing and content for such support can be difficult.

[1104] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1105] In this invention, the server includes means for acquiring a voice or text conversation with a child, means for analyzing the acquired conversation data and using an emotion engine to identify the child's emotions, means for inputting the emotion data identified by the emotion engine into a generative AI model to infer the child's mental state, means for notifying the parent of the child's mental state, means for monitoring the child's behavior and detecting abnormalities, and means for immediately notifying the parent if an abnormality is detected. This makes it possible to support the child's safety and mental growth by analyzing the child's mental state and behavior in real time and notifying the parent.

[1106] "Means for capturing voice or text conversations" refers to technology for capturing voice or text spoken by a child using a microphone, text input device, or the like.

[1107] An "emotion engine" refers to software or algorithms that use text and voice analysis to recognize and identify a child's emotions.

[1108] A "generative AI model" is an artificial intelligence model that generates new information and learning content based on acquired data.

[1109] "Means for inferring mental state" refers to a technology that uses emotional data identified by the emotion engine to infer a child's current mental state.

[1110] "Means for notifying parents" refers to technology that uses communication methods such as email and mobile notifications to inform parents of their child's mental state or abnormalities in real time.

[1111] "Means for monitoring behavior and detecting abnormalities" refers to technology that uses cameras and sensors to monitor children's movements and detect abnormal behavior.

[1112] "Means for generating learning content and presenting it to children" refers to technology that uses a generative AI model to generate learning content related to a subject and presents it to children at an appropriate time.

[1113] "Means for analyzing in real time and providing feedback" refers to technology that instantly analyzes acquired voice and behavioral data and provides feedback to the child based on the results.

[1114] This invention addresses the issue of children spending more time alone as the number of dual-income households increases. It provides a system that utilizes a generative AI model and an emotion engine to promote children's development and monitor safety within the home. This system is realized primarily by combining a server and a terminal.

[1115] Server Operation

[1116] 1. Load the generative AI model and emotion engine:

[1117] When the server starts up, it loads the generative AI model and emotion engine into memory. The AI ​​model and emotion engine are software that play a central role in the analysis and generation processes.

[1118] 2. Data Receipt and Storage:

[1119] The system receives the child's conversation and behavior data sent from the device and stores it in a database for later analysis.

[1120] 3. Emotion analysis:

[1121] The received conversation data is input into an emotion engine, which uses text and voice analysis to identify the child's emotions.

[1122] 4. Data Analysis:

[1123] The emotional data analyzed by the emotion engine is input into a generative AI model to predict the child's mental state. The generative AI model then uses the received data to more accurately predict the child's mental state.

[1124] 5. Notification function:

[1125] If an abnormal mental state is detected based on the analysis results, a notification will be sent to the parents via email or mobile notification.

[1126] 6. Learning content generation:

[1127] Based on the child's request, appropriate learning content is generated using a generative AI model and sent to the device.

[1128] 7. Behavioral Monitoring and Anomaly Detection:

[1129] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is immediately notified.

[1130] Device behavior

[1131] 1. Conversation Acquisition:

[1132] The system captures the child's speech using a microphone or text input device, and the captured data is sent to a server in real time.

[1133] 2. Data transmission:

[1134] The acquired conversation data and behavioral data are sent to the server.

[1135] 3. Learning Support:

[1136] The learning content received from the server is presented to the child, encouraging the child to study.

[1137] 4. Condition Monitoring:

[1138] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[1139] Specific examples

[1140] 1. Example conversation with a child:

[1141] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[1142] The terminal transmits this message to the server.

[1143] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[1144] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[1145] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[1146] 2. Learning support examples:

[1147] The user (child) tells the device, "I want to study math now."

[1148] The terminal sends this request to the server.

[1149] The server stores the received requests in a database and inputs them into a generative AI model.

[1150] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[1151] The server sends the generated questions to the terminal.

[1152] The terminal presents the received question to the child and waits for the child's answer.

[1153] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1154] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[1155] 3. Condition monitoring example:

[1156] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[1157] The device transmits behavioral data to the server in real time.

[1158] The server stores the received behavioral data in a database.

[1159] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[1160] The server sends an immediate notification to the parent if it detects an anomaly, such as an abnormally long period of inactivity or suspicious behavior.

[1161] Prompt Sentence Examples

[1162] Emotion Analysis Prompt: "Analyze this audio data to identify the emotion."

[1163] Learning content generation prompt: "Generate one basic math problem, such as an addition or subtraction problem."

[1164] This system uses a generative AI model and an emotion engine to analyze a child's mental state and behavior in real time, and provides parents with the information they need to support their child's growth and safety.

[1165] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1166] Step 1:

[1167] Starting the server and loading the model

[1168] When the server starts up, it loads the generative AI model and emotion engine into memory. This step makes the necessary software components available. The input is a server configuration file containing the generative AI model and emotion engine, and the output is the model and engine loaded into the server memory. Specifically, the server executes the model loading routine and allocates memory space.

[1169] Step 2:

[1170] Retrieving and sending conversation data

[1171] The device captures voice and text conversations from the user (child). As input, voice and text data are obtained using a microphone and text input device, and this is sent to the server. The output is raw data sent to the server. Specifically, the device captures data through a voice recording or text input interface, and sends the data to the server using a communication protocol.

[1172] Step 3:

[1173] Sentiment analysis of conversation data

[1174] The server inputs the received conversation data into an emotion engine to identify the child's emotions. The input is voice or text data sent from the device, which is analyzed by the emotion engine. The output is the identified emotion data. Specifically, the server runs the emotion engine's analysis algorithm and classifies emotions using text analysis and voice analysis technology.

[1175] Step 4:

[1176] Mental state inference from emotional data

[1177] The server inputs the emotion data identified by the emotion engine into the generative AI model to infer the child's mental state. The input is emotion data from the emotion engine, which is adapted and processed by the generative AI model. The output is the child's inferred mental state. Specifically, the server executes the generative AI model's data processing algorithm to infer the child's mental state based on the emotion data.

[1178] Step 5:

[1179] Mental status notification

[1180] The server sends a notification to the parent if an abnormality is detected based on the child's mental state. The input is the mental state data inferred by the generative AI model, and the information is conveyed to the parent via the notification system. The output is a notification message sent to the parent's device. Specifically, the server invokes the notification function to send an email or mobile notification.

[1181] Step 6:

[1182] Receiving a learning content generation request

[1183] When a user (child) inputs a learning request into the device, the device sends this request to the server. The input is voice or text data based on the user's learning request, which is sent to the server. The output is the request data sent to the server. Specifically, the device captures the request data using a voice or text input interface and sends it to the server using a communication protocol.

[1184] Step 7:

[1185] Generation and presentation of learning content

[1186] The server uses the generative AI model to generate appropriate learning content based on the received request and sends it to the device. The input is the learning request data from the user and the generative AI model, and learning content is created based on this. The output is the data that sends the generated learning content to the device. Specifically, the server uses the prompt sentences from the generative AI model to generate math problems and reading comprehension content and sends it to the device.

[1187] Step 8:

[1188] Implementing learning support

[1189] The device presents the learning content received from the server to the user (child) and allows them to work on it. The input is the learning content sent from the server, which is displayed on the user interface. The output is the answer data entered by the user. Specifically, the device displays the learning content using a display and speech synthesis function, and obtains the user's answers from the input device.

[1190] Step 9:

[1191] Monitoring and sending behavioral data

[1192] The device monitors the user's (child's) behavior using cameras and sensors and sends the data to a server. The input is the behavioral data acquired from the cameras and sensors, which is sent to the server. The output is the behavioral data sent to the server. Specifically, the device acquires video and sensor information in real time and uses a communication protocol to send it to the server.

[1193] Step 10:

[1194] Anomaly detection in behavioral data

[1195] The server analyzes the received behavioral data and immediately sends a notification to the parent if an abnormality is detected. The input is the behavioral data sent from the device, and an algorithm is applied to detect abnormalities based on this. The output is a notification message that is sent to the parent if an abnormality is detected. Specifically, the server runs the behavioral analysis algorithm, and if an abnormality is discovered, it notifies the parent via the notification system.

[1196] Through these steps, this system utilizes a generative AI model and emotion engine to analyze children's mental state and behavior in real time and provide parents with the necessary information to support their children's growth and safety.

[1197] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1198] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1199] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1200] [Third embodiment]

[1201] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1202] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1203] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1204] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1205] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1206] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1207] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1208] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1209] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1210] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1211] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1212] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1213] This invention addresses the issue of children spending more time alone as the number of dual-income households increases, by providing a system that utilizes generative AI to promote children's development and monitor safety within the home.

[1214] System configuration and operation

[1215] Server Operation

[1216] The server hosts the generative AI model and database management system and performs the following operations:

[1217] 1. Load the generative AI model:

[1218] The server loads the generative AI model into memory at startup.

[1219] 2. Data Receipt and Storage:

[1220] The system receives the child's conversation data sent from the device and stores it in a database.

[1221] 3. Data Analysis:

[1222] The received conversation data is input into a generative AI model to analyze the child's mental state.

[1223] 4. Notification function:

[1224] Based on the analysis results, if an abnormal mental state is detected, a notification will be sent to the parents.

[1225] 5. Learning content generation:

[1226] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[1227] 6. Behavioral monitoring and anomaly detection:

[1228] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is notified.

[1229] Device behavior

[1230] The device mediates interactions with the child and performs the following actions:

[1231] 1. Conversation Acquisition:

[1232] The child's speech is captured using a microphone or text input device.

[1233] 2. Data transmission:

[1234] The acquired conversation data is sent to the server.

[1235] 3. Learning Support:

[1236] The learning content received from the server is presented to the child and they are prompted to respond.

[1237] 4. Condition Monitoring:

[1238] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[1239] Usage example

[1240] 1. Example conversation with a child:

[1241] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[1242] The terminal transmits this message to the server.

[1243] The server stores the received data and analyzes the content of the statements using a generative AI model.

[1244] The server identifies the mental state "lonely" and sends a notification to the parent that "child feels lonely."

[1245] 2. Learning support examples:

[1246] The user (child) tells the device, "I want to study math now."

[1247] The terminal sends this request to the server.

[1248] The server uses a generative AI model to generate appropriate math problems and sends them to the device.

[1249] The device presents questions to the child and prompts them to answer.

[1250] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1251] 3. Condition monitoring example:

[1252] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[1253] The device transmits behavioral data to the server in real time.

[1254] The server analyzes the received behavioral data and notifies the parent if any abnormalities are detected.

[1255] summary

[1256] In this way, the system of this invention utilizes generative AI models to support children's development and monitor their safety, providing an environment where parents in dual-income households can feel secure knowing what is happening with their children. The specific implementation is programmed, interacting with children in real time, analyzing their mental state and behavior based on accumulated data, and providing necessary notifications and learning content. This makes children's alone time safer and more meaningful.

[1257] The processing flow will be explained below.

[1258] Specific explanation of system program processing

[1259] Retrieving conversations and sending data

[1260] Step 1:

[1261] The device receives the child's speech via a microphone or text input device. For example, the device receives a speech such as, "I was lonely at school today because my friends were not there."

[1262] Step 2:

[1263] The device sends the acquired speech data to the server, along with the speech content and a timestamp.

[1264] Step 3:

[1265] The server stores the data received from the device in a database, including the content of the message and a timestamp.

[1266] Step 4:

[1267] The server inputs the received data into a generative AI model to analyze the child's mental state. In this case, the emotion of "lonely" is identified.

[1268] Step 5:

[1269] The server stores the analysis results in a database and sends a notification to the parents if the child's mental state is determined to be "lonely."

[1270] Learning support and dialogue

[1271] Step 1:

[1272] The user (child) tells the device, "I want to study math now."

[1273] Step 2:

[1274] The device sends this request to the server, along with the request content and a timestamp.

[1275] Step 3:

[1276] The server stores the received requests in a database and inputs them into a generative AI model.

[1277] Step 4:

[1278] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[1279] Step 5:

[1280] The server sends the generated math problem to the terminal.

[1281] Step 6:

[1282] The device presents the received question to the child and waits for the child's answer.

[1283] Step 7:

[1284] The user (child) enters the answer into the terminal, and the terminal sends the answer to the server.

[1285] Step 8:

[1286] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[1287] Condition monitoring and anomaly detection

[1288] Step 1:

[1289] The device monitors children's behavior in real time through cameras and sensors.

[1290] Step 2:

[1291] The device sends the monitoring data, including the activity and a timestamp, to the server.

[1292] Step 3:

[1293] The server stores the received behavioral data in a database.

[1294] Step 4:

[1295] The server analyzes the accumulated data using a generative AI model and compares it with the normal range of behavior.

[1296] Step 5:

[1297] If the server detects an anomaly, it will immediately send a notification to the parent, for example if the device is stationary for an abnormally long period of time or if it is behaving suspiciously.

[1298] This allows the system to continuously monitor a child's mental state and behavior through daily interactions with the child and support their learning, and to support the child's growth and safety by promptly notifying parents if any abnormalities or dangers occur.

[1299] Example 1

[1300] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1301] With the increase in dual-income households, children spend more time alone, creating the need to properly monitor their growth and safety. There is also a lack of effective ways to monitor children's mental health and academic progress. This creates a need for a system that allows parents to monitor their children's condition while they are at work and respond quickly if something unusual occurs.

[1302] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1303] In this invention, the server includes means for acquiring conversations with children, means for transmitting the acquired conversation data to the server, means for storing the received conversation data in a database, means for analyzing the child's mental state using a generative AI model based on the stored conversation data, and means for notifying parents if an abnormal mental state is detected based on the analysis results.This enables parents in dual-income households to understand their children's mental state and behavior in real time even while at work, and to provide appropriate learning support and safety checks.

[1304] "Children" refers to minors who are eligible to use the System.

[1305] "Conversation data" refers to a recording of a child's speech in the form of text or audio data.

[1306] "Server" refers to the central computing unit that hosts the generative AI model and receives, stores, and analyzes data.

[1307] A "generative AI model" refers to a program model that uses artificial intelligence technology to perform natural language processing and data analysis.

[1308] "Database" refers to an information management system for storing received conversation data and behavioral data.

[1309] "Mental state" refers to the emotional and psychological state that can be inferred from a child's words and behavior.

[1310] "Notification" refers to an alert or informational message sent from the server to the parent.

[1311] "Learning content" refers to the learning questions and materials generated by the generative AI model in response to the child's requests.

[1312] "Behavioral data" refers to data such as a child's movements and location collected using cameras and sensors.

[1313] "Abnormal" refers to a child's behavior or mental state that is different from normal and indicates an unnatural or dangerous condition.

[1314] The present invention is a system that supports the safety and development of children by addressing the increasing amount of time they spend alone due to the increase in dual-income households. This system includes a device that interacts with children and a server that analyzes data and sends notifications.

[1315] Hardware and software used

[1316] Terminals consist of interaction devices equipped with input devices such as microphones, cameras, sensors, and touchscreens. Examples include general-purpose tablets, smart speakers, and surveillance cameras.

[1317] The server is a central computing unit that hosts the generative AI model and receives, stores, and analyzes data. Software used includes deep learning frameworks (e.g., TensorFlow, PyTorch), database management systems (e.g., MySQL, PostgreSQL), and various notification APIs (e.g., Twilio, SendGrid).

[1318] Conversation capture and analysis

[1319] The device uses a microphone and text input device to capture the child's speech. For example, if a child says, "I missed my friends at school today," the device uses voice recognition software to convert this speech into text data. The converted text data is then securely transmitted to a server via the HTTPS protocol.

[1320] The server stores the received text data in a database such as MySQL. It then analyzes the data using a generative AI model (e.g., BERT, GPT-3). At this time, the prompt "Please analyze the mental state when the user tells a sad story" is input to the model.

[1321] Mental state detection and notification

[1322] The server determines the child's mental state based on the analysis results. For example, if the emotion "lonely" is detected, a notification is sent to the parent using Twilio or SendGrid. The notification includes a message saying "Your child is feeling lonely."

[1323] Creating and presenting learning content

[1324] When a user (child) says to the device, "I want to study math now," the device sends this request to the server.

[1325] The server uses a generative AI model to generate appropriate math problems. The prompt "Generate simple math problems for elementary school students" is input to the model. The generated math problems are sent back to the device, which presents them to the child. The child enters the answers, and the answer data is sent back to the server.

[1326] Behavioral data monitoring and analysis

[1327] The device continuously monitors the child's behavior using cameras and sensors, for example, recording the child's behavior while they are in their room, and transmitting the monitoring data to a server in real time.

[1328] The server analyzes the received behavioral data and notifies the parent if abnormal behavior is detected. For example, if a child remains motionless for a long time, a notification stating "Child has not moved for a long time" is sent to the parent.

[1329] In this way, the system of the present invention utilizes a generative AI model to analyze a child's mental state and behavior, and provides parents with the necessary notifications and learning support, thereby providing an environment where parents in dual-income households can feel at ease knowing what is happening with their children.

[1330] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1331] Step 1: Obtaining conversation data

[1332] The device uses a microphone and text input device to capture what the child says. For example, if a child says, "I missed my friends at school today," the device uses voice recognition software to convert this speech into text data.

[1333] (Input) Child's speech (audio or text)

[1334] (Output) Text data (in the case of audio, data converted to text)

[1335] Step 2: Sending conversation data to the server

[1336] The device sends the retrieved text data to the server via an HTTP POST request, with the data being transferred securely using SSL encryption.

[1337] (Input) Text data

[1338] (Output) None (Transfer to server)

[1339] Step 3: Save your data

[1340] The server stores the received text data in MySQL (or other database management systems), including information such as what the child said, when they said it, and the speaker's ID.

[1341] (Input) Text data (transferred data)

[1342] (Output) Save data to database (persistent storage)

[1343] Step 4: Analyzing the conversation data

[1344] The server inputs the saved text data into a generative AI model to analyze the child's mental state. Specifically, the model is given a prompt sentence: "Please analyze the child's mental state when the user tells a sad story." This analysis uses the natural language processing capabilities of BERT and GPT-3.

[1345] (Input) Text data (saved conversation data), prompt sentence

[1346] (Output) Analysis results (mental state)

[1347] Step 5: Mental status notification

[1348] The server evaluates the child's mental state based on the analysis results, and if an abnormal state is detected, it immediately notifies the parent. For example, if the emotion "lonely" is detected, a notification stating "your child is feeling lonely" is sent to the parent using Twilio or SendGrid.

[1349] (Input) Analysis results (mental state)

[1350] (Output) Notification message (notification to parent)

[1351] Step 6: Request learning content

[1352] The user (child) requests learning content by telling the device, "I want to study math now."

[1353] (Input) Request statement ("I want to study math now.")

[1354] (Output) Request data (transferred to server)

[1355] Step 7: Generate learning content

[1356] The server receives the learning request and generates appropriate math problems using a generative AI model. For example, it generates problems by inputting a prompt sentence such as "Generate simple math problems for elementary school students."

[1357] (Input) Request data, prompt

[1358] (Output) Learning content (math problems)

[1359] Step 8: Present learning content and submit answers

[1360] The device receives a mathematical problem from the server and presents it to the child, prompting the child to answer it. The child enters the answer into the device, which then transmits the answer to the server.

[1361] (Input) Learning content (math problems)

[1362] (Output) Answer data (transferred to server)

[1363] Step 9: Monitor behavioral data

[1364] The device uses cameras and sensors to continuously monitor a child's behavior, for example by recording their movements and location in real time while they are in their room.

[1365] (Input) Behavioral data (data from cameras and sensors)

[1366] (Output) Behavioral data (transferred to server)

[1367] Step 10: Analyze behavioral data and detect anomalies

[1368] The server analyzes the received behavioral data and notifies parents if abnormal behavior is detected. For example, if a child stays in the same place for a long time, the server sends a message saying, "Your child has not moved for a long time."

[1369] (Input) Behavioral data (data monitored in real time)

[1370] (Output) Notification message (notification to parent)

[1371] (Application example 1)

[1372] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1373] With the increase in dual-income households, children are spending more time alone, making it necessary to provide appropriate supervision and learning support to ensure their growth and safety. It is also important to understand children's mental state and behavior in real time and detect abnormalities early. The present invention aims to provide a system that solves these problems.

[1374] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1375] In this invention, the server includes means for acquiring conversation data with a child, means for analyzing the acquired conversation data and using a generative AI model to infer the child's mental state, means for notifying a parent of the child's mental state, means for generating appropriate learning content and presenting it to the child, means for monitoring the child's behavior and detecting abnormalities, means for immediately notifying the parent when an abnormality is detected, and means for generating appropriate prompt sentences and inputting them into the generative AI model. This makes it possible to provide necessary learning support while monitoring the child's mental state and behavior in real time at home, and to respond quickly when an abnormality is detected.

[1376] "Conversation with children" refers to communication within the home that includes words and sentences spoken by children.

[1377] "Means of acquisition" refers to the functionality for collecting data using a microphone, text input device, or other sensor device.

[1378] A "generative AI model" is an artificial intelligence algorithm or software that generates new information or content from original data through natural language processing and data analysis.

[1379] "Mental state" refers to a child's psychological health and emotional changes, and is an indicator for assessing that state.

[1380] "Means of notifying parents" refers to the means of communication to promptly inform parents of analysis results or abnormal situations, including email, SMS, in-app notifications, etc.

[1381] "Learning content" refers to questions, reading materials, and other content designed to support a child's education and intellectual development.

[1382] "Presentation means" is a function for showing the generated learning content to children or conveying it to them by voice.

[1383] "Behavioral monitoring measures" are features that use cameras and sensors to continuously observe a child's physical movements and location.

[1384] "Means for detecting anomalies" refers to the ability to identify unusual behavior or situations and issue a warning when danger or problems occur.

[1385] A "prompt" is an instruction or question used to direct a generative AI model to perform a specific analysis or generation.

[1386] "Smart device" means a device with internet connectivity and advanced computing capabilities, including smartphones and tablets.

[1387] The present invention provides a system that uses a generative AI model to support the development and safety of children in dual-income households. This system is composed of a server and a terminal, and is realized using the following means.

[1388] Server Operation

[1389] The server hosts the generative AI model and database management system and performs the following operations:

[1390] 1. Load the generative AI model:

[1391] The server loads the generative AI model into memory at startup, allowing it to analyze conversational and behavioral data in real time.

[1392] 2. Data Receipt and Storage:

[1393] Conversation data and behavioral data with the child sent from the device is received and stored in a database.

[1394] 3. Data Analysis:

[1395] The received conversation data is input into a generative AI model to analyze the child's mental state, and based on this analysis, a notification is sent to the parent.

[1396] 4. Learning content generation:

[1397] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[1398] 5. Behavioral monitoring and anomaly detection:

[1399] The received behavioral data is analyzed, and if any abnormalities are detected, a notification is sent to the parent.

[1400] Device behavior

[1401] The device mediates interactions with the child and performs the following actions:

[1402] 1. Conversation Acquisition:

[1403] The child's speech is captured using a microphone or text input device.

[1404] 2. Data transmission:

[1405] The acquired conversation data is sent to the server.

[1406] 3. Learning Support:

[1407] The learning content received from the server is presented to the child and they are prompted to respond.

[1408] 4. Condition Monitoring:

[1409] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[1410] Specific use cases

[1411] 1. Example conversation with a child:

[1412] The child says to the device, "I was lonely at school today because my friends weren't there."

[1413] The terminal transmits this message to the server.

[1414] The server stores the received data and analyzes the content of the statements using a generative AI model.

[1415] The server identifies the mental state "lonely" and sends a notification to the parent that "child feels lonely."

[1416] 2. Learning support examples:

[1417] The child tells the device, "I want to study math now."

[1418] The terminal sends this request to the server.

[1419] The server uses a generative AI model to generate appropriate math problems and sends them to the device.

[1420] The device presents questions to the child and prompts them to answer.

[1421] The child enters the answer into the terminal, which then sends the answer to the server.

[1422] 3. Condition monitoring example:

[1423] Cameras and sensors monitor children's behavior while they are in their room.

[1424] The device transmits behavioral data to the server in real time.

[1425] The server analyzes the received behavioral data and notifies the parent if any abnormalities are detected.

[1426] Prompt Sentence Examples

[1427] "Analyze your child's mental state when they say, 'I missed my friends at school today.'"

[1428] "Generate math problems suitable for a child who said they want to start studying math now."

[1429] "Determine whether your child's behavior in the room is abnormal."

[1430] As described above, the system of the present invention utilizes a generative AI model to support children's growth and monitor their safety, thereby providing an environment where parents in dual-income households can keep track of their children's situations with peace of mind.

[1431] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1432] Step 1:

[1433] Loading a generative AI model

[1434] When the server starts up, it loads the generative AI model into memory. In this process, the model file is read from disk and expanded into memory. The input is the path to the model file, and the output is the loaded generative AI model. This allows subsequent data analysis processing to be performed.

[1435] Step 2:

[1436] Conversation Acquisition

[1437] The device captures conversations with the child through a microphone and text input device. In this process, voice data and text data are input and the captured conversation data is output. This allows the necessary conversation content to be sent to the server.

[1438] Step 3:

[1439] Sending conversation data

[1440] The terminal transmits the acquired conversation data to the server. In this process, the conversation data is input and the data transmitted to the server is output, allowing the server to analyze the conversation data.

[1441] Step 4:

[1442] Conversation data analysis

[1443] The server inputs the transmitted conversation data into a generative AI model for analysis. In this process, the conversation data is input and an analysis result such as "child's mental state" is output. Specifically, the generative AI model infers emotions and mental state from the content of the conversation and generates the result.

[1444] Step 5:

[1445] Parental Notification

[1446] Based on the analysis results, the server will send a notification to the parents if it detects an abnormal mental state, such as loneliness, in the process. This process inputs the analysis results and outputs a notification message. The notification will be sent via email or in-app notification.

[1447] Step 6:

[1448] Generating learning content

[1449] The server generates math problems and storytelling content in response to the child's request. This process takes a child's request as input and outputs generated learning content. Using a generative AI model, appropriate learning content is generated based on the prompt.

[1450] Step 7:

[1451] Presentation of learning content

[1452] The device presents the learning content received from the server to the child and prompts them to respond. In this process, the learning content is input and the results presented to the child are output. Specifically, the learning content is presented using a screen display or a voice read-out function.

[1453] Step 8:

[1454] Answers to learning content

[1455] The user (child) inputs the answer into the terminal. In this process, the child's answer is input and the answer sent to the terminal is output.

[1456] Step 9:

[1457] Sending answer data

[1458] The terminal transmits the child's answer to the server. In this process, answer data is input and the data transmitted to the server is output.

[1459] Step 10:

[1460] Analysis of answer data

[1461] The server analyzes the submitted answer data and determines whether it is correct. In this process, the answer data is input and the analysis results are output. A generative AI model is used to evaluate the correctness of the answer and generate feedback.

[1462] Step 11:

[1463] Behavioral monitoring

[1464] The device continuously monitors the child's behavior using cameras and sensors, and the process inputs behavioral data and outputs real-time monitored data.

[1465] Step 12:

[1466] Sending behavioral data

[1467] The terminal transmits the monitored behavioral data in real time to the server, where the behavioral data is input and the data transmitted to the server is output.

[1468] Step 13:

[1469] Analysis of behavioral data

[1470] The server analyzes the received behavioral data and detects anomalies. In this process, behavioral data is input and anomaly detection results are output.

[1471] Step 14:

[1472] Parent notification when an abnormality is detected

[1473] The server will immediately notify the parent if an anomaly is detected. This process inputs the anomaly detection result and outputs a notification message. Notifications can be sent via email or in-app notifications.

[1474] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1475] This invention addresses the issue of children spending more time alone as the number of dual-income households increases, by providing a system that utilizes generative AI and an emotion engine to promote children's development and monitor safety within the home.

[1476] System configuration and operation

[1477] Server Operation

[1478] The server hosts the generative AI model, emotion engine, and database management system, and performs the following operations:

[1479] 1. Load the generative AI model:

[1480] The server loads the generative AI model and emotion engine into memory at startup.

[1481] 2. Data Receipt and Storage:

[1482] The system receives the child's conversation data sent from the device and stores it in a database.

[1483] 3. Emotion analysis:

[1484] The received conversation data is input into an emotion engine to recognize the child's emotions. The emotion engine identifies emotions using, for example, text analysis or voice analysis.

[1485] 4. Data Analysis:

[1486] Emotional data analyzed by the emotion engine is input into a generative AI model to more accurately predict a child's mental state.

[1487] 5. Notification function:

[1488] Based on the analysis results, if an abnormal mental state is detected, a notification will be sent to the parents.

[1489] 6. Learning content generation:

[1490] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[1491] 7. Behavioral monitoring and anomaly detection:

[1492] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is notified.

[1493] Device behavior

[1494] The device mediates interactions with the child and performs the following actions:

[1495] 1. Conversation Acquisition:

[1496] The child's speech is captured using a microphone or text input device.

[1497] 2. Data transmission:

[1498] The acquired conversation data is sent to the server.

[1499] 3. Learning Support:

[1500] The learning content received from the server is presented to the child and they are prompted to respond.

[1501] 4. Condition Monitoring:

[1502] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[1503] Usage example

[1504] 1. Example conversation with a child:

[1505] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[1506] The terminal transmits this message to the server.

[1507] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[1508] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[1509] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[1510] 2. Learning support examples:

[1511] The user (child) tells the device, "I want to study math now."

[1512] The terminal sends this request to the server.

[1513] The server stores the received requests in a database and inputs them into a generative AI model.

[1514] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[1515] The server sends the generated mathematical problem to the terminal.

[1516] The terminal presents the received question to the child and waits for the child's answer.

[1517] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1518] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[1519] 3. Condition monitoring example:

[1520] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[1521] The device transmits behavioral data to the server in real time.

[1522] The server stores the received behavioral data in a database.

[1523] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[1524] The server immediately sends a notification to the parent if it detects an anomaly, such as an abnormally long period of inactivity or suspicious behavior.

[1525] This system utilizes a generative AI model and emotion engine to more accurately predict and monitor a child's mental state and behavior, and provides necessary information to parents, supporting their child's development and safety. The specific implementation is programmed, and it is possible to interact with children in real time.

[1526] The processing flow will be explained below.

[1527] A detailed explanation of the system's program processing (including the emotion engine)

[1528] Retrieving conversations and sending data

[1529] Step 1:

[1530] The device receives the child's speech via a microphone or text input device. For example, the device receives a speech such as, "I was lonely at school today because my friends were not there."

[1531] Step 2:

[1532] The device sends the acquired speech data to the server, along with the speech content and a timestamp.

[1533] Step 3:

[1534] The server stores the data received from the device in a database, including the content of the message and a timestamp.

[1535] Step 4:

[1536] The server inputs the received speech data into an emotion engine to analyze the child's emotions. The emotion engine recognizes emotions such as "lonely," "happy," and "sad" based on the characteristics of the text and voice.

[1537] Step 5:

[1538] The server inputs the emotion analysis data from the emotion engine into the generative AI model to more accurately predict the child's mental state. In this case, the mental state of "lonely" is identified.

[1539] Step 6:

[1540] The server stores the analysis results in a database, and if the child's mental state is determined to be "lonely," it sends a notification to the parents that "the child is feeling lonely."

[1541] Learning support and dialogue

[1542] Step 1:

[1543] The user (child) tells the device, "I want to study math now."

[1544] Step 2:

[1545] The device sends this request to the server, along with the request content and a timestamp.

[1546] Step 3:

[1547] The server stores the received requests in a database and inputs them into a generative AI model.

[1548] Step 4:

[1549] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[1550] Step 5:

[1551] The server sends the generated math problem to the terminal.

[1552] Step 6:

[1553] The device presents the received question to the child and waits for the child's answer.

[1554] Step 7:

[1555] The user (child) enters the answer into the terminal, and the terminal sends the answer to the server.

[1556] Step 8:

[1557] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it sends the feedback "That's correct!"

[1558] Condition monitoring and anomaly detection

[1559] Step 1:

[1560] The device monitors children's behavior in real time through cameras and sensors.

[1561] Step 2:

[1562] The device sends the monitoring data to the server, which includes the activity details and a timestamp.

[1563] Step 3:

[1564] The server stores the received behavioral data in a database.

[1565] Step 4:

[1566] The server analyzes the accumulated data and real-time data using a generative AI model and emotion engine, and compares it with the normal range of behavior.

[1567] Step 5:

[1568] If the server detects an anomaly, it will immediately send a notification to the parent, for example if the device is stationary for an abnormally long period of time or if it is behaving suspiciously.

[1569] Emotion analysis using an emotion engine

[1570] Step 1:

[1571] The device captures the child's speech through a microphone and text input device.

[1572] Step 2:

[1573] The terminal transmits the acquired speech data to the server.

[1574] Step 3:

[1575] The server inputs the received utterance data into the emotion engine.

[1576] Step 4:

[1577] The emotion engine analyzes the content of the speech and identifies emotion labels, such as "lonely," "happy," and "sad."

[1578] Step 5:

[1579] The server inputs the emotion labels obtained from the emotion engine into a generative AI model to more accurately predict the child's mental state.

[1580] Step 6:

[1581] The server sends notifications to parents based on information obtained from the emotion engine and generative AI model, and also generates and presents learning content and reading content as needed.

[1582] In this way, this system, which combines an emotion engine, can more accurately grasp a child's mental state and provide the necessary support and notifications, allowing them to spend more secure and meaningful time together.

[1583] Example 2

[1584] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1585] With the increase in dual-income households, there is a problem that children spend more time alone. In such situations, it is difficult for parents to grasp changes in their children's mental state and behavior in real time, raising concerns about their children's growth and safety. Furthermore, there is often a lack of appropriate support when children study alone. This invention solves these problems and provides a system that supports children's growth and safety.

[1586] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring a conversation with a child, a means for analyzing the acquired conversation data and using an emotion engine to recognize the child's emotions, a means for using a generative AI model to infer the child's mental state based on the recognized emotion data, and a means for notifying the parent of the child's mental state. This makes it possible to grasp the child's emotions and mental state in real time and to immediately notify the parent if an abnormality is detected. In addition, the generative AI model generates learning content related to a subject and presents it to the child, thereby realizing learning support. This makes it possible to comprehensively support the child's growth and safety.

[1587] "Means for acquiring a conversation with a child" refers to an input device such as a microphone or text input device for acquiring a child's speech as voice or text.

[1588] "Emotion engine that analyzes acquired conversation data and recognizes children's emotions" refers to a software component that uses text analysis and voice analysis of input conversation data to recognize children's emotions.

[1589] "A generative artificial intelligence model that predicts a child's mental state based on recognized emotional data" refers to a machine learning model that uses emotion recognition results as input to predict a child's mental state and psychological tendencies.

[1590] "Means of notifying parents of their child's mental state" refers to communication functions that send notifications to parents via email, SMS, or a dedicated app if an abnormal mental state is detected.

[1591] "Means for generating learning content related to a subject and presenting it to a child" refers to a function for generating learning questions and content using a generative AI model based on the child's requests and learning situation, and presenting it to the child.

[1592] "Means for monitoring children's behavior and detecting abnormalities" refers to a system that uses cameras and sensors to collect children's behavioral data in real time, analyzes it using machine learning models, and detects abnormalities.

[1593] "Means for immediately notifying parents if abnormalities are detected" refers to a communication function for quickly sending a notification to parents when abnormal behavior is detected.

[1594] This invention addresses the issue of children spending more time alone as the number of dual-income households increases. It provides a system that utilizes a generative AI model and an emotion engine to promote children's development and monitor safety within the home.

[1595] System Configuration

[1596] Server Operation

[1597] The server hosts the generative AI model, emotion engine, and database management system, and implements the following key functions:

[1598] 1. Load the generative AI model:

[1599] The server loads the generative AI model and emotion engine into memory at startup, enabling rapid data processing and analysis using deep learning libraries such as TensorFlow and PyTorch.

[1600] 2. Data Receipt and Storage:

[1601] The server receives the child's conversation data sent from the device and stores it in a database using HTTP requests and WebSockets. MySQL or PostgreSQL is used as the database management system.

[1602] 3. Emotion analysis:

[1603] The server inputs the received conversation data into an emotion engine to recognize the child's emotions. The emotion engine uses a natural language processing library (e.g., NLTK or spaCy) to analyze the text and identify emotions.

[1604] 4. Data Analysis:

[1605] The server inputs the emotional data analyzed by the emotion engine into a generative AI model to predict the child's mental state. The generative AI model then comprehensively analyzes past and real-time data to predict the child's mental state.

[1606] 5. Notification function:

[1607] Based on the analysis results, the server will send a notification to the parents if an abnormal mental state is detected. Notifications can be sent via email, SMS, or a dedicated app. For example, Amazon SES (Simple Email Service) can be used.

[1608] 6. Learning content generation:

[1609] The server generates learning content based on the child's request and sends it to the device. For example, math problems or reading content may be generated. For example, a question such as "Please solve the following equation: What is 5 + 7?" may be generated.

[1610] 7. Behavioral monitoring and anomaly detection:

[1611] The server analyzes the behavioral data received from the device and notifies the parent if an abnormality is detected. The behavioral data includes data obtained from cameras and motion sensors.

[1612] Device behavior

[1613] The device mediates interactions with the child and implements the following functions in cooperation with the server:

[1614] 1. Conversation Acquisition:

[1615] The device uses a microphone and text input device to capture the child's speech, which is then converted into text data using voice recognition software (e.g., Google Speech-to-Text API).

[1616] 2. Data transmission:

[1617] The device sends the acquired conversation data to the server via HTTP requests or WebSockets. The data is packaged in JSON format and sent to a specific endpoint.

[1618] 3. Learning Support:

[1619] The device receives learning content from the server and presents it to the child, prompting them to answer. For example, generated math problems are displayed on the device screen, allowing the child to enter their answers.

[1620] 4. Condition Monitoring:

[1621] The device uses cameras and sensors to monitor the child's behavior and notifies the server if it detects anything unusual, such as if the child remains motionless for a long period of time or if it detects suspicious movements.

[1622] Usage example

[1623] 1. Example conversation with a child:

[1624] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[1625] The terminal transmits this message to the server.

[1626] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[1627] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[1628] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[1629] 2. Learning support examples:

[1630] The user (child) tells the device, "I want to study math now."

[1631] The terminal sends this request to the server.

[1632] The server stores the received requests in a database and inputs them into a generative AI model.

[1633] The server uses a generative AI model to generate appropriate math problems, such as "Please solve the following equation: What is 5 + 7?"

[1634] The server sends the generated mathematical problem to the terminal.

[1635] The terminal presents the received question to the child and waits for the child's answer.

[1636] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1637] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[1638] 3. Condition monitoring example:

[1639] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[1640] The device transmits behavioral data to the server in real time.

[1641] The server stores the received behavioral data in a database.

[1642] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[1643] The server will immediately send a notification to its parent if it detects an anomaly, for example if it is quiet for an abnormally long period of time.

[1644] Example prompt

[1645] 1. "Explain what to do when your child tells you they're lonely and don't have any friends."

[1646] 2. "Explain how the system responds to a child's request to learn math."

[1647] This allows the system to utilize generative AI models and emotion engines to more accurately predict and monitor children's mental states and behaviors, providing parents with the information they need to support their children's growth and safety.

[1648] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1649] Step 1:

[1650] The device uses a microphone and a text input device to capture what the child says. For example, the child might say, "I was lonely at school today because my friends weren't there."

[1651] Input: Child's voice or text

[1652] Output: Conversation data in text format

[1653] Step 2:

[1654] The device sends the acquired conversation data to the server using HTTP requests or WebSockets. The data is packaged in JSON format and sent to a specific endpoint (e.g., / api / conversations).

[1655] Input: Text-based conversation data

[1656] Output: JSON formatted conversation data

[1657] Step 3:

[1658] The server receives the conversation data sent from the terminal and stores it in a database management system (e.g., MySQL, PostgreSQL).

[1659] Input: JSON format conversation data

[1660] Output: Conversation data stored in a database

[1661] Step 4:

[1662] The server retrieves conversation data from the database and inputs it into an emotion engine (e.g., NLTK, spaCy) to recognize the child's emotions. The emotion engine performs text analysis to identify emotions such as "lonely."

[1663] Input: Conversation data stored in the database

[1664] Output: Emotion data identified by the emotion engine

[1665] Step 5:

[1666] The server inputs the emotion data analyzed by the emotion engine into a generative AI model to infer the child's mental state. The generative AI model performs analysis using deep learning libraries (e.g., TensorFlow, PyTorch).

[1667] Input: Emotion data

[1668] Output: Inferred mental state data

[1669] Step 6:

[1670] The server then uses the estimated mental state data to notify parents if an abnormal state is detected. Notifications are sent via email, SMS, or a dedicated app (e.g., Amazon SES).

[1671] Input: Inferred mental state data

[1672] Output: Notification sent to parent

[1673] Step 7:

[1674] The server uses a generative AI model to generate learning content based on the child's requests, such as generating a math problem like "What is 5 + 7?"

[1675] Input: Child request data

[1676] Output: Generated learning content

[1677] Step 8:

[1678] The server sends the generated learning content to the device, which then presents it to the child and prompts them to respond.

[1679] Input: Generated learning content

[1680] Output: Learning content sent to the device

[1681] Step 9:

[1682] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1683] Input: Child's answer data

[1684] Output: Answer data sent to the server

[1685] Step 10:

[1686] The server stores the received answer data in a database, analyzes it using a generative AI model, creates feedback, and sends it to the device. For example, if the answer is correct, it displays "That's correct!"

[1687] Input: Child's answer data

[1688] Output: Feedback message

[1689] (Application example 2)

[1690] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1691] With the increase in dual-income households, children are spending more time alone. Under these circumstances, it is difficult to constantly monitor their children's mental state and safety, and there is a particular problem of delayed detection of dangers or abnormal behavior that may occur while parents are away. Furthermore, while it is important to make effective use of the time children spend alone and provide them with learning support, providing appropriate timing and content for such support can be difficult.

[1692] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1693] In this invention, the server includes means for acquiring a voice or text conversation with a child, means for analyzing the acquired conversation data and using an emotion engine to identify the child's emotions, means for inputting the emotion data identified by the emotion engine into a generative AI model to infer the child's mental state, means for notifying the parent of the child's mental state, means for monitoring the child's behavior and detecting abnormalities, and means for immediately notifying the parent if an abnormality is detected. This makes it possible to support the child's safety and mental growth by analyzing the child's mental state and behavior in real time and notifying the parent.

[1694] "Means for capturing voice or text conversations" refers to technology for capturing voice or text spoken by a child using a microphone, text input device, or the like.

[1695] An "emotion engine" refers to software or algorithms that use text and voice analysis to recognize and identify a child's emotions.

[1696] A "generative AI model" is an artificial intelligence model that generates new information and learning content based on acquired data.

[1697] "Means for inferring mental state" refers to a technology that uses emotional data identified by the emotion engine to infer a child's current mental state.

[1698] "Means for notifying parents" refers to technology that uses communication methods such as email and mobile notifications to inform parents of their child's mental state or abnormalities in real time.

[1699] "Means for monitoring behavior and detecting abnormalities" refers to technology that uses cameras and sensors to monitor children's movements and detect abnormal behavior.

[1700] "Means for generating learning content and presenting it to children" refers to technology that uses a generative AI model to generate learning content related to a subject and presents it to children at an appropriate time.

[1701] "Means for analyzing in real time and providing feedback" refers to technology that instantly analyzes acquired voice and behavioral data and provides feedback to the child based on the results.

[1702] This invention addresses the issue of children spending more time alone as the number of dual-income households increases. It provides a system that utilizes a generative AI model and an emotion engine to promote children's development and monitor safety within the home. This system is realized primarily by combining a server and a terminal.

[1703] Server Operation

[1704] 1. Load the generative AI model and emotion engine:

[1705] When the server starts up, it loads the generative AI model and emotion engine into memory. The AI ​​model and emotion engine are software that play a central role in the analysis and generation processes.

[1706] 2. Data Receipt and Storage:

[1707] The system receives the child's conversation and behavior data sent from the device and stores it in a database for later analysis.

[1708] 3. Emotion analysis:

[1709] The received conversation data is input into an emotion engine, which uses text and voice analysis to identify the child's emotions.

[1710] 4. Data Analysis:

[1711] The emotional data analyzed by the emotion engine is input into a generative AI model to predict the child's mental state. The generative AI model then uses the received data to more accurately predict the child's mental state.

[1712] 5. Notification function:

[1713] If an abnormal mental state is detected based on the analysis results, a notification will be sent to the parents via email or mobile notification.

[1714] 6. Learning content generation:

[1715] Based on the child's request, appropriate learning content is generated using a generative AI model and sent to the device.

[1716] 7. Behavioral Monitoring and Anomaly Detection:

[1717] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is immediately notified.

[1718] Device behavior

[1719] 1. Conversation Acquisition:

[1720] The system captures the child's speech using a microphone or text input device, and the captured data is sent to a server in real time.

[1721] 2. Data transmission:

[1722] The acquired conversation data and behavioral data are sent to the server.

[1723] 3. Learning Support:

[1724] The learning content received from the server is presented to the child, encouraging the child to study.

[1725] 4. Condition Monitoring:

[1726] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[1727] Specific examples

[1728] 1. Example conversation with a child:

[1729] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[1730] The terminal transmits this message to the server.

[1731] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[1732] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[1733] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[1734] 2. Learning support examples:

[1735] The user (child) tells the device, "I want to study math now."

[1736] The terminal sends this request to the server.

[1737] The server stores the received requests in a database and inputs them into a generative AI model.

[1738] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[1739] The server sends the generated questions to the terminal.

[1740] The terminal presents the received question to the child and waits for the child's answer.

[1741] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1742] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[1743] 3. Condition monitoring example:

[1744] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[1745] The device transmits behavioral data to the server in real time.

[1746] The server stores the received behavioral data in a database.

[1747] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[1748] The server sends an immediate notification to the parent if it detects an anomaly, such as an abnormally long period of inactivity or suspicious behavior.

[1749] Prompt Sentence Examples

[1750] Emotion Analysis Prompt: "Analyze this audio data to identify the emotion."

[1751] Learning content generation prompt: "Generate one basic math problem, such as an addition or subtraction problem."

[1752] This system uses a generative AI model and an emotion engine to analyze a child's mental state and behavior in real time, and provides parents with the information they need to support their child's growth and safety.

[1753] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1754] Step 1:

[1755] Starting the server and loading the model

[1756] When the server starts up, it loads the generative AI model and emotion engine into memory. This step makes the necessary software components available. The input is a server configuration file containing the generative AI model and emotion engine, and the output is the model and engine loaded into the server memory. Specifically, the server executes the model loading routine and allocates memory space.

[1757] Step 2:

[1758] Retrieving and sending conversation data

[1759] The device captures voice and text conversations from the user (child). As input, voice and text data are obtained using a microphone and text input device, and this is sent to the server. The output is raw data sent to the server. Specifically, the device captures data through a voice recording or text input interface, and sends the data to the server using a communication protocol.

[1760] Step 3:

[1761] Sentiment analysis of conversation data

[1762] The server inputs the received conversation data into an emotion engine to identify the child's emotions. The input is voice or text data sent from the device, which is analyzed by the emotion engine. The output is the identified emotion data. Specifically, the server runs the emotion engine's analysis algorithm and classifies emotions using text analysis and voice analysis technology.

[1763] Step 4:

[1764] Mental state inference from emotional data

[1765] The server inputs the emotion data identified by the emotion engine into the generative AI model to infer the child's mental state. The input is emotion data from the emotion engine, which is adapted and processed by the generative AI model. The output is the child's inferred mental state. Specifically, the server executes the generative AI model's data processing algorithm to infer the child's mental state based on the emotion data.

[1766] Step 5:

[1767] Mental status notification

[1768] The server sends a notification to the parent if an abnormality is detected based on the child's mental state. The input is the mental state data inferred by the generative AI model, and the information is conveyed to the parent via the notification system. The output is a notification message sent to the parent's device. Specifically, the server invokes the notification function to send an email or mobile notification.

[1769] Step 6:

[1770] Receiving a learning content generation request

[1771] When a user (child) inputs a learning request into the device, the device sends this request to the server. The input is voice or text data based on the user's learning request, which is sent to the server. The output is the request data sent to the server. Specifically, the device captures the request data using a voice or text input interface and sends it to the server using a communication protocol.

[1772] Step 7:

[1773] Generation and presentation of learning content

[1774] The server uses the generative AI model to generate appropriate learning content based on the received request and sends it to the device. The input is the learning request data from the user and the generative AI model, and learning content is created based on this. The output is the data that sends the generated learning content to the device. Specifically, the server uses the prompt sentences from the generative AI model to generate math problems and reading comprehension content and sends it to the device.

[1775] Step 8:

[1776] Implementing learning support

[1777] The device presents the learning content received from the server to the user (child) and allows them to work on it. The input is the learning content sent from the server, which is displayed on the user interface. The output is the answer data entered by the user. Specifically, the device displays the learning content using a display and speech synthesis function, and obtains the user's answers from the input device.

[1778] Step 9:

[1779] Monitoring and sending behavioral data

[1780] The device monitors the user's (child's) behavior using cameras and sensors and sends the data to a server. The input is the behavioral data acquired from the cameras and sensors, which is sent to the server. The output is the behavioral data sent to the server. Specifically, the device acquires video and sensor information in real time and uses a communication protocol to send it to the server.

[1781] Step 10:

[1782] Anomaly detection in behavioral data

[1783] The server analyzes the received behavioral data and immediately sends a notification to the parent if an abnormality is detected. The input is the behavioral data sent from the device, and an algorithm is applied to detect abnormalities based on this. The output is a notification message that is sent to the parent if an abnormality is detected. Specifically, the server executes the behavioral analysis algorithm, and if an abnormality is discovered, it notifies the parent via the notification system.

[1784] Through these steps, this system utilizes a generative AI model and emotion engine to analyze children's mental state and behavior in real time and provide parents with the necessary information to support their children's growth and safety.

[1785] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1786] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1787] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1788] [Fourth embodiment]

[1789] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1790] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1791] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1792] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1793] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1794] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1795] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1796] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1797] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1798] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1799] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1800] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1801] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1802] This invention addresses the issue of children spending more time alone as the number of dual-income households increases, by providing a system that utilizes generative AI to promote children's development and monitor safety within the home.

[1803] System configuration and operation

[1804] Server Operation

[1805] The server hosts the generative AI model and database management system and performs the following operations:

[1806] 1. Load the generative AI model:

[1807] The server loads the generative AI model into memory at startup.

[1808] 2. Data Receipt and Storage:

[1809] The system receives the child's conversation data sent from the device and stores it in a database.

[1810] 3. Data Analysis:

[1811] The received conversation data is input into a generative AI model to analyze the child's mental state.

[1812] 4. Notification function:

[1813] Based on the analysis results, if an abnormal mental state is detected, a notification will be sent to the parents.

[1814] 5. Learning content generation:

[1815] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[1816] 6. Behavioral monitoring and anomaly detection:

[1817] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is notified.

[1818] Device behavior

[1819] The device mediates interactions with the child and performs the following actions:

[1820] 1. Conversation Acquisition:

[1821] The child's speech is captured using a microphone or text input device.

[1822] 2. Data transmission:

[1823] The acquired conversation data is sent to the server.

[1824] 3. Learning Support:

[1825] The learning content received from the server is presented to the child and they are prompted to respond.

[1826] 4. Condition Monitoring:

[1827] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[1828] Usage example

[1829] 1. Example conversation with a child:

[1830] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[1831] The terminal transmits this message to the server.

[1832] The server stores the received data and analyzes the content of the statements using a generative AI model.

[1833] The server identifies the mental state "lonely" and sends a notification to the parent that "child feels lonely."

[1834] 2. Learning support examples:

[1835] The user (child) tells the device, "I want to study math now."

[1836] The terminal sends this request to the server.

[1837] The server uses a generative AI model to generate appropriate math problems and sends them to the device.

[1838] The device presents questions to the child and prompts them to answer.

[1839] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[1840] 3. Condition monitoring example:

[1841] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[1842] The device transmits behavioral data to the server in real time.

[1843] The server analyzes the received behavioral data and notifies the parent if any abnormalities are detected.

[1844] summary

[1845] In this way, the system of this invention utilizes generative AI models to support children's development and monitor their safety, providing an environment where parents in dual-income households can feel secure knowing what is happening with their children. The specific implementation is programmed, interacting with children in real time, analyzing their mental state and behavior based on accumulated data, and providing necessary notifications and learning content. This makes children's alone time safer and more meaningful.

[1846] The processing flow will be explained below.

[1847] Specific explanation of system program processing

[1848] Retrieving conversations and sending data

[1849] Step 1:

[1850] The device receives the child's speech via a microphone or text input device. For example, the device receives a speech such as, "I was lonely at school today because my friends were not there."

[1851] Step 2:

[1852] The device sends the acquired speech data to the server, along with the speech content and a timestamp.

[1853] Step 3:

[1854] The server stores the data received from the device in a database, including the content of the message and a timestamp.

[1855] Step 4:

[1856] The server inputs the received data into a generative AI model to analyze the child's mental state. In this case, the emotion of "lonely" is identified.

[1857] Step 5:

[1858] The server stores the analysis results in a database and sends a notification to the parents if the child's mental state is determined to be "lonely."

[1859] Learning support and dialogue

[1860] Step 1:

[1861] The user (child) tells the device, "I want to study math now."

[1862] Step 2:

[1863] The device sends this request to the server, along with the request content and a timestamp.

[1864] Step 3:

[1865] The server stores the received requests in a database and inputs them into a generative AI model.

[1866] Step 4:

[1867] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[1868] Step 5:

[1869] The server sends the generated math problem to the terminal.

[1870] Step 6:

[1871] The device presents the received question to the child and waits for the child's answer.

[1872] Step 7:

[1873] The user (child) enters the answer into the terminal, and the terminal sends the answer to the server.

[1874] Step 8:

[1875] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[1876] Condition monitoring and anomaly detection

[1877] Step 1:

[1878] The device monitors children's behavior in real time through cameras and sensors.

[1879] Step 2:

[1880] The device sends the monitoring data, including the activity and a timestamp, to the server.

[1881] Step 3:

[1882] The server stores the received behavioral data in a database.

[1883] Step 4:

[1884] The server analyzes the accumulated data using a generative AI model and compares it with the normal range of behavior.

[1885] Step 5:

[1886] If the server detects an anomaly, it will immediately send a notification to the parent, for example if the device is stationary for an abnormally long period of time or if it is behaving suspiciously.

[1887] This allows the system to continuously monitor a child's mental state and behavior through daily interactions with the child and support their learning, and to support the child's growth and safety by promptly notifying parents if any abnormalities or dangers occur.

[1888] Example 1

[1889] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1890] With the increase in dual-income households, children spend more time alone, creating the issue of the need to properly monitor their growth and safety. Furthermore, there is a lack of effective ways to monitor children's mental health and academic progress. This creates a need for a system that allows parents to monitor their children's condition while at work and respond quickly if there is an abnormality.

[1891] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1892] In this invention, the server includes means for acquiring conversations with children, means for transmitting the acquired conversation data to the server, means for storing the received conversation data in a database, means for analyzing the child's mental state using a generative AI model based on the stored conversation data, and means for notifying parents if an abnormal mental state is detected based on the analysis results. This enables parents in dual-income households to understand their children's mental state and behavior in real time even while at work, and to provide appropriate learning support and safety checks.

[1893] "Children" refers to minors who are eligible to use the System.

[1894] "Conversation data" refers to a recording of a child's speech in the form of text or audio data.

[1895] "Server" refers to the central computing unit that hosts the generative AI model and receives, stores, and analyzes data.

[1896] A "generative AI model" refers to a program model that uses artificial intelligence technology to perform natural language processing and data analysis.

[1897] "Database" refers to an information management system for storing received conversation data and behavioral data.

[1898] "Mental state" refers to the emotional and psychological state that can be inferred from a child's words and behavior.

[1899] "Notification" refers to an alert or informational message sent from the server to the parent.

[1900] "Learning content" refers to the learning questions and materials generated by the generative AI model in response to the child's requests.

[1901] "Behavioral data" refers to data such as a child's movements and location collected using cameras and sensors.

[1902] "Abnormal" refers to a child's behavior or mental state that is different from normal and indicates an unnatural or dangerous condition.

[1903] The present invention is a system that supports the safety and development of children by addressing the increasing amount of time they spend alone due to the increase in dual-income households. This system includes a device that interacts with children and a server that analyzes data and sends notifications.

[1904] Hardware and software used

[1905] Terminals consist of interaction devices equipped with input devices such as microphones, cameras, sensors, and touchscreens. Examples include general-purpose tablets, smart speakers, and surveillance cameras.

[1906] The server is a central computing unit that hosts the generative AI model and receives, stores, and analyzes data. Software used includes deep learning frameworks (e.g., TensorFlow, PyTorch), database management systems (e.g., MySQL, PostgreSQL), and various notification APIs (e.g., Twilio, SendGrid).

[1907] Conversation capture and analysis

[1908] The device uses a microphone and text input device to capture the child's speech. For example, if a child says, "I missed my friends at school today," the device uses voice recognition software to convert this speech into text data. The converted text data is then securely transmitted to a server via the HTTPS protocol.

[1909] The server stores the received text data in a database such as MySQL. It then analyzes the data using a generative AI model (e.g., BERT, GPT-3). At this time, the prompt "Please analyze the mental state when the user tells a sad story" is input to the model.

[1910] Mental state detection and notification

[1911] The server determines the child's mental state based on the analysis results. For example, if the emotion "lonely" is detected, a notification is sent to the parent using Twilio or SendGrid. The notification includes a message saying "Your child is feeling lonely."

[1912] Creating and presenting learning content

[1913] When a user (child) says to the device, "I want to study math now," the device sends this request to the server.

[1914] The server uses a generative AI model to generate appropriate math problems. The prompt "Generate simple math problems for elementary school students" is input to the model. The generated math problems are sent back to the device, which presents them to the child. The child enters the answers, and the answer data is sent back to the server.

[1915] Behavioral data monitoring and analysis

[1916] The device continuously monitors the child's behavior using cameras and sensors, for example, recording the child's behavior while they are in their room, and transmitting the monitoring data to a server in real time.

[1917] The server analyzes the received behavioral data and notifies the parent if abnormal behavior is detected. For example, if a child remains motionless for a long time, a notification stating "Child has not moved for a long time" is sent to the parent.

[1918] In this way, the system of the present invention utilizes a generative AI model to analyze a child's mental state and behavior, and provides parents with the necessary notifications and learning support, thereby providing an environment where parents in dual-income households can feel at ease knowing what is happening with their children.

[1919] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1920] Step 1: Obtaining conversation data

[1921] The device uses a microphone and text input device to capture what the child says. For example, if a child says, "I missed my friends at school today," the device uses voice recognition software to convert this speech into text data.

[1922] (Input) Child's speech (audio or text)

[1923] (Output) Text data (in the case of audio, data converted to text)

[1924] Step 2: Sending conversation data to the server

[1925] The device sends the retrieved text data to the server via an HTTP POST request, with the data being transferred securely using SSL encryption.

[1926] (Input) Text data

[1927] (Output) None (Transfer to server)

[1928] Step 3: Save your data

[1929] The server stores the received text data in MySQL (or other database management systems), including information such as what the child said, when they said it, and the speaker's ID.

[1930] (Input) Text data (transferred data)

[1931] (Output) Save data to database (persistent storage)

[1932] Step 4: Analyzing the conversation data

[1933] The server inputs the saved text data into a generative AI model to analyze the child's mental state. Specifically, the model is given a prompt sentence: "Please analyze the child's mental state when the user tells a sad story." This analysis uses the natural language processing capabilities of BERT and GPT-3.

[1934] (Input) Text data (saved conversation data), prompt sentence

[1935] (Output) Analysis results (mental state)

[1936] Step 5: Mental status notification

[1937] The server evaluates the child's mental state based on the analysis results, and if an abnormal state is detected, it immediately notifies the parent. For example, if the emotion "lonely" is detected, a notification stating "your child is feeling lonely" is sent to the parent using Twilio or SendGrid.

[1938] (Input) Analysis results (mental state)

[1939] (Output) Notification message (notification to parent)

[1940] Step 6: Request learning content

[1941] The user (child) requests learning content by telling the device, "I want to study math now."

[1942] (Input) Request statement ("I want to study math now.")

[1943] (Output) Request data (transferred to server)

[1944] Step 7: Generate learning content

[1945] The server receives the learning request and generates appropriate math problems using a generative AI model. For example, it generates problems by inputting a prompt sentence such as "Generate simple math problems for elementary school students."

[1946] (Input) Request data, prompt

[1947] (Output) Learning content (math problems)

[1948] Step 8: Present learning content and submit answers

[1949] The device presents the child with a mathematical problem received from the server and prompts the child to answer it. The child enters the answer into the device, which then transmits the answer to the server.

[1950] (Input) Learning content (math problems)

[1951] (Output) Answer data (transferred to server)

[1952] Step 9: Monitor behavioral data

[1953] The device uses cameras and sensors to continuously monitor a child's behavior, for example by recording their movements and location in real time while they are in their room.

[1954] (Input) Behavioral data (data from cameras and sensors)

[1955] (Output) Behavioral data (transferred to server)

[1956] Step 10: Analyze behavioral data and detect anomalies

[1957] The server analyzes the received behavioral data and notifies parents if abnormal behavior is detected. For example, if a child stays in the same place for a long time, the server sends a message saying, "Your child has not moved for a long time."

[1958] (Input) Behavioral data (real-time monitored data)

[1959] (Output) Notification message (notification to parent)

[1960] (Application example 1)

[1961] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1962] With the increase in dual-income households, children are spending more time alone, making it necessary to provide appropriate supervision and learning support to ensure their growth and safety. It is also important to understand children's mental state and behavior in real time and detect abnormalities early. The present invention aims to provide a system that solves these problems.

[1963] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1964] In this invention, the server includes means for acquiring conversation data with a child, means for analyzing the acquired conversation data and using a generative AI model to infer the child's mental state, means for notifying a parent of the child's mental state, means for generating appropriate learning content and presenting it to the child, means for monitoring the child's behavior and detecting abnormalities, means for immediately notifying the parent when an abnormality is detected, and means for generating appropriate prompt sentences and inputting them into the generative AI model. This makes it possible to provide necessary learning support while monitoring the child's mental state and behavior in real time at home, and to respond quickly when an abnormality is detected.

[1965] "Conversation with children" refers to communication within the home that includes words and sentences spoken by children.

[1966] "Means of acquisition" refers to the functionality for collecting data using a microphone, text input device, or other sensor device.

[1967] A "generative AI model" is an artificial intelligence algorithm or software that generates new information or content from original data through natural language processing and data analysis.

[1968] "Mental state" refers to a child's psychological health and emotional changes, and is an indicator for assessing that state.

[1969] "Means of notifying parents" refers to the means of communication to promptly inform parents of analysis results or abnormal situations, including email, SMS, in-app notifications, etc.

[1970] "Learning content" refers to questions, reading materials, and other content designed to support a child's education and intellectual development.

[1971] "Presentation means" is a function for showing the generated learning content to children or conveying it to them by voice.

[1972] "Behavioral monitoring measures" are features that use cameras and sensors to continuously observe a child's physical movements and location.

[1973] "Means for detecting anomalies" refers to the ability to identify unusual behavior or situations and issue a warning when danger or problems occur.

[1974] A "prompt" is an instruction or question used to direct a generative AI model to perform a specific analysis or generation.

[1975] "Smart device" means a device with internet connectivity and advanced computing capabilities, including smartphones and tablets.

[1976] The present invention provides a system that uses a generative AI model to support the development and safety of children in dual-income households. This system is composed of a server and a terminal, and is realized using the following means.

[1977] Server Operation

[1978] The server hosts the generative AI model and database management system and performs the following operations:

[1979] 1. Load the generative AI model:

[1980] The server loads the generative AI model into memory at startup, allowing it to analyze conversational and behavioral data in real time.

[1981] 2. Data Receipt and Storage:

[1982] Conversation data and behavioral data with the child sent from the device is received and stored in a database.

[1983] 3. Data Analysis:

[1984] The received conversation data is input into a generative AI model to analyze the child's mental state, and based on this analysis, a notification is sent to the parent.

[1985] 4. Learning content generation:

[1986] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[1987] 5. Behavioral monitoring and anomaly detection:

[1988] The received behavioral data is analyzed, and if any abnormalities are detected, a notification is sent to the parent.

[1989] Device behavior

[1990] The device mediates interactions with the child and performs the following actions:

[1991] 1. Conversation Acquisition:

[1992] The child's speech is captured using a microphone or text input device.

[1993] 2. Data transmission:

[1994] The acquired conversation data is sent to the server.

[1995] 3. Learning Support:

[1996] The learning content received from the server is presented to the child and they are prompted to respond.

[1997] 4. Condition Monitoring:

[1998] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[1999] Specific use cases

[2000] 1. Example conversation with a child:

[2001] The child says to the device, "I was lonely at school today because my friends weren't there."

[2002] The terminal transmits this message to the server.

[2003] The server stores the received data and analyzes the content of the statements using a generative AI model.

[2004] The server identifies the mental state "lonely" and sends a notification to the parent that "child feels lonely."

[2005] 2. Learning support examples:

[2006] The child tells the device, "I want to study math now."

[2007] The terminal sends this request to the server.

[2008] The server uses a generative AI model to generate appropriate math problems and sends them to the device.

[2009] The device presents questions to the child and prompts them to answer.

[2010] The child enters the answer into the terminal, which then sends the answer to the server.

[2011] 3. Condition monitoring example:

[2012] Cameras and sensors monitor children's behavior while they are in their room.

[2013] The device transmits behavioral data to the server in real time.

[2014] The server analyzes the received behavioral data and notifies the parent if any abnormalities are detected.

[2015] Prompt Sentence Examples

[2016] "Analyze your child's mental state when they say, 'I missed my friends at school today.'"

[2017] "Generate math problems suitable for a child who said they want to start studying math now."

[2018] "Determine whether your child's behavior in the room is abnormal."

[2019] As described above, the system of the present invention utilizes a generative AI model to support children's growth and monitor their safety, thereby providing an environment where parents in dual-income households can keep track of their children's situations with peace of mind.

[2020] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2021] Step 1:

[2022] Loading a generative AI model

[2023] When the server starts up, it loads the generative AI model into memory. In this process, the model file is read from disk and expanded into memory. The input is the path to the model file, and the output is the loaded generative AI model. This allows subsequent data analysis processing to be performed.

[2024] Step 2:

[2025] Conversation Acquisition

[2026] The device captures conversations with the child through a microphone and text input device. In this process, voice data and text data are input and the captured conversation data is output. This allows the necessary conversation content to be sent to the server.

[2027] Step 3:

[2028] Sending conversation data

[2029] The terminal transmits the acquired conversation data to the server. In this process, the conversation data is input and the data transmitted to the server is output, allowing the server to analyze the conversation data.

[2030] Step 4:

[2031] Conversation data analysis

[2032] The server inputs the transmitted conversation data into a generative AI model for analysis. In this process, the conversation data is input and an analysis result such as "child's mental state" is output. Specifically, the generative AI model infers emotions and mental state from the content of the conversation and generates the result.

[2033] Step 5:

[2034] Parental Notification

[2035] Based on the analysis results, the server will send a notification to the parents if it detects an abnormal mental state, such as loneliness, in the process. This process inputs the analysis results and outputs a notification message. The notification will be sent via email or in-app notification.

[2036] Step 6:

[2037] Generating learning content

[2038] The server generates math problems and storytelling content in response to the child's request. This process takes a child's request as input and outputs generated learning content. Using a generative AI model, appropriate learning content is generated based on the prompt.

[2039] Step 7:

[2040] Presentation of learning content

[2041] The device presents the learning content received from the server to the child and prompts them to respond. In this process, the learning content is input and the results presented to the child are output. Specifically, the learning content is presented using a screen display or a voice read-out function.

[2042] Step 8:

[2043] Answers to learning content

[2044] The user (child) inputs the answer into the terminal. In this process, the child's answer is input and the answer sent to the terminal is output.

[2045] Step 9:

[2046] Sending answer data

[2047] The terminal transmits the child's answer to the server. In this process, answer data is input and the data transmitted to the server is output.

[2048] Step 10:

[2049] Analysis of answer data

[2050] The server analyzes the submitted answer data and determines whether it is correct. In this process, the answer data is input and the analysis results are output. A generative AI model is used to evaluate the correctness of the answer and generate feedback.

[2051] Step 11:

[2052] Behavioral monitoring

[2053] The device continuously monitors the child's behavior using cameras and sensors, and the process inputs behavioral data and outputs real-time monitored data.

[2054] Step 12:

[2055] Sending behavioral data

[2056] The terminal transmits the monitored behavioral data in real time to the server, where the behavioral data is input and the data transmitted to the server is output.

[2057] Step 13:

[2058] Analysis of behavioral data

[2059] The server analyzes the received behavioral data and detects anomalies. In this process, behavioral data is input and anomaly detection results are output.

[2060] Step 14:

[2061] Parent notification when an abnormality is detected

[2062] The server will immediately notify the parent if an anomaly is detected. This process inputs the anomaly detection result and outputs a notification message. Notifications can be sent via email or in-app notifications.

[2063] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2064] This invention addresses the issue of children spending more time alone as the number of dual-income households increases, by providing a system that utilizes generative AI and an emotion engine to promote children's development and monitor safety within the home.

[2065] System configuration and operation

[2066] Server Operation

[2067] The server hosts the generative AI model, emotion engine, and database management system, and performs the following operations:

[2068] 1. Load the generative AI model:

[2069] The server loads the generative AI model and emotion engine into memory at startup.

[2070] 2. Data Receipt and Storage:

[2071] The system receives the child's conversation data sent from the device and stores it in a database.

[2072] 3. Emotion analysis:

[2073] The received conversation data is input into an emotion engine to recognize the child's emotions. The emotion engine identifies emotions using, for example, text analysis or voice analysis.

[2074] 4. Data Analysis:

[2075] Emotional data analyzed by the emotion engine is input into a generative AI model to more accurately predict a child's mental state.

[2076] 5. Notification function:

[2077] Based on the analysis results, if an abnormal mental state is detected, a notification will be sent to the parents.

[2078] 6. Learning content generation:

[2079] Based on the child's request, learning content such as math problems and reading content is generated and sent to the device.

[2080] 7. Behavioral monitoring and anomaly detection:

[2081] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is notified.

[2082] Device behavior

[2083] The device mediates interactions with the child and performs the following actions:

[2084] 1. Conversation Acquisition:

[2085] The child's speech is captured using a microphone or text input device.

[2086] 2. Data transmission:

[2087] The acquired conversation data is sent to the server.

[2088] 3. Learning Support:

[2089] The learning content received from the server is presented to the child and they are prompted to respond.

[2090] 4. Condition Monitoring:

[2091] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[2092] Usage example

[2093] 1. Example conversation with a child:

[2094] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[2095] The terminal transmits this message to the server.

[2096] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[2097] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[2098] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[2099] 2. Learning support examples:

[2100] The user (child) tells the device, "I want to study math now."

[2101] The terminal sends this request to the server.

[2102] The server stores the received requests in a database and inputs them into a generative AI model.

[2103] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[2104] The server sends the generated mathematical problem to the terminal.

[2105] The terminal presents the received question to the child and waits for the child's answer.

[2106] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[2107] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[2108] 3. Condition monitoring example:

[2109] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[2110] The device transmits behavioral data to the server in real time.

[2111] The server stores the received behavioral data in a database.

[2112] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[2113] The server immediately sends a notification to the parent if it detects an anomaly, such as an abnormally long period of inactivity or suspicious behavior.

[2114] This system utilizes a generative AI model and emotion engine to more accurately predict and monitor a child's mental state and behavior, and provides necessary information to parents, supporting their child's development and safety. The specific implementation is programmed, and it is possible to interact with children in real time.

[2115] The processing flow will be explained below.

[2116] A detailed explanation of the system's program processing (including the emotion engine)

[2117] Retrieving conversations and sending data

[2118] Step 1:

[2119] The device receives the child's speech via a microphone or text input device. For example, the device receives a speech such as, "I was lonely at school today because my friends were not there."

[2120] Step 2:

[2121] The device sends the acquired speech data to the server, along with the speech content and a timestamp.

[2122] Step 3:

[2123] The server stores the data received from the device in a database, including the content of the message and a timestamp.

[2124] Step 4:

[2125] The server inputs the received speech data into an emotion engine to analyze the child's emotions. The emotion engine recognizes emotions such as "lonely," "happy," and "sad" based on the characteristics of the text and voice.

[2126] Step 5:

[2127] The server inputs the emotion analysis data from the emotion engine into the generative AI model to more accurately predict the child's mental state. In this case, the mental state of "lonely" is identified.

[2128] Step 6:

[2129] The server stores the analysis results in a database, and if the child's mental state is determined to be "lonely," it sends a notification to the parents that "the child is feeling lonely."

[2130] Learning support and dialogue

[2131] Step 1:

[2132] The user (child) tells the device, "I want to study math now."

[2133] Step 2:

[2134] The device sends this request to the server, along with the request content and a timestamp.

[2135] Step 3:

[2136] The server stores the received requests in a database and inputs them into a generative AI model.

[2137] Step 4:

[2138] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[2139] Step 5:

[2140] The server sends the generated math problem to the terminal.

[2141] Step 6:

[2142] The device presents the received question to the child and waits for the child's answer.

[2143] Step 7:

[2144] The user (child) enters the answer into the terminal, and the terminal sends the answer to the server.

[2145] Step 8:

[2146] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it sends the feedback "That's correct!"

[2147] Condition monitoring and anomaly detection

[2148] Step 1:

[2149] The device monitors children's behavior in real time through cameras and sensors.

[2150] Step 2:

[2151] The device sends the monitoring data to the server, which includes the activity details and a timestamp.

[2152] Step 3:

[2153] The server stores the received behavioral data in a database.

[2154] Step 4:

[2155] The server analyzes the accumulated data and real-time data using a generative AI model and emotion engine, and compares it with the normal range of behavior.

[2156] Step 5:

[2157] If the server detects an anomaly, it will immediately send a notification to the parent, for example if the device is stationary for an abnormally long period of time or if it is behaving suspiciously.

[2158] Emotion analysis using an emotion engine

[2159] Step 1:

[2160] The device captures the child's speech through a microphone and text input device.

[2161] Step 2:

[2162] The terminal transmits the acquired speech data to the server.

[2163] Step 3:

[2164] The server inputs the received utterance data into the emotion engine.

[2165] Step 4:

[2166] The emotion engine analyzes the content of the speech and identifies emotion labels, such as "lonely," "happy," and "sad."

[2167] Step 5:

[2168] The server inputs the emotion labels obtained from the emotion engine into a generative AI model to more accurately predict the child's mental state.

[2169] Step 6:

[2170] The server sends notifications to parents based on information obtained from the emotion engine and generative AI model, and also generates and presents learning content and reading content as needed.

[2171] In this way, this system, which combines an emotion engine, can more accurately grasp a child's mental state and provide the necessary support and notifications, allowing them to spend more secure and meaningful time together.

[2172] Example 2

[2173] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2174] With the increase in dual-income households, there is a problem that children spend more time alone. In such situations, it is difficult for parents to grasp changes in their children's mental state and behavior in real time, raising concerns about their children's growth and safety. Furthermore, there is often a lack of appropriate support when children study alone. This invention solves these problems and provides a system that supports children's growth and safety.

[2175] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring a conversation with a child, a means for analyzing the acquired conversation data and using an emotion engine to recognize the child's emotions, a means for using a generative AI model to infer the child's mental state based on the recognized emotion data, and a means for notifying the parent of the child's mental state. This makes it possible to grasp the child's emotions and mental state in real time and to immediately notify the parent if an abnormality is detected. In addition, the generative AI model generates learning content related to a subject and presents it to the child, thereby realizing learning support. This makes it possible to comprehensively support the child's growth and safety.

[2176] "Means for acquiring a conversation with a child" refers to an input device such as a microphone or text input device for acquiring a child's speech as voice or text.

[2177] "Emotion engine that analyzes acquired conversation data and recognizes children's emotions" refers to a software component that uses text analysis and voice analysis of input conversation data to recognize children's emotions.

[2178] "A generative artificial intelligence model that predicts a child's mental state based on recognized emotional data" refers to a machine learning model that uses emotion recognition results as input to predict a child's mental state and psychological tendencies.

[2179] "Means of notifying parents of their child's mental state" refers to communication functions that send notifications to parents via email, SMS, or a dedicated app if an abnormal mental state is detected.

[2180] "Means for generating learning content related to a subject and presenting it to a child" refers to a function for generating learning questions and content using a generative AI model based on the child's requests and learning situation, and presenting it to the child.

[2181] "Means for monitoring children's behavior and detecting abnormalities" refers to a system that uses cameras and sensors to collect children's behavioral data in real time, analyzes it using machine learning models, and detects abnormalities.

[2182] "Means for immediately notifying parents if abnormalities are detected" refers to a communication function for quickly sending a notification to parents when abnormal behavior is detected.

[2183] This invention addresses the issue of children spending more time alone as the number of dual-income households increases. It provides a system that utilizes a generative AI model and an emotion engine to promote children's development and monitor safety within the home.

[2184] System Configuration

[2185] Server Operation

[2186] The server hosts the generative AI model, emotion engine, and database management system, and implements the following key functions:

[2187] 1. Load the generative AI model:

[2188] The server loads the generative AI model and emotion engine into memory at startup, enabling rapid data processing and analysis using deep learning libraries such as TensorFlow and PyTorch.

[2189] 2. Data Receipt and Storage:

[2190] The server receives the child's conversation data sent from the device and stores it in a database using HTTP requests and WebSockets. MySQL or PostgreSQL is used as the database management system.

[2191] 3. Emotion analysis:

[2192] The server inputs the received conversation data into an emotion engine to recognize the child's emotions. The emotion engine uses a natural language processing library (e.g., NLTK or spaCy) to analyze the text and identify emotions.

[2193] 4. Data Analysis:

[2194] The server inputs the emotional data analyzed by the emotion engine into a generative AI model to predict the child's mental state. The generative AI model then comprehensively analyzes past and real-time data to predict the child's mental state.

[2195] 5. Notification function:

[2196] Based on the analysis results, the server will send a notification to the parents if an abnormal mental state is detected. Notifications can be sent via email, SMS, or a dedicated app. For example, Amazon SES (Simple Email Service) can be used.

[2197] 6. Learning content generation:

[2198] The server generates learning content based on the child's request and sends it to the device. For example, math problems or reading content may be generated. For example, a question such as "Please solve the following equation: What is 5 + 7?" may be generated.

[2199] 7. Behavioral monitoring and anomaly detection:

[2200] The server analyzes the behavioral data received from the device and notifies the parent if an abnormality is detected. The behavioral data includes data obtained from cameras and motion sensors.

[2201] Device behavior

[2202] The device mediates interactions with the child and implements the following functions in cooperation with the server:

[2203] 1. Conversation Acquisition:

[2204] The device uses a microphone and text input device to capture the child's speech, which is then converted into text data using voice recognition software (e.g., Google Speech-to-Text API).

[2205] 2. Data transmission:

[2206] The device sends the acquired conversation data to the server via HTTP requests or WebSockets. The data is packaged in JSON format and sent to a specific endpoint.

[2207] 3. Learning Support:

[2208] The device receives learning content from the server and presents it to the child, prompting them to answer. For example, generated math problems are displayed on the device screen, allowing the child to enter their answers.

[2209] 4. Condition Monitoring:

[2210] The device uses cameras and sensors to monitor the child's behavior and notifies the server if it detects anything unusual, such as if the child remains motionless for a long period of time or if it detects suspicious movements.

[2211] Usage example

[2212] 1. Example conversation with a child:

[2213] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[2214] The terminal transmits this message to the server.

[2215] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[2216] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[2217] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[2218] 2. Learning support examples:

[2219] The user (child) tells the device, "I want to study math now."

[2220] The terminal sends this request to the server.

[2221] The server stores the received requests in a database and inputs them into a generative AI model.

[2222] The server uses a generative AI model to generate appropriate math problems, such as "Please solve the following equation: What is 5 + 7?"

[2223] The server sends the generated mathematical problem to the terminal.

[2224] The terminal presents the received question to the child and waits for the child's answer.

[2225] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[2226] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[2227] 3. Condition monitoring example:

[2228] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[2229] The device transmits behavioral data to the server in real time.

[2230] The server stores the received behavioral data in a database.

[2231] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[2232] The server will immediately send a notification to its parent if it detects an anomaly, for example if it is quiet for an abnormally long period of time.

[2233] Example prompt

[2234] 1. "Explain what to do when your child tells you they're lonely and don't have any friends."

[2235] 2. "Explain how the system responds to a child's request to learn math."

[2236] This allows the system to utilize generative AI models and emotion engines to more accurately predict and monitor children's mental states and behaviors, providing parents with the information they need to support their children's growth and safety.

[2237] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2238] Step 1:

[2239] The device uses a microphone and a text input device to capture what the child says. For example, the child might say, "I was lonely at school today because my friends weren't there."

[2240] Input: Child's voice or text

[2241] Output: Conversation data in text format

[2242] Step 2:

[2243] The device sends the acquired conversation data to the server using HTTP requests or WebSockets. The data is packaged in JSON format and sent to a specific endpoint (e.g., / api / conversations).

[2244] Input: Text-based conversation data

[2245] Output: JSON formatted conversation data

[2246] Step 3:

[2247] The server receives the conversation data sent from the terminal and stores it in a database management system (e.g., MySQL, PostgreSQL).

[2248] Input: JSON format conversation data

[2249] Output: Conversation data stored in a database

[2250] Step 4:

[2251] The server retrieves conversation data from the database and inputs it into an emotion engine (e.g., NLTK, spaCy) to recognize the child's emotions. The emotion engine performs text analysis to identify emotions such as "lonely."

[2252] Input: Conversation data stored in the database

[2253] Output: Emotion data identified by the emotion engine

[2254] Step 5:

[2255] The server inputs the emotion data analyzed by the emotion engine into a generative AI model to infer the child's mental state. The generative AI model performs analysis using deep learning libraries (e.g., TensorFlow, PyTorch).

[2256] Input: Emotion data

[2257] Output: Inferred mental state data

[2258] Step 6:

[2259] The server then uses the estimated mental state data to notify parents if an abnormal state is detected. Notifications are sent via email, SMS, or a dedicated app (e.g., Amazon SES).

[2260] Input: Inferred mental state data

[2261] Output: Notification sent to parent

[2262] Step 7:

[2263] The server uses a generative AI model to generate learning content based on the child's requests, such as generating a math problem like "What is 5 + 7?"

[2264] Input: Child request data

[2265] Output: Generated learning content

[2266] Step 8:

[2267] The server sends the generated learning content to the device, which then presents it to the child and prompts them to respond.

[2268] Input: Generated learning content

[2269] Output: Learning content sent to the device

[2270] Step 9:

[2271] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[2272] Input: Child's answer data

[2273] Output: Answer data sent to the server

[2274] Step 10:

[2275] The server stores the received answer data in a database, analyzes it using a generative AI model, creates feedback, and sends it to the device. For example, if the answer is correct, it displays "That's correct!"

[2276] Input: Child's answer data

[2277] Output: Feedback message

[2278] (Application example 2)

[2279] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2280] With the increase in dual-income households, children are spending more time alone. Under these circumstances, it is difficult to constantly monitor their children's mental state and safety, and there is a particular problem of delayed detection of dangers or abnormal behavior that may occur while parents are away. Furthermore, while it is important to make effective use of the time children spend alone and provide them with learning support, providing appropriate timing and content for such support can be difficult.

[2281] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2282] In this invention, the server includes means for acquiring a voice or text conversation with a child, means for analyzing the acquired conversation data and using an emotion engine to identify the child's emotions, means for inputting the emotion data identified by the emotion engine into a generative AI model to infer the child's mental state, means for notifying the parent of the child's mental state, means for monitoring the child's behavior and detecting abnormalities, and means for immediately notifying the parent if an abnormality is detected. This makes it possible to support the child's safety and mental growth by analyzing the child's mental state and behavior in real time and notifying the parent.

[2283] "Means for capturing voice or text conversations" refers to technology for capturing voice or text spoken by a child using a microphone, text input device, or the like.

[2284] An "emotion engine" refers to software or algorithms that use text and voice analysis to recognize and identify a child's emotions.

[2285] A "generative AI model" is an artificial intelligence model that generates new information and learning content based on acquired data.

[2286] "Means for inferring mental state" refers to a technology that uses emotional data identified by the emotion engine to infer a child's current mental state.

[2287] "Means for notifying parents" refers to technology that uses communication methods such as email and mobile notifications to inform parents of their child's mental state or abnormalities in real time.

[2288] "Means for monitoring behavior and detecting abnormalities" refers to technology that uses cameras and sensors to monitor children's movements and detect abnormal behavior.

[2289] "Means for generating learning content and presenting it to children" refers to technology that uses a generative AI model to generate learning content related to a subject and presents it to children at an appropriate time.

[2290] "Means for analyzing in real time and providing feedback" refers to technology that instantly analyzes acquired voice and behavioral data and provides feedback to the child based on the results.

[2291] This invention addresses the issue of children spending more time alone as the number of dual-income households increases. It provides a system that utilizes a generative AI model and an emotion engine to promote children's development and monitor safety within the home. This system is realized primarily by combining a server and a terminal.

[2292] Server Operation

[2293] 1. Load the generative AI model and emotion engine:

[2294] When the server starts up, it loads the generative AI model and emotion engine into memory. The AI ​​model and emotion engine are software that play a central role in the analysis and generation processes.

[2295] 2. Data Receipt and Storage:

[2296] The system receives the child's conversation and behavior data sent from the device and stores it in a database for later analysis.

[2297] 3. Emotion analysis:

[2298] The received conversation data is input into an emotion engine, which uses text and voice analysis to identify the child's emotions.

[2299] 4. Data Analysis:

[2300] The emotional data analyzed by the emotion engine is input into a generative AI model to predict the child's mental state. The generative AI model then uses the received data to more accurately predict the child's mental state.

[2301] 5. Notification function:

[2302] If an abnormal mental state is detected based on the analysis results, a notification will be sent to the parents via email or mobile notification.

[2303] 6. Learning content generation:

[2304] Based on the child's request, appropriate learning content is generated using a generative AI model and sent to the device.

[2305] 7. Behavioral Monitoring and Anomaly Detection:

[2306] The received behavioral data is analyzed, and if any abnormalities are detected, the parent is immediately notified.

[2307] Device behavior

[2308] 1. Conversation Acquisition:

[2309] The system captures the child's speech using a microphone or text input device, and the captured data is sent to a server in real time.

[2310] 2. Data transmission:

[2311] The acquired conversation data and behavioral data are sent to the server.

[2312] 3. Learning Support:

[2313] The learning content received from the server is presented to the child, encouraging the child to study.

[2314] 4. Condition Monitoring:

[2315] It continuously monitors children's behavior and speech, and notifies the server if any abnormalities are detected.

[2316] Specific examples

[2317] 1. Example conversation with a child:

[2318] The user (child) says to the device, "I was lonely at school today because my friends weren't there."

[2319] The terminal transmits this message to the server.

[2320] The server stores the received data in a database and uses an emotion engine to recognize the emotion "lonely."

[2321] The server inputs the emotional data of "lonely" into the generative AI model and analyzes the mental state.

[2322] If the server identifies the mental state "lonely," it sends a notification to the parent that "child feels lonely."

[2323] 2. Learning support examples:

[2324] The user (child) tells the device, "I want to study math now."

[2325] The terminal sends this request to the server.

[2326] The server stores the received requests in a database and inputs them into a generative AI model.

[2327] The server uses a generative AI model to generate appropriate math problems, such as "What is 2 + 3?"

[2328] The server sends the generated questions to the terminal.

[2329] The terminal presents the received question to the child and waits for the child's answer.

[2330] The user (child) inputs the answer into the terminal, and the terminal sends the answer to the server.

[2331] The server stores the received answer in a database, analyzes it with a generative AI model, and sends feedback. If the answer is correct, it displays "That's right!"

[2332] 3. Condition monitoring example:

[2333] While the user (child) is in the room, their behavior is monitored using cameras and sensors.

[2334] The device transmits behavioral data to the server in real time.

[2335] The server stores the received behavioral data in a database.

[2336] The server analyzes the accumulated and real-time data and compares it with normal behavioral ranges.

[2337] The server sends an immediate notification to the parent if it detects an anomaly, such as an abnormally long period of inactivity or suspicious behavior.

[2338] Prompt Sentence Examples

[2339] Emotion Analysis Prompt: "Analyze this audio data to identify the emotion."

[2340] Learning content generation prompt: "Generate one basic math problem, such as an addition or subtraction problem."

[2341] This system uses a generative AI model and an emotion engine to analyze a child's mental state and behavior in real time, and provides parents with the information they need to support their child's growth and safety.

[2342] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2343] Step 1:

[2344] Starting the server and loading the model

[2345] When the server starts up, it loads the generative AI model and emotion engine into memory. This step makes the necessary software components available. The input is a server configuration file containing the generative AI model and emotion engine, and the output is the model and engine loaded into the server memory. Specifically, the server executes the model loading routine and allocates memory space.

[2346] Step 2:

[2347] Retrieving and sending conversation data

[2348] The device captures voice and text conversations from the user (child). As input, voice and text data are obtained using a microphone and text input device, and this is sent to the server. The output is raw data sent to the server. Specifically, the device captures data through a voice recording or text input interface, and sends the data to the server using a communication protocol.

[2349] Step 3:

[2350] Sentiment analysis of conversation data

[2351] The server inputs the received conversation data into an emotion engine to identify the child's emotions. The input is voice or text data sent from the device, which is analyzed by the emotion engine. The output is the identified emotion data. Specifically, the server runs the emotion engine's analysis algorithm and classifies emotions using text analysis and voice analysis technology.

[2352] Step 4:

[2353] Mental state inference from emotional data

[2354] The server inputs the emotion data identified by the emotion engine into the generative AI model to infer the child's mental state. The input is emotion data from the emotion engine, which is adapted and processed by the generative AI model. The output is the child's inferred mental state. Specifically, the server executes the generative AI model's data processing algorithm to infer the child's mental state based on the emotion data.

[2355] Step 5:

[2356] Mental status notification

[2357] The server sends a notification to the parent if an abnormality is detected based on the child's mental state. The input is the mental state data inferred by the generative AI model, and the information is conveyed to the parent via the notification system. The output is a notification message sent to the parent's device. Specifically, the server invokes the notification function to send an email or mobile notification.

[2358] Step 6:

[2359] Receiving a learning content generation request

[2360] When a user (child) inputs a learning request into the device, the device sends this request to the server. The input is voice or text data based on the user's learning request, which is sent to the server. The output is the request data sent to the server. Specifically, the device captures the request data using a voice or text input interface and sends it to the server using a communication protocol.

[2361] Step 7:

[2362] Generation and presentation of learning content

[2363] The server uses the generative AI model to generate appropriate learning content based on the received request and sends it to the device. The input is the learning request data from the user and the generative AI model, and learning content is created based on this. The output is the data that sends the generated learning content to the device. Specifically, the server uses the prompt sentences from the generative AI model to generate math problems and reading comprehension content and sends it to the device.

[2364] Step 8:

[2365] Implementing learning support

[2366] The device presents the learning content received from the server to the user (child) and allows them to work on it. The input is the learning content sent from the server, which is displayed on the user interface. The output is the answer data entered by the user. Specifically, the device displays the learning content using a display and speech synthesis function, and obtains the user's answers from the input device.

[2367] Step 9:

[2368] Monitoring and sending behavioral data

[2369] The device monitors the user's (child's) behavior using cameras and sensors and sends the data to a server. The input is the behavioral data acquired from the cameras and sensors, which is sent to the server. The output is the behavioral data sent to the server. Specifically, the device acquires video and sensor information in real time and uses a communication protocol to send it to the server.

[2370] Step 10:

[2371] Anomaly detection in behavioral data

[2372] The server analyzes the received behavioral data and immediately sends a notification to the parent if an abnormality is detected. The input is the behavioral data sent from the device, and an algorithm is applied to detect abnormalities based on this. The output is a notification message that is sent to the parent if an abnormality is detected. Specifically, the server runs the behavioral analysis algorithm, and if an abnormality is discovered, it notifies the parent via the notification system.

[2373] Through these steps, this system utilizes a generative AI model and emotion engine to analyze children's mental state and behavior in real time and provide parents with the necessary information to support their children's growth and safety.

[2374] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2375] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2376] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2377] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2378] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2379] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between reli...

Claims

1. A means of obtaining conversations with children; a means for analyzing the acquired conversation data and using a generative artificial intelligence model to infer the child's mental state; a means of informing parents of their child's mental state; A system including:

2. The system of claim 1 , wherein the generative artificial intelligence model further comprises means for generating and presenting learning content related to a subject to the child.

3. a means for monitoring the child's behavior and detecting abnormalities; 10. The system of claim 1, further comprising means for immediately notifying a parent if an abnormality is detected.

4. 3. The system of claim 2, wherein the generative artificial intelligence model further comprises means for generating and presenting story content to the child.

5. 4. The system according to claim 3, further comprising means for storing the behavioral data of the child in a database and for performing behavioral analysis based on the stored data.

Images