System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting child-raising information; means for acquiring profile data of an individual child; means for analyzing the collected information based on the profile data of the child and generating individual advice; and means for notifying a user terminal of the individual advice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's busy society, parents often find it difficult to devote sufficient time to raising their children. It is also difficult to constantly obtain the latest and most accurate information necessary for child rearing and to provide appropriate support based on the child's growth and health. Furthermore, there is a lack of support systems to ensure children's safety and efficiently provide emotional support and health management. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means:

[0006] The system includes a means for collecting parenting information and a means for acquiring profile data for each child, a means for analyzing the collected information based on this information and the profile data, a means for generating individual advice, and a means for notifying the generated advice to the user terminal.

[0007] Furthermore, the system includes a means for receiving data from a surveillance camera or sensor, a means for analyzing the received data to detect abnormalities, and a means for issuing an alarm when an abnormality is detected, and a means for notifying the user terminal of the alarm.

[0008] The system also includes means for collecting data on the child's voice and facial expressions, means for analyzing the collected data to evaluate the child's emotional state, and means for generating an appropriate message based on the emotional state and notifying the user terminal of the message.

[0009] Furthermore, the system provides a means for inputting the child's dietary details and health status from a user terminal, a means for saving and accumulating the data, and a means for analyzing the accumulated data to identify nutritional balance and nutrient deficiencies. It also includes a means for generating meal suggestions based on the identification and notifying the user terminal.

[0010] Finally, we provide a system that provides comprehensive child-rearing support by providing a means for managing parents' schedules and children's event schedules, and a means for generating reminders based on the schedule and notifying them on the user's terminal.

[0011] "Parenting information" refers to the latest knowledge, advice, or guidelines related to parenting, collected from the internet or professional organizations.

[0012] "Profile Data" refers to detailed information related to an individual child, such as the child's age, health status, and allergy information.

[0013] "Analysis" refers to the processing of collected data and information to understand its contents and generate appropriate responses and advice.

[0014] "Individualized advice" refers to specific advice or suggestions tailored to each child's particular situation, based on collected and analyzed information.

[0015] "User terminal" refers to a device, such as a smartphone or tablet, that a user uses to receive information and as an interface.

[0016] "Surveillance camera" refers to a video camera installed to monitor children's movements and behavior.

[0017] A "sensor" refers to a device that detects changes in the physical environment (e.g., movement, sound, temperature, etc.) and collects them as data.

[0018] "Anomaly detection" refers to the process of identifying deviations from normal or unsafe conditions.

[0019] "Warning" refers to a notification or alert message that alerts the user when an abnormality or emergency occurs.

[0020] "Emotion analysis" refers to the data analysis process used to assess a child's emotional state based on voice and facial expression data.

[0021] "Message generation" refers to the process of formulating the content to be communicated to the user in natural language based on the analysis results.

[0022] "Dietary details" refers to details of the specific ingredients and dishes consumed by the child.

[0023] "Health status" refers to a child's current physical condition, medical history, and specific health indicators.

[0024] "Nutritional balance" refers to the criteria for evaluating whether the nutrients contained in a meal are adequate.

[0025] "Nutrient deficiency" refers to nutrients that do not meet the recommended amount required in the daily diet.

[0026] "Schedule management" refers to the process of effectively organizing and coordinating parent and child events and plans.

[0027] A "reminder" is a message or alert that notifies a user of an upcoming scheduled task or event. [Brief explanation of the drawings]

[0028] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0029] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0032] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0033] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0034] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0035] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0036] [First embodiment]

[0037] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0038] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0039] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0040] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0041] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0042] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0043] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0044] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0045] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0046] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0047] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0048] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] This invention is a system that provides comprehensive support for child-rearing through collaboration between an AI-equipped robot, a server, and a user device. The following describes the design of this system and the specific operation of each function.

[0050] 1. Gathering the latest information and providing personalized advice

[0051] The server regularly collects the latest articles and papers from reliable health and child-rearing information websites on the Internet. This information is stored in a database by category using text analysis tools. Each child's profile data (age, health condition, allergy information, etc.) is obtained, and the collected information is analyzed based on this to generate individualized advice. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[0052] Examples:

[0053] The server collects the latest articles on "How to prevent winter flu" and stores them in a database.

[0054] Extract relevant preventive measures based on the user's child (age 5, no allergies).

[0055] The system generates advice such as "It is important to take vitamin C" and notifies the user's device.

[0056] 2. Child safety checks using surveillance cameras and sensors

[0057] The server receives real-time data from surveillance cameras and motion sensors. The data is analyzed using image recognition models to detect abnormalities (such as a child climbing stairs). When an abnormality is detected, an alert is generated and sent to the user's device.

[0058] Examples:

[0059] A surveillance camera streams footage of a child accidentally climbing the stairs.

[0060] The server uses an image recognition model to identify anomalies and generates a warning that a child is in danger.

[0061] The warning is notified to the user terminal.

[0062] 3. Emotional support

[0063] The server collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm. It evaluates the child's emotional state (e.g., sadness or joy) and generates an appropriate message based on the results. The generated message is then sent to the user's device.

[0064] Examples:

[0065] The server collects the child's facial expression and voice data and evaluates the child as "sad."

[0066] Based on the analysis results, an encouraging message is generated, such as "It's okay, is there something bothering you?"

[0067] The emotional state and response content are notified to the user terminal.

[0068] 4. Health management and nutrition education support

[0069] The user device provides an interface where parents can input their child's dietary habits and health status. The server stores the input data and accumulates it daily. This data is then analyzed using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies. Based on the results, appropriate meal suggestions are generated and sent to the user device.

[0070] Examples:

[0071] Parents input their child's daily dietary information into a user terminal.

[0072] The server analyzes the data and identifies a lack of vitamin C intake.

[0073] "Eat more oranges and kiwis, which are richer in vitamin C," he suggests.

[0074] The user terminal is notified of the details of the proposal.

[0075] 5. Schedule management and reminder provision

[0076] The server manages the parent's schedule and the child's event schedule, generates reminders based on this schedule, and notifies the user's device as appropriate.

[0077] Examples:

[0078] Parents enter their child's vaccination schedule into a user terminal.

[0079] The server stores the schedule and generates a reminder as the day approaches: "Your vaccination is tomorrow."

[0080] A reminder is sent to the user's device.

[0081] In this way, the system of the present invention provides comprehensive support for child-rearing through functions such as collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and providing reminders.

[0082] The processing flow will be explained below.

[0083] 1. Gathering the latest information and providing personalized advice

[0084] Step 1:

[0085] Server: Regularly accesses reliable health and parenting information sites on the Internet and scrapes new articles and papers.

[0086] Step 2:

[0087] Server: Analyzes the content of collected articles using text analysis tools and stores them in a database by category.

[0088] Step 3:

[0089] Server: Obtains the child's individual profile data entered by the user (e.g., age, health status, allergy information).

[0090] Step 4:

[0091] Server: Based on the analyzed information, it generates personalized advice based on the child's profile data.

[0092] Step 5:

[0093] Server: Using natural language processing technology, the generated advice is converted into a message that is easy for the user to understand.

[0094] Step 6:

[0095] User device: The generated advice message is sent to the parent's smartphone or tablet.

[0096] Examples:

[0097] 1. The server collects the latest articles on "How to prevent winter influenza" and stores them in a database.

[0098] 2. The server retrieves the child's profile data and extracts relevant preventive measures.

[0099] 3. Generate advice such as "It is important to take vitamin C" and notify the user's device.

[0100] 2. Child safety checks using surveillance cameras and sensors

[0101] Step 1:

[0102] Server: Streams and receives real-time data from surveillance cameras and motion sensors.

[0103] Step 2:

[0104] Server: Analyzes received video data using an image recognition model to detect dangerous situations.

[0105] Step 3:

[0106] Server: Generates a warning alert if an anomaly is detected.

[0107] Step 4:

[0108] AI robot: Based on abnormality alerts received from the server, it issues a warning according to pre-set response methods.

[0109] Step 5:

[0110] User device: Real-time alert notifications are sent to parents' smartphones or tablets.

[0111] Examples:

[0112] 1. A security camera streams footage of a child accidentally climbing the stairs.

[0113] 2. The server uses an image recognition model to identify the child climbing the stairs and generates an anomaly alert.

[0114] 3. The AI ​​robot issues a warning that a child is in danger and notifies the user's device.

[0115] 3. Emotional support

[0116] Step 1:

[0117] Server: Using voice and facial recognition functions, collects children's voice and facial expression data in real time.

[0118] Step 2:

[0119] Server: Analyzes the collected data using an emotion analysis algorithm to evaluate the child's emotional state.

[0120] Step 3:

[0121] Server: Generates appropriate responses and support messages depending on the assessed emotional state.

[0122] Step 4:

[0123] AI robot: Speaks to children based on the generated message.

[0124] Step 5:

[0125] User device: Notifies parents of the child's emotional state and the AI ​​robot's response.

[0126] Examples:

[0127] 1. The server collects the child's facial expression and voice data and evaluates the child as "sad."

[0128] 2. The server generates an encouraging message based on the evaluation results, saying, "It's okay, is there something bothering you?"

[0129] 3. The AI ​​robot conveys the message to the child and notifies the user's device of their emotional state and response.

[0130] 4. Health management and nutrition education support

[0131] Step 1:

[0132] User terminal: Provides an interface where parents can input their child's diet and health status.

[0133] Step 2:

[0134] Server: Saves the entered data and accumulates daily data.

[0135] Step 3:

[0136] Server: Analyzes the accumulated data using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies.

[0137] Step 4:

[0138] Server: Generates appropriate meal suggestions based on the analysis results.

[0139] Step 5:

[0140] User terminal: Notifies parents of the generated meal suggestions and nutritional status reports.

[0141] Examples:

[0142] 1. Parents enter their child's daily dietary information into the user device.

[0143] 2. The server analyzes the input data and identifies that the person is lacking in vitamin C intake.

[0144] 3. "Incorporate more foods rich in vitamin C" and suggest specific recipes, and send a notification to the user's device.

[0145] 5. Schedule management and reminder provision

[0146] Step 1:

[0147] User terminal: Provides an interface for parents to input event schedules and schedules.

[0148] Step 2:

[0149] Server: Saves and manages the entered schedule data.

[0150] Step 3:

[0151] Server: Generates reminders based on schedule data.

[0152] Step 4:

[0153] User device: Notifies the parent of the generated reminder.

[0154] Examples:

[0155] 1. Parents enter their child's vaccination schedule into the user device.

[0156] 2. The server stores the appointment and generates a reminder when the appointment date approaches.

[0157] 3. The user device sends a reminder to the parent that "Vaccination is tomorrow."

[0158] Example 1

[0159] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] In child-rearing, it is often difficult for parents to receive the latest health information or appropriate advice based on the individual condition of their child, and children's safety and emotional care are often not adequately confirmed. This creates problems that make it difficult for parents to adequately support their children's health, safety, and emotional growth. To solve these problems, a comprehensive system of support for child-rearing is needed.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0162] In this invention, the server includes a means for collecting parenting information, a means for acquiring profile data for each child, and a means for classifying and saving the collected information using a text analysis tool. This allows parents to obtain the latest parenting information tailored to their individual circumstances in a timely manner. The server also includes a means for analyzing the collected information based on the child's profile data using a machine learning algorithm to generate personalized advice, a means for notifying a user terminal of the generated personalized advice using natural language processing technology, a means for receiving data from surveillance cameras and sensors, a means for analyzing the received data using an image recognition model to detect abnormalities, a means for generating a warning when an abnormality is detected and notifying the user terminal, a means for collecting voice and facial expression data of the child, a means for analyzing the collected data using an emotion analysis algorithm to evaluate the emotional state, a means for generating an appropriate message based on the emotional state using natural language processing technology, and a means for notifying the user terminal of the generated message. This enables comprehensive monitoring and analysis of a child's health, safety, and emotional state, and for providing appropriate advice and warnings in real time.

[0163] "Means for collecting parenting information" include devices and programs for collecting the latest articles and papers from reliable health and parenting information sites, specifically scraping tools and APIs.

[0164] "Means for obtaining individual child profile data" refers to a database and its interface for collecting and storing individual information such as each child's age, health condition, and allergy information, and for retrieving it as needed.

[0165] "Means for classifying and storing collected information using text analysis tools" refers to devices or programs that include text analysis tools for analyzing collected information, classifying it into categories, and storing it in a database.

[0166] "Means for analyzing using machine learning algorithms and generating personalized advice" refers to devices or programs that analyze collected data using machine learning algorithms and generate personalized advice based on each child's profile.

[0167] "Means for notifying a user terminal using natural language processing technology" refers to a device or program that uses natural language processing technology to generate a message from the generated advice in a form that is easy for the user to understand, and then sends that message to the user terminal.

[0168] "Means for receiving data from surveillance cameras and sensors" refers to devices and programs that transmit video and motion data from surveillance cameras and various sensors to a server in real time.

[0169] "Means for analyzing and detecting abnormalities using an image recognition model" refers to a device or program that analyzes received video and motion data using an image recognition model and detects abnormalities.

[0170] "Means for generating a warning when an abnormality is detected and notifying the user terminal" refers to a device or program that generates a warning message when an abnormality is detected and sends that message to the user terminal.

[0171] "Means for collecting children's voice and facial expression data" refers to devices such as cameras and microphones for collecting children's voice and facial expression data in real time, as well as programs for operating them.

[0172] "Means for evaluating a child's emotional state by analyzing using an emotion analysis algorithm" refers to a device or program that analyzes collected voice and facial expression data using an emotion analysis algorithm to evaluate a child's emotional state.

[0173] "Means for generating an appropriate message based on an emotional state using natural language processing technology" refers to a device or program that generates an appropriate message based on an evaluated emotional state using natural language processing technology.

[0174] The "means for notifying the user terminal of the generated message" refers to a device or program for transmitting the generated message to the user terminal.

[0175] This invention is a system that provides comprehensive support for child-rearing through collaboration between an AI-equipped robot, a server, and a user device. This system has multiple functions, including collecting the latest information and providing individualized advice, checking children's safety using surveillance cameras and sensors, providing emotional support, supporting health management and nutrition education, and managing schedules and providing reminders.

[0176] 1. Gathering the latest information and providing personalized advice

[0177] The server regularly collects the latest articles and papers from reliable health and parenting information websites. To do this, it uses Python scripts and the BeautifulSoup library to perform scraping. It then uses a text analysis tool (e.g., Apache OpenNLP) to categorize the collected information and store it in a database (e.g., MySQL). It obtains the user's child's profile data (e.g., age, health condition, allergy information, etc.) and analyzes it using machine learning algorithms such as scikit-learn. The generated advice is then converted into a user-friendly message using natural language processing technology (e.g., OpenAI's GPT-3) and sent to the user's device as a push notification.

[0178] Examples:

[0179] The server collects the latest articles on "How to prevent winter flu" and stores them in a database.

[0180] Extract relevant preventive measures based on the user's child (age 5, no allergies).

[0181] The system generates advice such as "It is important to take vitamin C" and notifies the user's device.

[0182] Example prompt sentence:

[0183] Input: Please tell me how to prevent winter flu in my child (5 years old, no allergies).

[0184] Output: Vitamin C intake is important. Eat plenty of oranges and kiwis.

[0185] 2. Child safety checks using surveillance cameras and sensors

[0186] The server receives real-time data from surveillance cameras (e.g., Nest Cam) and motion sensors (e.g., PIR sensors). The received data is analyzed using an image recognition model (e.g., TensorFlow's YOLO model) to detect anomalies. If an anomaly is detected, the server generates an alert and notifies the user device.

[0187] Examples:

[0188] A surveillance camera streams footage of a child accidentally climbing the stairs.

[0189] The server uses an image recognition model to identify anomalies and generates a warning that a child is in danger.

[0190] The warning is notified to the user terminal.

[0191] Example prompt sentence:

[0192] Input: Footage from a security camera shows a child starting to climb the stairs.

[0193] Output: Warning! Child in danger. Please act immediately.

[0194] 3. Emotional support

[0195] The server collects the child's voice and facial expression data and analyzes it using emotion analysis algorithms such as Microsoft Face API and Google Cloud Speech-to-Text. It evaluates the child's emotional state and generates an appropriate message based on the results. The message is then sent to the user's device.

[0196] Examples:

[0197] The server collects the child's facial expression and voice data and evaluates the child as "sad."

[0198] Based on the analysis results, an encouraging message is generated, such as "It's okay, is there something bothering you?"

[0199] The emotional state and response content are notified to the user terminal.

[0200] Example prompt sentence:

[0201] Input: How do you respond when your child is sad?

[0202] Output: Try gently saying, "It's okay. Is there something bothering you?"

[0203] 4. Health management and nutrition education support

[0204] The user device provides an interface where parents can input their child's dietary information and health status. The server stores the input data in a database and accumulates a daily diet history. This data is then analyzed using the Nutrition Data API to identify nutritional balance and nutrient deficiencies. Based on the results, appropriate meal suggestions are generated and notified to the user device.

[0205] Examples:

[0206] Parents input their child's daily dietary information into a user terminal.

[0207] The server analyzes the data and identifies a lack of vitamin C intake.

[0208] "Eat more oranges and kiwis, which are richer in vitamin C," he suggests.

[0209] The user terminal is notified of the details of the proposal.

[0210] Example prompt sentence:

[0211] Enter: If today's meal doesn't include oranges or kiwi, what are your suggestions?

[0212] Output: Eat more oranges and kiwis, which are rich in vitamin C.

[0213] 5. Schedule management and reminder provision

[0214] The server manages the parent's schedule and the child's event schedule, generates reminders based on this schedule, and notifies the user's device as appropriate.

[0215] Examples:

[0216] Parents enter their child's vaccination schedule into a user terminal.

[0217] The server stores the schedule and generates a reminder as the day approaches: "Your vaccination is tomorrow."

[0218] A reminder is sent to the user's device.

[0219] Example prompt sentence:

[0220] Enter: Set a reminder so you don't forget to schedule your vaccination appointment.

[0221] Output: Your vaccination is tomorrow. Don't forget.

[0222] In this way, the system of the present invention provides comprehensive support for child-rearing through functions such as collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders.

[0223] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0224] 1. Gathering the latest information and providing personalized advice

[0225] Step 1:

[0226] The server collects the latest articles and papers from reliable health and child-rearing information sites on the Internet.

[0227] Input: URL list to be collected

[0228] What it does: Use a Python script to parse the HTML and extract data from each URL using the BeautifulSoup library.

[0229] Output: Collected text data

[0230] Step 2:

[0231] The server uses text analysis tools to categorize the collected information.

[0232] Input: Collected text data

[0233] What it does: It uses Apache OpenNLP to analyze text and classify it into categories.

[0234] Output: Text data classified by category

[0235] Step 3:

[0236] The server stores the classified information in a database.

[0237] Input: Categorized text data

[0238] Specific behavior: Connects to a MySQL database and inserts data into the appropriate tables.

[0239] Output: Saved database entries

[0240] Step 4:

[0241] The server retrieves the profile data for each child from the database.

[0242] Input: Child's identification

[0243] Specific operation: Issues an SQL query and retrieves profile data.

[0244] Output: Child profile data

[0245] Step 5:

[0246] The server uses machine learning algorithms to analyze the profile data and collected information to generate personalized advice.

[0247] Input: Profile data, categorical text data

[0248] What it does: Uses the scikit-learn library to analyze information that matches the profile data.

[0249] Output: personalized advice

[0250] Step 6:

[0251] The server notifies the generated individual advice to the user terminal using natural language processing technology.

[0252] Input: Personalized Advice

[0253] Specific operation: Using OpenAI's GPT-3, messages are generated in natural language and sent to the user's device via the push notification API.

[0254] Output: Notification message to the user terminal

[0255] 2. Child safety checks using surveillance cameras and sensors

[0256] Step 1:

[0257] The server receives data in real time from surveillance cameras and sensors.

[0258] Input: Real-time camera and sensor data

[0259] Specific operation: Use the streaming API to receive data.

[0260] Output: Received real-time data

[0261] Step 2:

[0262] The server analyzes the received data using an image recognition model.

[0263] Input: Real-time data

[0264] Specific operation: Detect abnormal behavior using TensorFlow's YOLO model.

[0265] Output: Analysis results (normal / abnormal flag)

[0266] Step 3:

[0267] When the server detects an abnormality, it generates a warning and notifies the user terminal.

[0268] Input: Analysis results (abnormal flag)

[0269] Specific operation: If an abnormality is detected, a warning message is generated and sent to the user device via the push notification API.

[0270] Output: A warning message to the user's terminal.

[0271] 3. Emotional support

[0272] Step 1:

[0273] The server collects the child's voice and facial expression data.

[0274] Input: Camera and microphone data

[0275] Specific operation: Receives a data stream from a device.

[0276] Output: Collected voice and facial expression data

[0277] Step 2:

[0278] The server analyzes the collected data using a sentiment analysis algorithm.

[0279] Input: Voice and facial expression data

[0280] Specific behavior: Emotional state is assessed using Microsoft Face API and Google Cloud Speech-to-Text.

[0281] Output: Emotional state evaluation result

[0282] Step 3:

[0283] The server generates an appropriate message based on the emotional state using natural language processing techniques.

[0284] Input: Emotional state assessment results

[0285] Specific operation: Messages are generated using OpenAI's GPT-3.

[0286] Output: The generated message

[0287] Step 4:

[0288] The server notifies the user terminal of the generated message.

[0289] Input: The generated message

[0290] Specific operation: Send a message via the Push notification API.

[0291] Output: Notification message to the user terminal

[0292] 4. Health management and nutrition education support

[0293] Step 1:

[0294] The user inputs the child's diet and health condition into the user terminal.

[0295] Input: Data entered by the user

[0296] Specific operation: Fill in a form in an application on the user's device.

[0297] Output: Entered dietary and health data

[0298] Step 2:

[0299] The server saves the entered data in a database.

[0300] Input: Dietary and health status data

[0301] Specific action: Add a new entry to the database.

[0302] Output: Saved database records

[0303] Step 3:

[0304] The server analyzes the data using a nutritional analysis algorithm.

[0305] Input: Saved database records

[0306] Specific behavior: Uses the Nutrition Data API to identify nutritional balances and nutrient deficiencies.

[0307] Output: Analysis results (nutritional balance, nutrient deficiencies)

[0308] Step 4:

[0309] The server generates meal suggestions based on the analysis results and notifies the user terminal.

[0310] Input: Analysis results

[0311] Specific operation: Generates suggestions and sends them via the Push notification API.

[0312] Output: Meal suggestion message to user device

[0313] 5. Schedule management and reminder provision

[0314] Step 1:

[0315] The user inputs the child's event schedule and vaccination schedule into the user terminal.

[0316] Input: Schedule data entered by the user

[0317] Specific operation: Enter a schedule into an application on the user's device.

[0318] Output: Schedule data entered

[0319] Step 2:

[0320] The server stores the entered schedule data in a database.

[0321] Input: Schedule data

[0322] Specific Action: Adds a new schedule entry to the database.

[0323] Output: Saved database records

[0324] Step 3:

[0325] The server generates a reminder based on the schedule data and notifies the user terminal.

[0326] Input: Saved schedule data

[0327] Specific behavior: Generates reminder content and sends it via the Push notification API at the appropriate time.

[0328] Output: Reminder message to user device

[0329] (Application example 1)

[0330] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0331] Conventional methods for managing factory robots have had problems with insufficient efficient robot operation and safety. In particular, it has been difficult to provide appropriate advice and warnings based on the individual needs of each robot, and to properly manage the timing of maintenance. Furthermore, there has been a lack of systems for monitoring the operating status of robots in real time and taking appropriate action quickly. To solve these issues, a more advanced and comprehensive robot management system is needed.

[0332] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0333] In this invention, the server includes means for collecting information, means for acquiring individual profile data, means for analyzing the collected information based on the profile data and generating individual advice, means for notifying the user terminal of the individual advice, means for receiving and analyzing robot status data, means for generating an appropriate message based on the analysis results, and means for notifying the user terminal of the generated message, thereby enabling efficient operation of the robot, ensuring safety, and maintenance management.

[0334] "Information gathering means" are devices and programs that obtain up-to-date information and articles from reliable data sources.

[0335] The "means for acquiring profile data" refers to a device or program that collects information about a specific robot or target device and stores it as profile data.

[0336] The "means for analyzing information and generating personalized advice" refers to a device or program that analyzes the collected data and generates advice tailored to the specific needs of the subject.

[0337] The "means for notifying the user terminal of the advice" refers to a device or program that transmits the generated advice to the user terminal.

[0338] The "means for receiving and analyzing robot status data" refers to a device or program that collects data on the robot's operating status and performance and analyzes that data.

[0339] The "means for generating an appropriate message" refers to a device or program that generates an effective message to notify the user based on the analysis results.

[0340] The "means for notifying the user terminal of the generated message" refers to a device or program that quickly and reliably transmits the generated message to the user terminal.

[0341] "Means for receiving data from monitoring devices and sensors" refers to devices and programs that receive data collected from monitoring devices and various sensors within the factory.

[0342] "Means for detecting abnormalities" refers to a device or program that analyzes received data, identifies any abnormal conditions, and issues a warning.

[0343] "Means for analyzing efficiency" refers to devices or programs that analyze the operation data of each robot and evaluate the work efficiency.

[0344] The "means for generating advice for improving efficiency" is a device or program that provides specific advice for improving the work efficiency of the robot based on the analysis results.

[0345] "Means for managing maintenance schedules and sending reminders" refers to devices or programs that manage the timing of robot maintenance and notify users of maintenance when necessary.

[0346] The present invention relates to a factory robot management system that comprehensively supports the efficient operation and safety of factory robots by linking servers, monitoring devices, sensors, and user terminals.

[0347] System configuration

[0348] The server first has a means to collect the latest information and articles from reliable data sources (e.g., technology sites and industry information sites). The collected information is analyzed based on the profile data, and personalized advice is generated and sent to the user's device. This profile data is obtained based on the characteristics and demands of each individual robot.

[0349] Monitoring devices and sensors monitor the operating status of each robot in the factory and the surrounding environment in real time, and send the data to a server. The server analyzes the received data and issues an alert if an abnormality is detected, notifying the user's device. The server also analyzes the efficiency of the robots and generates advice for improving efficiency.

[0350] Furthermore, the server manages the robot's maintenance schedule and has a means for sending reminders, allowing the user to perform maintenance at the appropriate time.

[0351] Processing description

[0352] 1. Collection of information:

[0353] The server periodically retrieves up-to-date information from reliable data sources using the Python requests library, which is then parsed by text analysis tools and stored in a database.

[0354] 2. Analyzing data and generating advice:

[0355] Based on the profile data, the collected information is analyzed and personalized advice is generated based on each robot's condition and needs. The generated advice is then sent to the user's device using natural language processing technology.

[0356] 3. Monitoring and Alerting:

[0357] Data received from monitoring devices and sensors is analyzed using image recognition and motion detection models, and if an anomaly is detected, an alert is generated immediately and sent to the user's device.

[0358] 4. Maintenance Management:

[0359] The server manages the maintenance schedule for each robot and sends reminders as appropriate, based on a pre-set schedule.

[0360] Specific examples

[0361] Examples of gathering up-to-date information:

[0362] The server collects articles about "robot maintenance techniques," analyzes the information, and applies it to "Robot 1."

[0363] Safety monitoring example:

[0364] The monitoring device detects dangerous behavior of the robot while it is in operation and generates an alert to notify the administrator.

[0365] Efficiency analysis and advice examples:

[0366] The operation data of each robot is analyzed, and specific advice (e.g., how to optimize operation) to improve the efficiency of "Robot 2" is generated and notified to the user terminal.

[0367] Maintenance reminder example:

[0368] The server manages the maintenance schedule for "Robot 3" and sends a reminder "Maintenance is required" the day before the maintenance.

[0369] Prompt Sentence Examples

[0370] Prompt sentence to input to the generative AI model:

[0371] "Generate the latest maintenance advice based on the following robot ID and its current operational status: ID: robot_1, Status: Up, Maintenance Cycle: 1 month."

[0372] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0373] Step 1:

[0374] The server collects the latest information from a reliable data source. Here, the server uses Python's requests library to access URLs and retrieve article data on the latest technical information and maintenance techniques. The input is the URL, and the output is the retrieved latest information data.

[0375] Step 2:

[0376] The server analyzes the collected information using text analysis tools (e.g., natural language processing libraries). As a result of the analysis, the information is classified into categories. The input is the collected information data, and the output is the data classified by category.

[0377] Step 3:

[0378] The server obtains the profile data of each robot. This profile data includes information such as the robot's ID, characteristics, maintenance cycle, etc. The input is the robot's individual data request, and the output is the profile data.

[0379] Step 4:

[0380] The server generates appropriate personalized advice from the collected information based on the profile data. Specifically, it uses natural language processing technology to select relevant articles and information and restructures the content to suit the robot's needs. The input is profile data and categorized data, and the output is personalized advice.

[0381] Step 5:

[0382] The server notifies the user device of the generated personalized advice. Notifications are sent via APIs or messaging protocols, and users can receive them on their smartphones or tablets. The input is personalized advice, and the output is a notification sent to the user device.

[0383] Step 6:

[0384] Monitoring devices and sensors collect data in real time and send it to a server. Here, hardware such as cameras are used to monitor the robot's operation. The input is monitoring data, and the output is data sent to the server.

[0385] Step 7:

[0386] The server analyzes the received data and detects anomalies. Image recognition and motion detection models are used for the analysis. If an anomaly is detected, an alert is generated. The input is the monitoring data, and the output is the anomaly detection result.

[0387] Step 8:

[0388] If an abnormality is detected, the server immediately sends a warning to the user terminal. The user receives the warning message and can respond promptly. The input is the abnormality detection result, and the output is a warning notification to the user terminal.

[0389] Step 9:

[0390] The server analyzes the efficiency of each robot. Historical and real-time data are used for efficiency analysis to identify movement patterns and efficient operation methods. The input is movement data, and the output is the efficiency analysis results.

[0391] Step 10:

[0392] The server generates specific advice for improving efficiency and notifies the user terminal. This advice is created using natural language processing technology. The input is the efficiency analysis result, and the output is efficiency improvement advice.

[0393] Step 11:

[0394] The server manages the robot's maintenance schedule and sends reminders. Here, it notifies the robot of the need for maintenance based on a pre-defined schedule. The input is the maintenance schedule and the output is the reminder notification.

[0395] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0396] This invention is a system that comprehensively supports child-rearing by linking an AI-equipped robot, a server, a user terminal, and an emotion engine. The design of this system and the specific operation method of each function are explained below.

[0397] 1. Gathering the latest information and providing personalized advice

[0398] The server regularly collects the latest articles and papers from reliable health and child-rearing information sites and analyzes them using text analysis tools. The analyzed information is then stored in a database by category. Individual profile data (e.g., age, health condition, allergy information, emotional state) entered by the user for the child and the user themselves is then obtained, and the collected information is analyzed based on this to generate individual advice. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[0399] Examples:

[0400] The server collects the latest articles on "How to Prevent Influenza" and stores them in a database.

[0401] The user's child (5 years old, no allergies) and the user's emotional state are obtained, and relevant preventive measures are extracted.

[0402] It generates an "encouraging message according to the user's emotional state" and advice such as "taking vitamin C is important," and notifies the user's device.

[0403] 2. Child safety checks using surveillance cameras and sensors

[0404] The server receives real-time data from surveillance cameras and motion sensors, analyzes it with image recognition models to detect dangerous situations, and if an abnormality is detected, generates a warning alert and notifies the user's device.

[0405] Examples:

[0406] A surveillance camera streams footage of a child accidentally climbing the stairs.

[0407] The server uses an image recognition model to identify a child climbing stairs and generates an anomaly alert.

[0408] A warning that "child is in danger" is sent to the user terminal.

[0409] 3. Use of emotional support and emotional engines

[0410] The server collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm. It evaluates the child's emotional state (e.g., sadness, joy), and uses an emotion engine to generate an appropriate response or support message based on that state. The generated message, taking into account the user's emotional state, is then sent to the user's device, where the AI ​​robot then speaks to the child.

[0411] Examples:

[0412] The server collects the child's facial expressions and voice data, and uses an emotion analysis algorithm to assess whether the child is "sad."

[0413] Sentiment analysis algorithms also assess the user's emotional state and generate a comprehensive message.

[0414] The AI ​​robot will ask the child, "It's okay. Is there something bothering you?" and notify the user's device of its emotional state and response.

[0415] 4. Health management and nutrition education support

[0416] The user device provides an interface for parents to input their child's dietary habits and health status, and the server saves and stores the data. This data is analyzed using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies. Based on the analysis results, appropriate meal suggestions are generated and notified to the user device.

[0417] Examples:

[0418] Parents input their child's daily dietary information into a user terminal.

[0419] The server analyzes the data and identifies a lack of vitamin C intake.

[0420] "Incorporate more foods rich in vitamin C," he says, suggesting specific recipes.

[0421] 5. Schedule management and reminder provision

[0422] The server manages the parent's schedule and the child's event schedule, and generates reminders based on this schedule data. The generated reminders are then sent to the user's device.

[0423] Examples:

[0424] Parents enter their child's vaccination schedule into a user terminal.

[0425] The server stores the appointment and generates a reminder when the appointment date approaches.

[0426] The user terminal notifies the parent with a reminder that "Vaccination is tomorrow."

[0427] 6. User Emotion Recognition

[0428] The emotion engine collects and analyzes the user's voice and facial expression data to assess the user's emotional state, and tailors personalized advice and support messages based on the assessed emotional state to provide more effective support.

[0429] Examples:

[0430] The emotion engine analyzes the user's tone of voice and facial expressions and assesses them as "feeling stressed."

[0431] The server receives the evaluation results and generates advice such as, "We recommend that you spend some time relaxing with your child."

[0432] The advice is sent to the user's device, and music and activities that can help parents and children relax are suggested.

[0433] In this way, the system of the present invention provides comprehensive support for child-rearing by collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders, and recognizing users' emotions using an emotion engine.

[0434] The processing flow will be explained below.

[0435] 1. Gathering the latest information and providing personalized advice

[0436] Step 1:

[0437] Server: Regularly accesses reliable health and parenting information websites and scrapes new articles and papers.

[0438] Step 2:

[0439] Server: Analyzes the content of collected articles using text analysis tools and stores them in a database by category.

[0440] Step 3:

[0441] Server: Obtains individual profile data (e.g., age, health condition, allergy information, emotional state) of the child and the user themselves entered by the user.

[0442] Step 4:

[0443] Server: Analyzes the collected information and generates personalized advice based on the profile data.

[0444] Step 5:

[0445] Server: Using natural language processing technology, the generated advice is converted into a message that is easy for the user to understand.

[0446] Step 6:

[0447] User device: The generated advice message is sent to the parent's smartphone or tablet.

[0448] Examples:

[0449] 1. The server collects the latest articles about "How to prevent influenza" and stores them in a database.

[0450] 2. The server retrieves the child's profile data and extracts relevant preventive measures.

[0451] 3. Generate advice such as "It is important to take vitamin C" and notify the user's device.

[0452] 2. Child safety checks using surveillance cameras and sensors

[0453] Step 1:

[0454] Server: Streams and receives real-time data from surveillance cameras and motion sensors.

[0455] Step 2:

[0456] Server: Analyzes received video data using an image recognition model to detect dangerous situations.

[0457] Step 3:

[0458] Server: Generates a warning alert if an anomaly is detected.

[0459] Step 4:

[0460] AI robot: Based on abnormality alerts received from the server, it issues a warning according to pre-set response methods.

[0461] Step 5:

[0462] User device: Sends real-time alert notifications to parents' smartphones and tablets.

[0463] Examples:

[0464] 1. A security camera streams footage of a child accidentally climbing the stairs.

[0465] 2. The server uses an image recognition model to identify the child climbing the stairs and generates an anomaly alert.

[0466] 3. The AI ​​robot issues a warning that a child is in danger and notifies the user's device.

[0467] 3. Use of emotional support and emotional engines

[0468] Step 1:

[0469] Server: Using voice and facial recognition functions, collects children's voice and facial expression data in real time.

[0470] Step 2:

[0471] Server: Analyzes the collected data using an emotion analysis algorithm to evaluate the child's emotional state.

[0472] Step 3:

[0473] Server: Depending on the assessed emotional state, it uses an emotion engine to generate appropriate responses and support messages.

[0474] Step 4:

[0475] AI robot: Speaks to children based on the generated message.

[0476] Step 5:

[0477] User device: Notifies parents of the child's emotional state and the AI ​​robot's response.

[0478] Examples:

[0479] 1. The server collects the child's facial expression and voice data and uses an emotion analysis algorithm to evaluate the child as "sad."

[0480] 2. Based on the evaluation results, the server generates an encouraging message saying, "It's okay, is there anything you're having trouble with?"

[0481] 3. The AI ​​robot conveys the message to the child and notifies the user's device of their emotional state and response.

[0482] 4. Health management and nutrition education support

[0483] Step 1:

[0484] User terminal: Provides an interface for parents to input their child's dietary information and health status.

[0485] Step 2:

[0486] Server: Saves the entered data and accumulates daily data.

[0487] Step 3:

[0488] Server: Analyzes the accumulated data using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies.

[0489] Step 4:

[0490] Server: Generates appropriate meal suggestions based on the analysis results.

[0491] Step 5:

[0492] User terminal: Notifies parents of the generated meal suggestions and nutritional status report.

[0493] Examples:

[0494] 1. The user inputs the child's daily meal plan into the user terminal.

[0495] 2. The server analyzes the input data and identifies that the person is lacking in vitamin C intake.

[0496] 3. The server suggests specific recipes, such as "Incorporate more foods rich in vitamin C," and notifies the user's device.

[0497] 5. Schedule management and reminder provision

[0498] Step 1:

[0499] User terminal: Provides an interface for parents to input event schedules and schedules.

[0500] Step 2:

[0501] Server: Saves and manages the entered schedule data.

[0502] Step 3:

[0503] Server: Generates reminders based on schedule data.

[0504] Step 4:

[0505] User device: Notifies the parent of the generated reminder.

[0506] Examples:

[0507] 1. The user enters their child's vaccination schedule into the user terminal.

[0508] 2. The server stores the appointment and generates a reminder when the appointment date approaches.

[0509] 3. The user device sends a reminder to the parent that "Vaccination is tomorrow."

[0510] 6. User Emotion Recognition

[0511] Step 1:

[0512] Emotion engine: Collects and analyzes the user's voice and facial expression data.

[0513] Step 2:

[0514] Server: Receives data from the emotion engine and evaluates the user's emotional state.

[0515] Step 3:

[0516] Server: Tailors personalized advice and support messages based on the assessed emotional state.

[0517] Step 4:

[0518] User terminal: Notifies the parent of the generated support message.

[0519] Examples:

[0520] 1. The emotion engine analyzes the user's tone of voice and facial expressions and assesses them as "feeling stressed."

[0521] 2. The server receives the evaluation results and generates advice such as, "We recommend that you spend some time relaxing with your child."

[0522] 3. The user device will notify them of the advice and suggest relaxing music and activities.

[0523] This system allows AI-equipped robots, servers, user devices, and emotion engines to work together to provide comprehensive support for child-rearing.

[0524] Example 2

[0525] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0526] In modern society, raising children has become an extremely diverse and complex task. Busy parents often find it difficult to obtain accurate and up-to-date information on child rearing, making it even more difficult to comprehensively address their children's safety, health, and emotional support. In particular, many aspects must be considered simultaneously, including monitoring to ensure children's safety, providing emotional care, and managing nutritional balance. The present invention aims to provide a system that comprehensively solves these complex child rearing challenges.

[0527] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting child-rearing related information, means for acquiring profile data of each child, means for analyzing the collected information based on the profile data and generating individual advice, means for notifying the user terminal of the individual advice, means for collecting emotional data of family members and evaluating their emotional states using an emotion analysis algorithm, means for generating an appropriate response message based on their emotional states, and means for notifying the user terminal of the response message and responding to the user via a voice output device. This enables comprehensive support for child-rearing.

[0528] "Child-rearing related information" refers to the latest health, educational, and psychological information related to child-rearing, as well as other information useful for raising children.

[0529] "Profile Data" refers to individual data about each child and their parents, such as age, health status, allergy information, and emotional state.

[0530] An "emotion analysis algorithm" refers to a computational method for analyzing emotional states from voice data, facial expression data, etc., and evaluating the results.

[0531] A "response message" refers to an appropriate message to a user that is generated based on the user's emotional state and individual circumstances.

[0532] "Monitoring equipment" refers to equipment that uses surveillance devices such as cameras and sensors to monitor children's activities and surrounding conditions in real time.

[0533] A "nutritional analysis algorithm" refers to a calculation method that analyzes nutritional balance based on input dietary data and suggests necessary nutrients and appropriate meals.

[0534] "Reminder" refers to a warning or attention message that notifies the user based on a pre-set schedule.

[0535] The term "audio output device" refers to a device for actually transmitting the generated voice message to the user as voice.

[0536] The present invention provides a comprehensive support system for child rearing. This system realizes child safety confirmation, health management, emotional support, and schedule management by linking together a server, terminals, users, an emotion engine, a monitoring device, and a nutritional analysis algorithm.

[0537] The server periodically collects the latest articles and papers from reliable health and parenting information websites on the Internet. Web scraping tools and APIs are used for collection. Specifically, Beautiful Soup is used for web scraping, and the PubMed API is used for data collection. The collected data is analyzed using text analysis tools (e.g., NLTK, SpaCy) and stored by category in a database (e.g., MySQL, MongoDB).

[0538] Users enter their own and their children's profile data (e.g., age, health condition, allergy information, emotional state) on their device. The device sends the entered data to the server, which analyzes the data and generates personalized advice. The personalized advice is generated using natural language processing technology (e.g., GPT-4, BERT) and notified to the user's device.

[0539] For example, the server collects the latest articles on "How to prevent influenza" and stores them in a database. When a user inputs the profile data of a child (age 5, no allergies), the server analyzes the relevant prevention methods, generates advice such as "It is important to take vitamin C," and notifies the user's device.

[0540] Monitoring devices (e.g., cameras, motion sensors) send real-time data of children to a server. The server uses image recognition models (e.g., YOLO, OpenCV) to analyze the monitoring data and detect abnormalities. If an abnormality is detected, the server generates a warning alert and notifies the user's device.

[0541] Specifically, a surveillance camera receives footage of a child accidentally climbing stairs, and the server analyzes the footage to detect any abnormalities, then sends a warning to the user's device that the child is in danger.

[0542] The server also collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm (e.g., Affectiva, Microsoft Azure Emotion API). Based on the analysis results, the server evaluates the child's emotional state and generates an appropriate response message. This message is sent to the user's device, which responds to the child via the voice output device.

[0543] For example, the server collects a child's facial expression and voice data and uses an emotion analysis algorithm to evaluate the child as "sad." The server then generates a message saying, "It's okay. Is something bothering you?", and the AI ​​robot responds, notifying the user's device of the child's emotional state and the response.

[0544] Furthermore, the user device provides an interface for parents to input their child's dietary information and health status, and sends the data to a server. The server then analyzes the input data using a nutritional analysis algorithm (e.g., MyFitnessPal's API). Based on the analysis results, the nutritional balance is evaluated and appropriate meal suggestions are generated.

[0545] For example, if a parent inputs their child's diet and the server identifies a vitamin C deficiency, it will suggest specific recipes such as "Incorporate more vitamin C-rich foods."

[0546] The server manages schedule data, generates reminders based on this data, and notifies the user device. For example, when a parent inputs their child's vaccination schedule, the server saves the schedule and sends a reminder that "Tomorrow's vaccination date" when the scheduled date approaches.

[0547] The emotion engine then evaluates the user's emotional state and generates personalized advice and support messages based on the evaluation results, thereby providing optimal support that takes the user's emotional state into account.

[0548] Example prompt sentence:

[0549] 1. "What are some good sleep guidelines for a 1-year-old?"

[0550] 2. "What should I do if I'm deficient in vitamin C?"

[0551] 3. "What safety precautions should I take when climbing stairs?"

[0552] 4. "What are some ways parents and children can relax when they're feeling stressed?"

[0553] In this way, the system of the present invention provides comprehensive support for child-rearing by collecting the latest information and providing individual advice, checking children's safety using monitoring devices and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders, and recognizing users' emotions using an emotion engine.

[0554] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0555] Step 1:

[0556] The server collects child-rearing related information.

[0557] Input: Online health and child-rearing information sites

[0558] Data processing: Collecting data using web scraping tools and APIs

[0559] Output: Latest childcare-related information data

[0560] Specific behavior:

[0561] The server launches a web scraping tool (e.g., Beautiful Soup) to collect the required information from the specified site.

[0562] Analyze the collected data using text analysis tools (e.g., NLTK, SpaCy) and categorize the information.

[0563] Store the classified data in a database (e.g., MySQL, MongoDB).

[0564] Step 2:

[0565] Users enter profile data for their children and themselves.

[0566] Input: Data such as the child's and the user's age, health status, allergy information, and emotional state

[0567] Output: User profile data

[0568] Specific behavior:

[0569] The user enters the required profile data into the input form on the device.

[0570] The user terminal transmits the input data to the server.

[0571] Step 3:

[0572] The server generates personalized advice based on the profile data.

[0573] Input: User-entered profile data, latest parenting-related information stored on the server

[0574] Data calculation: Analyze and generate data using natural language processing techniques (e.g., GPT-4, BERT)

[0575] Output: Individual advice message

[0576] Specific behavior:

[0577] The server retrieves the user's profile data and updates from the database.

[0578] Natural language processing techniques are used to generate personalized advice.

[0579] The generated advice is notified to the user terminal.

[0580] Step 4:

[0581] The server receives and analyzes real-time data from the monitoring devices.

[0582] Input: Video data and sensor data from surveillance devices (e.g., cameras, motion sensors)

[0583] Data calculation: Analyze using image recognition models (e.g., YOLO, OpenCV)

[0584] Output: Analysis results of monitoring data

[0585] Specific behavior:

[0586] The monitoring device captures real-time video data and transmits it to a server.

[0587] The server launches an image recognition model and analyzes the video data.

[0588] Detect anomalies and save detailed data.

[0589] Step 5:

[0590] If the server detects an abnormality, it generates a warning alert and notifies the user.

[0591] Input: Analysis results of monitoring data

[0592] Data processing: Applying anomaly detection algorithms and generating warning alerts

[0593] Output: Warning alert notification

[0594] Specific behavior:

[0595] The server detects abnormal situations based on the analysis results.

[0596] Generate warning alerts in response to anomalies.

[0597] A warning alert is sent to the user terminal.

[0598] Step 6:

[0599] The server collects and analyzes the child's voice and facial expression data.

[0600] Input: Child's voice data and facial expression data

[0601] Data calculation: Analyze using emotion analysis algorithms (e.g., Affectiva, Microsoft Azure Emotion API)

[0602] Output: Emotional state evaluation result

[0603] Specific behavior:

[0604] Emotion sensors and cameras capture the child's voice and facial expressions.

[0605] The server analyzes the data using a sentiment analysis algorithm.

[0606] The analysis results are evaluated and stored in a database.

[0607] Step 7:

[0608] The server generates and notifies an appropriate response message based on the emotional state.

[0609] Input: Emotional state assessment results

[0610] Data Computation: Uses an emotion engine to generate appropriate response messages

[0611] Output: Response message notification

[0612] Specific behavior:

[0613] The server obtains the emotion analysis results.

[0614] Use an emotion engine to generate appropriate response messages.

[0615] The message is sent to the user terminal, and a response is given to the user via the voice output device.

[0616] Step 8:

[0617] The user terminal inputs information about the child's diet and health condition.

[0618] Input: Child's dietary information, health status data

[0619] Output: Dietary and health data

[0620] Specific behavior:

[0621] The user enters information about their child's diet and health status into the interface.

[0622] The user terminal transmits the input data to the server.

[0623] Step 9:

[0624] The server analyzes the data and generates appropriate meal suggestions.

[0625] Input: User-entered food data, stored health information

[0626] Data calculation: Analyze data using a nutritional analysis algorithm to evaluate nutritional balance

[0627] Output: Appropriate meal suggestions

[0628] Specific behavior:

[0629] The server retrieves dietary and health data from a database.

[0630] Nutrition analysis algorithms are used to analyze the data and identify nutrient deficiencies.

[0631] An appropriate meal suggestion is generated and notified to the user terminal.

[0632] Step 10:

[0633] The server manages the schedule data and generates reminders.

[0634] Input: Schedule data entered by the user

[0635] Data processing: Manage schedule data and generate reminders

[0636] Output: Reminder notification

[0637] Specific behavior:

[0638] The user inputs the child's schedule data.

[0639] The server manages the schedule data and generates reminders before important events.

[0640] Reminders are sent to the user's device.

[0641] Step 11:

[0642] The emotional engine analyzes the user's emotional state and adjusts the message based on the results.

[0643] Input: User's voice data and facial expression data

[0644] Data Computing: Emotion analysis algorithms assess the user's emotional state and tailor messages

[0645] Output: Adjusted support message

[0646] Specific behavior:

[0647] The emotion engine captures the user's voice and facial expression data.

[0648] The server obtains the evaluation result of the emotion engine.

[0649] A support message is adjusted based on the evaluation result and notified to the user terminal.

[0650] (Application example 2)

[0651] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0652] In modern society, parents are faced with a wide range of information and tasks related to child-rearing, which often leads to stress and anxiety. Furthermore, caring for children while shopping in a store can be even more exhausting for parents. Furthermore, managing children's safety in the store, providing health advice, and efficiently managing schedules are challenges. Currently, there are no systems that provide individual support while taking into account the emotional state of parents and children. Therefore, there is a need for a system that provides comprehensive child-rearing support and emotional support so that parents and children can enjoy shopping with peace of mind.

[0653] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0654] A means of collecting child-rearing information;

[0655] A means of obtaining profile data for each child; and

[0656] means for analyzing the collected information based on the child's profile data and generating personalized advice;

[0657] means for notifying a user terminal of the individual advice;

[0658] A means for assessing the emotional state of parents and children and suggesting appropriate support messages and products based on the emotional state;

[0659] A means for managing schedule data of a customer and notifying a schedule reminder to a user terminal;

[0660] This allows parents and children to enjoy shopping in-store with peace of mind, and also allows them to receive individual advice and support regarding child-rearing.

[0661] The "means for collecting child-rearing information" refers to a device or program that has the function of periodically collecting the latest articles and papers from reliable health information sites and child-rearing information sites and analyzing them using text analysis tools.

[0662] "Means for acquiring profile data of individual children" refers to a device or program for acquiring individual data such as the age, health condition, allergy information, emotional state, etc. of the child or the user himself / herself entered by the user.

[0663] "Means for analyzing collected information based on a child's profile data and generating individualized advice" refers to a device or program that has the function of analyzing collected information based on the obtained child's profile data and generating individually tailored advice from the results.

[0664] The "means for notifying the user terminal of the individual advice" is a device or program having a function for notifying the user terminal of the generated individual advice as a message that is easy for the user to understand.

[0665] "Means for assessing the emotional state of parents and children and suggesting appropriate support messages or products based on the emotional state" refers to a device or program that has the function of collecting and analyzing voice and facial expression data of parents and children, assessing their emotional state, and then suggesting the most appropriate support message or product.

[0666] "Means for managing customer schedule data and notifying schedule reminders to the user terminal" refers to a device or program that has the function of managing parent schedule data and children's event schedules and notifying the user terminal of reminders when the scheduled date approaches.

[0667] "Means for receiving data from surveillance cameras and sensors" refers to a device or program that has the function of receiving data in real time from surveillance cameras and sensors installed within the store.

[0668] "Means for analyzing received data and detecting abnormalities" refers to a device or program that has the function of analyzing data received from a surveillance camera or sensor and detecting dangerous situations or abnormalities.

[0669] The "means for issuing a warning when an abnormality is detected" refers to a device or program that has the function of immediately generating and issuing a warning alert when an abnormality is detected.

[0670] The "means for detecting an abnormality and notifying a user terminal of a warning" refers to a device or program having the function of notifying a user terminal of a warning upon detecting an abnormality.

[0671] The "means for collecting children's voice and facial expression data" refers to a device or program for collecting children's voice and facial expression data in real time.

[0672] "Means for analyzing collected data and assessing emotional state" refers to a device or program that has the function of assessing a child's emotional state (e.g., sadness, joy) based on collected data.

[0673] The "means for generating an appropriate message based on the emotional state" refers to a device or program that has the function of generating an appropriate response or support message depending on the evaluated emotional state.

[0674] The "means for notifying the user terminal of the generated message" refers to a device or program having a function for notifying the user terminal of the generated support message.

[0675] The "means for evaluating the emotional state of the parent and generating a support message based on the overall emotional state of the parent and child" refers to a device or program that has the function of analyzing the parent's voice and facial expression data, evaluating the overall emotional state of the parent and child, and generating an appropriate support message based on that.

[0676] The "means for the physical robot to notify the parent and child of the generated support message" is a device or program that has the function of a dedicated robot notifying the parent and child of the generated support message by voice or display.

[0677] The present invention is a system that provides comprehensive support for child-rearing in a brick-and-mortar store by linking an AI-equipped robot, a server, a user terminal, and an emotion engine. The system of the present invention includes the following main components and functions:

[0678] 1. Gathering the latest information and providing personalized advice

[0679] The server periodically collects the latest articles and papers from reliable health and parenting information sites and analyzes them using text analysis tools. The collected information is stored in a database, and personalized advice is generated based on the profile data of the child and parent entered by the user. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[0680] As a specific example, the server collects the latest articles on influenza prevention methods, and based on data entered by the parents about a 5-year-old child (with no allergies), generates advice such as "taking vitamin C is important," and notifies the user's device.

[0681] 2. Child safety checks using surveillance cameras and sensors

[0682] The server receives real-time data from surveillance cameras and motion sensors installed in the store and uses image recognition models to detect dangerous situations. If an abnormality is detected, the server generates a warning alert and notifies the user's device.

[0683] As a specific example, if a surveillance camera receives streaming footage of a child climbing stairs, the server will use an image recognition model to confirm that the child is climbing stairs and send a warning to the user device that "the child is in a dangerous situation."

[0684] 3. Use of emotional support and emotional engines

[0685] The server collects voice and facial expression data from parents and children and analyzes it using an emotion analysis algorithm. It evaluates their emotional state and uses an emotion engine to generate support messages and product suggestions based on that state. The generated messages are sent to the user's device, and the AI ​​robot then speaks to the parent or child.

[0686] As a specific example, if the server collects a child's facial expression and voice data and evaluates the child as "sad," the emotion engine will generate a message saying "It's okay. Is something bothering you?", and the AI ​​robot will speak this to the child and simultaneously notify the user's device.

[0687] 4. Health management and nutrition education support

[0688] The user device provides an interface for parents to input their child's dietary information and health status, and the server stores and analyzes the input data. It identifies nutritional balance and nutrient deficiencies, generates appropriate meal suggestions, and notifies the user device.

[0689] As a specific example, if a parent inputs their child's daily diet and the server identifies that their child is not getting enough vitamin C, the server will notify the user's device with a suggestion to "incorporate salads with oranges, which are rich in vitamin C."

[0690] 5. Schedule management and reminder provision

[0691] The server manages the parent's schedule and the child's event schedule, and generates reminders based on the schedule data, ensuring that parents do not forget important events.

[0692] As a specific example, parents input their child's vaccination schedule, the server stores the schedule, and when the scheduled date approaches, a reminder is sent to the user's device saying, "The vaccination is tomorrow."

[0693] 6. User Emotion Recognition

[0694] The emotion engine collects and analyzes the parent's voice and facial expression data to assess the parent's emotional state. Based on the assessed emotional state, it tailors individual advice and support messages to provide more effective support.

[0695] As a specific example, if a parent is assessed as feeling stressed, the server generates advice such as "We recommend that you make time to relax with your child" and notifies the user terminal of the advice.

[0696] Example of an input prompt for a generative AI model:

[0697] "Based on your child's health, please tell us the latest flu prevention methods and food suggestions."

[0698] This system allows parents and children to enjoy shopping in-store with peace of mind and receive comprehensive childcare support.

[0699] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0700] Step 1:

[0701] The user enters their child's profile data (e.g., age, health status, allergy information, etc.) into the device. The device sends the entered data to the server. At this stage, the input is the user's profile data, and the output is that data sent to the server.

[0702] Step 2:

[0703] The server periodically collects the latest articles and papers from reliable health and child-rearing information sites and analyzes them using text analysis tools. The input is the latest article and paper data from external information sites, and the output is a database that stores the analysis results. Specifically, the server periodically crawls information sites, collects text data, and analyzes and classifies the information using NLP (natural language processing) algorithms.

[0704] Step 3:

[0705] The server compares the user's child's profile data with the latest collected information to generate personalized advice. The input is the user's profile data and the analyzed latest information, and the output is personalized advice. At this stage, the server compares the profile to select relevant information and uses natural language generation (NLG) technology to generate advice messages in a format that is easy for the user to understand.

[0706] Step 4:

[0707] The generated individual advice is notified to the device and displayed to the user. The input is the generated advice message, and the output is a notification that the user sees. Specifically, the advice message is pushed from the server to the device and displayed as a pop-up on the device screen.

[0708] Step 5:

[0709] The server receives real-time data from surveillance cameras and motion sensors. The input is video and motion data from the surveillance cameras and sensors, and the output is a data stream for analysis. The server captures and stores the streaming data provided by the cameras and sensors.

[0710] Step 6:

[0711] The server analyzes the received data using an image recognition model to detect anomalies. The input is real-time data from surveillance cameras and sensors, and the output is the results of anomaly detection. Specifically, the server uses a CNN (convolutional neural network) model to detect abnormal behavior in the video (e.g., a child playing on the stairs).

[0712] Step 7:

[0713] If an anomaly is detected, the server generates a warning alert and notifies the user device. The input is the anomaly detection result, and the output is a warning alert message. When an anomaly is detected, the server constructs the alert message and sends a push notification to the user device.

[0714] Step 8:

[0715] The server collects the voice and facial expression data of the parent and child and analyzes it using an emotion analysis algorithm. The input is the voice and facial expression data of the parent and child, and the output is the evaluation result of the emotional state. The server evaluates the emotional state based on the collected voice and video data using an emotion analysis model (e.g., CNN or RNN).

[0716] Step 9:

[0717] Based on the emotional state, the server generates appropriate support messages and product suggestions and notifies the user device. The input is the evaluation result of the emotional state, and the output is the support message and product suggestions. Based on the evaluation result, the server generates appropriate support messages and notifies the user device.

[0718] Step 10:

[0719] The server manages the parent's schedule data and the child's event schedule and generates schedule reminders. The input is the parent's schedule data and event schedule, and the output is a reminder message. The server uses schedule management software to push reminder notifications to the user's device when the event date approaches.

[0720] Step 11:

[0721] A dedicated robot notifies the parent and child of the generated support message by voice or display. The input is the support message sent from the server, and the output is the notification by voice or display from the robot. Specifically, the robot uses voice synthesis technology to convey the support message to the parent and child, and simultaneously displays it on the display.

[0722] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0723] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0724] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0725] [Second embodiment]

[0726] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0727] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0728] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0729] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0730] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0731] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0732] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0733] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0734] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0735] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0736] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0737] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0738] This invention is a system that provides comprehensive support for child-rearing through collaboration between an AI-equipped robot, a server, and a user device. The following describes the design of this system and the specific operation of each function.

[0739] 1. Gathering the latest information and providing personalized advice

[0740] The server regularly collects the latest articles and papers from reliable health and child-rearing information websites on the Internet. This information is stored in a database by category using text analysis tools. Each child's profile data (age, health condition, allergy information, etc.) is obtained, and the collected information is analyzed based on this to generate individualized advice. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[0741] Examples:

[0742] The server collects the latest articles on "How to prevent winter flu" and stores them in a database.

[0743] Extract relevant preventive measures based on the user's child (age 5, no allergies).

[0744] The system generates advice such as "It is important to take vitamin C" and notifies the user's device.

[0745] 2. Child safety checks using surveillance cameras and sensors

[0746] The server receives real-time data from surveillance cameras and motion sensors. The data is analyzed using image recognition models to detect abnormalities (such as a child climbing stairs). When an abnormality is detected, an alert is generated and sent to the user's device.

[0747] Examples:

[0748] A surveillance camera streams footage of a child accidentally climbing the stairs.

[0749] The server uses an image recognition model to identify anomalies and generates a warning that a child is in danger.

[0750] The warning is notified to the user terminal.

[0751] 3. Emotional support

[0752] The server collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm. It evaluates the child's emotional state (e.g., sadness or joy) and generates an appropriate message based on the results. The generated message is then sent to the user's device.

[0753] Examples:

[0754] The server collects the child's facial expression and voice data and evaluates the child as "sad."

[0755] Based on the analysis results, an encouraging message is generated, such as "It's okay, is there something bothering you?"

[0756] The emotional state and response content are notified to the user terminal.

[0757] 4. Health management and nutrition education support

[0758] The user device provides an interface where parents can input their child's dietary habits and health status. The server stores the input data and accumulates it daily. This data is then analyzed using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies. Based on the results, appropriate meal suggestions are generated and sent to the user device.

[0759] Examples:

[0760] Parents input their child's daily dietary information into a user terminal.

[0761] The server analyzes the data and identifies a lack of vitamin C intake.

[0762] "Eat more oranges and kiwis, which are richer in vitamin C," he suggests.

[0763] The user terminal is notified of the details of the proposal.

[0764] 5. Schedule management and reminder provision

[0765] The server manages the parent's schedule and the child's event schedule, generates reminders based on this schedule, and notifies the user's device as appropriate.

[0766] Examples:

[0767] Parents enter their child's vaccination schedule into a user terminal.

[0768] The server stores the schedule and generates a reminder as the day approaches: "Your vaccination is tomorrow."

[0769] A reminder is sent to the user's device.

[0770] In this way, the system of the present invention provides comprehensive support for child-rearing through functions such as collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and providing reminders.

[0771] The processing flow will be explained below.

[0772] 1. Gathering the latest information and providing personalized advice

[0773] Step 1:

[0774] Server: Regularly accesses reliable health and parenting information sites on the Internet and scrapes new articles and papers.

[0775] Step 2:

[0776] Server: Analyzes the content of collected articles using text analysis tools and stores them in a database by category.

[0777] Step 3:

[0778] Server: Obtains the child's individual profile data entered by the user (e.g., age, health status, allergy information).

[0779] Step 4:

[0780] Server: Based on the analyzed information, it generates personalized advice based on the child's profile data.

[0781] Step 5:

[0782] Server: Using natural language processing technology, the generated advice is converted into a message that is easy for the user to understand.

[0783] Step 6:

[0784] User device: The generated advice message is sent to the parent's smartphone or tablet.

[0785] Examples:

[0786] 1. The server collects the latest articles on "How to prevent winter influenza" and stores them in a database.

[0787] 2. The server retrieves the child's profile data and extracts relevant preventive measures.

[0788] 3. Generate advice such as "It is important to take vitamin C" and notify the user's device.

[0789] 2. Child safety checks using surveillance cameras and sensors

[0790] Step 1:

[0791] Server: Streams and receives real-time data from surveillance cameras and motion sensors.

[0792] Step 2:

[0793] Server: Analyzes received video data using an image recognition model to detect dangerous situations.

[0794] Step 3:

[0795] Server: Generates a warning alert if an anomaly is detected.

[0796] Step 4:

[0797] AI robot: Based on abnormality alerts received from the server, it issues a warning according to pre-set response methods.

[0798] Step 5:

[0799] User device: Real-time alert notifications are sent to parents' smartphones or tablets.

[0800] Examples:

[0801] 1. A security camera streams footage of a child accidentally climbing the stairs.

[0802] 2. The server uses an image recognition model to identify the child climbing the stairs and generates an anomaly alert.

[0803] 3. The AI ​​robot issues a warning that a child is in danger and notifies the user's device.

[0804] 3. Emotional support

[0805] Step 1:

[0806] Server: Using voice and facial recognition functions, collects children's voice and facial expression data in real time.

[0807] Step 2:

[0808] Server: Analyzes the collected data using an emotion analysis algorithm to evaluate the child's emotional state.

[0809] Step 3:

[0810] Server: Generates appropriate responses and support messages depending on the assessed emotional state.

[0811] Step 4:

[0812] AI robot: Speaks to children based on the generated message.

[0813] Step 5:

[0814] User device: Notifies parents of the child's emotional state and the AI ​​robot's response.

[0815] Examples:

[0816] 1. The server collects the child's facial expression and voice data and evaluates the child as "sad."

[0817] 2. The server generates an encouraging message based on the evaluation results, saying, "It's okay, is there something bothering you?"

[0818] 3. The AI ​​robot conveys the message to the child and notifies the user's device of their emotional state and response.

[0819] 4. Health management and nutrition education support

[0820] Step 1:

[0821] User terminal: Provides an interface where parents can input their child's diet and health status.

[0822] Step 2:

[0823] Server: Saves the entered data and accumulates daily data.

[0824] Step 3:

[0825] Server: Analyzes the accumulated data using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies.

[0826] Step 4:

[0827] Server: Generates appropriate meal suggestions based on the analysis results.

[0828] Step 5:

[0829] User terminal: Notifies parents of the generated meal suggestions and nutritional status reports.

[0830] Examples:

[0831] 1. Parents enter their child's daily dietary information into the user device.

[0832] 2. The server analyzes the input data and identifies that the person is lacking in vitamin C intake.

[0833] 3. "Incorporate more foods rich in vitamin C" is suggested as a specific recipe, and a notification is sent to the user's device.

[0834] 5. Schedule management and reminder provision

[0835] Step 1:

[0836] User terminal: Provides an interface for parents to input event schedules and schedules.

[0837] Step 2:

[0838] Server: Saves and manages the entered schedule data.

[0839] Step 3:

[0840] Server: Generates reminders based on schedule data.

[0841] Step 4:

[0842] User device: Notifies the parent of the generated reminder.

[0843] Examples:

[0844] 1. Parents enter their child's vaccination schedule into the user device.

[0845] 2. The server stores the appointment and generates a reminder when the appointment date approaches.

[0846] 3. The user device sends a reminder to the parent that "Vaccination is tomorrow."

[0847] Example 1

[0848] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0849] In child-rearing, it is often difficult for parents to receive the latest health information or appropriate advice based on the individual condition of their child, and children's safety and emotional care are often not adequately confirmed. This creates problems that make it difficult for parents to adequately support their children's health, safety, and emotional growth. To solve these problems, a comprehensive system of support for child-rearing is needed.

[0850] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0851] In this invention, the server includes a means for collecting parenting information, a means for acquiring profile data for each child, and a means for classifying and saving the collected information using a text analysis tool. This allows parents to obtain the latest parenting information tailored to their individual circumstances in a timely manner. The server also includes a means for analyzing the collected information based on the child's profile data using a machine learning algorithm to generate personalized advice, a means for notifying a user terminal of the generated personalized advice using natural language processing technology, a means for receiving data from surveillance cameras and sensors, a means for analyzing the received data using an image recognition model to detect abnormalities, a means for generating a warning when an abnormality is detected and notifying the user terminal, a means for collecting voice and facial expression data of the child, a means for analyzing the collected data using an emotion analysis algorithm to evaluate the emotional state, a means for generating an appropriate message based on the emotional state using natural language processing technology, and a means for notifying the user terminal of the generated message. This enables comprehensive monitoring and analysis of a child's health, safety, and emotional state, and for providing appropriate advice and warnings in real time.

[0852] "Means for collecting parenting information" include devices and programs for collecting the latest articles and papers from reliable health and parenting information sites, specifically scraping tools and APIs.

[0853] "Means for obtaining individual child profile data" refers to a database and its interface for collecting and storing individual information such as each child's age, health condition, and allergy information, and for retrieving it as needed.

[0854] "Means for classifying and storing collected information using text analysis tools" refers to devices or programs that include text analysis tools for analyzing collected information, classifying it by category, and storing it in a database.

[0855] "Means for analyzing using machine learning algorithms and generating personalized advice" refers to devices or programs that analyze collected data using machine learning algorithms and generate personalized advice based on each child's profile.

[0856] "Means for notifying a user terminal using natural language processing technology" refers to a device or program that uses natural language processing technology to generate a message from the generated advice in a form that is easy for the user to understand, and then sends that message to the user terminal.

[0857] "Means for receiving data from surveillance cameras and sensors" refers to devices and programs that transmit video and motion data from surveillance cameras and various sensors to a server in real time.

[0858] "Means for analyzing and detecting abnormalities using an image recognition model" refers to a device or program that analyzes received video and motion data using an image recognition model and detects abnormalities.

[0859] "Means for generating a warning when an abnormality is detected and notifying the user terminal" refers to a device or program that generates a warning message when an abnormality is detected and sends that message to the user terminal.

[0860] "Means for collecting children's voice and facial expression data" refers to devices such as cameras and microphones for collecting children's voice and facial expression data in real time, as well as programs for operating them.

[0861] "Means for evaluating a child's emotional state by analyzing using an emotion analysis algorithm" refers to a device or program that analyzes collected voice and facial expression data using an emotion analysis algorithm to evaluate a child's emotional state.

[0862] "Means for generating an appropriate message based on an emotional state using natural language processing technology" refers to a device or program that generates an appropriate message based on an evaluated emotional state using natural language processing technology.

[0863] The "means for notifying the user terminal of the generated message" refers to a device or program for transmitting the generated message to the user terminal.

[0864] This invention is a system that provides comprehensive support for child-rearing through collaboration between an AI-equipped robot, a server, and a user device. This system has multiple functions, including collecting the latest information and providing individualized advice, checking children's safety using surveillance cameras and sensors, providing emotional support, supporting health management and nutrition education, and managing schedules and providing reminders.

[0865] 1. Gathering the latest information and providing personalized advice

[0866] The server regularly collects the latest articles and papers from reliable health and parenting information websites. To do this, it uses Python scripts and the BeautifulSoup library to perform scraping. It then uses a text analysis tool (e.g., Apache OpenNLP) to categorize the collected information and store it in a database (e.g., MySQL). It obtains the user's child's profile data (e.g., age, health condition, allergy information, etc.) and analyzes it using machine learning algorithms such as scikit-learn. The generated advice is then converted into a user-friendly message using natural language processing technology (e.g., OpenAI's GPT-3) and sent to the user's device as a push notification.

[0867] Examples:

[0868] The server collects the latest articles on "How to prevent winter flu" and stores them in a database.

[0869] Extract relevant preventive measures based on the user's child (age 5, no allergies).

[0870] The system generates advice such as "It is important to take vitamin C" and notifies the user's device.

[0871] Example prompt sentence:

[0872] Input: Please tell me how to prevent winter flu in my child (5 years old, no allergies).

[0873] Output: Vitamin C intake is important. Eat plenty of oranges and kiwis.

[0874] 2. Child safety checks using surveillance cameras and sensors

[0875] The server receives real-time data from surveillance cameras (e.g., Nest Cam) and motion sensors (e.g., PIR sensors). The received data is analyzed using an image recognition model (e.g., TensorFlow's YOLO model) to detect anomalies. If an anomaly is detected, the server generates an alert and notifies the user device.

[0876] Examples:

[0877] A surveillance camera streams footage of a child accidentally climbing the stairs.

[0878] The server uses an image recognition model to identify anomalies and generates a warning that a child is in danger.

[0879] The warning is notified to the user terminal.

[0880] Example prompt sentence:

[0881] Input: Footage from a security camera shows a child starting to climb the stairs.

[0882] Output: Warning! Child in danger. Please act immediately.

[0883] 3. Emotional support

[0884] The server collects the child's voice and facial expression data and analyzes it using emotion analysis algorithms such as Microsoft Face API and Google Cloud Speech-to-Text. It evaluates the child's emotional state and generates an appropriate message based on the results. The message is then sent to the user's device.

[0885] Examples:

[0886] The server collects the child's facial expression and voice data and evaluates the child as "sad."

[0887] Based on the analysis results, an encouraging message is generated, such as "It's okay, is there something bothering you?"

[0888] The emotional state and response content are notified to the user terminal.

[0889] Example prompt sentence:

[0890] Input: How do you respond when your child is sad?

[0891] Output: Try gently saying, "It's okay. Is there something bothering you?"

[0892] 4. Health management and nutrition education support

[0893] The user device provides an interface where parents can input their child's dietary information and health status. The server stores the input data in a database and accumulates a daily diet history. This data is then analyzed using the Nutrition Data API to identify nutritional balance and nutrient deficiencies. Based on the results, appropriate meal suggestions are generated and notified to the user device.

[0894] Examples:

[0895] Parents input their child's daily dietary information into a user terminal.

[0896] The server analyzes the data and identifies a lack of vitamin C intake.

[0897] "Eat more oranges and kiwis, which are richer in vitamin C," he suggests.

[0898] The user terminal is notified of the details of the proposal.

[0899] Example prompt sentence:

[0900] Enter: If today's meal doesn't include oranges or kiwi, what are your suggestions?

[0901] Output: Eat more oranges and kiwis, which are rich in vitamin C.

[0902] 5. Schedule management and reminder provision

[0903] The server manages the parent's schedule and the child's event schedule, generates reminders based on this schedule, and notifies the user's device as appropriate.

[0904] Examples:

[0905] Parents enter their child's vaccination schedule into a user terminal.

[0906] The server stores the schedule and generates a reminder as the day approaches: "Your vaccination is tomorrow."

[0907] A reminder is sent to the user's device.

[0908] Example prompt sentence:

[0909] Enter: Set a reminder so you don't forget to schedule your vaccination appointment.

[0910] Output: Your vaccination is tomorrow. Don't forget.

[0911] In this way, the system of the present invention provides comprehensive support for child-rearing through functions such as collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders.

[0912] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0913] 1. Gathering the latest information and providing personalized advice

[0914] Step 1:

[0915] The server collects the latest articles and papers from reliable health and child-rearing information sites on the Internet.

[0916] Input: List of URLs to be collected

[0917] What it does: Use a Python script to parse the HTML and extract data from each URL using the BeautifulSoup library.

[0918] Output: Collected text data

[0919] Step 2:

[0920] The server uses text analysis tools to categorize the collected information.

[0921] Input: Collected text data

[0922] What it does: It uses Apache OpenNLP to analyze text and classify it into categories.

[0923] Output: Text data classified by category

[0924] Step 3:

[0925] The server stores the classified information in a database.

[0926] Input: Categorized text data

[0927] Specific behavior: Connects to a MySQL database and inserts data into the appropriate tables.

[0928] Output: Saved database entries

[0929] Step 4:

[0930] The server retrieves the profile data for each child from the database.

[0931] Input: Child's identification

[0932] Specific operation: Issues an SQL query and retrieves profile data.

[0933] Output: Child profile data

[0934] Step 5:

[0935] The server uses machine learning algorithms to analyze the profile data and collected information to generate personalized advice.

[0936] Input: Profile data, categorical text data

[0937] What it does: Uses the scikit-learn library to analyze information that matches the profile data.

[0938] Output: personalized advice

[0939] Step 6:

[0940] The server notifies the generated individual advice to the user terminal using natural language processing technology.

[0941] Input: Personalized Advice

[0942] Specific operation: Using OpenAI's GPT-3, messages are generated in natural language and sent to the user's device via the push notification API.

[0943] Output: Notification message to the user terminal

[0944] 2. Child safety checks using surveillance cameras and sensors

[0945] Step 1:

[0946] The server receives data in real time from surveillance cameras and sensors.

[0947] Input: Real-time camera and sensor data

[0948] Specific operation: Use the streaming API to receive data.

[0949] Output: Received real-time data

[0950] Step 2:

[0951] The server analyzes the received data using an image recognition model.

[0952] Input: Real-time data

[0953] Specific operation: Detect abnormal behavior using TensorFlow's YOLO model.

[0954] Output: Analysis results (normal / abnormal flag)

[0955] Step 3:

[0956] When the server detects an abnormality, it generates a warning and notifies the user terminal.

[0957] Input: Analysis results (abnormal flag)

[0958] Specific operation: If an abnormality is detected, a warning message is generated and sent to the user device via the push notification API.

[0959] Output: A warning message to the user's terminal.

[0960] 3. Emotional support

[0961] Step 1:

[0962] The server collects the child's voice and facial expression data.

[0963] Input: Camera and microphone data

[0964] Specific operation: Receives a data stream from a device.

[0965] Output: Collected voice and facial expression data

[0966] Step 2:

[0967] The server analyzes the collected data using a sentiment analysis algorithm.

[0968] Input: Voice and facial expression data

[0969] Specific behavior: Emotional state is assessed using Microsoft Face API and Google Cloud Speech-to-Text.

[0970] Output: Emotional state evaluation result

[0971] Step 3:

[0972] The server generates an appropriate message based on the emotional state using natural language processing techniques.

[0973] Input: Emotional state assessment results

[0974] Specific operation: Messages are generated using OpenAI's GPT-3.

[0975] Output: The generated message

[0976] Step 4:

[0977] The server notifies the user terminal of the generated message.

[0978] Input: The generated message

[0979] Specific operation: Send a message via the Push notification API.

[0980] Output: Notification message to the user terminal

[0981] 4. Health management and nutrition education support

[0982] Step 1:

[0983] The user inputs the child's diet and health condition into the user terminal.

[0984] Input: Data entered by the user

[0985] Specific operation: Fill in a form in an application on the user's device.

[0986] Output: Entered dietary and health data

[0987] Step 2:

[0988] The server saves the entered data in a database.

[0989] Input: Dietary and health status data

[0990] Specific action: Add a new entry to the database.

[0991] Output: Saved database records

[0992] Step 3:

[0993] The server analyzes the data using a nutritional analysis algorithm.

[0994] Input: Saved database records

[0995] Specific behavior: Uses the Nutrition Data API to identify nutritional balances and nutrient deficiencies.

[0996] Output: Analysis results (nutritional balance, nutrient deficiencies)

[0997] Step 4:

[0998] The server generates meal suggestions based on the analysis results and notifies the user terminal.

[0999] Input: Analysis results

[1000] Specific operation: Generates suggestions and sends them via the Push notification API.

[1001] Output: Meal suggestion message to user device

[1002] 5. Schedule management and reminder provision

[1003] Step 1:

[1004] The user inputs the child's event schedule and vaccination schedule into the user terminal.

[1005] Input: Schedule data entered by the user

[1006] Specific operation: Enter a schedule into an application on the user's device.

[1007] Output: Schedule data entered

[1008] Step 2:

[1009] The server stores the entered schedule data in a database.

[1010] Input: Schedule data

[1011] Specific Action: Adds a new schedule entry to the database.

[1012] Output: Saved database records

[1013] Step 3:

[1014] The server generates a reminder based on the schedule data and notifies the user terminal.

[1015] Input: Saved schedule data

[1016] Specific behavior: Generates reminder content and sends it via the Push notification API at the appropriate time.

[1017] Output: Reminder message to user device

[1018] (Application example 1)

[1019] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1020] Conventional methods for managing factory robots have had problems with insufficient efficient robot operation and safety. In particular, it has been difficult to provide appropriate advice and warnings based on the individual needs of each robot, and to properly manage the timing of maintenance. Furthermore, there has been a lack of systems for monitoring the operating status of robots in real time and taking appropriate action quickly. To solve these issues, a more advanced and comprehensive robot management system is needed.

[1021] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1022] In this invention, the server includes means for collecting information, means for acquiring individual profile data, means for analyzing the collected information based on the profile data and generating individual advice, means for notifying the user terminal of the individual advice, means for receiving and analyzing robot status data, means for generating an appropriate message based on the analysis results, and means for notifying the user terminal of the generated message, thereby enabling efficient operation of the robot, ensuring safety, and maintenance management.

[1023] "Information gathering means" are devices and programs that obtain the latest information and articles from reliable data sources.

[1024] The "means for acquiring profile data" refers to a device or program that collects information about a specific robot or target device and saves it as profile data.

[1025] The "means for analyzing information and generating personalized advice" refers to a device or program that analyzes the collected data and generates advice tailored to the specific needs of the subject.

[1026] The "means for notifying the user terminal of the advice" refers to a device or program that transmits the generated advice to the user terminal.

[1027] The "means for receiving and analyzing robot status data" refers to a device or program that collects data on the robot's operating status and performance and analyzes that data.

[1028] The "means for generating an appropriate message" refers to a device or program that generates an effective message to notify the user based on the analysis results.

[1029] The "means for notifying the user terminal of the generated message" refers to a device or program that quickly and reliably transmits the generated message to the user terminal.

[1030] "Means for receiving data from monitoring devices and sensors" refers to devices and programs that receive data collected from monitoring devices and various sensors within the factory.

[1031] "Means for detecting abnormalities" refers to a device or program that analyzes received data, identifies any abnormal conditions, and issues a warning.

[1032] "Means for analyzing efficiency" refers to devices or programs that analyze the operation data of each robot and evaluate the work efficiency.

[1033] The "means for generating advice for improving efficiency" is a device or program that provides specific advice for improving the work efficiency of the robot based on the analysis results.

[1034] "Means for managing maintenance schedules and sending reminders" refers to devices or programs that manage the timing of robot maintenance and notify users of maintenance when necessary.

[1035] The present invention relates to a factory robot management system that comprehensively supports the efficient operation and safety of factory robots by linking servers, monitoring devices, sensors, and user terminals.

[1036] System configuration

[1037] The server first has a means to collect the latest information and articles from reliable data sources (e.g., technology sites and industry information sites). The collected information is analyzed based on the profile data, and personalized advice is generated and sent to the user's device. This profile data is obtained based on the characteristics and demands of each individual robot.

[1038] Monitoring devices and sensors monitor the operating status of each robot in the factory and the surrounding environment in real time, and send the data to a server. The server analyzes the received data and issues an alert if an abnormality is detected, notifying the user's device. The server also analyzes the efficiency of the robots and generates advice for improving efficiency.

[1039] Furthermore, the server manages the robot's maintenance schedule and has a means for sending reminders, allowing the user to perform maintenance at the appropriate time.

[1040] Processing description

[1041] 1. Collection of information:

[1042] The server periodically retrieves up-to-date information from reliable data sources using the Python requests library, which is then parsed by text analysis tools and stored in a database.

[1043] 2. Analyzing data and generating advice:

[1044] Based on the profile data, the collected information is analyzed and personalized advice is generated based on each robot's condition and needs. The generated advice is then sent to the user's device using natural language processing technology.

[1045] 3. Monitoring and Alerting:

[1046] Data received from monitoring devices and sensors is analyzed using image recognition and motion detection models, and if an anomaly is detected, an alert is generated immediately and sent to the user's device.

[1047] 4. Maintenance Management:

[1048] The server manages the maintenance schedule for each robot and sends reminders as appropriate, based on a pre-set schedule.

[1049] Specific examples

[1050] Examples of gathering up-to-date information:

[1051] The server collects articles about "robot maintenance techniques," analyzes the information, and applies it to "Robot 1."

[1052] Safety monitoring example:

[1053] The monitoring device detects dangerous behavior of the robot while it is in operation and generates an alert to notify the administrator.

[1054] Efficiency analysis and advice examples:

[1055] The operation data of each robot is analyzed, and specific advice (e.g., how to optimize operation) to improve the efficiency of "Robot 2" is generated and notified to the user terminal.

[1056] Maintenance reminder example:

[1057] The server manages the maintenance schedule for "Robot 3" and sends a reminder "Maintenance is required" the day before the maintenance.

[1058] Prompt Sentence Examples

[1059] Prompt sentence to input to the generative AI model:

[1060] "Generate the latest maintenance advice based on the following robot ID and its current operational status: ID: robot_1, Status: Up, Maintenance Cycle: 1 month."

[1061] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1062] Step 1:

[1063] The server collects the latest information from a reliable data source. Here, the server uses Python's requests library to access URLs and retrieve article data on the latest technical information and maintenance techniques. The input is the URL, and the output is the retrieved latest information data.

[1064] Step 2:

[1065] The server analyzes the collected information using a text analysis tool (e.g., a natural language processing library). As a result of the analysis, the information is classified into categories. The input is the collected information data, and the output is the data classified by category.

[1066] Step 3:

[1067] The server obtains the profile data of each robot. This profile data includes information such as the robot's ID, characteristics, maintenance cycle, etc. The input is the robot's individual data request, and the output is the profile data.

[1068] Step 4:

[1069] The server generates appropriate personalized advice from the collected information based on the profile data. Specifically, it uses natural language processing technology to select relevant articles and information and restructures the content to suit the robot's needs. The input is profile data and categorized data, and the output is personalized advice.

[1070] Step 5:

[1071] The server notifies the user device of the generated personalized advice. Notifications are sent via APIs or messaging protocols, and users can receive them on their smartphones or tablets. The input is personalized advice, and the output is a notification sent to the user device.

[1072] Step 6:

[1073] Monitoring devices and sensors collect data in real time and send it to a server. Here, hardware such as cameras are used to monitor the robot's operation. The input is monitoring data, and the output is data sent to the server.

[1074] Step 7:

[1075] The server analyzes the received data and detects anomalies. Image recognition and motion detection models are used for the analysis. If an anomaly is detected, an alert is generated. The input is the monitoring data, and the output is the anomaly detection result.

[1076] Step 8:

[1077] If an abnormality is detected, the server immediately sends a warning to the user terminal. The user receives the warning message and can respond promptly. The input is the abnormality detection result, and the output is a warning notification to the user terminal.

[1078] Step 9:

[1079] The server analyzes the efficiency of each robot. Historical and real-time data are used for efficiency analysis to identify movement patterns and efficient operation methods. The input is movement data, and the output is the efficiency analysis results.

[1080] Step 10:

[1081] The server generates specific advice for improving efficiency and notifies the user terminal. This advice is created using natural language processing technology. The input is the efficiency analysis result, and the output is efficiency improvement advice.

[1082] Step 11:

[1083] The server manages the robot's maintenance schedule and sends reminders. Here, it notifies the robot of the need for maintenance based on a pre-defined schedule. The input is the maintenance schedule and the output is the reminder notification.

[1084] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1085] This invention is a system that comprehensively supports child-rearing by linking an AI-equipped robot, a server, a user terminal, and an emotion engine. The design of this system and the specific operation method of each function are explained below.

[1086] 1. Gathering the latest information and providing personalized advice

[1087] The server regularly collects the latest articles and papers from reliable health and child-rearing information sites and analyzes them using text analysis tools. The analyzed information is then stored in a database by category. Individual profile data (e.g., age, health condition, allergy information, emotional state) entered by the user for the child and the user themselves is then obtained, and the collected information is analyzed based on this to generate individual advice. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[1088] Examples:

[1089] The server collects the latest articles on "How to Prevent Influenza" and stores them in a database.

[1090] The user's child (5 years old, no allergies) and the user's emotional state are obtained, and relevant preventive measures are extracted.

[1091] It generates an "encouraging message according to the user's emotional state" and advice such as "taking vitamin C is important," and notifies the user's device.

[1092] 2. Child safety checks using surveillance cameras and sensors

[1093] The server receives real-time data from surveillance cameras and motion sensors, analyzes it with image recognition models to detect dangerous situations, and if an abnormality is detected, generates a warning alert and notifies the user's device.

[1094] Examples:

[1095] A surveillance camera streams footage of a child accidentally climbing the stairs.

[1096] The server uses an image recognition model to identify a child climbing stairs and generates an anomaly alert.

[1097] A warning that "child is in danger" is sent to the user terminal.

[1098] 3. Use of emotional support and emotional engines

[1099] The server collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm. It evaluates the child's emotional state (e.g., sadness, joy), and uses an emotion engine to generate an appropriate response or support message based on that state. The generated message, taking into account the user's emotional state, is then sent to the user's device, where the AI ​​robot then speaks to the child.

[1100] Examples:

[1101] The server collects the child's facial expressions and voice data, and uses an emotion analysis algorithm to assess whether the child is "sad."

[1102] Sentiment analysis algorithms also assess the user's emotional state and generate a comprehensive message.

[1103] The AI ​​robot will ask the child, "It's okay. Is there something bothering you?" and notify the user's device of its emotional state and response.

[1104] 4. Health management and nutrition education support

[1105] The user device provides an interface for parents to input their child's dietary habits and health status, and the server saves and stores the data. This data is analyzed using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies. Based on the analysis results, appropriate meal suggestions are generated and notified to the user device.

[1106] Examples:

[1107] Parents input their child's daily dietary information into a user terminal.

[1108] The server analyzes the data and identifies a lack of vitamin C intake.

[1109] "Incorporate more foods rich in vitamin C," he says, suggesting specific recipes.

[1110] 5. Schedule management and reminder provision

[1111] The server manages the parent's schedule and the child's event schedule, and generates reminders based on this schedule data. The generated reminders are then sent to the user's device.

[1112] Examples:

[1113] Parents enter their child's vaccination schedule into a user terminal.

[1114] The server stores the appointment and generates a reminder when the appointment date approaches.

[1115] The user terminal notifies the parent with a reminder that "Vaccination is tomorrow."

[1116] 6. User Emotion Recognition

[1117] The emotion engine collects and analyzes the user's voice and facial expression data to assess the user's emotional state, and tailors personalized advice and support messages based on the assessed emotional state to provide more effective support.

[1118] Examples:

[1119] The emotion engine analyzes the user's tone of voice and facial expressions and assesses them as "feeling stressed."

[1120] The server receives the evaluation results and generates advice such as, "We recommend that you spend some time relaxing with your child."

[1121] The system notifies the user terminal of the advice and suggests music and activities that will help parents and children relax.

[1122] In this way, the system of the present invention provides comprehensive support for child-rearing by collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders, and recognizing users' emotions using an emotion engine.

[1123] The processing flow will be explained below.

[1124] 1. Gathering the latest information and providing personalized advice

[1125] Step 1:

[1126] Server: Regularly accesses reliable health and parenting information websites and scrapes new articles and papers.

[1127] Step 2:

[1128] Server: Analyzes the content of collected articles using text analysis tools and stores them in a database by category.

[1129] Step 3:

[1130] Server: Obtains individual profile data (e.g., age, health condition, allergy information, emotional state) of the child and the user themselves entered by the user.

[1131] Step 4:

[1132] Server: Analyzes the collected information and generates personalized advice based on the profile data.

[1133] Step 5:

[1134] Server: Using natural language processing technology, the generated advice is converted into a message that is easy for the user to understand.

[1135] Step 6:

[1136] User device: The generated advice message is sent to the parent's smartphone or tablet.

[1137] Examples:

[1138] 1. The server collects the latest articles about "How to prevent influenza" and stores them in a database.

[1139] 2. The server retrieves the child's profile data and extracts relevant preventive measures.

[1140] 3. Generate advice such as "It is important to take vitamin C" and notify the user's device.

[1141] 2. Child safety checks using surveillance cameras and sensors

[1142] Step 1:

[1143] Server: Streams and receives real-time data from surveillance cameras and motion sensors.

[1144] Step 2:

[1145] Server: Analyzes received video data using an image recognition model to detect dangerous situations.

[1146] Step 3:

[1147] Server: Generates a warning alert if an anomaly is detected.

[1148] Step 4:

[1149] AI robot: Based on abnormality alerts received from the server, it issues a warning according to pre-set response methods.

[1150] Step 5:

[1151] User device: Sends real-time alert notifications to parents' smartphones and tablets.

[1152] Examples:

[1153] 1. A security camera streams footage of a child accidentally climbing the stairs.

[1154] 2. The server uses an image recognition model to identify the child climbing the stairs and generates an anomaly alert.

[1155] 3. The AI ​​robot issues a warning that a child is in danger and notifies the user's device.

[1156] 3. Use of emotional support and emotional engines

[1157] Step 1:

[1158] Server: Using voice and facial recognition functions, collects children's voice and facial expression data in real time.

[1159] Step 2:

[1160] Server: Analyzes the collected data using an emotion analysis algorithm to evaluate the child's emotional state.

[1161] Step 3:

[1162] Server: Depending on the assessed emotional state, it uses an emotion engine to generate appropriate responses and support messages.

[1163] Step 4:

[1164] AI robot: Speaks to children based on the generated message.

[1165] Step 5:

[1166] User device: Notifies parents of the child's emotional state and the AI ​​robot's response.

[1167] Examples:

[1168] 1. The server collects the child's facial expression and voice data and uses an emotion analysis algorithm to evaluate the child as "sad."

[1169] 2. Based on the evaluation results, the server generates an encouraging message saying, "It's okay, is there anything you're having trouble with?"

[1170] 3. The AI ​​robot conveys the message to the child and notifies the user's device of their emotional state and response.

[1171] 4. Health management and nutrition education support

[1172] Step 1:

[1173] User terminal: Provides an interface for parents to input their child's dietary information and health status.

[1174] Step 2:

[1175] Server: Saves the entered data and accumulates daily data.

[1176] Step 3:

[1177] Server: Analyzes the accumulated data using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies.

[1178] Step 4:

[1179] Server: Generates appropriate meal suggestions based on the analysis results.

[1180] Step 5:

[1181] User terminal: Notifies parents of the generated meal suggestions and nutritional status report.

[1182] Examples:

[1183] 1. The user inputs the child's daily meal plan into the user terminal.

[1184] 2. The server analyzes the input data and identifies that the person is lacking in vitamin C intake.

[1185] 3. The server suggests specific recipes, such as "Incorporate more foods rich in vitamin C," and notifies the user's device.

[1186] 5. Schedule management and reminder provision

[1187] Step 1:

[1188] User terminal: Provides an interface for parents to input event schedules and schedules.

[1189] Step 2:

[1190] Server: Saves and manages the entered schedule data.

[1191] Step 3:

[1192] Server: Generates reminders based on schedule data.

[1193] Step 4:

[1194] User device: Notifies the parent of the generated reminder.

[1195] Examples:

[1196] 1. The user enters their child's vaccination schedule into the user terminal.

[1197] 2. The server stores the appointment and generates a reminder when the appointment date approaches.

[1198] 3. The user device sends a reminder to the parent that "Vaccination is tomorrow."

[1199] 6. User Emotion Recognition

[1200] Step 1:

[1201] Emotion engine: Collects and analyzes the user's voice and facial expression data.

[1202] Step 2:

[1203] Server: Receives data from the emotion engine and evaluates the user's emotional state.

[1204] Step 3:

[1205] Server: Tailors personalized advice and support messages based on the assessed emotional state.

[1206] Step 4:

[1207] User terminal: Notifies the parent of the generated support message.

[1208] Examples:

[1209] 1. The emotion engine analyzes the user's tone of voice and facial expressions and assesses them as "feeling stressed."

[1210] 2. The server receives the evaluation results and generates advice such as, "We recommend that you spend some time relaxing with your child."

[1211] 3. The user device will notify them of the advice and suggest relaxing music and activities.

[1212] This system allows AI-equipped robots, servers, user devices, and emotion engines to work together to provide comprehensive support for child-rearing.

[1213] Example 2

[1214] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1215] In modern society, raising children has become an extremely diverse and complex task. Busy parents often find it difficult to obtain accurate and up-to-date information on child rearing, making it even more difficult to comprehensively address their children's safety, health, and emotional support. In particular, many aspects must be considered simultaneously, including monitoring to ensure children's safety, providing emotional care, and managing nutritional balance. The present invention aims to provide a system that comprehensively solves these complex child rearing challenges.

[1216] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting child-rearing related information, means for acquiring profile data of each child, means for analyzing the collected information based on the profile data and generating individual advice, means for notifying the user terminal of the individual advice, means for collecting emotional data of family members and evaluating their emotional states using an emotion analysis algorithm, means for generating an appropriate response message based on their emotional states, and means for notifying the user terminal of the response message and responding to the user via a voice output device. This enables comprehensive support for child-rearing.

[1217] "Child-rearing related information" refers to the latest health, educational, and psychological information related to child-rearing, as well as other information useful for raising children.

[1218] "Profile Data" refers to individual data about each child and their parents, such as age, health status, allergy information, and emotional state.

[1219] An "emotion analysis algorithm" refers to a computational method for analyzing emotional states from voice data, facial expression data, etc., and evaluating the results.

[1220] A "response message" refers to an appropriate message to a user that is generated based on the user's emotional state and individual circumstances.

[1221] "Monitoring equipment" refers to equipment that uses surveillance devices such as cameras and sensors to monitor children's activities and surrounding conditions in real time.

[1222] A "nutritional analysis algorithm" refers to a calculation method that analyzes nutritional balance based on input dietary data and suggests necessary nutrients and appropriate meals.

[1223] "Reminder" refers to a warning or attention message that notifies the user based on a pre-set schedule.

[1224] The term "audio output device" refers to a device for actually transmitting the generated voice message to the user as voice.

[1225] The present invention provides a comprehensive support system for child rearing. This system realizes child safety confirmation, health management, emotional support, and schedule management by linking together a server, terminals, users, an emotion engine, a monitoring device, and a nutritional analysis algorithm.

[1226] The server periodically collects the latest articles and papers from reliable health and parenting information websites on the Internet. Web scraping tools and APIs are used for collection. Specifically, Beautiful Soup is used for web scraping, and the PubMed API is used for data collection. The collected data is analyzed using text analysis tools (e.g., NLTK, SpaCy) and stored by category in a database (e.g., MySQL, MongoDB).

[1227] Users enter their own and their children's profile data (e.g., age, health condition, allergy information, emotional state) on their device. The device sends the entered data to the server, which analyzes the data and generates personalized advice. The personalized advice is generated using natural language processing technology (e.g., GPT-4, BERT) and notified to the user's device.

[1228] For example, the server collects the latest articles on "How to prevent influenza" and stores them in a database. When a user inputs the profile data of a child (age 5, no allergies), the server analyzes the relevant prevention methods, generates advice such as "It is important to take vitamin C," and notifies the user's device.

[1229] Monitoring devices (e.g., cameras, motion sensors) send real-time data of children to a server. The server uses image recognition models (e.g., YOLO, OpenCV) to analyze the monitoring data and detect abnormalities. If an abnormality is detected, the server generates a warning alert and notifies the user's device.

[1230] Specifically, a surveillance camera receives footage of a child accidentally climbing stairs, and the server analyzes the footage to detect any abnormalities, then sends a warning to the user's device that the child is in danger.

[1231] The server also collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm (e.g., Affectiva, Microsoft Azure Emotion API). Based on the analysis results, the server evaluates the child's emotional state and generates an appropriate response message. This message is sent to the user's device, which responds to the child via the voice output device.

[1232] For example, the server collects a child's facial expression and voice data and uses an emotion analysis algorithm to evaluate the child as "sad." The server then generates a message saying, "It's okay. Is something bothering you?", and the AI ​​robot responds, notifying the user's device of the child's emotional state and the response.

[1233] Furthermore, the user device provides an interface for parents to input their child's dietary information and health status, and sends the data to a server. The server then analyzes the input data using a nutritional analysis algorithm (e.g., MyFitnessPal's API). Based on the analysis results, the nutritional balance is evaluated and appropriate meal suggestions are generated.

[1234] For example, if a parent inputs their child's diet and the server identifies a vitamin C deficiency, it will suggest specific recipes such as "Incorporate more vitamin C-rich foods."

[1235] The server manages schedule data, generates reminders based on this data, and notifies the user device. For example, when a parent inputs their child's vaccination schedule, the server saves the schedule and sends a reminder that "Tomorrow's vaccination date" when the scheduled date approaches.

[1236] The emotion engine then evaluates the user's emotional state and generates personalized advice and support messages based on the evaluation results, thereby providing optimal support that takes the user's emotional state into account.

[1237] Example prompt sentence:

[1238] 1. "What are some good sleep guidelines for a 1-year-old?"

[1239] 2. "What should I do if I'm deficient in vitamin C?"

[1240] 3. "What safety precautions should I take when climbing stairs?"

[1241] 4. "What are some ways parents and children can relax when they're feeling stressed?"

[1242] In this way, the system of the present invention provides comprehensive support for child-rearing by collecting the latest information and providing individual advice, checking children's safety using monitoring devices and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders, and recognizing users' emotions using an emotion engine.

[1243] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1244] Step 1:

[1245] The server collects child-rearing related information.

[1246] Input: Online health and child-rearing information sites

[1247] Data processing: Collecting data using web scraping tools and APIs

[1248] Output: Latest childcare-related information data

[1249] Specific behavior:

[1250] The server launches a web scraping tool (e.g., Beautiful Soup) to collect the required information from the specified site.

[1251] Analyze the collected data using text analysis tools (e.g., NLTK, SpaCy) and categorize the information.

[1252] Store the classified data in a database (e.g., MySQL, MongoDB).

[1253] Step 2:

[1254] Users enter profile data for their children and themselves.

[1255] Input: Data such as the child's and the user's age, health status, allergy information, and emotional state

[1256] Output: User profile data

[1257] Specific behavior:

[1258] The user enters the required profile data into the input form on the device.

[1259] The user terminal transmits the input data to the server.

[1260] Step 3:

[1261] The server generates personalized advice based on the profile data.

[1262] Input: User-entered profile data, latest parenting-related information stored on the server

[1263] Data calculation: Analyze and generate data using natural language processing techniques (e.g., GPT-4, BERT)

[1264] Output: Individual advice message

[1265] Specific behavior:

[1266] The server retrieves the user's profile data and updates from the database.

[1267] Natural language processing techniques are used to generate personalized advice.

[1268] The generated advice is notified to the user terminal.

[1269] Step 4:

[1270] The server receives and analyzes real-time data from the monitoring devices.

[1271] Input: Video data and sensor data from surveillance devices (e.g., cameras, motion sensors)

[1272] Data calculation: Analyze using image recognition models (e.g., YOLO, OpenCV)

[1273] Output: Analysis results of monitoring data

[1274] Specific behavior:

[1275] The monitoring device captures real-time video data and transmits it to a server.

[1276] The server launches an image recognition model and analyzes the video data.

[1277] Detect anomalies and save detailed data.

[1278] Step 5:

[1279] If the server detects an abnormality, it generates a warning alert and notifies the user.

[1280] Input: Analysis results of monitoring data

[1281] Data processing: Applying anomaly detection algorithms and generating warning alerts

[1282] Output: Warning alert notification

[1283] Specific behavior:

[1284] The server detects abnormal situations based on the analysis results.

[1285] Generate warning alerts in response to anomalies.

[1286] A warning alert is sent to the user terminal.

[1287] Step 6:

[1288] The server collects and analyzes the child's voice and facial expression data.

[1289] Input: Child's voice data and facial expression data

[1290] Data calculation: Analyze using emotion analysis algorithms (e.g., Affectiva, Microsoft Azure Emotion API)

[1291] Output: Emotional state evaluation result

[1292] Specific behavior:

[1293] Emotion sensors and cameras capture the child's voice and facial expressions.

[1294] The server analyzes the data using a sentiment analysis algorithm.

[1295] The analysis results are evaluated and stored in a database.

[1296] Step 7:

[1297] The server generates and notifies an appropriate response message based on the emotional state.

[1298] Input: Emotional state assessment results

[1299] Data Computation: Uses an emotion engine to generate appropriate response messages

[1300] Output: Response message notification

[1301] Specific behavior:

[1302] The server obtains the emotion analysis results.

[1303] Use an emotion engine to generate appropriate response messages.

[1304] The message is sent to the user terminal, and a response is given to the user via the voice output device.

[1305] Step 8:

[1306] The user terminal inputs information about the child's diet and health condition.

[1307] Input: Child's dietary information, health status data

[1308] Output: Dietary and health data

[1309] Specific behavior:

[1310] The user enters information about their child's diet and health status into the interface.

[1311] The user terminal transmits the input data to the server.

[1312] Step 9:

[1313] The server analyzes the data and generates appropriate meal suggestions.

[1314] Input: User-entered food data, stored health information

[1315] Data calculation: Analyze data using a nutritional analysis algorithm to evaluate nutritional balance

[1316] Output: Appropriate meal suggestions

[1317] Specific behavior:

[1318] The server retrieves dietary and health data from a database.

[1319] Nutrition analysis algorithms are used to analyze the data and identify nutrient deficiencies.

[1320] An appropriate meal suggestion is generated and notified to the user terminal.

[1321] Step 10:

[1322] The server manages the schedule data and generates reminders.

[1323] Input: Schedule data entered by the user

[1324] Data processing: Manage schedule data and generate reminders

[1325] Output: Reminder notification

[1326] Specific behavior:

[1327] The user inputs the child's schedule data.

[1328] The server manages the schedule data and generates reminders before important events.

[1329] Reminders are sent to the user's device.

[1330] Step 11:

[1331] The emotional engine analyzes the user's emotional state and adjusts the message based on the results.

[1332] Input: User's voice data and facial expression data

[1333] Data Computing: Emotion analysis algorithms assess the user's emotional state and tailor messages

[1334] Output: Adjusted support message

[1335] Specific behavior:

[1336] The emotion engine captures the user's voice and facial expression data.

[1337] The server obtains the evaluation result of the emotion engine.

[1338] A support message is adjusted based on the evaluation result and notified to the user terminal.

[1339] (Application example 2)

[1340] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1341] In modern society, parents are faced with a wide range of information and tasks related to child-rearing, which often leads to stress and anxiety. Furthermore, caring for children while shopping in a store can be even more exhausting for parents. Furthermore, managing children's safety in the store, providing health advice, and efficiently managing schedules are challenges. Currently, there are no systems that provide individual support while taking into account the emotional state of parents and children. Therefore, there is a need for a system that provides comprehensive child-rearing support and emotional support so that parents and children can enjoy shopping with peace of mind.

[1342] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1343] A means of collecting child-rearing information;

[1344] A means of obtaining profile data for each child; and

[1345] means for analyzing the collected information based on the child's profile data and generating personalized advice;

[1346] means for notifying a user terminal of the individual advice;

[1347] A means for assessing the emotional state of parents and children and suggesting appropriate support messages and products based on the emotional state;

[1348] A means for managing schedule data of a customer and notifying a schedule reminder to a user terminal;

[1349] This allows parents and children to enjoy shopping in-store with peace of mind, and also allows them to receive individual advice and support regarding child-rearing.

[1350] The "means for collecting child-rearing information" refers to a device or program that has the function of periodically collecting the latest articles and papers from reliable health information sites and child-rearing information sites and analyzing them using text analysis tools.

[1351] "Means for acquiring profile data of individual children" refers to a device or program for acquiring individual data such as the age, health condition, allergy information, emotional state, etc. of the child or the user himself / herself entered by the user.

[1352] "Means for analyzing collected information based on a child's profile data and generating individualized advice" refers to a device or program that has the function of analyzing collected information based on the obtained child's profile data and generating individually tailored advice from the results.

[1353] The "means for notifying the user terminal of the individual advice" is a device or program having a function for notifying the user terminal of the generated individual advice as a message that is easy for the user to understand.

[1354] "Means for assessing the emotional state of parents and children and suggesting appropriate support messages or products based on the emotional state" refers to a device or program that has the function of collecting and analyzing voice and facial expression data of parents and children, assessing their emotional state, and then suggesting the most appropriate support message or product.

[1355] "Means for managing customer schedule data and notifying schedule reminders to the user terminal" refers to a device or program that has the function of managing parent schedule data and children's event schedules and notifying the user terminal of reminders when the scheduled date approaches.

[1356] "Means for receiving data from surveillance cameras and sensors" refers to a device or program that has the function of receiving data in real time from surveillance cameras and sensors installed within the store.

[1357] "Means for analyzing received data and detecting abnormalities" refers to a device or program that has the function of analyzing data received from a surveillance camera or sensor and detecting dangerous situations or abnormalities.

[1358] The "means for issuing a warning when an abnormality is detected" refers to a device or program that has the function of immediately generating and issuing a warning alert when an abnormality is detected.

[1359] The "means for detecting an abnormality and notifying a user terminal of a warning" refers to a device or program having the function of notifying a user terminal of a warning upon detecting an abnormality.

[1360] The "means for collecting children's voice and facial expression data" refers to a device or program for collecting children's voice and facial expression data in real time.

[1361] "Means for analyzing collected data and assessing emotional state" refers to a device or program that has the function of assessing a child's emotional state (e.g., sadness, joy) based on collected data.

[1362] The "means for generating an appropriate message based on the emotional state" refers to a device or program that has the function of generating an appropriate response or support message depending on the evaluated emotional state.

[1363] The "means for notifying the user terminal of the generated message" refers to a device or program having a function for notifying the user terminal of the generated support message.

[1364] The "means for evaluating the emotional state of the parent and generating a support message based on the overall emotional state of the parent and child" refers to a device or program that has the function of analyzing the parent's voice and facial expression data, evaluating the overall emotional state of the parent and child, and generating an appropriate support message based on that.

[1365] The "means for the physical robot to notify the parent and child of the generated support message" is a device or program that has the function of a dedicated robot notifying the parent and child of the generated support message by voice or display.

[1366] The present invention is a system that provides comprehensive support for child-rearing in a brick-and-mortar store by linking an AI-equipped robot, a server, a user terminal, and an emotion engine. The system of the present invention includes the following main components and functions:

[1367] 1. Gathering the latest information and providing personalized advice

[1368] The server periodically collects the latest articles and papers from reliable health and parenting information sites and analyzes them using text analysis tools. The collected information is stored in a database, and personalized advice is generated based on the profile data of the child and parent entered by the user. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[1369] As a specific example, the server collects the latest articles on influenza prevention methods, and based on data entered by the parents about a 5-year-old child (with no allergies), generates advice such as "taking vitamin C is important," and notifies the user's device.

[1370] 2. Child safety checks using surveillance cameras and sensors

[1371] The server receives real-time data from surveillance cameras and motion sensors installed in the store and uses image recognition models to detect dangerous situations. If an abnormality is detected, the server generates a warning alert and notifies the user's device.

[1372] As a specific example, if a surveillance camera receives streaming footage of a child climbing stairs, the server will use an image recognition model to confirm that the child is climbing stairs and send a warning to the user device that "the child is in a dangerous situation."

[1373] 3. Use of emotional support and emotional engines

[1374] The server collects voice and facial expression data from parents and children and analyzes it using an emotion analysis algorithm. It evaluates their emotional state and uses an emotion engine to generate support messages and product suggestions based on that state. The generated messages are sent to the user's device, and the AI ​​robot then speaks to the parent or child.

[1375] As a specific example, if the server collects a child's facial expression and voice data and evaluates the child as "sad," the emotion engine will generate a message saying "It's okay. Is something bothering you?", and the AI ​​robot will speak this to the child and simultaneously notify the user's device.

[1376] 4. Health management and nutrition education support

[1377] The user device provides an interface for parents to input their child's dietary information and health status, and the server stores and analyzes the input data. It identifies nutritional balance and nutrient deficiencies, generates appropriate meal suggestions, and notifies the user device.

[1378] As a specific example, if a parent inputs their child's daily diet and the server identifies that their child is not getting enough vitamin C, the server will notify the user's device with a suggestion to "incorporate salads with oranges, which are rich in vitamin C."

[1379] 5. Schedule management and reminder provision

[1380] The server manages the parent's schedule and the child's event schedule, and generates reminders based on the schedule data, ensuring that parents do not forget important events.

[1381] As a specific example, parents input their child's vaccination schedule, the server stores the schedule, and when the scheduled date approaches, a reminder is sent to the user's device saying, "The vaccination is tomorrow."

[1382] 6. User Emotion Recognition

[1383] The emotion engine collects and analyzes the parent's voice and facial expression data to assess the parent's emotional state. Based on the assessed emotional state, it tailors individual advice and support messages to provide more effective support.

[1384] As a specific example, if a parent is assessed as feeling stressed, the server generates advice such as "We recommend that you make time to relax with your child" and notifies the user terminal of the advice.

[1385] Example of an input prompt for a generative AI model:

[1386] "Based on your child's health, please tell us the latest flu prevention methods and food suggestions."

[1387] This system allows parents and children to enjoy shopping in-store with peace of mind and receive comprehensive childcare support.

[1388] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1389] Step 1:

[1390] The user enters their child's profile data (e.g., age, health status, allergy information, etc.) into the device. The device sends the entered data to the server. At this stage, the input is the user's profile data, and the output is that data sent to the server.

[1391] Step 2:

[1392] The server periodically collects the latest articles and papers from reliable health and child-rearing information sites and analyzes them using text analysis tools. The input is the latest article and paper data from external information sites, and the output is a database that stores the analysis results. Specifically, the server periodically crawls information sites, collects text data, and analyzes and classifies the information using NLP (natural language processing) algorithms.

[1393] Step 3:

[1394] The server compares the user's child's profile data with the latest collected information to generate personalized advice. The input is the user's profile data and the analyzed latest information, and the output is personalized advice. At this stage, the server compares the profile to select relevant information and uses natural language generation (NLG) technology to generate advice messages in a format that is easy for the user to understand.

[1395] Step 4:

[1396] The generated individual advice is notified to the device and displayed to the user. The input is the generated advice message, and the output is a notification that the user sees. Specifically, the advice message is pushed from the server to the device and displayed as a pop-up on the device screen.

[1397] Step 5:

[1398] The server receives real-time data from surveillance cameras and motion sensors. The input is video and motion data from the surveillance cameras and sensors, and the output is a data stream for analysis. The server captures and stores the streaming data provided by the cameras and sensors.

[1399] Step 6:

[1400] The server analyzes the received data using an image recognition model to detect anomalies. The input is real-time data from surveillance cameras and sensors, and the output is the results of anomaly detection. Specifically, the server uses a CNN (convolutional neural network) model to detect abnormal behavior in the video (e.g., a child playing on the stairs).

[1401] Step 7:

[1402] If an anomaly is detected, the server generates a warning alert and notifies the user device. The input is the anomaly detection result, and the output is a warning alert message. When an anomaly is detected, the server constructs the alert message and sends a push notification to the user device.

[1403] Step 8:

[1404] The server collects the voice and facial expression data of the parent and child and analyzes it using an emotion analysis algorithm. The input is the voice and facial expression data of the parent and child, and the output is the evaluation result of the emotional state. The server evaluates the emotional state based on the collected voice and video data using an emotion analysis model (e.g., CNN or RNN).

[1405] Step 9:

[1406] Based on the emotional state, the server generates appropriate support messages and product suggestions and notifies the user device. The input is the evaluation result of the emotional state, and the output is the support message and product suggestions. Based on the evaluation result, the server generates appropriate support messages and notifies the user device.

[1407] Step 10:

[1408] The server manages the parent's schedule data and the child's event schedule and generates schedule reminders. The input is the parent's schedule data and event schedule, and the output is a reminder message. The server uses schedule management software to push reminder notifications to the user's device when the event date approaches.

[1409] Step 11:

[1410] A dedicated robot notifies the parent and child of the generated support message by voice or display. The input is the support message sent from the server, and the output is the notification by voice or display from the robot. Specifically, the robot uses voice synthesis technology to convey the support message to the parent and child, and simultaneously displays it on the display.

[1411] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1412] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1413] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1414] [Third embodiment]

[1415] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1416] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1417] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1418] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1419] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1420] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1421] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1422] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1423] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1424] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1425] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1426] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1427] This invention is a system that provides comprehensive support for child-rearing through collaboration between an AI-equipped robot, a server, and a user device. The following describes the design of this system and the specific operation of each function.

[1428] 1. Gathering the latest information and providing personalized advice

[1429] The server regularly collects the latest articles and papers from reliable health and child-rearing information websites on the Internet. This information is stored in a database by category using text analysis tools. Each child's profile data (age, health condition, allergy information, etc.) is obtained, and the collected information is analyzed based on this to generate individualized advice. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[1430] Examples:

[1431] The server collects the latest articles on "How to prevent winter flu" and stores them in a database.

[1432] Extract relevant preventive measures based on the user's child (age 5, no allergies).

[1433] The system generates advice such as "It is important to take vitamin C" and notifies the user's device.

[1434] 2. Child safety checks using surveillance cameras and sensors

[1435] The server receives real-time data from surveillance cameras and motion sensors. The data is analyzed using image recognition models to detect abnormalities (such as a child climbing stairs). When an abnormality is detected, an alert is generated and sent to the user's device.

[1436] Examples:

[1437] A surveillance camera streams footage of a child accidentally climbing the stairs.

[1438] The server uses an image recognition model to identify anomalies and generates a warning that a child is in danger.

[1439] The warning is notified to the user terminal.

[1440] 3. Emotional support

[1441] The server collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm. It evaluates the child's emotional state (e.g., sadness or joy) and generates an appropriate message based on the results. The generated message is then sent to the user's device.

[1442] Examples:

[1443] The server collects the child's facial expression and voice data and evaluates the child as "sad."

[1444] Based on the analysis results, an encouraging message is generated, such as "It's okay, is there something bothering you?"

[1445] The emotional state and response content are notified to the user terminal.

[1446] 4. Health management and nutrition education support

[1447] The user device provides an interface where parents can input their child's dietary habits and health status. The server stores the input data and accumulates it daily. This data is then analyzed using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies. Based on the results, appropriate meal suggestions are generated and sent to the user device.

[1448] Examples:

[1449] Parents input their child's daily dietary information into a user terminal.

[1450] The server analyzes the data and identifies a lack of vitamin C intake.

[1451] "Eat more oranges and kiwis, which are richer in vitamin C," he suggests.

[1452] The user terminal is notified of the details of the proposal.

[1453] 5. Schedule management and reminder provision

[1454] The server manages the parent's schedule and the child's event schedule, generates reminders based on this schedule, and notifies the user's device as appropriate.

[1455] Examples:

[1456] Parents enter their child's vaccination schedule into a user terminal.

[1457] The server stores the schedule and generates a reminder as the day approaches: "Your vaccination is tomorrow."

[1458] A reminder is sent to the user's device.

[1459] In this way, the system of the present invention provides comprehensive support for child-rearing through functions such as collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and providing reminders.

[1460] The processing flow will be explained below.

[1461] 1. Gathering the latest information and providing personalized advice

[1462] Step 1:

[1463] Server: Regularly accesses reliable health and parenting information sites on the Internet and scrapes new articles and papers.

[1464] Step 2:

[1465] Server: Analyzes the content of collected articles using text analysis tools and stores them in a database by category.

[1466] Step 3:

[1467] Server: Obtains the child's individual profile data entered by the user (e.g., age, health status, allergy information).

[1468] Step 4:

[1469] Server: Based on the analyzed information, it generates personalized advice based on the child's profile data.

[1470] Step 5:

[1471] Server: Using natural language processing technology, the generated advice is converted into a message that is easy for the user to understand.

[1472] Step 6:

[1473] User device: The generated advice message is sent to the parent's smartphone or tablet.

[1474] Examples:

[1475] 1. The server collects the latest articles on "How to prevent winter influenza" and stores them in a database.

[1476] 2. The server retrieves the child's profile data and extracts relevant preventive measures.

[1477] 3. Generate advice such as "It is important to take vitamin C" and notify the user's device.

[1478] 2. Child safety checks using surveillance cameras and sensors

[1479] Step 1:

[1480] Server: Streams and receives real-time data from surveillance cameras and motion sensors.

[1481] Step 2:

[1482] Server: Analyzes received video data using an image recognition model to detect dangerous situations.

[1483] Step 3:

[1484] Server: Generates a warning alert if an anomaly is detected.

[1485] Step 4:

[1486] AI robot: Based on abnormality alerts received from the server, it issues a warning according to pre-set response methods.

[1487] Step 5:

[1488] User device: Real-time alert notifications are sent to parents' smartphones or tablets.

[1489] Examples:

[1490] 1. A security camera streams footage of a child accidentally climbing the stairs.

[1491] 2. The server uses an image recognition model to identify the child climbing the stairs and generates an anomaly alert.

[1492] 3. The AI ​​robot issues a warning that a child is in danger and notifies the user's device.

[1493] 3. Emotional support

[1494] Step 1:

[1495] Server: Using voice and facial recognition functions, collects children's voice and facial expression data in real time.

[1496] Step 2:

[1497] Server: Analyzes the collected data using an emotion analysis algorithm to evaluate the child's emotional state.

[1498] Step 3:

[1499] Server: Generates appropriate responses and support messages depending on the assessed emotional state.

[1500] Step 4:

[1501] AI robot: Speaks to children based on the generated message.

[1502] Step 5:

[1503] User device: Notifies parents of the child's emotional state and the AI ​​robot's response.

[1504] Examples:

[1505] 1. The server collects the child's facial expression and voice data and evaluates the child as "sad."

[1506] 2. The server generates an encouraging message based on the evaluation results, saying, "It's okay, is there something bothering you?"

[1507] 3. The AI ​​robot conveys the message to the child and notifies the user's device of their emotional state and response.

[1508] 4. Health management and nutrition education support

[1509] Step 1:

[1510] User terminal: Provides an interface where parents can input their child's diet and health status.

[1511] Step 2:

[1512] Server: Saves the entered data and accumulates daily data.

[1513] Step 3:

[1514] Server: Analyzes the accumulated data using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies.

[1515] Step 4:

[1516] Server: Generates appropriate meal suggestions based on the analysis results.

[1517] Step 5:

[1518] User terminal: Notifies parents of the generated meal suggestions and nutritional status reports.

[1519] Examples:

[1520] 1. Parents enter their child's daily dietary information into the user device.

[1521] 2. The server analyzes the input data and identifies that the person is lacking in vitamin C intake.

[1522] 3. "Incorporate more foods rich in vitamin C" is suggested as a specific recipe, and a notification is sent to the user's device.

[1523] 5. Schedule management and reminder provision

[1524] Step 1:

[1525] User terminal: Provides an interface for parents to input event schedules and schedules.

[1526] Step 2:

[1527] Server: Saves and manages the entered schedule data.

[1528] Step 3:

[1529] Server: Generates reminders based on schedule data.

[1530] Step 4:

[1531] User device: Notifies the parent of the generated reminder.

[1532] Examples:

[1533] 1. Parents enter their child's vaccination schedule into the user device.

[1534] 2. The server stores the appointment and generates a reminder when the appointment date approaches.

[1535] 3. The user device sends a reminder to the parent that "Vaccination is tomorrow."

[1536] Example 1

[1537] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1538] In child-rearing, it is often difficult for parents to receive the latest health information or appropriate advice based on the individual condition of their child, and children's safety and emotional care are often not adequately confirmed. This creates problems that make it difficult for parents to adequately support their children's health, safety, and emotional growth. To solve these problems, a comprehensive system of support for child-rearing is needed.

[1539] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1540] In this invention, the server includes a means for collecting parenting information, a means for acquiring profile data for each child, and a means for classifying and saving the collected information using a text analysis tool. This allows parents to obtain the latest parenting information tailored to their individual circumstances in a timely manner. The server also includes a means for analyzing the collected information based on the child's profile data using a machine learning algorithm to generate personalized advice, a means for notifying a user terminal of the generated personalized advice using natural language processing technology, a means for receiving data from surveillance cameras and sensors, a means for analyzing the received data using an image recognition model to detect abnormalities, a means for generating a warning when an abnormality is detected and notifying the user terminal, a means for collecting voice and facial expression data of the child, a means for analyzing the collected data using an emotion analysis algorithm to evaluate the emotional state, a means for generating an appropriate message based on the emotional state using natural language processing technology, and a means for notifying the user terminal of the generated message. This enables comprehensive monitoring and analysis of a child's health, safety, and emotional state, and for providing appropriate advice and warnings in real time.

[1541] "Means for collecting parenting information" include devices and programs for collecting the latest articles and papers from reliable health and parenting information sites, specifically scraping tools and APIs.

[1542] "Means for obtaining individual child profile data" refers to a database and its interface for collecting and storing individual information such as each child's age, health condition, and allergy information, and for retrieving it as needed.

[1543] "Means for classifying and storing collected information using text analysis tools" refers to devices or programs that include text analysis tools for analyzing collected information, classifying it by category, and storing it in a database.

[1544] "Means for analyzing using machine learning algorithms and generating personalized advice" refers to devices or programs that analyze collected data using machine learning algorithms and generate personalized advice based on each child's profile.

[1545] "Means for notifying a user terminal using natural language processing technology" refers to a device or program that uses natural language processing technology to generate a message from the generated advice in a form that is easy for the user to understand, and then sends that message to the user terminal.

[1546] "Means for receiving data from surveillance cameras and sensors" refers to devices and programs that transmit video and motion data from surveillance cameras and various sensors to a server in real time.

[1547] "Means for analyzing and detecting abnormalities using an image recognition model" refers to a device or program that analyzes received video and motion data using an image recognition model and detects abnormalities.

[1548] "Means for generating a warning when an abnormality is detected and notifying the user terminal" refers to a device or program that generates a warning message when an abnormality is detected and sends that message to the user terminal.

[1549] "Means for collecting children's voice and facial expression data" refers to devices such as cameras and microphones for collecting children's voice and facial expression data in real time, as well as programs for operating them.

[1550] "Means for evaluating a child's emotional state by analyzing using an emotion analysis algorithm" refers to a device or program that analyzes collected voice and facial expression data using an emotion analysis algorithm to evaluate a child's emotional state.

[1551] "Means for generating an appropriate message based on an emotional state using natural language processing technology" refers to a device or program that generates an appropriate message based on an evaluated emotional state using natural language processing technology.

[1552] The "means for notifying the user terminal of the generated message" refers to a device or program for transmitting the generated message to the user terminal.

[1553] This invention is a system that provides comprehensive support for child-rearing through collaboration between an AI-equipped robot, a server, and a user device. This system has multiple functions, including collecting the latest information and providing individualized advice, checking children's safety using surveillance cameras and sensors, providing emotional support, supporting health management and nutrition education, and managing schedules and providing reminders.

[1554] 1. Gathering the latest information and providing personalized advice

[1555] The server regularly collects the latest articles and papers from reliable health and parenting information websites. To do this, it uses Python scripts and the BeautifulSoup library to perform scraping. It then uses a text analysis tool (e.g., Apache OpenNLP) to categorize the collected information and store it in a database (e.g., MySQL). It obtains the user's child's profile data (e.g., age, health condition, allergy information, etc.) and analyzes it using machine learning algorithms such as scikit-learn. The generated advice is then converted into a user-friendly message using natural language processing technology (e.g., OpenAI's GPT-3) and sent to the user's device as a push notification.

[1556] Examples:

[1557] The server collects the latest articles on "How to prevent winter flu" and stores them in a database.

[1558] Extract relevant preventive measures based on the user's child (age 5, no allergies).

[1559] The system generates advice such as "It is important to take vitamin C" and notifies the user's device.

[1560] Example prompt sentence:

[1561] Input: Please tell me how to prevent winter flu in my child (5 years old, no allergies).

[1562] Output: Vitamin C intake is important. Eat plenty of oranges and kiwis.

[1563] 2. Child safety checks using surveillance cameras and sensors

[1564] The server receives real-time data from surveillance cameras (e.g., Nest Cam) and motion sensors (e.g., PIR sensors). The received data is analyzed using an image recognition model (e.g., TensorFlow's YOLO model) to detect anomalies. If an anomaly is detected, the server generates an alert and notifies the user device.

[1565] Examples:

[1566] A surveillance camera streams footage of a child accidentally climbing the stairs.

[1567] The server uses an image recognition model to identify anomalies and generates a warning that a child is in danger.

[1568] The warning is notified to the user terminal.

[1569] Example prompt sentence:

[1570] Input: Footage from a security camera shows a child starting to climb the stairs.

[1571] Output: Warning! Child in danger. Please act immediately.

[1572] 3. Emotional support

[1573] The server collects the child's voice and facial expression data and analyzes it using emotion analysis algorithms such as Microsoft Face API and Google Cloud Speech-to-Text. It evaluates the child's emotional state and generates an appropriate message based on the results. The message is then sent to the user's device.

[1574] Examples:

[1575] The server collects the child's facial expression and voice data and evaluates the child as "sad."

[1576] Based on the analysis results, an encouraging message is generated, such as "It's okay, is there something bothering you?"

[1577] The emotional state and response content are notified to the user terminal.

[1578] Example prompt sentence:

[1579] Input: How do you respond when your child is sad?

[1580] Output: Try gently saying, "It's okay. Is there something bothering you?"

[1581] 4. Health management and nutrition education support

[1582] The user device provides an interface where parents can input their child's dietary information and health status. The server stores the input data in a database and accumulates a daily diet history. This data is then analyzed using the Nutrition Data API to identify nutritional balance and nutrient deficiencies. Based on the results, appropriate meal suggestions are generated and notified to the user device.

[1583] Examples:

[1584] Parents input their child's daily dietary information into a user terminal.

[1585] The server analyzes the data and identifies a lack of vitamin C intake.

[1586] "Eat more oranges and kiwis, which are richer in vitamin C," he suggests.

[1587] The user terminal is notified of the details of the proposal.

[1588] Example prompt sentence:

[1589] Enter: If today's meal doesn't include oranges or kiwi, what are your suggestions?

[1590] Output: Eat more oranges and kiwis, which are rich in vitamin C.

[1591] 5. Schedule management and reminder provision

[1592] The server manages the parent's schedule and the child's event schedule, generates reminders based on this schedule, and notifies the user's device as appropriate.

[1593] Examples:

[1594] Parents enter their child's vaccination schedule into a user terminal.

[1595] The server stores the schedule and generates a reminder as the day approaches: "Your vaccination is tomorrow."

[1596] A reminder is sent to the user's device.

[1597] Example prompt sentence:

[1598] Enter: Set a reminder so you don't forget to schedule your vaccination appointment.

[1599] Output: Your vaccination is tomorrow. Don't forget.

[1600] In this way, the system of the present invention provides comprehensive support for child-rearing through functions such as collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders.

[1601] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1602] 1. Gathering the latest information and providing personalized advice

[1603] Step 1:

[1604] The server collects the latest articles and papers from reliable health and child-rearing information sites on the Internet.

[1605] Input: List of URLs to be collected

[1606] What it does: Use a Python script to parse the HTML and extract data from each URL using the BeautifulSoup library.

[1607] Output: Collected text data

[1608] Step 2:

[1609] The server uses text analysis tools to categorize the collected information.

[1610] Input: Collected text data

[1611] What it does: It uses Apache OpenNLP to analyze text and classify it into categories.

[1612] Output: Text data classified by category

[1613] Step 3:

[1614] The server stores the classified information in a database.

[1615] Input: Categorized text data

[1616] Specific behavior: Connects to a MySQL database and inserts data into the appropriate tables.

[1617] Output: Saved database entries

[1618] Step 4:

[1619] The server retrieves the profile data for each child from the database.

[1620] Input: Child's identification

[1621] Specific operation: Issues an SQL query and retrieves profile data.

[1622] Output: Child profile data

[1623] Step 5:

[1624] The server uses machine learning algorithms to analyze the profile data and collected information to generate personalized advice.

[1625] Input: Profile data, categorical text data

[1626] What it does: Uses the scikit-learn library to analyze information that matches the profile data.

[1627] Output: personalized advice

[1628] Step 6:

[1629] The server notifies the generated individual advice to the user terminal using natural language processing technology.

[1630] Input: Personalized Advice

[1631] Specific operation: Using OpenAI's GPT-3, messages are generated in natural language and sent to the user's device via the push notification API.

[1632] Output: Notification message to the user terminal

[1633] 2. Child safety checks using surveillance cameras and sensors

[1634] Step 1:

[1635] The server receives data in real time from surveillance cameras and sensors.

[1636] Input: Real-time camera and sensor data

[1637] Specific operation: Use the streaming API to receive data.

[1638] Output: Received real-time data

[1639] Step 2:

[1640] The server analyzes the received data using an image recognition model.

[1641] Input: Real-time data

[1642] Specific operation: Detect abnormal behavior using TensorFlow's YOLO model.

[1643] Output: Analysis results (normal / abnormal flag)

[1644] Step 3:

[1645] When the server detects an abnormality, it generates a warning and notifies the user terminal.

[1646] Input: Analysis results (abnormal flag)

[1647] Specific operation: If an abnormality is detected, a warning message is generated and sent to the user device via the push notification API.

[1648] Output: A warning message to the user's terminal.

[1649] 3. Emotional support

[1650] Step 1:

[1651] The server collects the child's voice and facial expression data.

[1652] Input: Camera and microphone data

[1653] Specific operation: Receives a data stream from a device.

[1654] Output: Collected voice and facial expression data

[1655] Step 2:

[1656] The server analyzes the collected data using a sentiment analysis algorithm.

[1657] Input: Voice and facial expression data

[1658] Specific behavior: Emotional state is assessed using Microsoft Face API and Google Cloud Speech-to-Text.

[1659] Output: Emotional state evaluation result

[1660] Step 3:

[1661] The server generates an appropriate message based on the emotional state using natural language processing techniques.

[1662] Input: Emotional state assessment results

[1663] Specific operation: Messages are generated using OpenAI's GPT-3.

[1664] Output: The generated message

[1665] Step 4:

[1666] The server notifies the user terminal of the generated message.

[1667] Input: The generated message

[1668] Specific operation: Send a message via the Push notification API.

[1669] Output: Notification message to the user terminal

[1670] 4. Health management and nutrition education support

[1671] Step 1:

[1672] The user inputs the child's diet and health condition into the user terminal.

[1673] Input: Data entered by the user

[1674] Specific operation: Fill in a form in an application on the user's device.

[1675] Output: Entered dietary and health data

[1676] Step 2:

[1677] The server saves the entered data in a database.

[1678] Input: Dietary and health status data

[1679] Specific action: Add a new entry to the database.

[1680] Output: Saved database records

[1681] Step 3:

[1682] The server analyzes the data using a nutritional analysis algorithm.

[1683] Input: Saved database records

[1684] Specific behavior: Uses the Nutrition Data API to identify nutritional balances and nutrient deficiencies.

[1685] Output: Analysis results (nutritional balance, nutrient deficiencies)

[1686] Step 4:

[1687] The server generates meal suggestions based on the analysis results and notifies the user terminal.

[1688] Input: Analysis results

[1689] Specific operation: Generates suggestions and sends them via the Push notification API.

[1690] Output: Meal suggestion message to user device

[1691] 5. Schedule management and reminder provision

[1692] Step 1:

[1693] The user inputs the child's event schedule and vaccination schedule into the user terminal.

[1694] Input: Schedule data entered by the user

[1695] Specific operation: Enter a schedule into an application on the user's device.

[1696] Output: Schedule data entered

[1697] Step 2:

[1698] The server stores the entered schedule data in a database.

[1699] Input: Schedule data

[1700] Specific Action: Adds a new schedule entry to the database.

[1701] Output: Saved database records

[1702] Step 3:

[1703] The server generates a reminder based on the schedule data and notifies the user terminal.

[1704] Input: Saved schedule data

[1705] Specific behavior: Generates reminder content and sends it via the Push notification API at the appropriate time.

[1706] Output: Reminder message to user device

[1707] (Application example 1)

[1708] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1709] Conventional methods for managing factory robots have had problems with insufficient efficient robot operation and safety. In particular, it has been difficult to provide appropriate advice and warnings based on the individual needs of each robot, and to properly manage the timing of maintenance. Furthermore, there has been a lack of systems for monitoring the operating status of robots in real time and taking appropriate action quickly. To solve these issues, a more advanced and comprehensive robot management system is needed.

[1710] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1711] In this invention, the server includes means for collecting information, means for acquiring individual profile data, means for analyzing the collected information based on the profile data and generating individual advice, means for notifying the user terminal of the individual advice, means for receiving and analyzing robot status data, means for generating an appropriate message based on the analysis results, and means for notifying the user terminal of the generated message, thereby enabling efficient operation of the robot, ensuring safety, and maintenance management.

[1712] "Information gathering means" are devices and programs that obtain the latest information and articles from reliable data sources.

[1713] The "means for acquiring profile data" refers to a device or program that collects information about a specific robot or target device and saves it as profile data.

[1714] The "means for analyzing information and generating personalized advice" refers to a device or program that analyzes the collected data and generates advice tailored to the specific needs of the subject.

[1715] The "means for notifying the user terminal of the advice" refers to a device or program that transmits the generated advice to the user terminal.

[1716] The "means for receiving and analyzing robot status data" refers to a device or program that collects data on the robot's operating status and performance and analyzes that data.

[1717] The "means for generating an appropriate message" refers to a device or program that generates an effective message to notify the user based on the analysis results.

[1718] The "means for notifying the user terminal of the generated message" refers to a device or program that quickly and reliably transmits the generated message to the user terminal.

[1719] "Means for receiving data from monitoring devices and sensors" refers to devices and programs that receive data collected from monitoring devices and various sensors within the factory.

[1720] "Means for detecting abnormalities" refers to a device or program that analyzes received data, identifies any abnormal conditions, and issues a warning.

[1721] "Means for analyzing efficiency" refers to devices or programs that analyze the operation data of each robot and evaluate the work efficiency.

[1722] The "means for generating advice for improving efficiency" is a device or program that provides specific advice for improving the work efficiency of the robot based on the analysis results.

[1723] "Means for managing maintenance schedules and sending reminders" refers to devices or programs that manage the timing of robot maintenance and notify users of maintenance when necessary.

[1724] The present invention relates to a factory robot management system that comprehensively supports the efficient operation and safety of factory robots by linking servers, monitoring devices, sensors, and user terminals.

[1725] System configuration

[1726] The server first has a means to collect the latest information and articles from reliable data sources (e.g., technology sites and industry information sites). The collected information is analyzed based on the profile data, and personalized advice is generated and sent to the user's device. This profile data is obtained based on the characteristics and demands of each individual robot.

[1727] Monitoring devices and sensors monitor the operating status of each robot in the factory and the surrounding environment in real time, and send the data to a server. The server analyzes the received data and issues an alert if an abnormality is detected, notifying the user's device. The server also analyzes the efficiency of the robots and generates advice for improving efficiency.

[1728] Furthermore, the server manages the robot's maintenance schedule and has a means for sending reminders, allowing the user to perform maintenance at the appropriate time.

[1729] Processing description

[1730] 1. Collection of information:

[1731] The server periodically retrieves up-to-date information from reliable data sources using the Python requests library, which is then parsed by text analysis tools and stored in a database.

[1732] 2. Analyzing data and generating advice:

[1733] Based on the profile data, the collected information is analyzed and personalized advice is generated based on each robot's condition and needs. The generated advice is then sent to the user's device using natural language processing technology.

[1734] 3. Monitoring and Alerting:

[1735] Data received from monitoring devices and sensors is analyzed using image recognition and motion detection models, and if an anomaly is detected, an alert is generated immediately and sent to the user's device.

[1736] 4. Maintenance Management:

[1737] The server manages the maintenance schedule for each robot and sends reminders as appropriate, based on a pre-set schedule.

[1738] Specific examples

[1739] Examples of gathering up-to-date information:

[1740] The server collects articles about "robot maintenance techniques," analyzes the information, and applies it to "Robot 1."

[1741] Safety monitoring example:

[1742] The monitoring device detects dangerous behavior of the robot while it is in operation and generates an alert to notify the administrator.

[1743] Efficiency analysis and advice examples:

[1744] The operation data of each robot is analyzed, and specific advice (e.g., how to optimize operation) to improve the efficiency of "Robot 2" is generated and notified to the user terminal.

[1745] Maintenance reminder example:

[1746] The server manages the maintenance schedule for "Robot 3" and sends a reminder "Maintenance is required" the day before the maintenance.

[1747] Prompt Sentence Examples

[1748] Prompt sentence to input to the generative AI model:

[1749] "Generate the latest maintenance advice based on the following robot ID and its current operational status: ID: robot_1, Status: Up, Maintenance Cycle: 1 month."

[1750] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1751] Step 1:

[1752] The server collects the latest information from a reliable data source. Here, the server uses Python's requests library to access URLs and retrieve article data on the latest technical information and maintenance techniques. The input is the URL, and the output is the retrieved latest information data.

[1753] Step 2:

[1754] The server analyzes the collected information using a text analysis tool (e.g., a natural language processing library). As a result of the analysis, the information is classified into categories. The input is the collected information data, and the output is the data classified by category.

[1755] Step 3:

[1756] The server obtains the profile data of each robot. This profile data includes information such as the robot's ID, characteristics, maintenance cycle, etc. The input is the robot's individual data request, and the output is the profile data.

[1757] Step 4:

[1758] The server generates appropriate personalized advice from the collected information based on the profile data. Specifically, it uses natural language processing technology to select relevant articles and information and restructures the content to suit the robot's needs. The input is profile data and categorized data, and the output is personalized advice.

[1759] Step 5:

[1760] The server notifies the user device of the generated personalized advice. Notifications are sent via APIs or messaging protocols, and users can receive them on their smartphones or tablets. The input is personalized advice, and the output is a notification sent to the user device.

[1761] Step 6:

[1762] Monitoring devices and sensors collect data in real time and send it to a server. Here, hardware such as cameras are used to monitor the robot's operation. The input is monitoring data, and the output is data sent to the server.

[1763] Step 7:

[1764] The server analyzes the received data and detects anomalies. Image recognition and motion detection models are used for the analysis. If an anomaly is detected, an alert is generated. The input is the monitoring data, and the output is the anomaly detection result.

[1765] Step 8:

[1766] If an abnormality is detected, the server immediately sends a warning to the user terminal. The user receives the warning message and can respond promptly. The input is the abnormality detection result, and the output is a warning notification to the user terminal.

[1767] Step 9:

[1768] The server analyzes the efficiency of each robot. Historical and real-time data are used for efficiency analysis to identify movement patterns and efficient operation methods. The input is movement data, and the output is the efficiency analysis results.

[1769] Step 10:

[1770] The server generates specific advice for improving efficiency and notifies the user terminal. This advice is created using natural language processing technology. The input is the efficiency analysis result, and the output is efficiency improvement advice.

[1771] Step 11:

[1772] The server manages the robot's maintenance schedule and sends reminders. Here, it notifies the robot of the need for maintenance based on a pre-defined schedule. The input is the maintenance schedule and the output is the reminder notification.

[1773] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1774] This invention is a system that comprehensively supports child-rearing by linking an AI-equipped robot, a server, a user terminal, and an emotion engine. The design of this system and the specific operation method of each function are explained below.

[1775] 1. Gathering the latest information and providing personalized advice

[1776] The server regularly collects the latest articles and papers from reliable health and child-rearing information sites and analyzes them using text analysis tools. The analyzed information is then stored in a database by category. Individual profile data (e.g., age, health condition, allergy information, emotional state) entered by the user for the child and the user themselves is then obtained, and the collected information is analyzed based on this to generate individual advice. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[1777] Examples:

[1778] The server collects the latest articles on "How to Prevent Influenza" and stores them in a database.

[1779] The user's child (5 years old, no allergies) and the user's emotional state are obtained, and relevant preventive measures are extracted.

[1780] It generates an "encouraging message according to the user's emotional state" and advice such as "taking vitamin C is important," and notifies the user's device.

[1781] 2. Child safety checks using surveillance cameras and sensors

[1782] The server receives real-time data from surveillance cameras and motion sensors, analyzes it with image recognition models to detect dangerous situations, and if an abnormality is detected, generates a warning alert and notifies the user's device.

[1783] Examples:

[1784] A surveillance camera streams footage of a child accidentally climbing the stairs.

[1785] The server uses an image recognition model to identify a child climbing stairs and generates an anomaly alert.

[1786] A warning that "child is in danger" is sent to the user terminal.

[1787] 3. Use of emotional support and emotional engines

[1788] The server collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm. It evaluates the child's emotional state (e.g., sadness, joy), and uses an emotion engine to generate an appropriate response or support message based on that state. The generated message, taking into account the user's emotional state, is then sent to the user's device, where the AI ​​robot then speaks to the child.

[1789] Examples:

[1790] The server collects the child's facial expressions and voice data, and uses an emotion analysis algorithm to assess whether the child is "sad."

[1791] Sentiment analysis algorithms also assess the user's emotional state and generate a comprehensive message.

[1792] The AI ​​robot will ask the child, "It's okay. Is there something bothering you?" and notify the user's device of its emotional state and response.

[1793] 4. Health management and nutrition education support

[1794] The user device provides an interface for parents to input their child's dietary habits and health status, and the server saves and stores the data. This data is analyzed using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies. Based on the analysis results, appropriate meal suggestions are generated and notified to the user device.

[1795] Examples:

[1796] Parents input their child's daily dietary information into a user terminal.

[1797] The server analyzes the data and identifies a lack of vitamin C intake.

[1798] "Incorporate more foods rich in vitamin C," he says, suggesting specific recipes.

[1799] 5. Schedule management and reminder provision

[1800] The server manages the parent's schedule and the child's event schedule, and generates reminders based on this schedule data. The generated reminders are then sent to the user's device.

[1801] Examples:

[1802] Parents enter their child's vaccination schedule into a user terminal.

[1803] The server stores the appointment and generates a reminder when the appointment date approaches.

[1804] The user terminal notifies the parent with a reminder that "Vaccination is tomorrow."

[1805] 6. User Emotion Recognition

[1806] The emotion engine collects and analyzes the user's voice and facial expression data to assess the user's emotional state, and tailors personalized advice and support messages based on the assessed emotional state to provide more effective support.

[1807] Examples:

[1808] The emotion engine analyzes the user's tone of voice and facial expressions and assesses them as "feeling stressed."

[1809] The server receives the evaluation results and generates advice such as, "We recommend that you spend some time relaxing with your child."

[1810] The system notifies the user terminal of the advice and suggests music and activities that will help parents and children relax.

[1811] In this way, the system of the present invention provides comprehensive support for child-rearing by collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders, and recognizing users' emotions using an emotion engine.

[1812] The processing flow will be explained below.

[1813] 1. Gathering the latest information and providing personalized advice

[1814] Step 1:

[1815] Server: Regularly accesses reliable health and parenting information websites and scrapes new articles and papers.

[1816] Step 2:

[1817] Server: Analyzes the content of collected articles using text analysis tools and stores them in a database by category.

[1818] Step 3:

[1819] Server: Obtains individual profile data (e.g., age, health condition, allergy information, emotional state) of the child and the user themselves entered by the user.

[1820] Step 4:

[1821] Server: Analyzes the collected information and generates personalized advice based on the profile data.

[1822] Step 5:

[1823] Server: Using natural language processing technology, the generated advice is converted into a message that is easy for the user to understand.

[1824] Step 6:

[1825] User device: The generated advice message is sent to the parent's smartphone or tablet.

[1826] Examples:

[1827] 1. The server collects the latest articles about "How to prevent influenza" and stores them in a database.

[1828] 2. The server retrieves the child's profile data and extracts relevant preventive measures.

[1829] 3. Generate advice such as "It is important to take vitamin C" and notify the user's device.

[1830] 2. Child safety checks using surveillance cameras and sensors

[1831] Step 1:

[1832] Server: Streams and receives real-time data from surveillance cameras and motion sensors.

[1833] Step 2:

[1834] Server: Analyzes received video data using an image recognition model to detect dangerous situations.

[1835] Step 3:

[1836] Server: Generates a warning alert if an anomaly is detected.

[1837] Step 4:

[1838] AI robot: Based on abnormality alerts received from the server, it issues a warning according to pre-set response methods.

[1839] Step 5:

[1840] User device: Sends real-time alert notifications to parents' smartphones and tablets.

[1841] Examples:

[1842] 1. A security camera streams footage of a child accidentally climbing the stairs.

[1843] 2. The server uses an image recognition model to identify the child climbing the stairs and generates an anomaly alert.

[1844] 3. The AI ​​robot issues a warning that a child is in danger and notifies the user's device.

[1845] 3. Use of emotional support and emotional engines

[1846] Step 1:

[1847] Server: Using voice and facial recognition functions, collects children's voice and facial expression data in real time.

[1848] Step 2:

[1849] Server: Analyzes the collected data using an emotion analysis algorithm to evaluate the child's emotional state.

[1850] Step 3:

[1851] Server: Depending on the assessed emotional state, it uses an emotion engine to generate appropriate responses and support messages.

[1852] Step 4:

[1853] AI robot: Speaks to children based on the generated message.

[1854] Step 5:

[1855] User device: Notifies parents of the child's emotional state and the AI ​​robot's response.

[1856] Examples:

[1857] 1. The server collects the child's facial expression and voice data and uses an emotion analysis algorithm to evaluate the child as "sad."

[1858] 2. Based on the evaluation results, the server generates an encouraging message saying, "It's okay, is there anything you're having trouble with?"

[1859] 3. The AI ​​robot conveys the message to the child and notifies the user's device of their emotional state and response.

[1860] 4. Health management and nutrition education support

[1861] Step 1:

[1862] User terminal: Provides an interface for parents to input their child's dietary information and health status.

[1863] Step 2:

[1864] Server: Saves the entered data and accumulates daily data.

[1865] Step 3:

[1866] Server: Analyzes the accumulated data using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies.

[1867] Step 4:

[1868] Server: Generates appropriate meal suggestions based on the analysis results.

[1869] Step 5:

[1870] User terminal: Notifies parents of the generated meal suggestions and nutritional status report.

[1871] Examples:

[1872] 1. The user inputs the child's daily meal plan into the user terminal.

[1873] 2. The server analyzes the input data and identifies that the person is lacking in vitamin C intake.

[1874] 3. The server suggests specific recipes, such as "Incorporate more foods rich in vitamin C," and notifies the user's device.

[1875] 5. Schedule management and reminder provision

[1876] Step 1:

[1877] User terminal: Provides an interface for parents to input event schedules and schedules.

[1878] Step 2:

[1879] Server: Saves and manages the entered schedule data.

[1880] Step 3:

[1881] Server: Generates reminders based on schedule data.

[1882] Step 4:

[1883] User device: Notifies the parent of the generated reminder.

[1884] Examples:

[1885] 1. The user enters their child's vaccination schedule into the user terminal.

[1886] 2. The server stores the appointment and generates a reminder when the appointment date approaches.

[1887] 3. The user device sends a reminder to the parent that "Vaccination is tomorrow."

[1888] 6. User Emotion Recognition

[1889] Step 1:

[1890] Emotion engine: Collects and analyzes the user's voice and facial expression data.

[1891] Step 2:

[1892] Server: Receives data from the emotion engine and evaluates the user's emotional state.

[1893] Step 3:

[1894] Server: Tailors personalized advice and support messages based on the assessed emotional state.

[1895] Step 4:

[1896] User terminal: Notifies the parent of the generated support message.

[1897] Examples:

[1898] 1. The emotion engine analyzes the user's tone of voice and facial expressions and assesses them as "feeling stressed."

[1899] 2. The server receives the evaluation results and generates advice such as, "We recommend that you spend some time relaxing with your child."

[1900] 3. The user device will notify them of the advice and suggest relaxing music and activities.

[1901] This system allows AI-equipped robots, servers, user devices, and emotion engines to work together to provide comprehensive support for child-rearing.

[1902] Example 2

[1903] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1904] In modern society, raising children has become an extremely diverse and complex task. Busy parents often find it difficult to obtain accurate and up-to-date information on child rearing, making it even more difficult to comprehensively address their children's safety, health, and emotional support. In particular, many aspects must be considered simultaneously, including monitoring to ensure children's safety, providing emotional care, and managing nutritional balance. The present invention aims to provide a system that comprehensively solves these complex child rearing challenges.

[1905] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting child-rearing related information, means for acquiring profile data of each child, means for analyzing the collected information based on the profile data and generating individual advice, means for notifying the user terminal of the individual advice, means for collecting emotional data of family members and evaluating their emotional states using an emotion analysis algorithm, means for generating an appropriate response message based on their emotional states, and means for notifying the user terminal of the response message and responding to the user via a voice output device. This enables comprehensive support for child-rearing.

[1906] "Child-rearing related information" refers to the latest health, educational, and psychological information related to child-rearing, as well as other information useful for raising children.

[1907] "Profile Data" refers to individual data about each child and their parents, such as age, health status, allergy information, and emotional state.

[1908] An "emotion analysis algorithm" refers to a computational method for analyzing emotional states from voice data, facial expression data, etc., and evaluating the results.

[1909] A "response message" refers to an appropriate message to a user that is generated based on the user's emotional state and individual circumstances.

[1910] "Monitoring equipment" refers to equipment that uses surveillance devices such as cameras and sensors to monitor children's activities and surrounding conditions in real time.

[1911] A "nutritional analysis algorithm" refers to a calculation method that analyzes nutritional balance based on input dietary data and suggests necessary nutrients and appropriate meals.

[1912] "Reminder" refers to a warning or attention message that notifies the user based on a pre-set schedule.

[1913] The term "audio output device" refers to a device for actually transmitting the generated voice message to the user as voice.

[1914] The present invention provides a comprehensive support system for child rearing. This system realizes child safety confirmation, health management, emotional support, and schedule management by linking together a server, terminals, users, an emotion engine, a monitoring device, and a nutritional analysis algorithm.

[1915] The server periodically collects the latest articles and papers from reliable health and parenting information websites on the Internet. Web scraping tools and APIs are used for collection. Specifically, Beautiful Soup is used for web scraping, and the PubMed API is used for data collection. The collected data is analyzed using text analysis tools (e.g., NLTK, SpaCy) and stored by category in a database (e.g., MySQL, MongoDB).

[1916] Users enter their own and their children's profile data (e.g., age, health condition, allergy information, emotional state) on their device. The device sends the entered data to the server, which analyzes the data and generates personalized advice. The personalized advice is generated using natural language processing technology (e.g., GPT-4, BERT) and notified to the user's device.

[1917] For example, the server collects the latest articles on "How to prevent influenza" and stores them in a database. When a user inputs the profile data of a child (age 5, no allergies), the server analyzes the relevant prevention methods, generates advice such as "It is important to take vitamin C," and notifies the user's device.

[1918] Monitoring devices (e.g., cameras, motion sensors) send real-time data of children to a server. The server uses image recognition models (e.g., YOLO, OpenCV) to analyze the monitoring data and detect abnormalities. If an abnormality is detected, the server generates a warning alert and notifies the user's device.

[1919] Specifically, a surveillance camera receives footage of a child accidentally climbing stairs, and the server analyzes the footage to detect any abnormalities, then sends a warning to the user's device that the child is in danger.

[1920] The server also collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm (e.g., Affectiva, Microsoft Azure Emotion API). Based on the analysis results, the server evaluates the child's emotional state and generates an appropriate response message. This message is sent to the user's device, which responds to the child via the voice output device.

[1921] For example, the server collects a child's facial expression and voice data and uses an emotion analysis algorithm to evaluate the child as "sad." The server then generates a message saying, "It's okay. Is something bothering you?", and the AI ​​robot responds, notifying the user's device of the child's emotional state and the response.

[1922] Furthermore, the user device provides an interface for parents to input their child's dietary information and health status, and sends the data to a server. The server then analyzes the input data using a nutritional analysis algorithm (e.g., MyFitnessPal's API). Based on the analysis results, the nutritional balance is evaluated and appropriate meal suggestions are generated.

[1923] For example, if a parent inputs their child's diet and the server identifies a vitamin C deficiency, it will suggest specific recipes such as "Incorporate more vitamin C-rich foods."

[1924] The server manages schedule data, generates reminders based on this data, and notifies the user device. For example, when a parent inputs their child's vaccination schedule, the server saves the schedule and sends a reminder that "Tomorrow's vaccination date" when the scheduled date approaches.

[1925] The emotion engine then evaluates the user's emotional state and generates personalized advice and support messages based on the evaluation results, thereby providing optimal support that takes the user's emotional state into account.

[1926] Example prompt sentence:

[1927] 1. "What are some good sleep guidelines for a 1-year-old?"

[1928] 2. "What should I do if I'm deficient in vitamin C?"

[1929] 3. "What safety precautions should I take when climbing stairs?"

[1930] 4. "What are some ways parents and children can relax when they're feeling stressed?"

[1931] In this way, the system of the present invention provides comprehensive support for child-rearing by collecting the latest information and providing individual advice, checking children's safety using monitoring devices and sensors, providing emotional support, health management and nutritional education support, schedule management and reminders, and recognizing users' emotions using an emotion engine.

[1932] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1933] Step 1:

[1934] The server collects child-rearing related information.

[1935] Input: Online health and child-rearing information sites

[1936] Data processing: Collecting data using web scraping tools and APIs

[1937] Output: Latest childcare-related information data

[1938] Specific behavior:

[1939] The server launches a web scraping tool (e.g., Beautiful Soup) to collect the required information from the specified site.

[1940] Analyze the collected data using text analysis tools (e.g., NLTK, SpaCy) and categorize the information.

[1941] Store the classified data in a database (e.g., MySQL, MongoDB).

[1942] Step 2:

[1943] Users enter profile data for their children and themselves.

[1944] Input: Data such as the child's and the user's age, health status, allergy information, and emotional state

[1945] Output: User profile data

[1946] Specific behavior:

[1947] The user enters the required profile data into the input form on the device.

[1948] The user terminal transmits the input data to the server.

[1949] Step 3:

[1950] The server generates personalized advice based on the profile data.

[1951] Input: User-entered profile data, latest parenting-related information stored on the server

[1952] Data calculation: Analyze and generate data using natural language processing techniques (e.g., GPT-4, BERT)

[1953] Output: Individual advice message

[1954] Specific behavior:

[1955] The server retrieves the user's profile data and updates from the database.

[1956] Natural language processing techniques are used to generate personalized advice.

[1957] The generated advice is notified to the user terminal.

[1958] Step 4:

[1959] The server receives and analyzes real-time data from the monitoring devices.

[1960] Input: Video data and sensor data from surveillance devices (e.g., cameras, motion sensors)

[1961] Data calculation: Analyze using image recognition models (e.g., YOLO, OpenCV)

[1962] Output: Analysis results of monitoring data

[1963] Specific behavior:

[1964] The monitoring device captures real-time video data and transmits it to a server.

[1965] The server launches an image recognition model and analyzes the video data.

[1966] Detect anomalies and save detailed data.

[1967] Step 5:

[1968] If the server detects an abnormality, it generates a warning alert and notifies the user.

[1969] Input: Analysis results of monitoring data

[1970] Data processing: Apply anomaly detection algorithms and generate warning alerts

[1971] Output: Warning alert notification

[1972] Specific behavior:

[1973] The server detects abnormal situations based on the analysis results.

[1974] Generate warning alerts in response to anomalies.

[1975] A warning alert is sent to the user terminal.

[1976] Step 6:

[1977] The server collects and analyzes the child's voice and facial expression data.

[1978] Input: Child's voice data and facial expression data

[1979] Data calculation: Analyze using emotion analysis algorithms (e.g., Affectiva, Microsoft Azure Emotion API)

[1980] Output: Emotional state evaluation result

[1981] Specific behavior:

[1982] Emotion sensors and cameras capture the child's voice and facial expressions.

[1983] The server analyzes the data using a sentiment analysis algorithm.

[1984] The analysis results are evaluated and stored in a database.

[1985] Step 7:

[1986] The server generates and notifies an appropriate response message based on the emotional state.

[1987] Input: Emotional state assessment results

[1988] Data Computation: Uses an emotion engine to generate appropriate response messages

[1989] Output: Response message notification

[1990] Specific behavior:

[1991] The server obtains the emotion analysis results.

[1992] Use an emotion engine to generate appropriate response messages.

[1993] The message is sent to the user terminal, and a response is given to the user via the voice output device.

[1994] Step 8:

[1995] The user terminal inputs information about the child's diet and health condition.

[1996] Input: Child's dietary information, health status data

[1997] Output: Dietary and health data

[1998] Specific behavior:

[1999] The user enters information about their child's diet and health status into the interface.

[2000] The user terminal transmits the input data to the server.

[2001] Step 9:

[2002] The server analyzes the data and generates appropriate meal suggestions.

[2003] Input: User-entered food data, stored health information

[2004] Data calculation: Analyze data using a nutritional analysis algorithm to evaluate nutritional balance

[2005] Output: Appropriate meal suggestions

[2006] Specific behavior:

[2007] The server retrieves dietary and health data from a database.

[2008] Nutrition analysis algorithms are used to analyze the data and identify nutrient deficiencies.

[2009] An appropriate meal suggestion is generated and notified to the user terminal.

[2010] Step 10:

[2011] The server manages the schedule data and generates reminders.

[2012] Input: Schedule data entered by the user

[2013] Data processing: Manage schedule data and generate reminders

[2014] Output: Reminder notification

[2015] Specific behavior:

[2016] The user inputs the child's schedule data.

[2017] The server manages the schedule data and generates reminders before important events.

[2018] Reminders are sent to the user's device.

[2019] Step 11:

[2020] The emotional engine analyzes the user's emotional state and adjusts the message based on the results.

[2021] Input: User's voice data and facial expression data

[2022] Data Computing: Emotion analysis algorithms assess the user's emotional state and tailor messages

[2023] Output: Adjusted support message

[2024] Specific behavior:

[2025] The emotion engine captures the user's voice and facial expression data.

[2026] The server obtains the evaluation result of the emotion engine.

[2027] A support message is adjusted based on the evaluation result and notified to the user terminal.

[2028] (Application example 2)

[2029] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2030] In modern society, parents are faced with a wide range of information and tasks related to child-rearing, which often leads to stress and anxiety. Furthermore, caring for children while shopping in a store can be even more exhausting for parents. Furthermore, managing children's safety in the store, providing health advice, and efficiently managing schedules are challenges. Currently, there are no systems that provide individual support while taking into account the emotional state of parents and children. Therefore, there is a need for a system that provides comprehensive child-rearing support and emotional support so that parents and children can enjoy shopping with peace of mind.

[2031] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2032] A means of collecting child-rearing information;

[2033] A means of obtaining profile data for each child; and

[2034] means for analyzing the collected information based on the child's profile data and generating personalized advice;

[2035] means for notifying a user terminal of the individual advice;

[2036] A means for assessing the emotional state of parents and children and suggesting appropriate support messages and products based on the emotional state;

[2037] A means for managing schedule data of a customer and notifying a schedule reminder to a user terminal;

[2038] This allows parents and children to enjoy shopping in-store with peace of mind, and also allows them to receive individual advice and support regarding child-rearing.

[2039] The "means for collecting child-rearing information" refers to a device or program that has the function of periodically collecting the latest articles and papers from reliable health information sites and child-rearing information sites and analyzing them using text analysis tools.

[2040] "Means for acquiring profile data of individual children" refers to a device or program for acquiring individual data such as the age, health condition, allergy information, emotional state, etc. of the child or the user himself / herself entered by the user.

[2041] "Means for analyzing collected information based on a child's profile data and generating individualized advice" refers to a device or program that has the function of analyzing collected information based on the obtained child's profile data and generating individually tailored advice from the results.

[2042] The "means for notifying the user terminal of the individual advice" is a device or program having a function for notifying the user terminal of the generated individual advice as a message that is easy for the user to understand.

[2043] "Means for assessing the emotional state of parents and children and suggesting appropriate support messages or products based on the emotional state" refers to a device or program that has the function of collecting and analyzing voice and facial expression data of parents and children, assessing their emotional state, and then suggesting the most appropriate support message or product.

[2044] "Means for managing customer schedule data and notifying schedule reminders to the user terminal" refers to a device or program that has the function of managing parent schedule data and children's event schedules and notifying the user terminal of reminders when the scheduled date approaches.

[2045] "Means for receiving data from surveillance cameras and sensors" refers to a device or program that has the function of receiving data in real time from surveillance cameras and sensors installed within the store.

[2046] "Means for analyzing received data and detecting abnormalities" refers to a device or program that has the function of analyzing data received from a surveillance camera or sensor and detecting dangerous situations or abnormalities.

[2047] The "means for issuing a warning when an abnormality is detected" refers to a device or program that has the function of immediately generating and issuing a warning alert when an abnormality is detected.

[2048] The "means for detecting an abnormality and notifying a user terminal of a warning" refers to a device or program having the function of notifying a user terminal of a warning upon detecting an abnormality.

[2049] The "means for collecting children's voice and facial expression data" refers to a device or program for collecting children's voice and facial expression data in real time.

[2050] "Means for analyzing collected data and assessing emotional state" refers to a device or program that has the function of assessing a child's emotional state (e.g., sadness, joy) based on collected data.

[2051] The "means for generating an appropriate message based on the emotional state" refers to a device or program that has the function of generating an appropriate response or support message depending on the evaluated emotional state.

[2052] The "means for notifying the user terminal of the generated message" is a device or program having a function for notifying the user terminal of the generated support message.

[2053] The "means for evaluating the emotional state of the parent and generating a support message based on the overall emotional state of the parent and child" refers to a device or program that has the function of analyzing the parent's voice and facial expression data, evaluating the overall emotional state of the parent and child, and generating an appropriate support message based on that.

[2054] The "means for the physical robot to notify the parent and child of the generated support message" is a device or program that has the function of a dedicated robot notifying the parent and child of the generated support message by voice or display.

[2055] The present invention is a system that provides comprehensive support for child-rearing in a brick-and-mortar store by linking an AI-equipped robot, a server, a user terminal, and an emotion engine. The system of the present invention includes the following main components and functions:

[2056] 1. Gathering the latest information and providing personalized advice

[2057] The server periodically collects the latest articles and papers from reliable health and parenting information sites and analyzes them using text analysis tools. The collected information is stored in a database, and personalized advice is generated based on the profile data of the child and parent entered by the user. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[2058] As a specific example, the server collects the latest articles on influenza prevention methods, and based on data entered by the parents about a 5-year-old child (with no allergies), generates advice such as "taking vitamin C is important," and notifies the user's device.

[2059] 2. Child safety checks using surveillance cameras and sensors

[2060] The server receives real-time data from surveillance cameras and motion sensors installed in the store and uses image recognition models to detect dangerous situations. If an abnormality is detected, the server generates a warning alert and notifies the user's device.

[2061] As a specific example, if a surveillance camera receives streaming footage of a child climbing stairs, the server will use an image recognition model to confirm that the child is climbing stairs and send a warning to the user device that the child is in a dangerous situation.

[2062] 3. Use of emotional support and emotional engines

[2063] The server collects voice and facial expression data from parents and children and analyzes it using an emotion analysis algorithm. It evaluates their emotional state and uses an emotion engine to generate support messages and product suggestions based on that state. The generated messages are sent to the user's device, and the AI ​​robot then speaks to the parent or child.

[2064] As a specific example, if the server collects a child's facial expression and voice data and evaluates the child as "sad," the emotion engine will generate a message saying "It's okay. Is something bothering you?", and the AI ​​robot will speak this to the child and simultaneously notify the user's device.

[2065] 4. Health management and nutrition education support

[2066] The user device provides an interface for parents to input their child's dietary information and health status, and the server stores and analyzes the input data. It identifies nutritional balance and nutrient deficiencies, generates appropriate meal suggestions, and notifies the user device.

[2067] As a specific example, if a parent inputs their child's daily diet and the server identifies that their child is not getting enough vitamin C, the server will notify the user's device with a suggestion to "incorporate salads with oranges, which are rich in vitamin C."

[2068] 5. Schedule management and reminder provision

[2069] The server manages the parent's schedule and the child's event schedule, and generates reminders based on the schedule data, ensuring that parents do not forget important events.

[2070] As a specific example, parents input their child's vaccination schedule, the server stores the schedule, and when the scheduled date approaches, a reminder is sent to the user's device saying, "The vaccination is tomorrow."

[2071] 6. User Emotion Recognition

[2072] The emotion engine collects and analyzes the parent's voice and facial expression data to assess the parent's emotional state. Based on the assessed emotional state, it tailors individual advice and support messages to provide more effective support.

[2073] As a specific example, if a parent is assessed as feeling stressed, the server generates advice such as "We recommend that you make time to relax with your child" and notifies the user terminal of the advice.

[2074] Example of an input prompt for a generative AI model:

[2075] "Based on your child's health, please tell us the latest flu prevention methods and food suggestions."

[2076] This system allows parents and children to enjoy shopping in-store with peace of mind and receive comprehensive childcare support.

[2077] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2078] Step 1:

[2079] The user enters their child's profile data (e.g., age, health status, allergy information, etc.) into the device. The device sends the entered data to the server. At this stage, the input is the user's profile data, and the output is that data sent to the server.

[2080] Step 2:

[2081] The server periodically collects the latest articles and papers from reliable health and child-rearing information sites and analyzes them using text analysis tools. The input is the latest article and paper data from external information sites, and the output is a database that stores the analysis results. Specifically, the server periodically crawls information sites, collects text data, and analyzes and classifies the information using NLP (natural language processing) algorithms.

[2082] Step 3:

[2083] The server compares the user's child's profile data with the latest collected information to generate personalized advice. The input is the user's profile data and the analyzed latest information, and the output is personalized advice. At this stage, the server compares the profile to select relevant information and uses natural language generation (NLG) technology to generate advice messages in a format that is easy for the user to understand.

[2084] Step 4:

[2085] The generated individual advice is notified to the device and displayed to the user. The input is the generated advice message, and the output is a notification that the user sees. Specifically, the advice message is pushed from the server to the device and displayed as a pop-up on the device screen.

[2086] Step 5:

[2087] The server receives real-time data from surveillance cameras and motion sensors. The input is video and motion data from the surveillance cameras and sensors, and the output is a data stream for analysis. The server captures and stores the streaming data provided by the cameras and sensors.

[2088] Step 6:

[2089] The server analyzes the received data using an image recognition model to detect anomalies. The input is real-time data from surveillance cameras and sensors, and the output is the results of anomaly detection. Specifically, the server uses a CNN (convolutional neural network) model to detect abnormal behavior in the video (e.g., a child playing on the stairs).

[2090] Step 7:

[2091] If an anomaly is detected, the server generates a warning alert and notifies the user device. The input is the anomaly detection result, and the output is a warning alert message. When an anomaly is detected, the server constructs the alert message and sends a push notification to the user device.

[2092] Step 8:

[2093] The server collects the voice and facial expression data of the parent and child and analyzes it using an emotion analysis algorithm. The input is the voice and facial expression data of the parent and child, and the output is the evaluation result of the emotional state. The server evaluates the emotional state based on the collected voice and video data using an emotion analysis model (e.g., CNN or RNN).

[2094] Step 9:

[2095] Based on the emotional state, the server generates appropriate support messages and product suggestions and notifies the user device. The input is the evaluation result of the emotional state, and the output is the support message and product suggestions. Based on the evaluation result, the server generates appropriate support messages and notifies the user device.

[2096] Step 10:

[2097] The server manages the parent's schedule data and the child's event schedule and generates schedule reminders. The input is the parent's schedule data and event schedule, and the output is a reminder message. The server uses schedule management software to push reminder notifications to the user's device when the event date approaches.

[2098] Step 11:

[2099] A dedicated robot notifies the parent and child of the generated support message by voice or display. The input is the support message sent from the server, and the output is the notification by voice or display from the robot. Specifically, the robot uses voice synthesis technology to convey the support message to the parent and child, and simultaneously displays it on the display.

[2100] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2101] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2102] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2103] [Fourth embodiment]

[2104] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2105] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2106] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2107] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2108] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2109] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2110] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2111] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2112] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2113] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2114] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2115] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2116] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2117] This invention is a system that provides comprehensive support for child-rearing through collaboration between an AI-equipped robot, a server, and a user device. The following describes the design of this system and the specific operation of each function.

[2118] 1. Gathering the latest information and providing personalized advice

[2119] The server regularly collects the latest articles and papers from reliable health and child-rearing information websites on the Internet. This information is stored in a database by category using text analysis tools. Each child's profile data (age, health condition, allergy information, etc.) is obtained, and the collected information is analyzed based on this to generate individualized advice. The generated advice is then sent to the user's device as an easy-to-understand message using natural language processing technology.

[2120] Examples:

[2121] The server collects the latest articles on "How to prevent winter flu" and stores them in a database.

[2122] Extract relevant preventive measures based on the user's child (age 5, no allergies).

[2123] The system generates advice such as "It is important to take vitamin C" and notifies the user's device.

[2124] 2. Child safety checks using surveillance cameras and sensors

[2125] The server receives real-time data from surveillance cameras and motion sensors. The data is analyzed using image recognition models to detect abnormalities (such as a child climbing stairs). When an abnormality is detected, an alert is generated and sent to the user's device.

[2126] Examples:

[2127] A surveillance camera streams footage of a child accidentally climbing the stairs.

[2128] The server uses an image recognition model to identify anomalies and generates a warning that a child is in danger.

[2129] The warning is notified to the user terminal.

[2130] 3. Emotional support

[2131] The server collects the child's voice and facial expression data and analyzes it using an emotion analysis algorithm. It evaluates the child's emotional state (e.g., sadness or joy) and generates an appropriate message based on the results. The generated message is then sent to the user's device.

[2132] Examples:

[2133] The server collects the child's facial expression and voice data and evaluates the child as "sad."

[2134] Based on the analysis results, an encouraging message is generated, such as "It's okay, is there something bothering you?"

[2135] The emotional state and response content are notified to the user terminal.

[2136] 4. Health management and nutrition education support

[2137] The user device provides an interface where parents can input their child's dietary habits and health status. The server stores the input data and accumulates it daily. This data is then analyzed using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies. Based on the results, appropriate meal suggestions are generated and sent to the user device.

[2138] Examples:

[2139] Parents input their child's daily dietary information into a user terminal.

[2140] The server analyzes the data and identifies a lack of vitamin C intake.

[2141] "Eat more oranges and kiwis, which are richer in vitamin C," he suggests.

[2142] The user terminal is notified of the details of the proposal.

[2143] 5. Schedule management and reminder provision

[2144] The server manages the parent's schedule and the child's event schedule, generates reminders based on this schedule, and notifies the user's device as appropriate.

[2145] Examples:

[2146] Parents enter their child's vaccination schedule into a user terminal.

[2147] The server stores the schedule and generates a reminder as the day approaches: "Your vaccination is tomorrow."

[2148] A reminder is sent to the user's device.

[2149] In this way, the system of the present invention provides comprehensive support for child-rearing through functions such as collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and providing reminders.

[2150] The processing flow will be explained below.

[2151] 1. Gathering the latest information and providing personalized advice

[2152] Step 1:

[2153] Server: Regularly accesses reliable health and parenting information sites on the Internet and scrapes new articles and papers.

[2154] Step 2:

[2155] Server: Analyzes the content of collected articles using text analysis tools and stores them in a database by category.

[2156] Step 3:

[2157] Server: Obtains the child's individual profile data entered by the user (e.g., age, health status, allergy information).

[2158] Step 4:

[2159] Server: Based on the analyzed information, it generates personalized advice based on the child's profile data.

[2160] Step 5:

[2161] Server: Using natural language processing technology, the generated advice is converted into a message that is easy for the user to understand.

[2162] Step 6:

[2163] User device: The generated advice message is sent to the parent's smartphone or tablet.

[2164] Examples:

[2165] 1. The server collects the latest articles on "How to prevent winter influenza" and stores them in a database.

[2166] 2. The server retrieves the child's profile data and extracts relevant preventive measures.

[2167] 3. Generate advice such as "It is important to take vitamin C" and notify the user's device.

[2168] 2. Child safety checks using surveillance cameras and sensors

[2169] Step 1:

[2170] Server: Streams and receives real-time data from surveillance cameras and motion sensors.

[2171] Step 2:

[2172] Server: Analyzes received video data using an image recognition model to detect dangerous situations.

[2173] Step 3:

[2174] Server: Generates a warning alert if an anomaly is detected.

[2175] Step 4:

[2176] AI robot: Based on abnormality alerts received from the server, it issues a warning according to pre-set response methods.

[2177] Step 5:

[2178] User device: Real-time alert notifications are sent to parents' smartphones or tablets.

[2179] Examples:

[2180] 1. A security camera streams footage of a child accidentally climbing the stairs.

[2181] 2. The server uses an image recognition model to identify the child climbing the stairs and generates an anomaly alert.

[2182] 3. The AI ​​robot issues a warning that a child is in danger and notifies the user's device.

[2183] 3. Emotional support

[2184] Step 1:

[2185] Server: Using voice and facial recognition functions, collects children's voice and facial expression data in real time.

[2186] Step 2:

[2187] Server: Analyzes the collected data using an emotion analysis algorithm to evaluate the child's emotional state.

[2188] Step 3:

[2189] Server: Generates appropriate responses and support messages depending on the assessed emotional state.

[2190] Step 4:

[2191] AI robot: Speaks to children based on the generated message.

[2192] Step 5:

[2193] User device: Notifies parents of the child's emotional state and the AI ​​robot's response.

[2194] Examples:

[2195] 1. The server collects the child's facial expression and voice data and evaluates the child as "sad."

[2196] 2. The server generates an encouraging message based on the evaluation results, saying, "It's okay, is there something bothering you?"

[2197] 3. The AI ​​robot conveys the message to the child and notifies the user's device of their emotional state and response.

[2198] 4. Health management and nutrition education support

[2199] Step 1:

[2200] User terminal: Provides an interface where parents can input their child's diet and health status.

[2201] Step 2:

[2202] Server: Saves the entered data and accumulates daily data.

[2203] Step 3:

[2204] Server: Analyzes the accumulated data using a nutritional analysis algorithm to identify nutritional balance and nutrient deficiencies.

[2205] Step 4:

[2206] Server: Generates appropriate meal suggestions based on the analysis results.

[2207] Step 5:

[2208] User terminal: Notifies parents of the generated meal suggestions and nutritional status reports.

[2209] Examples:

[2210] 1. Parents enter their child's daily dietary information into the user device.

[2211] 2. The server analyzes the input data and identifies that the person is lacking in vitamin C intake.

[2212] 3. "Incorporate more foods rich in vitamin C" is suggested as a specific recipe, and a notification is sent to the user's device.

[2213] 5. Schedule management and reminder provision

[2214] Step 1:

[2215] User terminal: Provides an interface for parents to input event schedules and schedules.

[2216] Step 2:

[2217] Server: Saves and manages the entered schedule data.

[2218] Step 3:

[2219] Server: Generates reminders based on schedule data.

[2220] Step 4:

[2221] User device: Notifies the parent of the generated reminder.

[2222] Examples:

[2223] 1. Parents enter their child's vaccination schedule into the user device.

[2224] 2. The server stores the appointment and generates a reminder when the appointment date approaches.

[2225] 3. The user device sends a reminder to the parent that "Vaccination is tomorrow."

[2226] Example 1

[2227] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2228] In child-rearing, it is often difficult for parents to receive the latest health information or appropriate advice based on the individual condition of their child, and children's safety and emotional care are often not adequately confirmed. This creates problems that make it difficult for parents to adequately support their children's health, safety, and emotional growth. To solve these problems, a comprehensive system of support for child-rearing is needed.

[2229] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2230] In this invention, the server includes a means for collecting parenting information, a means for acquiring profile data for each child, and a means for classifying and saving the collected information using a text analysis tool. This allows parents to obtain the latest parenting information tailored to their individual circumstances in a timely manner. The server also includes a means for analyzing the collected information based on the child's profile data using a machine learning algorithm to generate personalized advice, a means for notifying a user terminal of the generated personalized advice using natural language processing technology, a means for receiving data from surveillance cameras and sensors, a means for analyzing the received data using an image recognition model to detect abnormalities, a means for generating a warning when an abnormality is detected and notifying the user terminal, a means for collecting voice and facial expression data of the child, a means for analyzing the collected data using an emotion analysis algorithm to evaluate the emotional state, a means for generating an appropriate message based on the emotional state using natural language processing technology, and a means for notifying the user terminal of the generated message. This enables comprehensive monitoring and analysis of a child's health, safety, and emotional state, and for providing appropriate advice and warnings in real time.

[2231] "Means for collecting parenting information" include devices and programs for collecting the latest articles and papers from reliable health and parenting information sites, specifically scraping tools and APIs.

[2232] "Means for obtaining individual child profile data" refers to a database and its interface for collecting and storing individual information such as each child's age, health condition, and allergy information, and for retrieving it as needed.

[2233] "Means for classifying and storing collected information using text analysis tools" refers to devices or programs that include text analysis tools for analyzing collected information, classifying it into categories, and storing it in a database.

[2234] "Means for analyzing using machine learning algorithms and generating personalized advice" refers to devices or programs that analyze collected data using machine learning algorithms and generate personalized advice based on each child's profile.

[2235] "Means for notifying a user terminal using natural language processing technology" refers to a device or program that uses natural language processing technology to generate a message from the generated advice in a form that is easy for the user to understand, and then sends that message to the user terminal.

[2236] "Means for receiving data from surveillance cameras and sensors" refers to devices and programs that transmit video and motion data from surveillance cameras and various sensors to a server in real time.

[2237] "Means for analyzing and detecting abnormalities using an image recognition model" refers to a device or program that analyzes received video and motion data using an image recognition model and detects abnormalities.

[2238] "Means for generating a warning when an abnormality is detected and notifying the user terminal" refers to a device or program that generates a warning message when an abnormality is detected and sends that message to the user terminal.

[2239] "Means for collecting children's voice and facial expression data" refers to devices such as cameras and microphones for collecting children's voice and facial expression data in real time, as well as programs for operating them.

[2240] "Means for evaluating a child's emotional state by analyzing using an emotion analysis algorithm" refers to a device or program that analyzes collected voice and facial expression data using an emotion analysis algorithm to evaluate a child's emotional state.

[2241] "Means for generating an appropriate message based on an emotional state using natural language processing technology" refers to a device or program that generates an appropriate message based on an evaluated emotional state using natural language processing technology.

[2242] The "means for notifying the user terminal of the generated message" refers to a device or program for transmitting the generated message to the user terminal.

[2243] This invention is a system that provides comprehensive support for child-rearing through collaboration between an AI-equipped robot, a server, and a user device. This system has multiple functions, including collecting the latest information and providing individualized advice, checking children's safety using surveillance cameras and sensors, providing emotional support, supporting health management and nutrition education, and managing schedules and providing reminders.

[2244] 1. Gathering the latest information and providing personalized advice

[2245] The server regularly collects the latest articles and papers from reliable health and parenting information websites. To do this, it uses Python scripts and the BeautifulSoup library to perform scraping. It then uses a text analysis tool (e.g., Apache OpenNLP) to categorize the collected information and store it in a database (e.g., MySQL). It obtains the user's child's profile data (e.g., age, health condition, allergy information, etc.) and analyzes it using machine learning algorithms such as scikit-learn. The generated advice is then converted into a user-friendly message using natural language processing technology (e.g., OpenAI's GPT-3) and sent to the user's device as a push notification.

[2246] Examples:

[2247] The server collects the latest articles on "How to prevent winter flu" and stores them in a database.

[2248] Extract relevant preventive measures based on the user's child (age 5, no allergies).

[2249] The system generates advice such as "It is important to take vitamin C" and notifies the user's device.

[2250] Example prompt sentence:

[2251] Input: Please tell me how to prevent winter flu in my child (5 years old, no allergies).

[2252] Output: Vitamin C intake is important. Eat plenty of oranges and kiwis.

[2253] 2. Child safety checks using surveillance cameras and sensors

[2254] The server receives real-time data from surveillance cameras (e.g., Nest Cam) and motion sensors (e.g., PIR sensors). The received data is analyzed using an image recognition model (e.g., TensorFlow's YOLO model) to detect anomalies. If an anomaly is detected, the server generates an alert and notifies the user device.

[2255] Examples:

[2256] A surveillance camera streams footage of a child accidentally climbing the stairs.

[2257] The server uses an image recognition model to identify anomalies and generates a warning that a child is in danger.

[2258] The warning is notified to the user terminal.

[2259] Example prompt sentence:

[2260] Input: Footage from a security camera shows a child starting to climb the stairs.

[2261] Output: Warning! Child in danger. Please act immediately.

[2262] 3. Emotional support

[2263] The server collects the child's voice and facial expression data and analyzes it using emotion analysis algorithms such as Microsoft Face API and Google Cloud Speech-to-Text. It evaluates the child's emotional state and generates an appropriate message based on the results. The message is then sent to the user's device.

[2264] Examples:

[2265] The server collects the child's facial expression and voice data and evaluates the child as "sad."

[2266] Based on the analysis results, an encouraging message is generated, such as "It's okay, is there something bothering you?"

[2267] The emotional state and response content are notified to the user terminal.

[2268] Example prompt sentence:

[2269] Input: How do you respond when your child is sad?

[2270] Output: Try gently saying, "It's okay. Is there something bothering you?"

[2271] 4. Health management and nutrition education support

[2272] The user device provides an interface where parents can input their child's dietary information and health status. The server stores the input data in a database and accumulates a daily diet history. This data is then analyzed using the Nutrition Data API to identify nutritional balance and nutrient deficiencies. Based on the results, appropriate meal suggestions are generated and notified to the user device.

[2273] Examples:

[2274] Parents input their child's daily dietary information into a user terminal.

[2275] The server analyzes the data and identifies a lack of vitamin C intake.

[2276] "Eat more oranges and kiwis, which are richer in vitamin C," he suggests.

[2277] The user terminal is notified of the details of the proposal.

[2278] Example prompt sentence:

[2279] Enter: If today's meal doesn't include oranges or kiwi, what are your suggestions?

[2280] Output: Eat more oranges and kiwis, which are rich in vitamin C.

[2281] 5. Schedule management and reminder provision

[2282] The server manages the parent's schedule and the child's event schedule, generates reminders based on this schedule, and notifies the user's device as appropriate.

[2283] Examples:

[2284] Parents enter their child's vaccination schedule into a user terminal.

[2285] The server stores the schedule and generates a reminder as the day approaches: "Your vaccination is tomorrow."

[2286] A reminder is sent to the user's device.

[2287] Example prompt sentence:

[2288] Enter: Set a reminder so you don't forget to schedule your vaccination appointment.

[2289] Output: Your vaccination is tomorrow. Don't forget.

[2290] In this way, the system of the present invention provides comprehensive support for child-rearing through functions such as collecting the latest information and providing individual advice, checking children's safety using surveillance cameras and sensors, providing emotional support, health management and nutritional education support, schedule management and providing reminders.

[2291] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2292] 1. Gathering the latest information and providing personalized advice

[2293] Step 1:

[2294] The server collects the latest articles and papers from reliable health and child-rearing information sites on the Internet.

[2295] Input: List of URLs to be collected

[2296] What it does: Use a Python script to parse the HTML and extract data from each URL using the BeautifulSoup library.

[2297] Output: Collected text data

[2298] Step 2:

[2299] The server uses text analysis tools to categorize the information it collects.

[2300] Input: Collected text data

[2301] What it does: It uses Apache OpenNLP to analyze text and classify it into categories.

[2302] Output: Text data classified by category

[2303] Step 3:

[2304] The server stores the classified information in a database.

[2305] Input: Categorized text data

[2306] Specific behavior: Connects to a MySQL database and inserts data into the appropriate tables.

[2307] Output: Saved database entries

[2308] Step 4:

[2309] The server retrieves the profile data for each child from the database.

[2310] Input: Child's identification

[2311] Specific operation: Issues an SQL query and retrieves profile data.

[2312] Output: Child profile data

[2313] Step 5:

[2314] The server uses machine learning algorithms to analyze the profile data and collected information to generate personalized advice.

[2315] Input: Profile data, categorical text data

[2316] What it does: Uses the scikit-learn library to analyze information that matches the profile data.

[2317] Output: personalized advice

[2318] Step 6:

[2319] The server notifies the generated individual advice to the user terminal using natural language ...

Claims

1. A means of collecting child-rearing information; A means of obtaining profile data for each child; and means for analyzing the collected information based on the child's profile data and generating personalized advice; means for notifying a user terminal of the individual advice; A system including:

2. A means for receiving data from surveillance cameras and sensors; means for analyzing the received data and detecting anomalies; a means for issuing an alarm when an abnormality is detected; means for notifying a user terminal of the detection of an abnormality and a warning; The system of claim 1 , comprising:

3. A means of collecting children's voice and facial expression data, a means for analyzing the collected data to assess emotional state; means for generating an appropriate message based on the emotional state; means for notifying a user terminal of the generated message; The system of claim 1 , comprising:

4. A means for inputting the child's dietary content and health condition from a user terminal; means for saving and storing the input data; A method for analyzing accumulated data to identify nutritional balance and nutrient deficiencies, and A means for generating meal suggestions based on nutritional balance; means for notifying a user terminal of the generated meal suggestions; The system of claim 1 , comprising:

5. In order to check the safety of children, we will provide a means for receiving, analyzing, detecting abnormalities and notifying data from surveillance cameras and sensors. In emotional support, collection of voice and facial expression data, emotion analysis, message generation and notification means, In health management and nutrition education support, it is a means of inputting, saving, accumulating, analyzing, suggesting meals and notifying users about their dietary information and health status. The system of claim 1 , comprising:

6. A way to manage parental schedules and children's event schedules, means for generating reminders based on a schedule; means for notifying a user terminal of the generated reminder; The system of claim 1 , comprising:

Images