system

Abstract

To provide a system that can reduce the burden of child-rearing on parents and provide an environment where parents can focus on child-rearing with peace of mind by managing a baby's behavior in real time and notifying them of countermeasures. [Solution] A system including: a means for collecting voice data of the baby; a means for collecting sleep data of the baby; a means for transmitting the collected voice data and sleep data to a server; a means for analyzing the voice data and sleep data and learning the baby's behavioral patterns; a means for generating behavioral predictions and countermeasures based on the learned behavioral patterns; a means for notifying a user terminal of the generated behavioral predictions and countermeasures; a means for displaying the notified behavioral predictions and countermeasures on a user interface; a means for providing a community function that allows the user to share information with others; and a means for consulting with an expert free of charge within a dedicated app.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern families raising children, especially those with no parenting experience or dual-income households, face the challenge of accurately understanding a baby's behavior and condition and responding appropriately. Conventional baby monitors and childcare support tools lack the functionality to analyze a baby's behavioral patterns in real time and provide individualized solutions. This leaves parents relying on their own experience, general reference books, and online information, often resulting in anxiety and stress. The present invention aims to solve these issues by managing a baby's behavior in real time and providing a system that notifies parents of appropriate solutions, thereby reducing the burden of childcare and providing an environment in which parents can focus on childcare with peace of mind. [Means for solving the problem]

[0005] The present invention is a system including means for collecting baby voice data, means for collecting baby sleep data, means for transmitting the data to a server, means for analyzing the received data to learn the baby's behavioral patterns, means for generating behavioral predictions and countermeasures based on the learned behavioral patterns, means for notifying a user terminal of the generated behavioral predictions and countermeasures, means for displaying the notified behavioral predictions and countermeasures on a user interface, means for providing a community function for users to share information with others, and means for free consultation with experts within a dedicated app. The system also includes a function for identifying a baby's crying pattern through voice analysis and estimating the baby's condition based on the identified crying pattern, and a function for analyzing the sleep data to extract the degree of deep sleep and sleep duration and provide an appropriate sleep pattern, thereby achieving comprehensive health management for babies and support for parents.

[0006] "Audio data" refers to sound information such as a baby's cries and vocalizations.

[0007] "Sleep data" refers to sleep-related information such as the baby's rolling over, heart rate, and breathing patterns.

[0008] "Server" refers to a central device that receives, analyzes, stores, and notifies data sent from baby monitors and sleep monitors.

[0009] "Behavioral patterns" refer to the tendencies and regularities in a baby's behavior that are observed at specific times and in specific situations.

[0010] "Behavioral prediction" refers to predicting a baby's future behavior based on learned behavioral patterns.

[0011] "Countermeasures" refer to appropriate countermeasures for predicted behavior.

[0012] "User device" refers to a device used by a parent to receive and display notifications, such as a smartphone or tablet.

[0013] "User interface" refers to the screen and operating means that allow a user to view information and perform operations.

[0014] "Community function" refers to the bulletin board and chat functions within the system that allow users with the same child-rearing environment to share information with each other.

[0015] "A means to consult with a specialist for free" refers to a function that allows users to consult with a specialist doctor or counselor online for free about questions or concerns about child-rearing. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] A specific embodiment of the present invention will be described. The present invention is a system for managing a baby's behavior in real time and notifying parents of appropriate countermeasures. This system collects and analyzes the baby's voice data and sleep data to learn and predict the baby's behavioral patterns and provide countermeasures.

[0038] System configuration

[0039] The system consists of the following main components:

[0040] Devices (baby monitors, sleep monitors):

[0041] The baby monitor collects the baby's cries and vocalizations and sends them to a server as audio data.

[0042] The sleep monitor collects the baby's movements, heart rate, and breathing patterns and sends the data to a server.

[0043] server:

[0044] Data reception and storage: Receives voice and sleep data sent from the device and stores it in a database.

[0045] Data analysis: The received data is analyzed using an AI algorithm to learn the baby's behavioral patterns.

[0046] Behavior prediction and generation of countermeasures: Behavior prediction is made based on learned behavior patterns, and countermeasures are generated based on that prediction.

[0047] Sending notification: The generated behavior prediction and how to deal with it are notified to the user device.

[0048] User device (dedicated app):

[0049] Receive and display notifications: Receive notifications sent from the server and display them in an easy-to-read format for the user.

[0050] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[0051] Specialist consultation function: Provides a function to consult with a specialist online about questions regarding childcare.

[0052] Program processing

[0053] The processing performed by the program of this system will be explained below with specific examples.

[0054] Devices (baby monitors, sleep monitors)

[0055] A baby monitor detects a baby's crying and sends the audio data to a server. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[0056] The sleep monitor detects the baby's heart rate and tossing and turning, and sends sleep data to a server, for example, when the baby goes into deep sleep or wakes up.

[0057] server

[0058] The server receives the crying data sent by the baby monitor and analyzes it, for example to determine if the crying matches a certain pattern (e.g., hungry, sleepy, etc.).

[0059] Analyze sleep data to extract the baby's deep sleep level and sleep duration. For example, analyze whether the baby is sleeping soundly in the middle of the night.

[0060] Based on the analysis results, the system predicts the baby's behavior and generates countermeasures. For example, if the baby is crying because he or she is hungry, the system generates a countermeasure such as "Please prepare milk."

[0061] The generated behavior prediction and how to deal with it are notified to the user device. For example, a notification saying "Your baby has woken up. Please try to put him / her back to sleep" is sent.

[0062] User device (dedicated app)

[0063] The dedicated app receives notifications from the server and displays them on the user interface. For example, a parent can check a notification on their smartphone saying, "The baby has woken up."

[0064] Through the community function, parents can share parenting information with other parents. For example, they can ask questions about parenting problems and receive advice from other parents in the community.

[0065] You can use the specialist consultation function to consult with a specialist online. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[0066] The above is an embodiment of the present invention. This system allows parents to understand their baby's condition in real time and take appropriate measures. As a result, the burden of childcare on parents is reduced, allowing them to focus on raising their child with peace of mind.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The device (baby monitor) collects the baby's cries and vocalizations. For example, if the baby cries, the cry is recognized as audio data and recorded.

[0070] Step 2:

[0071] The device (sleep monitor) collects data on the baby's movements, heart rate, and breathing patterns. For example, if the baby rolls over or their heart rate changes, it records that data.

[0072] Step 3:

[0073] The devices (baby monitors and sleep monitors) transmit collected audio and sleep data to a server in real time, using Wi-Fi, Bluetooth, or other wireless communication methods.

[0074] Step 4:

[0075] The server receives the voice and sleep data sent from the device. For example, data on a baby's crying and heart rate is sent to the server.

[0076] Step 5:

[0077] The server stores the received data in a database, where each piece of data is linked to a baby's profile.

[0078] Step 6:

[0079] The server analyzes the stored data using AI algorithms to learn the baby's behavioral patterns. For example, it analyzes crying data over a certain period of time to identify the time of day and crying patterns of the baby.

[0080] Step 7:

[0081] The server predicts the baby's future behavior based on the learned behavioral patterns. For example, it predicts the time of day when the baby will cry based on past data and prepares to notify the user of the prediction.

[0082] Step 8:

[0083] The server generates an appropriate response to the predicted behavior. For example, if a pattern of "crying because the baby is hungry" is detected, the server generates a message saying "Please prepare milk" as a response.

[0084] Step 9:

[0085] The server then notifies the user device (dedicated app) of the generated behavior predictions and countermeasures. The notification content is sent in real time.

[0086] Step 10:

[0087] The device (dedicated app) receives notifications sent from the server. For example, a notification saying "The baby has woken up" is received.

[0088] Step 11:

[0089] The device (dedicated app) displays the received notification on the user interface, and the parent checks the notification and decides what to do next.

[0090] Step 12:

[0091] The user (parent) can take appropriate action based on the notification. For example, if the notification says "Prepare milk," the parent prepares milk and gives it to the baby.

[0092] Step 13:

[0093] The device (dedicated app) provides a community function that allows users to share information with other parents, and users can exchange their parenting experiences and questions within the community.

[0094] Step 14:

[0095] Users (parents) can consult with a specialist through a dedicated app about questions and concerns about childcare, and based on the specialist's answers, they can take more specific measures.

[0096] This enables the system to analyze a baby's behavioral patterns in detail and provide appropriate childcare support to parents in real time.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] When raising a baby, parents need to constantly monitor their baby's condition and respond appropriately. However, accurately understanding the reason for a baby's crying or sleep state can be difficult and often places a burden on parents. In addition, there are limited ways to obtain information and advice on childcare from other parents or experts. To solve these problems, a system is needed that monitors a baby's condition in real time and notifies parents of the appropriate course of action.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes means for collecting baby's voice data, means for collecting baby's physiological data, means for transmitting the collected voice data and physiological data to the server, means for analyzing the voice data and physiological data to learn the baby's behavioral patterns, means for generating a behavior prediction and a countermeasure based on the learned behavioral patterns, means for notifying the generated behavior prediction and countermeasure to a user terminal, means for displaying the notified behavior prediction and countermeasure on a user interface, means for providing a community function for the user to share information with others, and means for consulting with an expert within a dedicated app. This allows parents to monitor their baby's condition in real time and receive specific instructions for appropriate countermeasures. Furthermore, by sharing information with other parents and experts, parents can receive support to alleviate concerns and questions about childcare.

[0102] "Means for collecting baby voice data" refers to a device or function that records a baby's cries and vocalizations in real time and stores and transmits them in digital form.

[0103] "Means for collecting baby physiological data" means a device or function that records physiological information such as a baby's heart rate, breathing patterns, and rolling over in real time, and stores and transmits this information in digital form.

[0104] The "means for transmitting the collected voice data and physiological data to a server" refers to a communication device or function for transmitting the voice and physiological data collected by the terminal to a server via the Internet.

[0105] The "means for analyzing the voice data and physiological data and learning the baby's behavioral patterns" refers to a function that uses an AI algorithm to analyze the data received by the server and predict the baby's behavior and condition.

[0106] The "means for predicting behavior and generating a countermeasure based on the learned behavior pattern" is a function for predicting the baby's future behavior based on the analysis results and automatically generating an appropriate countermeasure.

[0107] The "means for notifying the user terminal of the generated behavior prediction and countermeasure method" is a communication device or function for transmitting the behavior prediction and countermeasure method generated by the server to the user terminal in real time.

[0108] The "means for displaying the notified predicted behavior and countermeasure method on a user interface" is an interface function for displaying the notification received by the user terminal in a form that is easy for the user to see.

[0109] "Means for providing a community function that enables the user to share information with others" refers to a bulletin board or chat function within the dedicated app that enables users to share information and concerns about child-rearing with other parents and exchange opinions.

[0110] "Means to consult with experts within the dedicated app" is a function that allows users to send questions about childcare to experts through the dedicated app and receive advice online.

[0111] This invention is a system that manages a baby's behavior in real time and notifies parents of appropriate measures. This system learns and predicts the baby's behavioral patterns by collecting and analyzing the baby's voice data and physiological data, and provides appropriate measures.

[0112] System configuration

[0113] The present invention consists of the following major components:

[0114] Devices (baby monitors, sleep monitors):

[0115] Baby monitor: Collects a baby's cries and vocalizations and sends them to a server as audio data. For example, if a baby cries in the middle of the night, the audio data is sent to the server in real time.

[0116] Sleep monitor: Collects physiological data such as the baby's movements, heart rate, breathing patterns, etc., and sends it to a server. For example, when the baby enters a deep sleep or wakes up, the data is collected and sent to the server.

[0117] server:

[0118] Data reception and storage: Receives the voice data and physiological data sent from the device and stores them in a database. For example, the server receives the baby's crying data and heart rate data and stores them in the appropriate format.

[0119] Data analysis: The received audio and physiological data is analyzed using AI algorithms. For example, the server analyzes the audio data to determine if the baby's crying matches a specific pattern (hungry, sleepy, etc.). The server also analyzes physiological data to assess the baby's sleep cycle and how soundly they are sleeping.

[0120] Behavior prediction and generation of countermeasures: Based on the analysis results, the system predicts the baby's behavior and generates appropriate countermeasures. For example, if the baby's crying is analyzed as "hungry," the system generates the countermeasure "Please prepare milk." Also, if the sleep data indicates that the baby is in a deep sleep, the system generates a notification saying "The baby is sleeping deeply. Please be quiet."

[0121] Notification sending: The server notifies the user device of the generated behavior prediction and how to deal with it. For example, it sends a notification saying, "The baby is crying. He may be hungry. Please prepare some milk."

[0122] User device (dedicated app):

[0123] Receiving and displaying notifications: The user device receives notifications from the server and displays them on the user interface. The user can check the notifications and take appropriate action. For example, the user can see a message on the smartphone app saying, "Your baby has woken up. Please try to put him / her back to sleep."

[0124] Community function: Users can use the community function to share parenting information with other parents. For example, they can ask questions about how to deal with nighttime crying and receive advice from other parents.

[0125] Specialist consultation function: Users can use the specialist consultation function of the dedicated app to consult with experts online about childcare-related questions. For example, they can send a message to a specialist saying, "My baby wakes up frequently during the night," and receive specific advice.

[0126] Specific examples

[0127] For example, if a baby starts crying in the middle of the night, the system will operate as follows:

[0128] 1. The baby monitor detects the baby's crying and sends the audio data to the server in real time.

[0129] 2. The server receives the crying data and stores it in a database. At the same time, it uses an AI algorithm to analyze the crying data and identify behavioral patterns that indicate "hungry."

[0130] 3. If the server determines that the user is hungry, it generates a solution such as "Please prepare milk" and notifies this information to the user's terminal.

[0131] 4. The user device (dedicated app) receives a notification from the server and displays a message to the parent saying, "The baby is crying. He may be hungry. Please prepare some milk."

[0132] Example input to a generative AI model

[0133] Below are some example prompts to input to the generative AI model:

[0134] User: "My baby started crying in the middle of the night. What should I do?"

[0135] AI: "Let's find out why your baby is crying. First, he may need milk, so get some ready. If he still doesn't stop crying, check to see if he has a wet diaper or is uncomfortable."

[0136] This invention allows parents to monitor their baby's condition in real time and take appropriate measures promptly. In addition, by sharing information with other parents and experts, parents can alleviate concerns and questions about childcare, allowing them to approach childcare with peace of mind.

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1:

[0139] Data collection from devices (baby monitors, sleep monitors)

[0140] How a baby monitor works: A baby monitor collects a baby's crying sounds in real time. Specifically, the baby monitor's microphone detects the crying sounds and converts the audio data into a digital format. The input is the baby's crying sounds, and the output is digital audio data.

[0141] How a sleep monitor works: A sleep monitor monitors a baby's heart rate, breathing patterns, and rolling over. This data is collected using digital sensors. The input is the baby's physiological movements, and the output is digital physiological data.

[0142] Step 2:

[0143] Sending data from the device to the server

[0144] How the baby monitor works: The baby monitor sends the collected audio data to a server via the Internet. Specifically, the communication module in the device sends the data. The input is digital audio data, and the output is the audio data sent to the server.

[0145] Operation of the sleep monitor: The sleep monitor also sends the collected physiological data to a server. Specifically, a communication module sends the data via the Internet. The input is digital physiological data, and the output is the physiological data sent to the server.

[0146] Step 3:

[0147] Receiving and storing data by the server

[0148] Data reception: The server receives data sent from the baby monitor or sleep monitor. Specifically, the server's communication module receives the data. The input is audio data and physiological data sent via the Internet, and the output is data stored in the server's memory.

[0149] Data Storage Operation: The received audio and physiological data is stored in a database. Specifically, the database management system stores the data in an appropriate format. The input is the received data and the output is the stored database entry.

[0150] Step 4:

[0151] Data analysis by server

[0152] Analysis of audio data: The server analyzes the audio data using an AI algorithm. Specifically, it compares the crying data with specific patterns (hungry, sleepy, etc.). The input is the stored audio data, and the output is the analysis result indicating the cause of the crying.

[0153] Physiological data analysis: The server analyzes the physiological data to assess the baby's sleep cycle and deepness. For example, it analyzes heart rate and breathing patterns to determine whether the baby is sleeping deeply. The input is the stored physiological data, and the output is the analysis result indicating the state of sleep.

[0154] Step 5:

[0155] Server predicts behavior and generates countermeasures

[0156] Generation of behavior prediction: The server predicts the baby's behavior based on the analysis results. For example, if the analysis of a baby's cry indicates that the baby is hungry, a behavior prediction is made based on that information. The input is the analysis results of the voice data and physiological data, and the output is a specific behavior prediction.

[0157] Generation of action plan: The server generates an appropriate action plan based on the behavior prediction. For example, if the predicted result is "I'm hungry," the server generates the action plan "Please prepare some milk." The input is the behavior prediction result, and the output is a specific action plan.

[0158] Step 6:

[0159] Notification from the server to the user device

[0160] Sending notification: The server notifies the user device of the generated behavior prediction and countermeasures. Specifically, the communication module sends the notification data. The input is the behavior prediction and countermeasure data, and the output is a notification to the user device.

[0161] Step 7:

[0162] Receiving and displaying notifications on user devices

[0163] Receiving notifications: The user device receives notifications from the server. The communication module receives the notification data, and the dedicated application uses that data. The input is the notification data sent from the server, and the output is the display on the user interface.

[0164] Displaying a notification: The dedicated app displays a notification on the user interface. Specifically, the app displays a message on the screen saying, "Your baby is crying. He might be hungry. Please prepare some milk." The input is the received notification data, and the output is the notification displayed to the user.

[0165] Step 8:

[0166] Additional functions for user devices (community function, specialist consultation function)

[0167] Community function: Users can share parenting information with other parents through the community function. For example, they can ask questions about specific parenting concerns and receive advice from other parents. The input volume is the questions and information entered by the user, and the output volume is the answers and shared information from other users.

[0168] Specialist consultation function: Users can use the specialist consultation function of the dedicated app to consult with an expert online about questions about childcare. For example, they can send a question such as "My baby wakes up frequently during the night" to a specialist and receive specific advice. The input is the question entered by the user, and the output is advice from the specialist.

[0169] (Application example 1)

[0170] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0171] Modern parents, especially those raising children for the first time, face a great deal of anxiety and stress. While they need to accurately understand their baby's condition and take appropriate measures, choosing the right childcare products for their baby while shopping in a physical store can be a difficult task. Furthermore, when expert advice is needed, there is a lack of means to respond quickly. There is a need to address these issues.

[0172] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0173] In this invention, the server includes means for analyzing the baby's voice data and sleep data to learn the baby's behavioral patterns, means for predicting behavior and generating countermeasures based on the learned behavioral patterns, and means for suggesting childcare products suitable for the baby's condition in a physical store. This allows parents to understand the baby's condition in real time and take appropriate measures, and also makes it possible to suggest childcare products that are optimal for the baby while shopping in the physical store.

[0174] "Baby voice data" is a digital recording of sounds made by a baby, such as crying or vocalizations.

[0175] "Baby sleep data" is a digital record of physiological information related to a baby's sleep, such as their heart rate, breathing patterns, and rolling over.

[0176] "Server" means the central system that receives, analyzes, stores, and sends notifications to user devices.

[0177] A "means for learning behavioral patterns" is an algorithm or method for analyzing collected data, identifying consistencies and changes in a baby's behavior, and learning those patterns.

[0178] The "means for predicting behavior and generating countermeasures" refers to an algorithm and method for predicting future behavior based on learned behavioral patterns and generating appropriate countermeasures for the behavior.

[0179] A "user terminal" is a mobile device such as a smartphone or tablet used by a parent or caregiver, which receives and displays notifications, provides community functions, and provides expert consultation functions.

[0180] "Means for providing community functions" are online bulletin boards and chat functions that allow parenters to share information and receive advice from each other.

[0181] "A way to consult with an expert for free" is a way to ask questions or get advice about child-rearing from an expert online for free.

[0182] The "means for suggesting childcare products in a physical store" refers to a system and method for suggesting appropriate childcare products to parents who are shopping in a store based on the real-time status of their baby.

[0183] A "generative AI model" is a supervised or unsupervised artificial intelligence model used for data analysis and behavior prediction.

[0184] A "prompt" is a textual instruction used as input to an AI model.

[0185] The following describes in detail the embodiments of the present invention. The present invention is a system that manages a baby's behavior in real time and suggests appropriate ways to deal with the baby and childcare products to parents. This system learns and predicts the baby's behavioral patterns by collecting and analyzing the baby's voice data and sleep data, and then suggests ways to deal with the baby and childcare products.

[0186] System configuration

[0187] The system consists of the following main components:

[0188] Devices (baby monitors, sleep monitors):

[0189] Baby monitors collect the baby's cries and vocalizations and send them to a server as audio data. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[0190] The sleep monitor collects the baby's heart rate and movements, and sends the data to a server as sleep data, such as when the baby falls into a deep sleep or wakes up.

[0191] server:

[0192] Data reception and storage: The voice and sleep data sent from the device is received and stored in a database using software such as a common database management system (DBMS) or cloud service.

[0193] Data analysis: The received data is analyzed using a generative AI model to learn the baby's behavioral patterns. The software used is machine learning libraries such as TENSORFLOW (registered trademark) and PyTorch.

[0194] Behavior prediction and generation of coping methods: Behavior prediction is made based on learned behavior patterns, and coping methods and childcare product suggestions are generated based on the prediction.

[0195] Sending notification: The generated behavior prediction and how to deal with it are notified to the user terminal.

[0196] User device (dedicated app):

[0197] Receiving and displaying notifications: Receives notifications sent from the server and displays them in the user interface.

[0198] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[0199] Expert consultation function: Provides a function to consult with experts online about questions regarding childcare.

[0200] A function to suggest childcare products in physical stores: Recommends the most suitable childcare products based on the baby's condition in real time.

[0201] Program processing

[0202] The server receives and analyzes the baby's voice and sleep data. For example, if the baby monitor detects the baby crying, the data is sent to the server. The server analyzes the data and determines whether the crying matches a certain pattern (e.g., hungry, sleepy, etc.).

[0203] The server then predicts behavior based on the behavioral patterns and generates suggested solutions and childcare items. This is done using a generative AI model. For example, if the baby is crying because it is hungry, the server predicts the solution, suggesting "prepare milk" and "a baby bottle."

[0204] The generated solutions and suggestions are sent to the user's device, which can be a smartphone, tablet, or other device, and displayed in a dedicated app. Additionally, if the childcare product suggestion function is enabled in the physical store, shelves with suitable childcare products will be displayed in the app, facilitating smooth purchasing for parents.

[0205] Specific examples

[0206] For example, when a baby starts crying in the morning, a baby monitor captures the sound. This data is sent to a server for analysis. If the cry is identified as a sign of hunger, the server sends a notification to the user's device suggesting a baby bottle as an appropriate childcare item, along with a message saying, "Prepare some milk."

[0207] An example of a prompt is, "Generate a notification message for a parent based on baby's crying sound, which indicates possible hunger." This allows users to quickly respond to their baby's needs and easily find and purchase the necessary childcare products.

[0208] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0209] Step 1:

[0210] Collects baby's audio and sleep data. The baby monitor captures the baby's crying in real time, and the sleep monitor collects sleep data such as the baby's heart rate, rolling over, and breathing patterns. The input is the crying audio data and heart rate and movement data, and the output is a digital file containing these data.

[0211] Step 2:

[0212] The device sends collected audio and sleep data to the server. The audio and sleep data sent from the baby monitor and sleep monitor are sent to the server. The input is the collected digital file, and the output is the data stored on the server.

[0213] Step 3:

[0214] The server analyzes the received audio and sleep data and learns the baby's behavioral patterns. The software used is a machine learning library such as TensorFlow or PyTorch. The input is the audio and sleep data stored on the server, and the output is a model that shows the baby's behavioral patterns.

[0215] Step 4:

[0216] The server generates behavior predictions and countermeasures based on the learned behavior patterns. A generative AI model is used for this. The input is a model showing the behavior patterns, and the output is text data containing the behavior predictions and countermeasures. Specific childcare products are also suggested.

[0217] Step 5:

[0218] The server notifies the user device of the generated behavior prediction and countermeasures using a dedicated notification API. The input is text data containing the behavior prediction and countermeasures, and the output is a push notification to the user device.

[0219] Step 6:

[0220] The user terminal displays the notified behavior prediction and countermeasures on the user interface. The input is the notification data sent from the server, and the output is a notification message displayed on the user interface.

[0221] Step 7:

[0222] Through a dedicated app, users are recommended childcare products suited to their baby's condition in a physical store. The input is the store's situation and real-time baby data, and the output is a list of recommended childcare products.

[0223] Step 8:

[0224] The system provides a community function that allows users to share information with others and a function that allows users to consult with experts online for free about child-rearing questions. The input is the user's questions and child-rearing information, and the output is answers and advice from other users and experts.

[0225] This trend will enable parents to understand their baby's condition in real time, take appropriate measures, and quickly select the necessary childcare products in physical stores.

[0226] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0227] A specific embodiment for carrying out the present invention will now be described. The present invention is a system that combines a basic system that manages a baby's behavior in real time with an emotion engine that recognizes the user's emotions and maximizes the effectiveness of childcare support.

[0228] System configuration

[0229] The system consists of the following main components:

[0230] Devices (baby monitors, sleep monitors, user devices):

[0231] The baby monitor collects the baby's cries and vocalizations and sends them to a server as audio data.

[0232] The sleep monitor collects the baby's movements, heart rate, and breathing patterns and sends the data to a server.

[0233] The user device collects the user's voice and facial expressions and sends them to the server as emotional data.

[0234] server:

[0235] Data reception and storage: Receives voice data, sleep data, and emotion data sent from the device and stores them in a database.

[0236] Data analysis: The received data is analyzed using AI algorithms to learn the baby's behavioral patterns and the user's emotional state.

[0237] Behavior prediction and response generation: Generate behavior prediction and response methods based on learned behavioral patterns and emotional states. Adjust the content of notifications and response methods based on the user's emotional state.

[0238] Sending notification: The generated behavior prediction and how to deal with it are notified to the user device.

[0239] User device (dedicated app):

[0240] Receiving and displaying notifications: Receive notifications sent from the server and display them in a user-friendly format. The notification content is adjusted based on the user's emotional state.

[0241] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[0242] Specialist consultation function: Provides a function to consult with a specialist online about questions regarding childcare.

[0243] Program processing

[0244] The processing performed by the program of this system will be explained below with specific examples.

[0245] Terminals (baby monitors, sleep monitors, user devices)

[0246] A baby monitor detects a baby's crying and sends the audio data to a server. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[0247] The sleep monitor detects the baby's heart rate and tossing and turning, and sends sleep data to a server, for example, when the baby goes into deep sleep or wakes up.

[0248] The user device collects the user's voice and facial expressions in real time and transmits the emotional data to the server. For example, if the user is feeling stressed, the facial expression and tone of voice are transmitted as data.

[0249] server

[0250] The server receives data sent from the baby monitor, the sleep monitor, and the user device, such as the baby's crying data, heart rate data, and user emotion data.

[0251] The server stores the received data in a database, where each data item is linked to the baby's profile and the user's profile.

[0252] The server analyzes the stored data using AI algorithms to learn the baby's behavioral patterns and the user's emotional state. For example, by analyzing crying data and the user's emotional data over a certain period of time, it can identify the baby's crying times and crying patterns, as well as the user's stress patterns.

[0253] The server predicts the baby's future behavior based on the learned behavioral patterns and emotional state. For example, it predicts the time of day when the baby will cry based on past data and prepares to notify the user of the prediction.

[0254] The server generates and adjusts appropriate responses to predicted behaviors based on the user's emotional state. For example, if a "crying because they're hungry" pattern is detected, the server generates a message saying "Prepare milk" as a response. If the user is feeling stressed, the server may also include more detailed instructions and tips on how to relax.

[0255] The server then notifies the user device (dedicated app) of the generated behavior predictions and countermeasures. The notification content is sent in real time.

[0256] User device (dedicated app)

[0257] The dedicated app receives notifications from the server and displays them on the user interface. For example, a parent can check a notification on their smartphone saying, "The baby has woken up."

[0258] Through the community function, parents can share parenting information with other parents. For example, they can ask questions about parenting problems and receive advice from other parents in the community.

[0259] You can use the specialist consultation function to consult with a specialist online. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[0260] Specific examples

[0261] For example, if a baby starts crying in the middle of the night, the baby monitor collects audio data and sends it to a server. The server analyzes the audio data and determines that the baby is crying because it is hungry. At the same time, it analyzes emotional data sent from the user's device and recognizes that the user is tired. The server generates a notification that includes instructions such as "Prepare milk" and relaxation techniques such as "Take a slow, deep breath, then prepare milk," and sends it to a dedicated app. The parent can check the notification in the dedicated app, follow the instructions to prepare milk and feed the baby, while also reducing their own stress.

[0262] In this way, the system can not only monitor the baby's condition but also provide comprehensive childcare support that takes into account the parents' emotional state.

[0263] The processing flow will be explained below.

[0264] Step 1:

[0265] The device (baby monitor) collects the baby's cries and vocalizations. For example, if the baby cries, the cry is recorded as audio data.

[0266] Step 2:

[0267] The device (sleep monitor) collects data on the baby's movements, heart rate, and breathing patterns. For example, if the baby moves or their heart rate changes, it will detect and record that data.

[0268] Step 3:

[0269] The terminal (user device) collects the user's voice and facial expression data. For example, when a parent talks to their baby in the living room, the voice and facial expression data are recorded.

[0270] Step 4:

[0271] The devices (baby monitors, sleep monitors, user devices) transmit the collected data to the server in real time using wireless communication methods such as Wi-Fi and Bluetooth.

[0272] Step 5:

[0273] The server receives the voice data, sleep data, and emotion data sent from the device. For example, the server receives data on a baby's crying and heart rate, as well as data on the user's tone of voice and facial expression.

[0274] Step 6:

[0275] The server stores the received data in a database, where each data point is linked to the baby and user profile.

[0276] Step 7:

[0277] The server analyzes the stored data using AI algorithms. For example, it analyzes crying patterns to identify behavioral patterns such as "hungry" or "sleepy." At the same time, it analyzes the user's tone of voice and facial expressions to identify the user's emotional state (e.g., stress).

[0278] Step 8:

[0279] The server predicts the baby's future behavior based on the learned behavioral patterns. For example, it predicts the baby's tendency to wake up in the middle of the night based on past data, and prepares the next response based on the prediction results.

[0280] Step 9:

[0281] The server generates an appropriate response to the predicted behavior. For example, if the baby is crying because it is hungry, it generates a message saying, "Prepare milk." At the same time, it adjusts the response based on the user's emotional state. For example, if the user is feeling stressed, it includes detailed instructions such as, "First, take a deep breath. Then, prepare milk."

[0282] Step 10:

[0283] The server then notifies the user device (dedicated app) of the generated behavior prediction and how to respond. For example, it sends a notification saying, "Your baby has woken up and is hungry. First, take a deep breath, then prepare some milk."

[0284] Step 11:

[0285] The device (dedicated app) receives notifications sent from the server. For example, it receives a notification that "your baby has woken up and is hungry."

[0286] Step 12:

[0287] The device (dedicated app) displays the received notification on the user interface, and the parent checks the notification and decides what to do next.

[0288] Step 13:

[0289] The user (parent) can take appropriate action based on the notification. For example, they can follow the notification and take steps such as "First, take a deep breath, then prepare the milk."

[0290] Step 14:

[0291] The device (dedicated app) provides a community function that allows users to share information with other parents, allowing them to exchange parenting experiences and questions within the community.

[0292] Step 15:

[0293] Users (parents) can consult with a specialist through a dedicated app about questions and concerns about childcare, and based on the specialist's answers, they can take more specific measures.

[0294] This enables the system to perform detailed analysis of the baby's behavioral patterns and the user's emotional state, providing parents with appropriate and tailored parenting support in real time.

[0295] Example 2

[0296] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0297] Conventional childcare support systems focused on monitoring the baby's behavior and condition, but did not take into account the parent's emotional state or stress. As a result, there was a problem that the information and coping methods for baby care were not optimized for the parent's condition. This raised concerns that parents might experience excessive stress from childcare, leading to a decline in the quality of childcare.

[0298] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0299] In this invention, the server includes means for analyzing the voice data, sleep data, and user emotional data to learn the baby's behavioral patterns and the user's emotional state, means for generating baby behavior predictions and countermeasures based on the learned behavioral patterns and emotional state and adjusting them according to the user's emotional state, and means for notifying the user terminal of the generated behavior predictions and countermeasures, thereby making it possible not only to monitor the baby's condition but also to provide optimal support based on the parent's emotional state.

[0300] "Audio data" is digitized sound information such as a baby's vocalizations and cries.

[0301] "Sleep data" is sleep-related information such as a baby's rolling over, heart rate, and breathing patterns that is collected and recorded digitally.

[0302] A "server" is a computer system that receives, stores, and analyzes collected data, generates behavior predictions and countermeasures, and notifies the user terminal.

[0303] A "behavioral pattern" is a specific pattern or tendency extracted from a series of data on a baby's behavior or condition.

[0304] "Emotional state" is data that indicates the user's (parent's) psychological state and stress level, and is collected from voice and facial expressions.

[0305] A "user terminal" is a device that receives notifications from the server and displays them on a user interface, and includes smartphones and tablets.

[0306] The "community function" is a function that allows users to share information and communicate with each other.

[0307] "Means for consulting with experts on childcare" is a function that allows users to consult with experts online about questions about childcare through a dedicated app and receive advice.

[0308] "Behavior prediction" means that the server predicts the baby's future behavior and state in advance.

[0309] "Solutions" are specific steps and advice on caring for the baby that are generated by the server based on behavioral predictions.

[0310] A "user interface" refers to a screen and operating means that allows a user to check and operate notifications and data from a server via a terminal.

[0311] Specific embodiments of the present invention will be described below. The present invention is a system that combines a system for managing a baby's behavior in real time with an emotion engine that recognizes the user's emotions and maximizes the effectiveness of childcare support.

[0312] This system consists of the following main components: terminals (baby monitor, sleep monitor, user device), server, and user terminal (dedicated app).

[0313] Terminals (baby monitors, sleep monitors, user devices)

[0314] The devices include baby monitors that collect the baby's cries and vocalizations, sleep monitors that monitor the baby's movements, heart rate, and breathing patterns, and user devices that collect the user's voice and facial expressions.Specific examples of the use of the following devices are given below.

[0315] Baby monitor: General baby monitor device

[0316] Sleep monitor: General sleep monitor device

[0317] User devices: smartphones and tablets

[0318] Examples:

[0319] For example, if a baby starts crying in the middle of the night, a baby monitor will collect the sound data in real time and send it to a server. A sleep monitor will collect data on the baby's heart rate and how often the baby turns over in bed and send it to a server. A user device will collect the user's stress level from their facial expressions and voice and send it to a server as emotional data.

[0320] server

[0321] The server is a central control unit that receives, stores, and analyzes data sent from the terminals. The server has the following functions:

[0322] 1. Data Reception and Storage: The server receives the baby's voice data, sleep data, and user's emotion data and stores them in a database. The collected data is linked to the baby's and user's profiles.

[0323] 2. Data Analysis: The server uses AI algorithms such as GOOGLE TENSOR® Flow to analyze the received data and learn the baby's behavioral patterns and the user's emotional state.

[0324] 3. Behavior prediction and response generation: Based on the learned behavioral patterns and emotional state, the server generates optimal behavior prediction and response methods. It also adjusts the notification content according to the user's emotional state.

[0325] 4. Notification transmission: The generated behavior prediction and countermeasures are notified to the user terminal in real time.

[0326] Examples:

[0327] For example, when a baby starts crying and audio data is sent to the server, the server determines that the baby is crying because it is hungry. At the same time, emotional data indicating that the user is tired is analyzed and a notification is generated instructing the user to prepare milk. The notification also includes relaxation tips, such as "Take a slow, deep breath, and then prepare the milk."

[0328] User device (dedicated app)

[0329] The user terminal is a dedicated application for receiving notifications from the server and displaying them on the user interface. This application includes the following functions:

[0330] 1. Receiving and displaying notifications: Receives predicted behavior and countermeasures sent from the server and displays them in an easy-to-read format for the user.

[0331] 2. Community function: A function for sharing information with other users. Parenting information can be exchanged through message boards and chats.

[0332] 3. Specialist consultation function: Provides a function that allows users to consult with a specialist online about questions regarding childcare.

[0333] Examples:

[0334] For example, the dedicated app will receive a notification that the baby has woken up, and the parent can check the notification on their smartphone. They can also get advice from other parents through the community function or consult with a specialist.

[0335] Example prompts for generative AI models

[0336] "My baby started crying in the middle of the night. Please suggest appropriate ways to handle this situation. Also, include ways to support the parents if they are feeling stressed."

[0337] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0338] System program processing flow

[0339] Terminals (baby monitors, sleep monitors, user devices)

[0340] Step 1: Collect data

[0341] A baby monitor collects the baby's cries and vocalizations. For example, if a baby starts crying in the middle of the night, the built-in microphone captures the sound. The input is the baby's cry, and the output is audio data.

[0342] A sleep monitor monitors a baby's movements, heart rate, and breathing patterns. For example, a heart rate sensor measures the baby's heart rate in real time. The input is the baby's physical activity, and the output is sleep data.

[0343] The user device detects the user's voice and facial expressions. For example, the built-in camera and microphone collect the user's speaking voice and facial expressions. The input is the user's facial expressions and voice, and the output is emotional data.

[0344] Step 2: Sending data

[0345] The baby monitor transmits the collected audio data to a server, for example, using a Wi-Fi connection to transmit the audio data in real time. The collected audio data is the input, and the transmitted audio data is the output.

[0346] The sleep monitor sends the collected heart rate and sleep movement data to a server. For example, it transfers the data using Bluetooth or Wi-Fi. The collected sleep data is the input, and the sent sleep data is the output.

[0347] The user device sends the collected voice and facial expression data to a server, e.g., uploading emotional data in real time using an internet connection. The collected emotional data is the input, and the sent emotional data is the output.

[0348] server

[0349] Step 3: Receiving the data

[0350] The server receives the voice data, sleep data, and emotion data sent from the device. For example, crying data from a baby monitor, heart rate data from a sleep monitor, and emotion data from the user device are simultaneously received. Various data from the device are input, and a list of received data is output.

[0351] Step 4: Save your data

[0352] The server stores the received data in a database, for example linking it to the baby's profile and storing voice data, sleep data, and emotion data. Data received as input, data records stored as output.

[0353] Step 5: Data analysis

[0354] The server analyzes the stored data using an AI algorithm. The input is the stored data, and the output is the baby's behavioral patterns and the user's emotional state. Using Google® TensorFlow, the system learns the baby's behavioral patterns based on crying data and heart rate data. The system also analyzes the user's emotional data to understand the user's emotional state.

[0355] Step 6: Predicting behavior and generating countermeasures

[0356] The server generates behavior predictions and countermeasures based on the learned behavioral patterns and emotional state. The analysis results are input, and the generated behavior predictions and countermeasures are output. For example, it predicts the next time the baby will cry and generates specific countermeasures such as preparing milk. Furthermore, if the user is tired, it generates a notification that includes relaxation techniques.

[0357] Step 7: Sending notifications

[0358] The server notifies the user device of the generated behavior prediction and the countermeasure. The generated behavior prediction and countermeasure are input, and the sent notification is output. For example, a notification that "The baby is crying" and the contents "Please prepare milk" and "Please take slow, deep breaths" are sent to a dedicated app in real time.

[0359] User device (dedicated app)

[0360] Step 8: Receiving and Viewing Notifications

[0361] The user device receives notifications from the server and displays them on the user interface. The notification sent is the input, and the notification content displayed is the output. For example, a parent sees a notification on their smartphone saying "The baby has woken up" and acts accordingly.

[0362] Step 9: Community Features

[0363] The user terminal provides a community function for sharing information with other users. For example, a user can post a question about childcare on a community bulletin board and receive advice from other users. The input is the user's question, and the output is the answer from other users.

[0364] Step 10: Specialist consultation function

[0365] The user device provides a function for online childcare consultation with experts. For example, a user can consult a specialist about a problem such as "my baby is not sleeping well" and receive specific advice. The user's consultation content is input, and the expert's advice is output.

[0366] (Application example 2)

[0367] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0368] For parents who go shopping in physical stores with their babies, it is a significant burden to properly manage their baby's condition while shopping. Furthermore, a lack of support tailored to the baby's condition and the parent's emotional state increases parental stress. Therefore, there is a need for a system that can manage the condition of both the baby and the parent in real time and provide appropriate support.

[0369] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0370] In this invention, the server includes means for collecting baby's voice data, means for collecting baby's sleep data, means for transmitting the collected voice data and sleep data to the server, means for analyzing the voice data and sleep data to learn the baby's behavioral patterns, means for predicting behavior and generating countermeasures based on the learned behavioral patterns, means for collecting parent's voice and facial expressions and transmitting them to the server as emotional data, means for analyzing the emotional data to learn the parent's emotional state, and means for adjusting the behavior predictions and countermeasures based on the learned emotional state. This makes it possible to manage the baby's condition and the parent's emotional state in real time and provide appropriate support and specific countermeasures.

[0371] "Baby voice data" refers to audio information of a baby's cries and vocalizations.

[0372] "Baby sleep data" refers to information about a baby's sleep state, such as the baby's rolling over, heart rate, and breathing patterns.

[0373] "Server" refers to a central processing unit for receiving, storing, and analyzing collected data.

[0374] "Means for learning behavioral patterns" refers to algorithms that analyze collected baby data and identify certain patterns and trends.

[0375] "Means for behavior prediction and countermeasure generation" refers to software or algorithms that predict future behavior and generate appropriate countermeasures based on learned behavioral patterns.

[0376] "User terminal" refers to a device (smartphone, tablet, etc.) used to provide notifications and advice to parents.

[0377] "Means for collecting parent's voice and facial expression" refers to devices or sensors for collecting parent's voice and facial expression data in order to understand the parent's emotional state.

[0378] "Emotional data" refers to information about the parent's emotional state obtained by analyzing their voice and facial expressions.

[0379] "Means for learning emotional states" refers to algorithms that analyze collected emotional data and identify patterns and trends in parents' emotions.

[0380] "Means for adjusting behavioral predictions and coping strategies" refers to software or algorithms that appropriately adjust a baby's behavioral predictions and coping strategies to match the parent's emotional state.

[0381] "Means to link with in-store services" refers to the ability to link with services at physical stores and have employees provide support when parents need it.

[0382] MODE FOR CARRYING OUT THE INVENTION

[0383] The present invention is described in detail below with respect to an embodiment thereof. The present system aims to reduce the burden of child-rearing by managing baby behavior data and parental emotional data in real time and providing appropriate support.

[0384] The system consists of the following main components:

[0385] Terminal

[0386] Baby monitor: A device that collects a baby's cries and vocalizations and sends them to a server as audio data. For example, if a baby starts crying in a store, the device can collect the crying sound.

[0387] Sleep monitor: A device that collects a baby's movements, heart rate, and breathing patterns and sends them to a server as sleep data. For example, it records when a baby moves or wakes up while sleeping.

[0388] User device: A device such as a smartphone or tablet that collects the parent's voice and facial expressions and sends them to a server as emotional data. For example, when a parent is feeling stressed, their facial expression and tone of voice can be recognized.

[0389] server

[0390] The server receives data sent from the baby monitor, sleep monitor, and user device and stores it in a database.

[0391] The received data is analyzed by an AI algorithm to learn the baby's behavioral patterns and the parent's emotional state. Specifically, it uses machine learning models to analyze patterns in the data.

[0392] Based on the stored data, the system predicts the baby's future behavior and generates and adjusts countermeasures based on the parent's emotional state. For example, it suggests appropriate countermeasures based on the "time of day the baby cries" and "pattern of crying."

[0393] The server notifies the user terminal of the generated behavior prediction and the countermeasure, and displays the notified content in an easy-to-read format for the parent.

[0394] User terminal

[0395] Receive notifications sent from the server and display them on the unit or application so that parents can act immediately.

[0396] Through the community function, parents can share parenting information with other parents. They can also exchange opinions using the message board and chat function.

[0397] By using the specialist consultation function, you can consult with a specialist online about your child-rearing questions. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[0398] It also has the ability to link with in-store services, allowing employees to provide support as needed.

[0399] Software and hardware used

[0400] OpenCV: An image processing library for face recognition and expression analysis.

[0401] Keras: A machine learning library used to implement emotion recognition models.

[0402] TensorFlow: A deep learning framework that acts as a backend for Keras.

[0403] Smartphone: A device used for parental emotion recognition and application integration.

[0404] Baby monitor: A device used to collect audio data from a baby.

[0405] Sleep monitor: A device used to collect data such as a baby's heart rate and how often they turn over.

[0406] Specific examples

[0407] For example, imagine a parent shopping in a physical store with their baby and the baby suddenly starts crying. The baby monitor detects the crying and sends the audio data to the server. At the same time, the smartphone camera analyzes the parent's facial expressions and sends the parent's emotional data to the server. The server analyzes this data, determines that the baby is crying because it is hungry, and generates a countermeasure such as "prepare milk." If the parent is feeling tired or stressed, it may also notify them of relaxation techniques such as "take a slow, deep breath, then prepare milk." The notification content is displayed in real time on the smartphone app, allowing the parent to take appropriate action. Furthermore, if support from an employee in the store is needed, they can request help from an employee through the app.

[0408] Prompt Sentence Examples

[0409] Generate specific instructions based on what kind of support the baby needs when crying, taking into account the parents' emotional data.

[0410] This allows the system to simultaneously monitor the baby's condition and the parent's emotions, providing appropriate childcare support even while shopping in a physical store.

[0411] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0412] Step 1:

[0413] Collect the baby's voice data. In this step, the baby monitor detects the baby's cries and vocalizations and collects them as voice data. Specifically, the microphone captures the sound and converts it into a digital format. The input is the baby's voice, and the output is digital voice data.

[0414] Step 2:

[0415] Collecting the baby's sleep data. In this step, the sleep monitor monitors the baby's movements, heart rate, and breathing patterns and collects this data. Specifically, sensors detect the baby's body movements and biological signals and convert them into data. The input is the baby's body movements and biological signals, and the output is digital data.

[0416] Step 3:

[0417] Collect parent's voice and facial expression data. In this step, the user device (smartphone or tablet) detects the parent's voice and facial expression and collects that data. Specifically, the camera captures the parent's face and the microphone records the voice. The input is the parent's voice and facial expression, and the output is digital emotional data.

[0418] Step 4:

[0419] The collected voice data, sleep data, and emotion data are sent to a server. Specifically, the user device uploads these data to the server via the Internet. The input is the collected data, and the output is the data stored on the server.

[0420] Step 5:

[0421] The server stores the received data in a database. Specifically, the database system classifies and stores the data correctly. The input is the transmitted data, and the output is the data stored in the database.

[0422] Step 6:

[0423] The received data is analyzed using an AI algorithm to learn the baby's behavioral patterns and the parent's emotional state. Specifically, a machine learning model (for example, a model using Keras or TensorFlow) analyzes the data. The input is various stored data, and the output is the learned behavioral patterns and emotional state.

[0424] Step 7:

[0425] The server generates behavior predictions and countermeasures based on the learned behavior patterns. Specifically, the predictive model predicts future behavior and generates appropriate countermeasures. The input is the learned behavior pattern, and the output is the predicted behavior and its countermeasures.

[0426] Step 8:

[0427] The server adjusts behavior predictions and response methods based on the learned emotional state. Specifically, it adjusts the content of response methods and notification methods according to the parent's emotional state. The input is the learned emotional state, behavior predictions, and response methods, and the output is the adjusted response methods.

[0428] Step 9:

[0429] The generated behavior prediction and countermeasures are notified to the user device. Specifically, the notification system sends information to the user's device in real time. The input is the adjusted countermeasures, and the output is a notification to the user device.

[0430] Step 10:

[0431] The notification content is displayed on the user's device to help the parent take appropriate action. Specifically, the notification content is displayed on the smartphone screen. The input is the received notification data, and the output is the content displayed to the parent.

[0432] Step 11:

[0433] In-store services are linked and staff members provide support. In this step, a function is implemented that allows parents to request help from store staff if necessary. The input is a request for help, and the output is the provision of support by staff.

[0434] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0435] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0436] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0437] [Second embodiment]

[0438] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0439] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0440] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0441] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0442] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0443] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0444] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0445] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0446] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0447] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0448] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0449] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0450] A specific embodiment of the present invention will be described. The present invention is a system for managing a baby's behavior in real time and notifying parents of appropriate countermeasures. This system collects and analyzes the baby's voice data and sleep data to learn and predict the baby's behavioral patterns and provide countermeasures.

[0451] System configuration

[0452] The system consists of the following main components:

[0453] Devices (baby monitors, sleep monitors):

[0454] The baby monitor collects the baby's cries and vocalizations and sends them to a server as audio data.

[0455] The sleep monitor collects the baby's movements, heart rate, and breathing patterns and sends the data to a server.

[0456] server:

[0457] Data reception and storage: Receives voice and sleep data sent from the device and stores it in a database.

[0458] Data analysis: The received data is analyzed using an AI algorithm to learn the baby's behavioral patterns.

[0459] Behavior prediction and generation of countermeasures: Behavior prediction is made based on learned behavior patterns, and countermeasures are generated based on that prediction.

[0460] Sending notification: The generated behavior prediction and how to deal with it are notified to the user device.

[0461] User device (dedicated app):

[0462] Receive and display notifications: Receive notifications sent from the server and display them in an easy-to-read format for the user.

[0463] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[0464] Specialist consultation function: Provides a function to consult with a specialist online about questions regarding childcare.

[0465] Program processing

[0466] The processing performed by the program of this system will be explained below with specific examples.

[0467] Devices (baby monitors, sleep monitors)

[0468] A baby monitor detects a baby's crying and sends the audio data to a server. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[0469] The sleep monitor detects the baby's heart rate and tossing and turning, and sends sleep data to a server, for example, when the baby goes into deep sleep or wakes up.

[0470] server

[0471] The server receives the crying data sent by the baby monitor and analyzes it, for example to determine if the crying matches a certain pattern (e.g., hungry, sleepy, etc.).

[0472] Analyze sleep data to extract the baby's deep sleep level and sleep duration. For example, analyze whether the baby is sleeping soundly in the middle of the night.

[0473] Based on the analysis results, the system predicts the baby's behavior and generates countermeasures. For example, if the baby is crying because he or she is hungry, the system generates a countermeasure such as "Please prepare milk."

[0474] The generated behavior prediction and how to deal with it are notified to the user device. For example, a notification saying "Your baby has woken up. Please try to put him / her back to sleep" is sent.

[0475] User device (dedicated app)

[0476] The dedicated app receives notifications from the server and displays them on the user interface. For example, a parent can check a notification on their smartphone saying, "The baby has woken up."

[0477] Through the community function, parents can share parenting information with other parents. For example, they can ask questions about parenting problems and receive advice from other parents in the community.

[0478] You can use the specialist consultation function to consult with a specialist online. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[0479] The above is an embodiment of the present invention. This system allows parents to understand their baby's condition in real time and take appropriate measures. As a result, the burden of childcare on parents is reduced, allowing them to focus on raising their child with peace of mind.

[0480] The processing flow will be explained below.

[0481] Step 1:

[0482] The device (baby monitor) collects the baby's cries and vocalizations. For example, if the baby cries, the cry is recognized as audio data and recorded.

[0483] Step 2:

[0484] The device (sleep monitor) collects data on the baby's movements, heart rate, and breathing patterns. For example, if the baby rolls over or their heart rate changes, it records that data.

[0485] Step 3:

[0486] The devices (baby monitors and sleep monitors) transmit collected audio and sleep data to a server in real time, using Wi-Fi, Bluetooth, or other wireless communication methods.

[0487] Step 4:

[0488] The server receives the voice and sleep data sent from the device. For example, data on a baby's crying and heart rate is sent to the server.

[0489] Step 5:

[0490] The server stores the received data in a database, where each piece of data is linked to a baby's profile.

[0491] Step 6:

[0492] The server analyzes the stored data using AI algorithms to learn the baby's behavioral patterns. For example, it analyzes crying data over a certain period of time to identify the time of day and crying patterns of the baby.

[0493] Step 7:

[0494] The server predicts the baby's future behavior based on the learned behavioral patterns. For example, it predicts the time of day when the baby will cry based on past data and prepares to notify the user of the prediction.

[0495] Step 8:

[0496] The server generates an appropriate response to the predicted behavior. For example, if a pattern of "crying because the baby is hungry" is detected, the server generates a message saying "Please prepare milk" as a response.

[0497] Step 9:

[0498] The server then notifies the user device (dedicated app) of the generated behavior predictions and countermeasures. The notification content is sent in real time.

[0499] Step 10:

[0500] The device (dedicated app) receives notifications sent from the server. For example, a notification saying "The baby has woken up" is received.

[0501] Step 11:

[0502] The device (dedicated app) displays the received notification on the user interface, and the parent checks the notification and decides what to do next.

[0503] Step 12:

[0504] The user (parent) can take appropriate action based on the notification. For example, if the notification says "Prepare milk," the parent prepares milk and gives it to the baby.

[0505] Step 13:

[0506] The device (dedicated app) provides a community function that allows users to share information with other parents, and users can exchange their parenting experiences and questions within the community.

[0507] Step 14:

[0508] Users (parents) can consult with a specialist through a dedicated app about questions and concerns about childcare, and based on the specialist's answers, they can take more specific measures.

[0509] This enables the system to analyze a baby's behavioral patterns in detail and provide appropriate childcare support to parents in real time.

[0510] Example 1

[0511] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0512] When raising a baby, parents need to constantly monitor their baby's condition and respond appropriately. However, accurately understanding the reason for a baby's crying or sleep state can be difficult and often places a burden on parents. In addition, there are limited ways to obtain information and advice on childcare from other parents or experts. To solve these problems, a system is needed that monitors a baby's condition in real time and notifies parents of the appropriate course of action.

[0513] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0514] In this invention, the server includes means for collecting baby's voice data, means for collecting baby's physiological data, means for transmitting the collected voice data and physiological data to the server, means for analyzing the voice data and physiological data to learn the baby's behavioral patterns, means for generating a behavior prediction and a countermeasure based on the learned behavioral patterns, means for notifying the generated behavior prediction and countermeasure to a user terminal, means for displaying the notified behavior prediction and countermeasure on a user interface, means for providing a community function for the user to share information with others, and means for consulting with an expert within a dedicated app. This allows parents to monitor their baby's condition in real time and receive specific instructions for appropriate countermeasures. Furthermore, by sharing information with other parents and experts, parents can receive support to alleviate concerns and questions about childcare.

[0515] "Means for collecting baby voice data" refers to a device or function that records a baby's cries and vocalizations in real time and stores and transmits them in digital form.

[0516] "Means for collecting baby physiological data" means a device or function that records physiological information such as a baby's heart rate, breathing patterns, and rolling over in real time, and stores and transmits this information in digital form.

[0517] The "means for transmitting the collected voice data and physiological data to a server" refers to a communication device or function for transmitting the voice and physiological data collected by the terminal to a server via the Internet.

[0518] The "means for analyzing the voice data and physiological data and learning the baby's behavioral patterns" refers to a function that uses an AI algorithm to analyze the data received by the server and predict the baby's behavior and condition.

[0519] The "means for predicting behavior and generating a countermeasure based on the learned behavior pattern" is a function for predicting the baby's future behavior based on the analysis results and automatically generating an appropriate countermeasure.

[0520] The "means for notifying the user terminal of the generated behavior prediction and countermeasure method" is a communication device or function for transmitting the behavior prediction and countermeasure method generated by the server to the user terminal in real time.

[0521] The "means for displaying the notified predicted behavior and countermeasure method on a user interface" is an interface function for displaying the notification received by the user terminal in a form that is easy for the user to see.

[0522] "Means for providing a community function that enables the user to share information with others" refers to a bulletin board or chat function within the dedicated app that enables users to share information and concerns about child-rearing with other parents and exchange opinions.

[0523] "Means to consult with experts within the dedicated app" is a function that allows users to send questions about childcare to experts through the dedicated app and receive advice online.

[0524] This invention is a system that manages a baby's behavior in real time and notifies parents of appropriate measures. This system learns and predicts the baby's behavioral patterns by collecting and analyzing the baby's voice data and physiological data, and provides appropriate measures.

[0525] System configuration

[0526] The present invention consists of the following major components:

[0527] Devices (baby monitors, sleep monitors):

[0528] Baby monitor: Collects a baby's cries and vocalizations and sends them to a server as audio data. For example, if a baby cries in the middle of the night, the audio data is sent to the server in real time.

[0529] Sleep monitor: Collects physiological data such as the baby's movements, heart rate, breathing patterns, etc., and sends it to a server. For example, when the baby enters a deep sleep or wakes up, the data is collected and sent to the server.

[0530] server:

[0531] Data reception and storage: Receives the voice data and physiological data sent from the device and stores them in a database. For example, the server receives the baby's crying data and heart rate data and stores them in the appropriate format.

[0532] Data analysis: The received audio and physiological data is analyzed using AI algorithms. For example, the server analyzes the audio data to determine if the baby's crying matches a specific pattern (hungry, sleepy, etc.). The server also analyzes physiological data to assess the baby's sleep cycle and how soundly they are sleeping.

[0533] Behavior prediction and generation of countermeasures: Based on the analysis results, the system predicts the baby's behavior and generates appropriate countermeasures. For example, if the baby's crying is analyzed as "hungry," the system generates the countermeasure "Please prepare milk." Also, if the sleep data indicates that the baby is in a deep sleep, the system generates a notification saying "The baby is sleeping deeply. Please be quiet."

[0534] Notification sending: The server notifies the user device of the generated behavior prediction and how to deal with it. For example, it sends a notification saying, "The baby is crying. He may be hungry. Please prepare some milk."

[0535] User device (dedicated app):

[0536] Receiving and displaying notifications: The user device receives notifications from the server and displays them on the user interface. The user can check the notifications and take appropriate action. For example, the user can see a message on the smartphone app saying, "Your baby has woken up. Please try to put him / her back to sleep."

[0537] Community function: Users can use the community function to share parenting information with other parents. For example, they can ask questions about how to deal with nighttime crying and receive advice from other parents.

[0538] Specialist consultation function: Users can use the specialist consultation function of the dedicated app to consult with experts online about childcare-related questions. For example, they can send a message to a specialist saying, "My baby wakes up frequently during the night," and receive specific advice.

[0539] Specific examples

[0540] For example, if a baby starts crying in the middle of the night, the system will operate as follows:

[0541] 1. The baby monitor detects the baby's crying and sends the audio data to the server in real time.

[0542] 2. The server receives the crying data and stores it in a database. At the same time, it uses an AI algorithm to analyze the crying data and identify behavioral patterns that indicate "hungry."

[0543] 3. If the server determines that the user is hungry, it generates a solution such as "Please prepare milk" and notifies this information to the user's terminal.

[0544] 4. The user device (dedicated app) receives a notification from the server and displays a message to the parent saying, "The baby is crying. He may be hungry. Please prepare some milk."

[0545] Example input to a generative AI model

[0546] Below are some example prompts to input to the generative AI model:

[0547] User: "My baby started crying in the middle of the night. What should I do?"

[0548] AI: "Let's find out why your baby is crying. First, he may need milk, so get some ready. If he still doesn't stop crying, check to see if he has a wet diaper or is uncomfortable."

[0549] This invention allows parents to monitor their baby's condition in real time and take appropriate measures promptly. In addition, by sharing information with other parents and experts, parents can alleviate concerns and questions about childcare, allowing them to approach childcare with peace of mind.

[0550] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0551] Step 1:

[0552] Data collection from devices (baby monitors, sleep monitors)

[0553] How a baby monitor works: A baby monitor collects a baby's crying sounds in real time. Specifically, the baby monitor's microphone detects the crying sounds and converts the audio data into a digital format. The input is the baby's crying sounds, and the output is digital audio data.

[0554] How a sleep monitor works: A sleep monitor monitors a baby's heart rate, breathing patterns, and rolling over. This data is collected using digital sensors. The input is the baby's physiological movements, and the output is digital physiological data.

[0555] Step 2:

[0556] Sending data from the device to the server

[0557] How the baby monitor works: The baby monitor sends the collected audio data to a server via the Internet. Specifically, the communication module in the device sends the data. The input is digital audio data, and the output is the audio data sent to the server.

[0558] Operation of the sleep monitor: The sleep monitor also sends the collected physiological data to a server. Specifically, a communication module sends the data via the Internet. The input is digital physiological data, and the output is the physiological data sent to the server.

[0559] Step 3:

[0560] Receiving and storing data by the server

[0561] Data reception: The server receives data sent from the baby monitor or sleep monitor. Specifically, the server's communication module receives the data. The input is audio data and physiological data sent via the Internet, and the output is data stored in the server's memory.

[0562] Data Storage Operation: The received audio and physiological data is stored in a database. Specifically, the database management system stores the data in an appropriate format. The input is the received data and the output is the stored database entry.

[0563] Step 4:

[0564] Data analysis by server

[0565] Analysis of audio data: The server analyzes the audio data using an AI algorithm. Specifically, it compares the crying data with specific patterns (hungry, sleepy, etc.). The input is the stored audio data, and the output is the analysis result indicating the cause of the crying.

[0566] Physiological data analysis: The server analyzes the physiological data to assess the baby's sleep cycle and deepness. For example, it analyzes heart rate and breathing patterns to determine whether the baby is sleeping deeply. The input is the stored physiological data, and the output is the analysis result indicating the state of sleep.

[0567] Step 5:

[0568] Server predicts behavior and generates countermeasures

[0569] Generation of behavior prediction: The server predicts the baby's behavior based on the analysis results. For example, if the analysis of a baby's cry indicates that the baby is hungry, a behavior prediction is made based on that information. The input is the analysis results of the voice data and physiological data, and the output is a specific behavior prediction.

[0570] Generation of action plan: The server generates an appropriate action plan based on the behavior prediction. For example, if the predicted result is "I'm hungry," the server generates the action plan "Please prepare some milk." The input is the behavior prediction result, and the output is a specific action plan.

[0571] Step 6:

[0572] Notification from the server to the user device

[0573] Sending notification: The server notifies the user device of the generated behavior prediction and countermeasures. Specifically, the communication module sends the notification data. The input is the behavior prediction and countermeasure data, and the output is a notification to the user device.

[0574] Step 7:

[0575] Receiving and displaying notifications on user devices

[0576] Receiving notifications: The user device receives notifications from the server. The communication module receives the notification data, and the dedicated application uses that data. The input is the notification data sent from the server, and the output is the display on the user interface.

[0577] Displaying a notification: The dedicated app displays a notification on the user interface. Specifically, the app displays a message on the screen saying, "Your baby is crying. He might be hungry. Please prepare some milk." The input is the received notification data, and the output is the notification displayed to the user.

[0578] Step 8:

[0579] Additional functions for user devices (community function, specialist consultation function)

[0580] Community function: Users can share parenting information with other parents through the community function. For example, they can ask questions about specific parenting concerns and receive advice from other parents. The input volume is the questions and information entered by the user, and the output volume is the answers and shared information from other users.

[0581] Specialist consultation function: Users can use the specialist consultation function of the dedicated app to consult with an expert online about questions about childcare. For example, they can send a question such as "My baby wakes up frequently during the night" to a specialist and receive specific advice. The input is the question entered by the user, and the output is advice from the specialist.

[0582] (Application example 1)

[0583] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0584] Modern parents, especially those raising children for the first time, face a great deal of anxiety and stress. While they need to accurately understand their baby's condition and take appropriate measures, choosing the right childcare products for their baby while shopping in a physical store can be a difficult task. Furthermore, when expert advice is needed, there is a lack of means to respond quickly. There is a need to address these issues.

[0585] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0586] In this invention, the server includes means for analyzing the baby's voice data and sleep data to learn the baby's behavioral patterns, means for predicting behavior and generating countermeasures based on the learned behavioral patterns, and means for suggesting childcare products suitable for the baby's condition in a physical store. This allows parents to understand the baby's condition in real time and take appropriate measures, and also makes it possible to suggest childcare products that are optimal for the baby while shopping in the physical store.

[0587] "Baby voice data" is a digital recording of sounds made by a baby, such as crying or vocalizations.

[0588] "Baby sleep data" is a digital record of physiological information related to a baby's sleep, such as their heart rate, breathing patterns, and rolling over.

[0589] "Server" means the central system that receives, analyzes, stores, and sends notifications to user devices.

[0590] A "means for learning behavioral patterns" is an algorithm or method for analyzing collected data, identifying consistencies and changes in a baby's behavior, and learning those patterns.

[0591] The "means for predicting behavior and generating countermeasures" refers to an algorithm and method for predicting future behavior based on learned behavioral patterns and generating appropriate countermeasures for the behavior.

[0592] A "user terminal" is a mobile device such as a smartphone or tablet used by a parent or caregiver, which receives and displays notifications, provides community functions, and provides expert consultation functions.

[0593] "Means for providing community functions" are online bulletin boards and chat functions that allow parenters to share information and receive advice from each other.

[0594] "A way to consult with an expert for free" is a way to ask questions or get advice about child-rearing from an expert online for free.

[0595] The "means for suggesting childcare products in a physical store" refers to a system and method for suggesting appropriate childcare products to parents who are shopping in a store based on the real-time status of their baby.

[0596] A "generative AI model" is a supervised or unsupervised artificial intelligence model used for data analysis and behavior prediction.

[0597] A "prompt" is a textual instruction used as input to an AI model.

[0598] The following describes in detail the embodiments of the present invention. The present invention is a system that manages a baby's behavior in real time and suggests appropriate ways to deal with the baby and childcare products to parents. This system learns and predicts the baby's behavioral patterns by collecting and analyzing the baby's voice data and sleep data, and then suggests ways to deal with the baby and childcare products.

[0599] System configuration

[0600] The system consists of the following main components:

[0601] Devices (baby monitors, sleep monitors):

[0602] Baby monitors collect the baby's cries and vocalizations and send them to a server as audio data. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[0603] The sleep monitor collects the baby's heart rate and movements, and sends the data to a server as sleep data, such as when the baby falls into a deep sleep or wakes up.

[0604] server:

[0605] Data reception and storage: The voice and sleep data sent from the device is received and stored in a database using software such as a common database management system (DBMS) or cloud service.

[0606] Data analysis: The received data is analyzed using a generative AI model to learn the baby's behavioral patterns. The software used is a machine learning library such as TensorFlow or PyTorch.

[0607] Behavior prediction and generation of coping methods: Behavior prediction is made based on learned behavior patterns, and coping methods and childcare product suggestions are generated based on the prediction.

[0608] Sending notification: The generated behavior prediction and how to deal with it are notified to the user terminal.

[0609] User device (dedicated app):

[0610] Receiving and displaying notifications: Receives notifications sent from the server and displays them in the user interface.

[0611] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[0612] Expert consultation function: Provides a function to consult with experts online about questions regarding childcare.

[0613] A function to suggest childcare products in physical stores: Recommends the most suitable childcare products based on the baby's condition in real time.

[0614] Program processing

[0615] The server receives and analyzes the baby's voice and sleep data. For example, if the baby monitor detects the baby crying, the data is sent to the server. The server analyzes the data and determines whether the crying matches a certain pattern (e.g., hungry, sleepy, etc.).

[0616] The server then predicts behavior based on the behavioral patterns and generates suggested solutions and childcare items. This is done using a generative AI model. For example, if the baby is crying because it is hungry, the server predicts the solution, suggesting "prepare milk" and "a baby bottle."

[0617] The generated solutions and suggestions are sent to the user's device, which can be a smartphone, tablet, or other device, and displayed in a dedicated app. Additionally, if the childcare product suggestion function is enabled in the physical store, shelves with suitable childcare products will be displayed in the app, facilitating smooth purchasing for parents.

[0618] Specific examples

[0619] For example, when a baby starts crying in the morning, a baby monitor captures the sound. This data is sent to a server for analysis. If the cry is identified as a sign of hunger, the server sends a notification to the user's device suggesting a baby bottle as an appropriate childcare item, along with a message saying, "Prepare some milk."

[0620] An example of a prompt is, "Generate a notification message for a parent based on baby's crying sound, which indicates possible hunger." This allows users to quickly respond to their baby's needs and easily find and purchase the necessary childcare products.

[0621] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0622] Step 1:

[0623] Collects baby's audio and sleep data. The baby monitor captures the baby's crying in real time, and the sleep monitor collects sleep data such as the baby's heart rate, rolling over, and breathing patterns. The input is the crying audio data and heart rate and movement data, and the output is a digital file containing these data.

[0624] Step 2:

[0625] The device sends collected audio and sleep data to the server. The audio and sleep data sent from the baby monitor and sleep monitor are sent to the server. The input is the collected digital file, and the output is the data stored on the server.

[0626] Step 3:

[0627] The server analyzes the received audio and sleep data and learns the baby's behavioral patterns. The software used is a machine learning library such as TensorFlow or PyTorch. The input is the audio and sleep data stored on the server, and the output is a model that shows the baby's behavioral patterns.

[0628] Step 4:

[0629] The server generates behavior predictions and countermeasures based on the learned behavior patterns. A generative AI model is used for this. The input is a model showing the behavior patterns, and the output is text data containing the behavior predictions and countermeasures. Specific childcare products are also suggested.

[0630] Step 5:

[0631] The server notifies the user device of the generated behavior prediction and countermeasures using a dedicated notification API. The input is text data containing the behavior prediction and countermeasures, and the output is a push notification to the user device.

[0632] Step 6:

[0633] The user terminal displays the notified behavior prediction and countermeasures on the user interface. The input is the notification data sent from the server, and the output is a notification message displayed on the user interface.

[0634] Step 7:

[0635] Through a dedicated app, users are recommended childcare products suited to their baby's condition in a physical store. The input is the store's situation and real-time baby data, and the output is a list of recommended childcare products.

[0636] Step 8:

[0637] The system provides a community function that allows users to share information with others and a function that allows users to consult with experts online for free about child-rearing questions. The input is the user's questions and child-rearing information, and the output is answers and advice from other users and experts.

[0638] This trend will enable parents to understand their baby's condition in real time, take appropriate measures, and quickly select the necessary childcare products in physical stores.

[0639] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0640] A specific embodiment for carrying out the present invention will now be described. The present invention is a system that combines a basic system that manages a baby's behavior in real time with an emotion engine that recognizes the user's emotions and maximizes the effectiveness of childcare support.

[0641] System configuration

[0642] The system consists of the following main components:

[0643] Devices (baby monitors, sleep monitors, user devices):

[0644] The baby monitor collects the baby's cries and vocalizations and sends them to a server as audio data.

[0645] The sleep monitor collects the baby's movements, heart rate, and breathing patterns and sends the data to a server.

[0646] The user device collects the user's voice and facial expressions and sends them to the server as emotional data.

[0647] server:

[0648] Data reception and storage: Receives voice data, sleep data, and emotion data sent from the device and stores them in a database.

[0649] Data analysis: The received data is analyzed using AI algorithms to learn the baby's behavioral patterns and the user's emotional state.

[0650] Behavior prediction and response generation: Generate behavior prediction and response methods based on learned behavioral patterns and emotional states. Adjust the content of notifications and response methods based on the user's emotional state.

[0651] Sending notification: The generated behavior prediction and how to deal with it are notified to the user device.

[0652] User device (dedicated app):

[0653] Receiving and displaying notifications: Receive notifications sent from the server and display them in a user-friendly format. The notification content is adjusted based on the user's emotional state.

[0654] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[0655] Specialist consultation function: Provides a function to consult with a specialist online about questions regarding childcare.

[0656] Program processing

[0657] The processing performed by the program of this system will be explained below with specific examples.

[0658] Terminals (baby monitors, sleep monitors, user devices)

[0659] A baby monitor detects a baby's crying and sends the audio data to a server. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[0660] The sleep monitor detects the baby's heart rate and tossing and turning, and sends sleep data to a server, for example, when the baby goes into deep sleep or wakes up.

[0661] The user device collects the user's voice and facial expressions in real time and transmits the emotional data to the server. For example, if the user is feeling stressed, the facial expression and tone of voice are transmitted as data.

[0662] server

[0663] The server receives data sent from the baby monitor, the sleep monitor, and the user device, such as the baby's crying data, heart rate data, and user emotion data.

[0664] The server stores the received data in a database, where each data item is linked to the baby's profile and the user's profile.

[0665] The server analyzes the stored data using AI algorithms to learn the baby's behavioral patterns and the user's emotional state. For example, by analyzing crying data and the user's emotional data over a certain period of time, it can identify the baby's crying times and crying patterns, as well as the user's stress patterns.

[0666] The server predicts the baby's future behavior based on the learned behavioral patterns and emotional state. For example, it predicts the time of day when the baby will cry based on past data and prepares to notify the user of the prediction.

[0667] The server generates and adjusts appropriate responses to predicted behaviors based on the user's emotional state. For example, if a "crying because they're hungry" pattern is detected, the server generates a message saying "Prepare milk" as a response. If the user is feeling stressed, the server may also include more detailed instructions and tips on how to relax.

[0668] The server then notifies the user device (dedicated app) of the generated behavior predictions and countermeasures. The notification content is sent in real time.

[0669] User device (dedicated app)

[0670] The dedicated app receives notifications from the server and displays them on the user interface. For example, a parent can check a notification on their smartphone saying, "The baby has woken up."

[0671] Through the community function, parents can share parenting information with other parents. For example, they can ask questions about parenting problems and receive advice from other parents in the community.

[0672] You can use the specialist consultation function to consult with a specialist online. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[0673] Specific examples

[0674] For example, if a baby starts crying in the middle of the night, the baby monitor collects audio data and sends it to a server. The server analyzes the audio data and determines that the baby is crying because it is hungry. At the same time, it analyzes emotional data sent from the user's device and recognizes that the user is tired. The server generates a notification that includes instructions such as "Prepare milk" and relaxation techniques such as "Take a slow, deep breath, then prepare milk," and sends it to a dedicated app. The parent can check the notification in the dedicated app, follow the instructions to prepare milk and feed the baby, while also reducing their own stress.

[0675] In this way, the system can not only monitor the baby's condition but also provide comprehensive childcare support that takes into account the parents' emotional state.

[0676] The processing flow will be explained below.

[0677] Step 1:

[0678] The device (baby monitor) collects the baby's cries and vocalizations. For example, if the baby cries, the cry is recorded as audio data.

[0679] Step 2:

[0680] The device (sleep monitor) collects data on the baby's movements, heart rate, and breathing patterns. For example, if the baby moves or their heart rate changes, it will detect and record that data.

[0681] Step 3:

[0682] The terminal (user device) collects the user's voice and facial expression data. For example, when a parent talks to their baby in the living room, the voice and facial expression data are recorded.

[0683] Step 4:

[0684] The devices (baby monitors, sleep monitors, user devices) transmit the collected data to the server in real time using wireless communication methods such as Wi-Fi and Bluetooth.

[0685] Step 5:

[0686] The server receives the voice data, sleep data, and emotion data sent from the device. For example, the server receives data on a baby's crying and heart rate, as well as data on the user's tone of voice and facial expression.

[0687] Step 6:

[0688] The server stores the received data in a database, where each data point is linked to the baby and user profile.

[0689] Step 7:

[0690] The server analyzes the stored data using AI algorithms. For example, it analyzes crying patterns to identify behavioral patterns such as "hungry" or "sleepy." At the same time, it analyzes the user's tone of voice and facial expressions to identify the user's emotional state (e.g., stress).

[0691] Step 8:

[0692] The server predicts the baby's future behavior based on the learned behavioral patterns. For example, it predicts the baby's tendency to wake up in the middle of the night based on past data, and prepares the next response based on the prediction results.

[0693] Step 9:

[0694] The server generates an appropriate response to the predicted behavior. For example, if the baby is crying because it is hungry, it generates a message saying, "Prepare milk." At the same time, it adjusts the response based on the user's emotional state. For example, if the user is feeling stressed, it includes detailed instructions such as, "First, take a deep breath. Then, prepare milk."

[0695] Step 10:

[0696] The server then notifies the user device (dedicated app) of the generated behavior prediction and how to respond. For example, it sends a notification saying, "Your baby has woken up and is hungry. First, take a deep breath, then prepare some milk."

[0697] Step 11:

[0698] The device (dedicated app) receives notifications sent from the server. For example, it receives a notification that "your baby has woken up and is hungry."

[0699] Step 12:

[0700] The device (dedicated app) displays the received notification on the user interface, and the parent checks the notification and decides what to do next.

[0701] Step 13:

[0702] The user (parent) can take appropriate action based on the notification. For example, they can follow the notification and take steps such as "First, take a deep breath, then prepare the milk."

[0703] Step 14:

[0704] The device (dedicated app) provides a community function that allows users to share information with other parents, allowing them to exchange parenting experiences and questions within the community.

[0705] Step 15:

[0706] Users (parents) can consult with a specialist through a dedicated app about questions and concerns about childcare, and based on the specialist's answers, they can take more specific measures.

[0707] This enables the system to perform detailed analysis of the baby's behavioral patterns and the user's emotional state, providing parents with appropriate and tailored parenting support in real time.

[0708] Example 2

[0709] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0710] Conventional childcare support systems focused on monitoring the baby's behavior and condition, but did not take into account the parent's emotional state or stress. As a result, there was a problem that the information and coping methods for baby care were not optimized for the parent's condition. This raised concerns that parents might experience excessive stress from childcare, leading to a decline in the quality of childcare.

[0711] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0712] In this invention, the server includes means for analyzing the voice data, sleep data, and user emotional data to learn the baby's behavioral patterns and the user's emotional state, means for generating baby behavior predictions and countermeasures based on the learned behavioral patterns and emotional state and adjusting them according to the user's emotional state, and means for notifying the user terminal of the generated behavior predictions and countermeasures, thereby making it possible not only to monitor the baby's condition but also to provide optimal support based on the parent's emotional state.

[0713] "Audio data" is digitized sound information such as a baby's vocalizations and cries.

[0714] "Sleep data" is sleep-related information such as a baby's rolling over, heart rate, and breathing patterns that is collected and recorded digitally.

[0715] A "server" is a computer system that receives, stores, and analyzes collected data, generates behavior predictions and countermeasures, and notifies the user terminal.

[0716] A "behavioral pattern" is a specific pattern or tendency extracted from a series of data on a baby's behavior or condition.

[0717] "Emotional state" is data that indicates the user's (parent's) psychological state and stress level, and is collected from voice and facial expressions.

[0718] A "user terminal" is a device that receives notifications from the server and displays them on a user interface, and includes smartphones and tablets.

[0719] The "community function" is a function that allows users to share information and communicate with each other.

[0720] "Means for consulting with experts on childcare" is a function that allows users to consult with experts online about questions about childcare through a dedicated app and receive advice.

[0721] "Behavior prediction" means that the server predicts the baby's future behavior and state in advance.

[0722] "Solutions" are specific steps and advice on caring for the baby that are generated by the server based on behavioral predictions.

[0723] A "user interface" refers to a screen and operating means that allows a user to check and operate notifications and data from a server via a terminal.

[0724] Specific embodiments of the present invention will be described below. The present invention is a system that combines a system for managing a baby's behavior in real time with an emotion engine that recognizes the user's emotions and maximizes the effectiveness of childcare support.

[0725] This system consists of the following main components: terminals (baby monitor, sleep monitor, user device), server, and user terminal (dedicated app).

[0726] Terminals (baby monitors, sleep monitors, user devices)

[0727] The devices include baby monitors that collect the baby's cries and vocalizations, sleep monitors that monitor the baby's movements, heart rate, and breathing patterns, and user devices that collect the user's voice and facial expressions.Specific examples of the use of the following devices are given below.

[0728] Baby monitor: General baby monitor device

[0729] Sleep monitor: General sleep monitor device

[0730] User devices: smartphones and tablets

[0731] Examples:

[0732] For example, if a baby starts crying in the middle of the night, a baby monitor will collect the sound data in real time and send it to a server. A sleep monitor will collect data on the baby's heart rate and how often the baby turns over in bed and send it to a server. A user device will collect the user's stress level from their facial expressions and voice and send it to a server as emotional data.

[0733] server

[0734] The server is a central control unit that receives, stores, and analyzes data sent from the terminals. The server has the following functions:

[0735] 1. Data Reception and Storage: The server receives the baby's voice data, sleep data, and user's emotion data and stores them in a database. The collected data is linked to the baby's and user's profiles.

[0736] 2. Data analysis: The server uses AI algorithms such as Google TensorFlow to analyze the incoming data and learn the baby's behavioral patterns and the user's emotional state.

[0737] 3. Behavior prediction and response generation: Based on the learned behavioral patterns and emotional state, the server generates optimal behavior prediction and response methods. It also adjusts the notification content according to the user's emotional state.

[0738] 4. Notification transmission: The generated behavior prediction and countermeasures are notified to the user terminal in real time.

[0739] Examples:

[0740] For example, when a baby starts crying and audio data is sent to the server, the server determines that the baby is crying because it is hungry. At the same time, emotional data indicating that the user is tired is analyzed and a notification is generated instructing the user to prepare milk. The notification also includes relaxation tips, such as "Take a slow, deep breath, and then prepare the milk."

[0741] User device (dedicated app)

[0742] The user terminal is a dedicated application for receiving notifications from the server and displaying them on the user interface. This application includes the following functions:

[0743] 1. Receiving and displaying notifications: Receives predicted behavior and countermeasures sent from the server and displays them in an easy-to-read format for the user.

[0744] 2. Community function: A function for sharing information with other users. Parenting information can be exchanged through message boards and chats.

[0745] 3. Specialist consultation function: Provides a function that allows users to consult with a specialist online about questions regarding childcare.

[0746] Examples:

[0747] For example, the dedicated app will receive a notification that the baby has woken up, and the parent can check the notification on their smartphone. They can also get advice from other parents through the community function or consult with a specialist.

[0748] Example prompts for generative AI models

[0749] "My baby started crying in the middle of the night. Please suggest appropriate ways to handle this situation. Also, include ways to support the parents if they are feeling stressed."

[0750] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0751] System program processing flow

[0752] Terminals (baby monitors, sleep monitors, user devices)

[0753] Step 1: Collect data

[0754] A baby monitor collects the baby's cries and vocalizations. For example, if a baby starts crying in the middle of the night, the built-in microphone captures the sound. The input is the baby's cry, and the output is audio data.

[0755] A sleep monitor monitors a baby's movements, heart rate, and breathing patterns. For example, a heart rate sensor measures the baby's heart rate in real time. The input is the baby's physical activity, and the output is sleep data.

[0756] The user device detects the user's voice and facial expressions. For example, the built-in camera and microphone collect the user's speaking voice and facial expressions. The input is the user's facial expressions and voice, and the output is emotional data.

[0757] Step 2: Sending data

[0758] The baby monitor transmits the collected audio data to a server, for example, using a Wi-Fi connection to transmit the audio data in real time. The collected audio data is the input, and the transmitted audio data is the output.

[0759] The sleep monitor sends the collected heart rate and sleep movement data to a server. For example, it transfers the data using Bluetooth or Wi-Fi. The collected sleep data is the input, and the sent sleep data is the output.

[0760] The user device sends the collected voice and facial expression data to a server, e.g., uploading emotional data in real time using an internet connection. The collected emotional data is the input, and the sent emotional data is the output.

[0761] server

[0762] Step 3: Receiving the data

[0763] The server receives the voice data, sleep data, and emotion data sent from the device. For example, crying data from a baby monitor, heart rate data from a sleep monitor, and emotion data from the user device are simultaneously received. Various data from the device are input, and a list of received data is output.

[0764] Step 4: Save your data

[0765] The server stores the received data in a database, for example linking it to the baby's profile and storing voice data, sleep data, and emotion data. Data received as input, data records stored as output.

[0766] Step 5: Data analysis

[0767] The server analyzes the stored data using an AI algorithm. The input is the stored data, and the output is the baby's behavioral patterns and the user's emotional state. Using Google TensorFlow, the server learns the baby's behavioral patterns based on crying data and heart rate data. The server also analyzes the user's emotional data to understand the user's emotional state.

[0768] Step 6: Predicting behavior and generating countermeasures

[0769] The server generates behavior predictions and countermeasures based on the learned behavioral patterns and emotional state. The analysis results are input, and the generated behavior predictions and countermeasures are output. For example, it predicts the next time the baby will cry and generates specific countermeasures such as preparing milk. Furthermore, if the user is tired, it generates a notification that includes relaxation techniques.

[0770] Step 7: Sending notifications

[0771] The server notifies the user device of the generated behavior prediction and the countermeasure. The generated behavior prediction and countermeasure are input, and the sent notification is output. For example, a notification that "The baby is crying" and the contents "Please prepare milk" and "Please take slow, deep breaths" are sent to a dedicated app in real time.

[0772] User device (dedicated app)

[0773] Step 8: Receiving and Viewing Notifications

[0774] The user device receives notifications from the server and displays them on the user interface. The notification sent is the input, and the notification content displayed is the output. For example, a parent sees a notification on their smartphone saying "The baby has woken up" and acts accordingly.

[0775] Step 9: Community Features

[0776] The user terminal provides a community function for sharing information with other users. For example, a user can post a question about childcare on a community bulletin board and receive advice from other users. The input is the user's question, and the output is the answer from other users.

[0777] Step 10: Specialist consultation function

[0778] The user device provides a function for online childcare consultation with experts. For example, a user can consult a specialist about a problem such as "my baby is not sleeping well" and receive specific advice. The user's consultation content is input, and the expert's advice is output.

[0779] (Application example 2)

[0780] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0781] For parents who go shopping in physical stores with their babies, it is a significant burden to properly manage their baby's condition while shopping. Furthermore, a lack of support tailored to the baby's condition and the parent's emotional state increases parental stress. Therefore, there is a need for a system that can manage the condition of both the baby and the parent in real time and provide appropriate support.

[0782] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0783] In this invention, the server includes means for collecting baby's voice data, means for collecting baby's sleep data, means for transmitting the collected voice data and sleep data to the server, means for analyzing the voice data and sleep data to learn the baby's behavioral patterns, means for predicting behavior and generating countermeasures based on the learned behavioral patterns, means for collecting parent's voice and facial expressions and transmitting them to the server as emotional data, means for analyzing the emotional data to learn the parent's emotional state, and means for adjusting the behavior predictions and countermeasures based on the learned emotional state. This makes it possible to manage the baby's condition and the parent's emotional state in real time and provide appropriate support and specific countermeasures.

[0784] "Baby voice data" refers to audio information of a baby's cries and vocalizations.

[0785] "Baby sleep data" refers to information about a baby's sleep state, such as the baby's rolling over, heart rate, and breathing patterns.

[0786] "Server" refers to a central processing unit for receiving, storing, and analyzing collected data.

[0787] "Means for learning behavioral patterns" refers to algorithms that analyze collected baby data and identify certain patterns and trends.

[0788] "Means for behavior prediction and countermeasure generation" refers to software or algorithms that predict future behavior and generate appropriate countermeasures based on learned behavioral patterns.

[0789] "User terminal" refers to a device (smartphone, tablet, etc.) used to provide notifications and advice to parents.

[0790] "Means for collecting parent's voice and facial expression" refers to devices or sensors for collecting parent's voice and facial expression data in order to understand the parent's emotional state.

[0791] "Emotional data" refers to information about the parent's emotional state obtained by analyzing their voice and facial expressions.

[0792] "Means for learning emotional states" refers to algorithms that analyze collected emotional data and identify patterns and trends in parents' emotions.

[0793] "Means for adjusting behavioral predictions and coping strategies" refers to software or algorithms that appropriately adjust a baby's behavioral predictions and coping strategies to match the parent's emotional state.

[0794] "Means to link with in-store services" refers to the ability to link with services at physical stores and have employees provide support when parents need it.

[0795] MODE FOR CARRYING OUT THE INVENTION

[0796] The present invention is described in detail below with respect to an embodiment thereof. The present system aims to reduce the burden of child-rearing by managing baby behavior data and parental emotional data in real time and providing appropriate support.

[0797] The system consists of the following main components:

[0798] Terminal

[0799] Baby monitor: A device that collects a baby's cries and vocalizations and sends them to a server as audio data. For example, if a baby starts crying in a store, the device can collect the crying sound.

[0800] Sleep monitor: A device that collects a baby's movements, heart rate, and breathing patterns and sends them to a server as sleep data. For example, it records when a baby moves or wakes up while sleeping.

[0801] User device: A device such as a smartphone or tablet that collects the parent's voice and facial expressions and sends them to a server as emotional data. For example, when a parent is feeling stressed, their facial expression and tone of voice can be recognized.

[0802] server

[0803] The server receives data sent from the baby monitor, sleep monitor, and user device and stores it in a database.

[0804] The received data is analyzed by an AI algorithm to learn the baby's behavioral patterns and the parent's emotional state. Specifically, it uses machine learning models to analyze patterns in the data.

[0805] Based on the stored data, the system predicts the baby's future behavior and generates and adjusts countermeasures based on the parent's emotional state. For example, it suggests appropriate countermeasures based on the "time of day the baby cries" and "pattern of crying."

[0806] The server notifies the user terminal of the generated behavior prediction and the countermeasure, and displays the notified content in an easy-to-read format for the parent.

[0807] User terminal

[0808] Receive notifications sent from the server and display them on the unit or application so that parents can act immediately.

[0809] Through the community function, parents can share parenting information with other parents. They can also exchange opinions using the message board and chat function.

[0810] By using the specialist consultation function, you can consult with a specialist online about your child-rearing questions. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[0811] It also has the ability to link with in-store services, allowing employees to provide support as needed.

[0812] Software and hardware used

[0813] OpenCV: An image processing library for face recognition and expression analysis.

[0814] Keras: A machine learning library used to implement emotion recognition models.

[0815] TensorFlow: A deep learning framework that acts as a backend for Keras.

[0816] Smartphone: A device used for parental emotion recognition and application integration.

[0817] Baby monitor: A device used to collect audio data from a baby.

[0818] Sleep monitor: A device used to collect data such as a baby's heart rate and how often they turn over.

[0819] Specific examples

[0820] For example, imagine a parent shopping in a physical store with their baby and the baby suddenly starts crying. The baby monitor detects the crying and sends the audio data to the server. At the same time, the smartphone camera analyzes the parent's facial expressions and sends the parent's emotional data to the server. The server analyzes this data, determines that the baby is crying because it is hungry, and generates a countermeasure such as "prepare milk." If the parent is feeling tired or stressed, it may also notify them of relaxation techniques such as "take a slow, deep breath, then prepare milk." The notification content is displayed in real time on the smartphone app, allowing the parent to take appropriate action. Furthermore, if support from an employee in the store is needed, they can request help from an employee through the app.

[0821] Prompt Sentence Examples

[0822] Generate specific instructions based on what kind of support the baby needs when crying, taking into account the parents' emotional data.

[0823] This allows the system to simultaneously monitor the baby's condition and the parent's emotions, providing appropriate childcare support even while shopping in a physical store.

[0824] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0825] Step 1:

[0826] Collect the baby's voice data. In this step, the baby monitor detects the baby's cries and vocalizations and collects them as voice data. Specifically, the microphone captures the sound and converts it into a digital format. The input is the baby's voice, and the output is digital voice data.

[0827] Step 2:

[0828] Collecting the baby's sleep data. In this step, the sleep monitor monitors the baby's movements, heart rate, and breathing patterns and collects this data. Specifically, sensors detect the baby's body movements and biological signals and convert them into data. The input is the baby's body movements and biological signals, and the output is digital data.

[0829] Step 3:

[0830] Collect parent's voice and facial expression data. In this step, the user device (smartphone or tablet) detects the parent's voice and facial expression and collects that data. Specifically, the camera captures the parent's face and the microphone records the voice. The input is the parent's voice and facial expression, and the output is digital emotional data.

[0831] Step 4:

[0832] The collected voice data, sleep data, and emotion data are sent to a server. Specifically, the user device uploads these data to the server via the Internet. The input is the collected data, and the output is the data stored on the server.

[0833] Step 5:

[0834] The server stores the received data in a database. Specifically, the database system classifies and stores the data correctly. The input is the transmitted data, and the output is the data stored in the database.

[0835] Step 6:

[0836] The received data is analyzed using an AI algorithm to learn the baby's behavioral patterns and the parent's emotional state. Specifically, a machine learning model (for example, a model using Keras or TensorFlow) analyzes the data. The input is various stored data, and the output is the learned behavioral patterns and emotional state.

[0837] Step 7:

[0838] The server generates behavior predictions and countermeasures based on the learned behavior patterns. Specifically, the predictive model predicts future behavior and generates appropriate countermeasures. The input is the learned behavior pattern, and the output is the predicted behavior and its countermeasures.

[0839] Step 8:

[0840] The server adjusts behavior predictions and response methods based on the learned emotional state. Specifically, it adjusts the content of response methods and notification methods according to the parent's emotional state. The input is the learned emotional state, behavior predictions, and response methods, and the output is the adjusted response methods.

[0841] Step 9:

[0842] The generated behavior prediction and countermeasures are notified to the user device. Specifically, the notification system sends information to the user's device in real time. The input is the adjusted countermeasures, and the output is a notification to the user device.

[0843] Step 10:

[0844] The notification content is displayed on the user's device to help the parent take appropriate action. Specifically, the notification content is displayed on the smartphone screen. The input is the received notification data, and the output is the content displayed to the parent.

[0845] Step 11:

[0846] In-store services are linked and staff members provide support. In this step, a function is implemented that allows parents to request help from store staff if necessary. The input is a request for help, and the output is the provision of support by staff.

[0847] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0848] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0849] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0850] [Third embodiment]

[0851] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0852] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0853] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0854] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0855] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0856] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0857] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0858] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0859] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0860] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0861] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0862] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0863] A specific embodiment of the present invention will be described. The present invention is a system for managing a baby's behavior in real time and notifying parents of appropriate countermeasures. This system collects and analyzes the baby's voice data and sleep data to learn and predict the baby's behavioral patterns and provide countermeasures.

[0864] System configuration

[0865] The system consists of the following main components:

[0866] Devices (baby monitors, sleep monitors):

[0867] The baby monitor collects the baby's cries and vocalizations and sends them to a server as audio data.

[0868] The sleep monitor collects the baby's movements, heart rate, and breathing patterns and sends the data to a server.

[0869] server:

[0870] Data reception and storage: Receives voice and sleep data sent from the device and stores it in a database.

[0871] Data analysis: The received data is analyzed using an AI algorithm to learn the baby's behavioral patterns.

[0872] Behavior prediction and generation of countermeasures: Behavior prediction is made based on learned behavior patterns, and countermeasures are generated based on that prediction.

[0873] Sending notification: The generated behavior prediction and how to deal with it are notified to the user device.

[0874] User device (dedicated app):

[0875] Receive and display notifications: Receive notifications sent from the server and display them in an easy-to-read format for the user.

[0876] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[0877] Specialist consultation function: Provides a function to consult with a specialist online about questions regarding childcare.

[0878] Program processing

[0879] The processing performed by the program of this system will be explained below with specific examples.

[0880] Devices (baby monitors, sleep monitors)

[0881] A baby monitor detects a baby's crying and sends the audio data to a server. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[0882] The sleep monitor detects the baby's heart rate and tossing and turning, and sends sleep data to a server, for example, when the baby goes into deep sleep or wakes up.

[0883] server

[0884] The server receives the crying data sent by the baby monitor and analyzes it, for example to determine if the crying matches a certain pattern (e.g., hungry, sleepy, etc.).

[0885] Analyze sleep data to extract the baby's deep sleep level and sleep duration. For example, analyze whether the baby is sleeping soundly in the middle of the night.

[0886] Based on the analysis results, the system predicts the baby's behavior and generates countermeasures. For example, if the baby is crying because he or she is hungry, the system generates a countermeasure such as "Please prepare milk."

[0887] The generated behavior prediction and how to deal with it are notified to the user device. For example, a notification saying "Your baby has woken up. Please try to put him / her back to sleep" is sent.

[0888] User device (dedicated app)

[0889] The dedicated app receives notifications from the server and displays them on the user interface. For example, a parent can check a notification on their smartphone saying, "The baby has woken up."

[0890] Through the community function, parents can share parenting information with other parents. For example, they can ask questions about parenting problems and receive advice from other parents in the community.

[0891] You can use the specialist consultation function to consult with a specialist online. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[0892] The above is an embodiment of the present invention. This system allows parents to understand their baby's condition in real time and take appropriate measures. As a result, the burden of childcare on parents is reduced, allowing them to focus on raising their child with peace of mind.

[0893] The processing flow will be explained below.

[0894] Step 1:

[0895] The device (baby monitor) collects the baby's cries and vocalizations. For example, if the baby cries, the cry is recognized as audio data and recorded.

[0896] Step 2:

[0897] The device (sleep monitor) collects data on the baby's movements, heart rate, and breathing patterns. For example, if the baby rolls over or their heart rate changes, it records that data.

[0898] Step 3:

[0899] The devices (baby monitors and sleep monitors) transmit collected audio and sleep data to a server in real time, using Wi-Fi, Bluetooth, or other wireless communication methods.

[0900] Step 4:

[0901] The server receives the voice and sleep data sent from the device. For example, data on a baby's crying and heart rate is sent to the server.

[0902] Step 5:

[0903] The server stores the received data in a database, where each piece of data is linked to a baby's profile.

[0904] Step 6:

[0905] The server analyzes the stored data using AI algorithms to learn the baby's behavioral patterns. For example, it analyzes crying data over a certain period of time to identify the time of day and crying patterns of the baby.

[0906] Step 7:

[0907] The server predicts the baby's future behavior based on the learned behavioral patterns. For example, it predicts the time of day when the baby will cry based on past data and prepares to notify the user of the prediction.

[0908] Step 8:

[0909] The server generates an appropriate response to the predicted behavior. For example, if a pattern of "crying because the baby is hungry" is detected, the server generates a message saying "Please prepare milk" as a response.

[0910] Step 9:

[0911] The server then notifies the user device (dedicated app) of the generated behavior predictions and countermeasures. The notification content is sent in real time.

[0912] Step 10:

[0913] The device (dedicated app) receives notifications sent from the server. For example, a notification saying "The baby has woken up" is received.

[0914] Step 11:

[0915] The device (dedicated app) displays the received notification on the user interface, and the parent checks the notification and decides what to do next.

[0916] Step 12:

[0917] The user (parent) can take appropriate action based on the notification. For example, if the notification says "Prepare milk," the parent prepares milk and gives it to the baby.

[0918] Step 13:

[0919] The device (dedicated app) provides a community function that allows users to share information with other parents, and users can exchange their parenting experiences and questions within the community.

[0920] Step 14:

[0921] Users (parents) can consult with a specialist through a dedicated app about questions and concerns about childcare, and based on the specialist's answers, they can take more specific measures.

[0922] This enables the system to analyze a baby's behavioral patterns in detail and provide appropriate childcare support to parents in real time.

[0923] Example 1

[0924] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0925] When raising a baby, parents need to constantly monitor their baby's condition and respond appropriately. However, accurately understanding the reason for a baby's crying or sleep state can be difficult and often places a burden on parents. In addition, there are limited ways to obtain information and advice on childcare from other parents or experts. To solve these problems, a system is needed that monitors a baby's condition in real time and notifies parents of the appropriate course of action.

[0926] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0927] In this invention, the server includes means for collecting baby's voice data, means for collecting baby's physiological data, means for transmitting the collected voice data and physiological data to the server, means for analyzing the voice data and physiological data to learn the baby's behavioral patterns, means for generating a behavior prediction and a countermeasure based on the learned behavioral patterns, means for notifying the generated behavior prediction and countermeasure to a user terminal, means for displaying the notified behavior prediction and countermeasure on a user interface, means for providing a community function for the user to share information with others, and means for consulting with an expert within a dedicated app. This allows parents to monitor their baby's condition in real time and receive specific instructions for appropriate countermeasures. Furthermore, by sharing information with other parents and experts, parents can receive support to alleviate concerns and questions about childcare.

[0928] "Means for collecting baby voice data" refers to a device or function that records a baby's cries and vocalizations in real time and stores and transmits them in digital form.

[0929] "Means for collecting baby physiological data" means a device or function that records physiological information such as a baby's heart rate, breathing patterns, and rolling over in real time, and stores and transmits this information in digital form.

[0930] The "means for transmitting the collected voice data and physiological data to a server" refers to a communication device or function for transmitting the voice and physiological data collected by the terminal to a server via the Internet.

[0931] The "means for analyzing the voice data and physiological data and learning the baby's behavioral patterns" refers to a function that uses an AI algorithm to analyze the data received by the server and predict the baby's behavior and condition.

[0932] The "means for predicting behavior and generating a countermeasure based on the learned behavior pattern" is a function for predicting the baby's future behavior based on the analysis results and automatically generating an appropriate countermeasure.

[0933] The "means for notifying the user terminal of the generated behavior prediction and countermeasure method" is a communication device or function for transmitting the behavior prediction and countermeasure method generated by the server to the user terminal in real time.

[0934] The "means for displaying the notified predicted behavior and countermeasure method on a user interface" is an interface function for displaying the notification received by the user terminal in a form that is easy for the user to see.

[0935] "Means for providing a community function that enables the user to share information with others" refers to a bulletin board or chat function within the dedicated app that enables users to share information and concerns about child-rearing with other parents and exchange opinions.

[0936] "Means to consult with experts within the dedicated app" is a function that allows users to send questions about childcare to experts through the dedicated app and receive advice online.

[0937] This invention is a system that manages a baby's behavior in real time and notifies parents of appropriate measures. This system learns and predicts the baby's behavioral patterns by collecting and analyzing the baby's voice data and physiological data, and provides appropriate measures.

[0938] System configuration

[0939] The present invention consists of the following major components:

[0940] Devices (baby monitors, sleep monitors):

[0941] Baby monitor: Collects a baby's cries and vocalizations and sends them to a server as audio data. For example, if a baby cries in the middle of the night, the audio data is sent to the server in real time.

[0942] Sleep monitor: Collects physiological data such as the baby's movements, heart rate, breathing patterns, etc., and sends it to a server. For example, when the baby enters a deep sleep or wakes up, the data is collected and sent to the server.

[0943] server:

[0944] Data reception and storage: Receives the voice data and physiological data sent from the device and stores them in a database. For example, the server receives the baby's crying data and heart rate data and stores them in the appropriate format.

[0945] Data analysis: The received audio and physiological data is analyzed using AI algorithms. For example, the server analyzes the audio data to determine if the baby's crying matches a specific pattern (hungry, sleepy, etc.). The server also analyzes physiological data to assess the baby's sleep cycle and how soundly they are sleeping.

[0946] Behavior prediction and generation of countermeasures: Based on the analysis results, the system predicts the baby's behavior and generates appropriate countermeasures. For example, if the baby's crying is analyzed as "hungry," the system generates the countermeasure "Please prepare milk." Also, if the sleep data indicates that the baby is in a deep sleep, the system generates a notification saying "The baby is sleeping deeply. Please be quiet."

[0947] Notification sending: The server notifies the user device of the generated behavior prediction and how to deal with it. For example, it sends a notification saying, "The baby is crying. He may be hungry. Please prepare some milk."

[0948] User device (dedicated app):

[0949] Receiving and displaying notifications: The user device receives notifications from the server and displays them on the user interface. The user can check the notifications and take appropriate action. For example, the user can see a message on the smartphone app saying, "Your baby has woken up. Please try to put him / her back to sleep."

[0950] Community function: Users can use the community function to share parenting information with other parents. For example, they can ask questions about how to deal with nighttime crying and receive advice from other parents.

[0951] Specialist consultation function: Users can use the specialist consultation function of the dedicated app to consult with experts online about childcare-related questions. For example, they can send a message to a specialist saying, "My baby wakes up frequently during the night," and receive specific advice.

[0952] Specific examples

[0953] For example, if a baby starts crying in the middle of the night, the system will operate as follows:

[0954] 1. The baby monitor detects the baby's crying and sends the audio data to the server in real time.

[0955] 2. The server receives the crying data and stores it in a database. At the same time, it uses an AI algorithm to analyze the crying data and identify behavioral patterns that indicate "hungry."

[0956] 3. If the server determines that the user is hungry, it generates a solution such as "Please prepare milk" and notifies this information to the user's terminal.

[0957] 4. The user device (dedicated app) receives a notification from the server and displays a message to the parent saying, "The baby is crying. He may be hungry. Please prepare some milk."

[0958] Example input to a generative AI model

[0959] Below are some example prompts to input to the generative AI model:

[0960] User: "My baby started crying in the middle of the night. What should I do?"

[0961] AI: "Let's find out why your baby is crying. First, he may need milk, so get some ready. If he still doesn't stop crying, check to see if he has a wet diaper or is uncomfortable."

[0962] This invention allows parents to monitor their baby's condition in real time and take appropriate measures promptly. In addition, by sharing information with other parents and experts, parents can alleviate concerns and questions about childcare, allowing them to approach childcare with peace of mind.

[0963] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0964] Step 1:

[0965] Data collection from devices (baby monitors, sleep monitors)

[0966] How a baby monitor works: A baby monitor collects a baby's crying sounds in real time. Specifically, the baby monitor's microphone detects the crying sounds and converts the audio data into a digital format. The input is the baby's crying sounds, and the output is digital audio data.

[0967] How a sleep monitor works: A sleep monitor monitors a baby's heart rate, breathing patterns, and rolling over. This data is collected using digital sensors. The input is the baby's physiological movements, and the output is digital physiological data.

[0968] Step 2:

[0969] Sending data from the device to the server

[0970] How the baby monitor works: The baby monitor sends the collected audio data to a server via the Internet. Specifically, the communication module in the device sends the data. The input is digital audio data, and the output is the audio data sent to the server.

[0971] Operation of the sleep monitor: The sleep monitor also sends the collected physiological data to a server. Specifically, a communication module sends the data via the Internet. The input is digital physiological data, and the output is the physiological data sent to the server.

[0972] Step 3:

[0973] Receiving and storing data by the server

[0974] Data reception: The server receives data sent from the baby monitor or sleep monitor. Specifically, the server's communication module receives the data. The input is audio data and physiological data sent via the Internet, and the output is data stored in the server's memory.

[0975] Data Storage Operation: The received audio and physiological data is stored in a database. Specifically, the database management system stores the data in an appropriate format. The input is the received data and the output is the stored database entry.

[0976] Step 4:

[0977] Data analysis by server

[0978] Analysis of audio data: The server analyzes the audio data using an AI algorithm. Specifically, it compares the crying data with specific patterns (hungry, sleepy, etc.). The input is the stored audio data, and the output is the analysis result indicating the cause of the crying.

[0979] Physiological data analysis: The server analyzes the physiological data to assess the baby's sleep cycle and deepness. For example, it analyzes heart rate and breathing patterns to determine whether the baby is sleeping deeply. The input is the stored physiological data, and the output is the analysis result indicating the state of sleep.

[0980] Step 5:

[0981] Server predicts behavior and generates countermeasures

[0982] Generation of behavior prediction: The server predicts the baby's behavior based on the analysis results. For example, if the analysis of a baby's cry indicates that the baby is hungry, a behavior prediction is made based on that information. The input is the analysis results of the voice data and physiological data, and the output is a specific behavior prediction.

[0983] Generation of action plan: The server generates an appropriate action plan based on the behavior prediction. For example, if the predicted result is "I'm hungry," the server generates the action plan "Please prepare some milk." The input is the behavior prediction result, and the output is a specific action plan.

[0984] Step 6:

[0985] Notification from the server to the user device

[0986] Sending notification: The server notifies the user device of the generated behavior prediction and countermeasures. Specifically, the communication module sends the notification data. The input is the behavior prediction and countermeasure data, and the output is a notification to the user device.

[0987] Step 7:

[0988] Receiving and displaying notifications on user devices

[0989] Receiving notifications: The user device receives notifications from the server. The communication module receives the notification data, and the dedicated application uses that data. The input is the notification data sent from the server, and the output is the display on the user interface.

[0990] Displaying a notification: The dedicated app displays a notification on the user interface. Specifically, the app displays a message on the screen saying, "Your baby is crying. He might be hungry. Please prepare some milk." The input is the received notification data, and the output is the notification displayed to the user.

[0991] Step 8:

[0992] Additional functions for user devices (community function, specialist consultation function)

[0993] Community function: Users can share parenting information with other parents through the community function. For example, they can ask questions about specific parenting concerns and receive advice from other parents. The input volume is the questions and information entered by the user, and the output volume is the answers and shared information from other users.

[0994] Specialist consultation function: Users can use the specialist consultation function of the dedicated app to consult with an expert online about questions about childcare. For example, they can send a question such as "My baby wakes up frequently during the night" to a specialist and receive specific advice. The input is the question entered by the user, and the output is advice from the specialist.

[0995] (Application example 1)

[0996] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0997] Modern parents, especially those raising children for the first time, face a great deal of anxiety and stress. While they need to accurately understand their baby's condition and take appropriate measures, choosing the right childcare products for their baby while shopping in a physical store can be a difficult task. Furthermore, when expert advice is needed, there is a lack of means to respond quickly. There is a need to address these issues.

[0998] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0999] In this invention, the server includes means for analyzing the baby's voice data and sleep data to learn the baby's behavioral patterns, means for predicting behavior and generating countermeasures based on the learned behavioral patterns, and means for suggesting childcare products suitable for the baby's condition in a physical store. This allows parents to understand the baby's condition in real time and take appropriate measures, and also makes it possible to suggest childcare products that are optimal for the baby while shopping in the physical store.

[1000] "Baby voice data" is a digital recording of sounds made by a baby, such as crying or vocalizations.

[1001] "Baby sleep data" is a digital record of physiological information related to a baby's sleep, such as their heart rate, breathing patterns, and rolling over.

[1002] "Server" means the central system that receives, analyzes, stores, and sends notifications to user devices.

[1003] A "means for learning behavioral patterns" is an algorithm or method for analyzing collected data, identifying consistencies and changes in a baby's behavior, and learning those patterns.

[1004] The "means for predicting behavior and generating countermeasures" refers to an algorithm and method for predicting future behavior based on learned behavioral patterns and generating appropriate countermeasures for the behavior.

[1005] A "user terminal" is a mobile device such as a smartphone or tablet used by a parent or caregiver, which receives and displays notifications, provides community functions, and provides expert consultation functions.

[1006] "Means for providing community functions" are online bulletin boards and chat functions that allow parenters to share information and receive advice from each other.

[1007] "A way to consult with an expert for free" is a way to ask questions or get advice about child-rearing from an expert online for free.

[1008] The "means for suggesting childcare products in a physical store" refers to a system and method for suggesting appropriate childcare products to parents who are shopping in a store based on the real-time status of their baby.

[1009] A "generative AI model" is a supervised or unsupervised artificial intelligence model used for data analysis and behavior prediction.

[1010] A "prompt" is a textual instruction used as input to an AI model.

[1011] The following describes in detail the embodiments of the present invention. The present invention is a system that manages a baby's behavior in real time and suggests appropriate ways to deal with the baby and childcare products to parents. This system learns and predicts the baby's behavioral patterns by collecting and analyzing the baby's voice data and sleep data, and then suggests ways to deal with the baby and childcare products.

[1012] System configuration

[1013] The system consists of the following main components:

[1014] Devices (baby monitors, sleep monitors):

[1015] Baby monitors collect the baby's cries and vocalizations and send them to a server as audio data. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[1016] The sleep monitor collects the baby's heart rate and movements, and sends the data to a server as sleep data, such as when the baby falls into a deep sleep or wakes up.

[1017] server:

[1018] Data reception and storage: The voice and sleep data sent from the device is received and stored in a database using software such as a common database management system (DBMS) or cloud service.

[1019] Data analysis: The received data is analyzed using a generative AI model to learn the baby's behavioral patterns. The software used is a machine learning library such as TensorFlow or PyTorch.

[1020] Behavior prediction and generation of coping methods: Behavior prediction is made based on learned behavior patterns, and coping methods and childcare product suggestions are generated based on the prediction.

[1021] Sending notification: The generated behavior prediction and how to deal with it are notified to the user terminal.

[1022] User device (dedicated app):

[1023] Receiving and displaying notifications: Receives notifications sent from the server and displays them in the user interface.

[1024] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[1025] Expert consultation function: Provides a function to consult with experts online about questions regarding childcare.

[1026] A function to suggest childcare products in physical stores: Recommends the most suitable childcare products based on the baby's condition in real time.

[1027] Program processing

[1028] The server receives and analyzes the baby's voice and sleep data. For example, if the baby monitor detects the baby crying, the data is sent to the server. The server analyzes the data and determines whether the crying matches a certain pattern (e.g., hungry, sleepy, etc.).

[1029] The server then predicts behavior based on the behavioral patterns and generates suggested solutions and childcare items. This is done using a generative AI model. For example, if the baby is crying because it is hungry, the server predicts the solution, suggesting "prepare milk" and "a baby bottle."

[1030] The generated solutions and suggestions are sent to the user's device, which can be a smartphone, tablet, or other device, and displayed in a dedicated app. Additionally, if the childcare product suggestion function is enabled in the physical store, shelves with suitable childcare products will be displayed in the app, facilitating smooth purchasing for parents.

[1031] Specific examples

[1032] For example, when a baby starts crying in the morning, a baby monitor captures the sound. This data is sent to a server for analysis. If the cry is identified as a sign of hunger, the server sends a notification to the user's device suggesting a baby bottle as an appropriate childcare item, along with a message saying, "Prepare some milk."

[1033] An example of a prompt is, "Generate a notification message for a parent based on baby's crying sound, which indicates possible hunger." This allows users to quickly respond to their baby's needs and easily find and purchase the necessary childcare products.

[1034] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1035] Step 1:

[1036] Collects baby's audio and sleep data. The baby monitor captures the baby's crying in real time, and the sleep monitor collects sleep data such as the baby's heart rate, rolling over, and breathing patterns. The input is the crying audio data and heart rate and movement data, and the output is a digital file containing these data.

[1037] Step 2:

[1038] The device sends collected audio and sleep data to the server. The audio and sleep data sent from the baby monitor and sleep monitor are sent to the server. The input is the collected digital file, and the output is the data stored on the server.

[1039] Step 3:

[1040] The server analyzes the received audio and sleep data and learns the baby's behavioral patterns. The software used is a machine learning library such as TensorFlow or PyTorch. The input is the audio and sleep data stored on the server, and the output is a model that shows the baby's behavioral patterns.

[1041] Step 4:

[1042] The server generates behavior predictions and countermeasures based on the learned behavior patterns. A generative AI model is used for this. The input is a model showing the behavior patterns, and the output is text data containing the behavior predictions and countermeasures. Specific childcare products are also suggested.

[1043] Step 5:

[1044] The server notifies the user device of the generated behavior prediction and countermeasures using a dedicated notification API. The input is text data containing the behavior prediction and countermeasures, and the output is a push notification to the user device.

[1045] Step 6:

[1046] The user terminal displays the notified behavior prediction and countermeasures on the user interface. The input is the notification data sent from the server, and the output is a notification message displayed on the user interface.

[1047] Step 7:

[1048] Through a dedicated app, users are recommended childcare products suited to their baby's condition in a physical store. The input is the store's situation and real-time baby data, and the output is a list of recommended childcare products.

[1049] Step 8:

[1050] The system provides a community function that allows users to share information with others and a function that allows users to consult with experts online for free about child-rearing questions. The input is the user's questions and child-rearing information, and the output is answers and advice from other users and experts.

[1051] This trend will enable parents to understand their baby's condition in real time, take appropriate measures, and quickly select the necessary childcare products in physical stores.

[1052] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1053] A specific embodiment for carrying out the present invention will now be described. The present invention is a system that combines a basic system that manages a baby's behavior in real time with an emotion engine that recognizes the user's emotions and maximizes the effectiveness of childcare support.

[1054] System configuration

[1055] The system consists of the following main components:

[1056] Devices (baby monitors, sleep monitors, user devices):

[1057] The baby monitor collects the baby's cries and vocalizations and sends them to a server as audio data.

[1058] The sleep monitor collects the baby's movements, heart rate, and breathing patterns and sends the data to a server.

[1059] The user device collects the user's voice and facial expressions and sends them to the server as emotional data.

[1060] server:

[1061] Data reception and storage: Receives voice data, sleep data, and emotion data sent from the device and stores them in a database.

[1062] Data analysis: The received data is analyzed using AI algorithms to learn the baby's behavioral patterns and the user's emotional state.

[1063] Behavior prediction and response generation: Generate behavior prediction and response methods based on learned behavioral patterns and emotional states. Adjust the content of notifications and response methods based on the user's emotional state.

[1064] Sending notification: The generated behavior prediction and how to deal with it are notified to the user device.

[1065] User device (dedicated app):

[1066] Receiving and displaying notifications: Receive notifications sent from the server and display them in a user-friendly format. The notification content is adjusted based on the user's emotional state.

[1067] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[1068] Specialist consultation function: Provides a function to consult with a specialist online about questions regarding childcare.

[1069] Program processing

[1070] The processing performed by the program of this system will be explained below with specific examples.

[1071] Terminals (baby monitors, sleep monitors, user devices)

[1072] A baby monitor detects a baby's crying and sends the audio data to a server. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[1073] The sleep monitor detects the baby's heart rate and tossing and turning, and sends sleep data to a server, for example, when the baby goes into deep sleep or wakes up.

[1074] The user device collects the user's voice and facial expressions in real time and transmits the emotional data to the server. For example, if the user is feeling stressed, the facial expression and tone of voice are transmitted as data.

[1075] server

[1076] The server receives data sent from the baby monitor, the sleep monitor, and the user device, such as the baby's crying data, heart rate data, and user emotion data.

[1077] The server stores the received data in a database, where each data item is linked to the baby's profile and the user's profile.

[1078] The server analyzes the stored data using AI algorithms to learn the baby's behavioral patterns and the user's emotional state. For example, by analyzing crying data and the user's emotional data over a certain period of time, it can identify the baby's crying times and crying patterns, as well as the user's stress patterns.

[1079] The server predicts the baby's future behavior based on the learned behavioral patterns and emotional state. For example, it predicts the time of day when the baby will cry based on past data and prepares to notify the user of the prediction.

[1080] The server generates and adjusts appropriate responses to predicted behaviors based on the user's emotional state. For example, if a "crying because they're hungry" pattern is detected, the server generates a message saying "Prepare milk" as a response. If the user is feeling stressed, the server may also include more detailed instructions and tips on how to relax.

[1081] The server then notifies the user device (dedicated app) of the generated behavior predictions and countermeasures. The notification content is sent in real time.

[1082] User device (dedicated app)

[1083] The dedicated app receives notifications from the server and displays them on the user interface. For example, a parent can check a notification on their smartphone saying, "The baby has woken up."

[1084] Through the community function, parents can share parenting information with other parents. For example, they can ask questions about parenting problems and receive advice from other parents in the community.

[1085] You can use the specialist consultation function to consult with a specialist online. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[1086] Specific examples

[1087] For example, if a baby starts crying in the middle of the night, the baby monitor collects audio data and sends it to a server. The server analyzes the audio data and determines that the baby is crying because it is hungry. At the same time, it analyzes emotional data sent from the user's device and recognizes that the user is tired. The server generates a notification that includes instructions such as "Prepare milk" and relaxation techniques such as "Take a slow, deep breath, then prepare milk," and sends it to a dedicated app. The parent can check the notification in the dedicated app, follow the instructions to prepare milk and feed the baby, while also reducing their own stress.

[1088] In this way, the system can not only monitor the baby's condition but also provide comprehensive childcare support that takes into account the parents' emotional state.

[1089] The processing flow will be explained below.

[1090] Step 1:

[1091] The device (baby monitor) collects the baby's cries and vocalizations. For example, if the baby cries, the cry is recorded as audio data.

[1092] Step 2:

[1093] The device (sleep monitor) collects data on the baby's movements, heart rate, and breathing patterns. For example, if the baby moves or their heart rate changes, it will detect and record that data.

[1094] Step 3:

[1095] The terminal (user device) collects the user's voice and facial expression data. For example, when a parent talks to their baby in the living room, the voice and facial expression data are recorded.

[1096] Step 4:

[1097] The devices (baby monitors, sleep monitors, user devices) transmit the collected data to the server in real time using wireless communication methods such as Wi-Fi and Bluetooth.

[1098] Step 5:

[1099] The server receives the voice data, sleep data, and emotion data sent from the device. For example, the server receives data on a baby's crying and heart rate, as well as data on the user's tone of voice and facial expression.

[1100] Step 6:

[1101] The server stores the received data in a database, where each data point is linked to the baby and user profile.

[1102] Step 7:

[1103] The server analyzes the stored data using AI algorithms. For example, it analyzes crying patterns to identify behavioral patterns such as "hungry" or "sleepy." At the same time, it analyzes the user's tone of voice and facial expressions to identify the user's emotional state (e.g., stress).

[1104] Step 8:

[1105] The server predicts the baby's future behavior based on the learned behavioral patterns. For example, it predicts the baby's tendency to wake up in the middle of the night based on past data, and prepares the next response based on the prediction results.

[1106] Step 9:

[1107] The server generates an appropriate response to the predicted behavior. For example, if the baby is crying because it is hungry, it generates a message saying, "Prepare milk." At the same time, it adjusts the response based on the user's emotional state. For example, if the user is feeling stressed, it includes detailed instructions such as, "First, take a deep breath. Then, prepare milk."

[1108] Step 10:

[1109] The server then notifies the user device (dedicated app) of the generated behavior prediction and how to respond. For example, it sends a notification saying, "Your baby has woken up and is hungry. First, take a deep breath, then prepare some milk."

[1110] Step 11:

[1111] The device (dedicated app) receives notifications sent from the server. For example, it receives a notification that "your baby has woken up and is hungry."

[1112] Step 12:

[1113] The device (dedicated app) displays the received notification on the user interface, and the parent checks the notification and decides what to do next.

[1114] Step 13:

[1115] The user (parent) can take appropriate action based on the notification. For example, they can follow the notification and take steps such as "First, take a deep breath, then prepare the milk."

[1116] Step 14:

[1117] The device (dedicated app) provides a community function that allows users to share information with other parents, allowing them to exchange parenting experiences and questions within the community.

[1118] Step 15:

[1119] Users (parents) can consult with a specialist through a dedicated app about questions and concerns about childcare, and based on the specialist's answers, they can take more specific measures.

[1120] This enables the system to perform detailed analysis of the baby's behavioral patterns and the user's emotional state, providing parents with appropriate and tailored parenting support in real time.

[1121] Example 2

[1122] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1123] Conventional childcare support systems focused on monitoring the baby's behavior and condition, but did not take into account the parent's emotional state or stress. As a result, there was a problem that the information and coping methods for baby care were not optimized for the parent's condition. This raised concerns that parents might experience excessive stress from childcare, leading to a decline in the quality of childcare.

[1124] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1125] In this invention, the server includes means for analyzing the voice data, sleep data, and user emotional data to learn the baby's behavioral patterns and the user's emotional state, means for generating baby behavior predictions and countermeasures based on the learned behavioral patterns and emotional state and adjusting them according to the user's emotional state, and means for notifying the user terminal of the generated behavior predictions and countermeasures, thereby making it possible not only to monitor the baby's condition but also to provide optimal support based on the parent's emotional state.

[1126] "Audio data" is digitized sound information such as a baby's vocalizations and cries.

[1127] "Sleep data" is sleep-related information such as a baby's rolling over, heart rate, and breathing patterns that is collected and recorded digitally.

[1128] A "server" is a computer system that receives, stores, and analyzes collected data, generates behavior predictions and countermeasures, and notifies the user terminal.

[1129] A "behavioral pattern" is a specific pattern or tendency extracted from a series of data on a baby's behavior or condition.

[1130] "Emotional state" is data that indicates the user's (parent's) psychological state and stress level, and is collected from voice and facial expressions.

[1131] A "user terminal" is a device that receives notifications from the server and displays them on a user interface, and includes smartphones and tablets.

[1132] The "community function" is a function that allows users to share information and communicate with each other.

[1133] "Means for consulting with experts on childcare" is a function that allows users to consult with experts online about questions about childcare through a dedicated app and receive advice.

[1134] "Behavior prediction" means that the server predicts the baby's future behavior and state in advance.

[1135] "Solutions" are specific steps and advice on caring for the baby that are generated by the server based on behavioral predictions.

[1136] A "user interface" refers to a screen and operating means that allows a user to check and operate notifications and data from a server via a terminal.

[1137] Specific embodiments of the present invention will be described below. The present invention is a system that combines a system for managing a baby's behavior in real time with an emotion engine that recognizes the user's emotions and maximizes the effectiveness of childcare support.

[1138] This system consists of the following main components: terminals (baby monitor, sleep monitor, user device), server, and user terminal (dedicated app).

[1139] Terminals (baby monitors, sleep monitors, user devices)

[1140] The devices include baby monitors that collect the baby's cries and vocalizations, sleep monitors that monitor the baby's movements, heart rate, and breathing patterns, and user devices that collect the user's voice and facial expressions.Specific examples of the use of the following devices are given below.

[1141] Baby monitor: General baby monitor device

[1142] Sleep monitor: General sleep monitor device

[1143] User devices: smartphones and tablets

[1144] Examples:

[1145] For example, if a baby starts crying in the middle of the night, a baby monitor will collect the sound data in real time and send it to a server. A sleep monitor will collect data on the baby's heart rate and how often the baby turns over in bed and send it to a server. A user device will collect the user's stress level from their facial expressions and voice and send it to a server as emotional data.

[1146] server

[1147] The server is a central control unit that receives, stores, and analyzes data sent from the terminals. The server has the following functions:

[1148] 1. Data Reception and Storage: The server receives the baby's voice data, sleep data, and user's emotion data and stores them in a database. The collected data is linked to the baby's and user's profiles.

[1149] 2. Data analysis: The server uses AI algorithms such as Google TensorFlow to analyze the incoming data and learn the baby's behavioral patterns and the user's emotional state.

[1150] 3. Behavior prediction and response generation: Based on the learned behavioral patterns and emotional state, the server generates optimal behavior prediction and response methods. It also adjusts the notification content according to the user's emotional state.

[1151] 4. Notification transmission: The generated behavior prediction and countermeasures are notified to the user terminal in real time.

[1152] Examples:

[1153] For example, when a baby starts crying and audio data is sent to the server, the server determines that the baby is crying because it is hungry. At the same time, emotional data indicating that the user is tired is analyzed and a notification is generated instructing the user to prepare milk. The notification also includes relaxation tips, such as "Take a slow, deep breath, and then prepare the milk."

[1154] User device (dedicated app)

[1155] The user terminal is a dedicated application for receiving notifications from the server and displaying them on the user interface. This application includes the following functions:

[1156] 1. Receiving and displaying notifications: Receives predicted behavior and countermeasures sent from the server and displays them in an easy-to-read format for the user.

[1157] 2. Community function: A function for sharing information with other users. Parenting information can be exchanged through message boards and chats.

[1158] 3. Specialist consultation function: Provides a function that allows users to consult with a specialist online about questions regarding childcare.

[1159] Examples:

[1160] For example, the dedicated app will receive a notification that the baby has woken up, and the parent can check the notification on their smartphone. They can also get advice from other parents through the community function or consult with a specialist.

[1161] Example prompts for generative AI models

[1162] "My baby started crying in the middle of the night. Please suggest appropriate ways to handle this situation. Also, include ways to support the parents if they are feeling stressed."

[1163] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1164] System program processing flow

[1165] Terminals (baby monitors, sleep monitors, user devices)

[1166] Step 1: Collect data

[1167] A baby monitor collects the baby's cries and vocalizations. For example, if a baby starts crying in the middle of the night, the built-in microphone captures the sound. The input is the baby's cry, and the output is audio data.

[1168] A sleep monitor monitors a baby's movements, heart rate, and breathing patterns. For example, a heart rate sensor measures the baby's heart rate in real time. The input is the baby's physical activity, and the output is sleep data.

[1169] The user device detects the user's voice and facial expressions. For example, the built-in camera and microphone collect the user's speaking voice and facial expressions. The input is the user's facial expressions and voice, and the output is emotional data.

[1170] Step 2: Sending data

[1171] The baby monitor transmits the collected audio data to a server, for example, using a Wi-Fi connection to transmit the audio data in real time. The collected audio data is the input, and the transmitted audio data is the output.

[1172] The sleep monitor sends the collected heart rate and sleep movement data to a server. For example, it transfers the data using Bluetooth or Wi-Fi. The collected sleep data is the input, and the sent sleep data is the output.

[1173] The user device sends the collected voice and facial expression data to a server, e.g., uploading emotional data in real time using an internet connection. The collected emotional data is the input, and the sent emotional data is the output.

[1174] server

[1175] Step 3: Receiving the data

[1176] The server receives the voice data, sleep data, and emotion data sent from the device. For example, crying data from a baby monitor, heart rate data from a sleep monitor, and emotion data from the user device are simultaneously received. Various data from the device are input, and a list of received data is output.

[1177] Step 4: Save your data

[1178] The server stores the received data in a database, for example linking it to the baby's profile and storing voice data, sleep data, and emotion data. Data received as input, data records stored as output.

[1179] Step 5: Data analysis

[1180] The server analyzes the stored data using an AI algorithm. The input is the stored data, and the output is the baby's behavioral patterns and the user's emotional state. Using Google TensorFlow, the server learns the baby's behavioral patterns based on crying data and heart rate data. The server also analyzes the user's emotional data to understand the user's emotional state.

[1181] Step 6: Predicting behavior and generating countermeasures

[1182] The server generates behavior predictions and countermeasures based on the learned behavioral patterns and emotional state. The analysis results are input, and the generated behavior predictions and countermeasures are output. For example, it predicts the next time the baby will cry and generates specific countermeasures such as preparing milk. Furthermore, if the user is tired, it generates a notification that includes relaxation techniques.

[1183] Step 7: Sending notifications

[1184] The server notifies the user device of the generated behavior prediction and the countermeasure. The generated behavior prediction and countermeasure are input, and the sent notification is output. For example, a notification that "The baby is crying" and the contents "Please prepare milk" and "Please take slow, deep breaths" are sent to a dedicated app in real time.

[1185] User device (dedicated app)

[1186] Step 8: Receiving and Viewing Notifications

[1187] The user device receives notifications from the server and displays them on the user interface. The notification sent is the input, and the notification content displayed is the output. For example, a parent sees a notification on their smartphone saying "The baby has woken up" and acts accordingly.

[1188] Step 9: Community Features

[1189] The user terminal provides a community function for sharing information with other users. For example, a user can post a question about childcare on a community bulletin board and receive advice from other users. The input is the user's question, and the output is the answer from other users.

[1190] Step 10: Specialist consultation function

[1191] The user device provides a function for online childcare consultation with experts. For example, a user can consult a specialist about a problem such as "my baby is not sleeping well" and receive specific advice. The user's consultation content is input, and the expert's advice is output.

[1192] (Application example 2)

[1193] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1194] For parents who go shopping in physical stores with their babies, it is a significant burden to properly manage their baby's condition while shopping. Furthermore, a lack of support tailored to the baby's condition and the parent's emotional state increases parental stress. Therefore, there is a need for a system that can manage the condition of both the baby and the parent in real time and provide appropriate support.

[1195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1196] In this invention, the server includes means for collecting baby's voice data, means for collecting baby's sleep data, means for transmitting the collected voice data and sleep data to the server, means for analyzing the voice data and sleep data to learn the baby's behavioral patterns, means for predicting behavior and generating countermeasures based on the learned behavioral patterns, means for collecting parent's voice and facial expressions and transmitting them to the server as emotional data, means for analyzing the emotional data to learn the parent's emotional state, and means for adjusting the behavior predictions and countermeasures based on the learned emotional state. This makes it possible to manage the baby's condition and the parent's emotional state in real time and provide appropriate support and specific countermeasures.

[1197] "Baby voice data" refers to audio information of a baby's cries and vocalizations.

[1198] "Baby sleep data" refers to information about a baby's sleep state, such as the baby's rolling over, heart rate, and breathing patterns.

[1199] "Server" refers to a central processing unit for receiving, storing, and analyzing collected data.

[1200] "Means for learning behavioral patterns" refers to algorithms that analyze collected baby data and identify certain patterns and trends.

[1201] "Means for behavior prediction and countermeasure generation" refers to software or algorithms that predict future behavior and generate appropriate countermeasures based on learned behavioral patterns.

[1202] "User terminal" refers to a device (smartphone, tablet, etc.) used to provide notifications and advice to parents.

[1203] "Means for collecting parent's voice and facial expression" refers to devices or sensors for collecting parent's voice and facial expression data in order to understand the parent's emotional state.

[1204] "Emotional data" refers to information about the parent's emotional state obtained by analyzing their voice and facial expressions.

[1205] "Means for learning emotional states" refers to algorithms that analyze collected emotional data and identify patterns and trends in parents' emotions.

[1206] "Means for adjusting behavioral predictions and coping strategies" refers to software or algorithms that appropriately adjust a baby's behavioral predictions and coping strategies to match the parent's emotional state.

[1207] "Means to link with in-store services" refers to the ability to link with services at physical stores and have employees provide support when parents need it.

[1208] MODE FOR CARRYING OUT THE INVENTION

[1209] The present invention is described in detail below with respect to an embodiment thereof. The present system aims to reduce the burden of child-rearing by managing baby behavior data and parental emotional data in real time and providing appropriate support.

[1210] The system consists of the following main components:

[1211] Terminal

[1212] Baby monitor: A device that collects a baby's cries and vocalizations and sends them to a server as audio data. For example, if a baby starts crying in a store, the device can collect the crying sound.

[1213] Sleep monitor: A device that collects a baby's movements, heart rate, and breathing patterns and sends them to a server as sleep data. For example, it records when a baby moves or wakes up while sleeping.

[1214] User device: A device such as a smartphone or tablet that collects the parent's voice and facial expressions and sends them to a server as emotional data. For example, when a parent is feeling stressed, their facial expression and tone of voice can be recognized.

[1215] server

[1216] The server receives data sent from the baby monitor, sleep monitor, and user device and stores it in a database.

[1217] The received data is analyzed by an AI algorithm to learn the baby's behavioral patterns and the parent's emotional state. Specifically, it uses machine learning models to analyze patterns in the data.

[1218] Based on the stored data, the system predicts the baby's future behavior and generates and adjusts countermeasures based on the parent's emotional state. For example, it suggests appropriate countermeasures based on the "time of day the baby cries" and "pattern of crying."

[1219] The server notifies the user terminal of the generated behavior prediction and the countermeasure, and displays the notified content in an easy-to-read format for the parent.

[1220] User terminal

[1221] Receive notifications sent from the server and display them on the unit or application so that parents can act immediately.

[1222] Through the community function, parents can share parenting information with other parents. They can also exchange opinions using the message board and chat function.

[1223] By using the specialist consultation function, you can consult with a specialist online about your child-rearing questions. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[1224] It also has the ability to link with in-store services, allowing employees to provide support as needed.

[1225] Software and hardware used

[1226] OpenCV: An image processing library for face recognition and expression analysis.

[1227] Keras: A machine learning library used to implement emotion recognition models.

[1228] TensorFlow: A deep learning framework that acts as a backend for Keras.

[1229] Smartphone: A device used for parental emotion recognition and application integration.

[1230] Baby monitor: A device used to collect audio data from a baby.

[1231] Sleep monitor: A device used to collect data such as a baby's heart rate and how often they turn over.

[1232] Specific examples

[1233] For example, imagine a parent shopping in a physical store with their baby and the baby suddenly starts crying. The baby monitor detects the crying and sends the audio data to the server. At the same time, the smartphone camera analyzes the parent's facial expressions and sends the parent's emotional data to the server. The server analyzes this data, determines that the baby is crying because it is hungry, and generates a countermeasure such as "prepare milk." If the parent is feeling tired or stressed, it may also notify them of relaxation techniques such as "take a slow, deep breath, then prepare milk." The notification content is displayed in real time on the smartphone app, allowing the parent to take appropriate action. Furthermore, if support from an employee in the store is needed, they can request help from an employee through the app.

[1234] Prompt Sentence Examples

[1235] Generate specific instructions based on what kind of support the baby needs when crying, taking into account the parents' emotional data.

[1236] This allows the system to simultaneously monitor the baby's condition and the parent's emotions, providing appropriate childcare support even while shopping in a physical store.

[1237] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1238] Step 1:

[1239] Collect the baby's voice data. In this step, the baby monitor detects the baby's cries and vocalizations and collects them as voice data. Specifically, the microphone captures the sound and converts it into a digital format. The input is the baby's voice, and the output is digital voice data.

[1240] Step 2:

[1241] Collecting the baby's sleep data. In this step, the sleep monitor monitors the baby's movements, heart rate, and breathing patterns and collects this data. Specifically, sensors detect the baby's body movements and biological signals and convert them into data. The input is the baby's body movements and biological signals, and the output is digital data.

[1242] Step 3:

[1243] Collect parent's voice and facial expression data. In this step, the user device (smartphone or tablet) detects the parent's voice and facial expression and collects that data. Specifically, the camera captures the parent's face and the microphone records the voice. The input is the parent's voice and facial expression, and the output is digital emotional data.

[1244] Step 4:

[1245] The collected voice data, sleep data, and emotion data are sent to a server. Specifically, the user device uploads these data to the server via the Internet. The input is the collected data, and the output is the data stored on the server.

[1246] Step 5:

[1247] The server stores the received data in a database. Specifically, the database system classifies and stores the data correctly. The input is the transmitted data, and the output is the data stored in the database.

[1248] Step 6:

[1249] The received data is analyzed using an AI algorithm to learn the baby's behavioral patterns and the parent's emotional state. Specifically, a machine learning model (for example, a model using Keras or TensorFlow) analyzes the data. The input is various stored data, and the output is the learned behavioral patterns and emotional state.

[1250] Step 7:

[1251] The server generates behavior predictions and countermeasures based on the learned behavior patterns. Specifically, the predictive model predicts future behavior and generates appropriate countermeasures. The input is the learned behavior pattern, and the output is the predicted behavior and its countermeasures.

[1252] Step 8:

[1253] The server adjusts behavior predictions and response methods based on the learned emotional state. Specifically, it adjusts the content of response methods and notification methods according to the parent's emotional state. The input is the learned emotional state, behavior predictions, and response methods, and the output is the adjusted response methods.

[1254] Step 9:

[1255] The generated behavior prediction and countermeasures are notified to the user device. Specifically, the notification system sends information to the user's device in real time. The input is the adjusted countermeasures, and the output is a notification to the user device.

[1256] Step 10:

[1257] The notification content is displayed on the user's device to help the parent take appropriate action. Specifically, the notification content is displayed on the smartphone screen. The input is the received notification data, and the output is the content displayed to the parent.

[1258] Step 11:

[1259] In-store services are linked and staff members provide support. In this step, a function is implemented that allows parents to request help from store staff if necessary. The input is a request for help, and the output is the provision of support by staff.

[1260] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1261] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1262] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1263] [Fourth embodiment]

[1264] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1265] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1266] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1267] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1268] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1269] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1270] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1271] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1272] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1273] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1274] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1275] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1276] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1277] A specific embodiment of the present invention will be described. The present invention is a system for managing a baby's behavior in real time and notifying parents of appropriate countermeasures. This system collects and analyzes the baby's voice data and sleep data to learn and predict the baby's behavioral patterns and provide countermeasures.

[1278] System configuration

[1279] The system consists of the following main components:

[1280] Devices (baby monitors, sleep monitors):

[1281] The baby monitor collects the baby's cries and vocalizations and sends them to a server as audio data.

[1282] The sleep monitor collects the baby's movements, heart rate, and breathing patterns and sends the data to a server.

[1283] server:

[1284] Data reception and storage: Receives voice and sleep data sent from the device and stores it in a database.

[1285] Data analysis: The received data is analyzed using an AI algorithm to learn the baby's behavioral patterns.

[1286] Behavior prediction and generation of countermeasures: Behavior prediction is made based on learned behavior patterns, and countermeasures are generated based on that prediction.

[1287] Sending notification: The generated behavior prediction and how to deal with it are notified to the user device.

[1288] User device (dedicated app):

[1289] Receive and display notifications: Receive notifications sent from the server and display them in an easy-to-read format for the user.

[1290] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[1291] Specialist consultation function: Provides a function to consult with a specialist online about questions regarding childcare.

[1292] Program processing

[1293] The processing performed by the program of this system will be explained below with specific examples.

[1294] Devices (baby monitors, sleep monitors)

[1295] A baby monitor detects a baby's crying and sends the audio data to a server. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[1296] The sleep monitor detects the baby's heart rate and tossing and turning, and sends sleep data to a server, for example, when the baby goes into deep sleep or wakes up.

[1297] server

[1298] The server receives the crying data sent by the baby monitor and analyzes it, for example to determine if the crying matches a certain pattern (e.g., hungry, sleepy, etc.).

[1299] Analyze sleep data to extract the baby's deep sleep level and sleep duration. For example, analyze whether the baby is sleeping soundly in the middle of the night.

[1300] Based on the analysis results, the system predicts the baby's behavior and generates countermeasures. For example, if the baby is crying because he or she is hungry, the system generates a countermeasure such as "Please prepare milk."

[1301] The generated behavior prediction and how to deal with it are notified to the user device. For example, a notification saying "Your baby has woken up. Please try to put him / her back to sleep" is sent.

[1302] User device (dedicated app)

[1303] The dedicated app receives notifications from the server and displays them on the user interface. For example, a parent can check a notification on their smartphone saying, "The baby has woken up."

[1304] Through the community function, parents can share parenting information with other parents. For example, they can ask questions about parenting problems and receive advice from other parents in the community.

[1305] You can use the specialist consultation function to consult with a specialist online. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[1306] The above is an embodiment of the present invention. This system allows parents to understand their baby's condition in real time and take appropriate measures. As a result, the burden of childcare on parents is reduced, allowing them to focus on raising their child with peace of mind.

[1307] The processing flow will be explained below.

[1308] Step 1:

[1309] The device (baby monitor) collects the baby's cries and vocalizations. For example, if the baby cries, the cry is recognized as audio data and recorded.

[1310] Step 2:

[1311] The device (sleep monitor) collects data on the baby's movements, heart rate, and breathing patterns. For example, if the baby rolls over or their heart rate changes, it records that data.

[1312] Step 3:

[1313] The devices (baby monitors and sleep monitors) transmit collected audio and sleep data to a server in real time, using Wi-Fi, Bluetooth, or other wireless communication methods.

[1314] Step 4:

[1315] The server receives the voice and sleep data sent from the device. For example, data on a baby's crying and heart rate is sent to the server.

[1316] Step 5:

[1317] The server stores the received data in a database, where each piece of data is linked to a baby's profile.

[1318] Step 6:

[1319] The server analyzes the stored data using AI algorithms to learn the baby's behavioral patterns. For example, it analyzes crying data over a certain period of time to identify the time of day and crying patterns of the baby.

[1320] Step 7:

[1321] The server predicts the baby's future behavior based on the learned behavioral patterns. For example, it predicts the time of day when the baby will cry based on past data and prepares to notify the user of the prediction.

[1322] Step 8:

[1323] The server generates an appropriate response to the predicted behavior. For example, if a pattern of "crying because the baby is hungry" is detected, the server generates a message saying "Please prepare milk" as a response.

[1324] Step 9:

[1325] The server then notifies the user device (dedicated app) of the generated behavior predictions and countermeasures. The notification content is sent in real time.

[1326] Step 10:

[1327] The device (dedicated app) receives notifications sent from the server. For example, a notification saying "The baby has woken up" is received.

[1328] Step 11:

[1329] The device (dedicated app) displays the received notification on the user interface, and the parent checks the notification and decides what to do next.

[1330] Step 12:

[1331] The user (parent) can take appropriate action based on the notification. For example, if the notification says "Prepare milk," the parent prepares milk and gives it to the baby.

[1332] Step 13:

[1333] The device (dedicated app) provides a community function that allows users to share information with other parents, and users can exchange their parenting experiences and questions within the community.

[1334] Step 14:

[1335] Users (parents) can consult with a specialist through a dedicated app about questions and concerns about childcare, and based on the specialist's answers, they can take more specific measures.

[1336] This enables the system to analyze a baby's behavioral patterns in detail and provide appropriate childcare support to parents in real time.

[1337] Example 1

[1338] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1339] When raising a baby, parents need to constantly monitor their baby's condition and respond appropriately. However, accurately understanding the reason for a baby's crying or sleep state can be difficult and often places a burden on parents. In addition, there are limited ways to obtain information and advice on childcare from other parents or experts. To solve these problems, a system is needed that monitors a baby's condition in real time and notifies parents of the appropriate course of action.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1341] In this invention, the server includes means for collecting baby's voice data, means for collecting baby's physiological data, means for transmitting the collected voice data and physiological data to the server, means for analyzing the voice data and physiological data to learn the baby's behavioral patterns, means for generating a behavior prediction and a countermeasure based on the learned behavioral patterns, means for notifying the generated behavior prediction and countermeasure to a user terminal, means for displaying the notified behavior prediction and countermeasure on a user interface, means for providing a community function for the user to share information with others, and means for consulting with an expert within a dedicated app. This allows parents to monitor their baby's condition in real time and receive specific instructions for appropriate countermeasures. Furthermore, by sharing information with other parents and experts, parents can receive support to alleviate concerns and questions about childcare.

[1342] "Means for collecting baby voice data" refers to a device or function that records a baby's cries and vocalizations in real time and stores and transmits them in digital form.

[1343] "Means for collecting baby physiological data" means a device or function that records physiological information such as a baby's heart rate, breathing patterns, and rolling over in real time, and stores and transmits this information in digital form.

[1344] The "means for transmitting the collected voice data and physiological data to a server" refers to a communication device or function for transmitting the voice and physiological data collected by the terminal to a server via the Internet.

[1345] The "means for analyzing the voice data and physiological data and learning the baby's behavioral patterns" refers to a function that uses an AI algorithm to analyze the data received by the server and predict the baby's behavior and condition.

[1346] The "means for predicting behavior and generating a countermeasure based on the learned behavior pattern" is a function for predicting the baby's future behavior based on the analysis results and automatically generating an appropriate countermeasure.

[1347] The "means for notifying the user terminal of the generated behavior prediction and countermeasure method" is a communication device or function for transmitting the behavior prediction and countermeasure method generated by the server to the user terminal in real time.

[1348] The "means for displaying the notified predicted behavior and countermeasure method on a user interface" is an interface function for displaying the notification received by the user terminal in a form that is easy for the user to see.

[1349] "Means for providing a community function that enables the user to share information with others" refers to a bulletin board or chat function within the dedicated app that enables users to share information and concerns about child-rearing with other parents and exchange opinions.

[1350] "Means to consult with experts within the dedicated app" is a function that allows users to send questions about childcare to experts through the dedicated app and receive advice online.

[1351] This invention is a system that manages a baby's behavior in real time and notifies parents of appropriate measures. This system learns and predicts the baby's behavioral patterns by collecting and analyzing the baby's voice data and physiological data, and provides appropriate measures.

[1352] System configuration

[1353] The present invention consists of the following major components:

[1354] Devices (baby monitors, sleep monitors):

[1355] Baby monitor: Collects a baby's cries and vocalizations and sends them to a server as audio data. For example, if a baby cries in the middle of the night, the audio data is sent to the server in real time.

[1356] Sleep monitor: Collects physiological data such as the baby's movements, heart rate, breathing patterns, etc., and sends it to a server. For example, when the baby enters a deep sleep or wakes up, the data is collected and sent to the server.

[1357] server:

[1358] Data reception and storage: Receives the voice data and physiological data sent from the device and stores them in a database. For example, the server receives the baby's crying data and heart rate data and stores them in the appropriate format.

[1359] Data analysis: The received audio and physiological data is analyzed using AI algorithms. For example, the server analyzes the audio data to determine if the baby's crying matches a specific pattern (hungry, sleepy, etc.). The server also analyzes physiological data to assess the baby's sleep cycle and how soundly they are sleeping.

[1360] Behavior prediction and generation of countermeasures: Based on the analysis results, the system predicts the baby's behavior and generates appropriate countermeasures. For example, if the baby's crying is analyzed as "hungry," the system generates the countermeasure "Please prepare milk." Also, if the sleep data indicates that the baby is in a deep sleep, the system generates a notification saying "The baby is sleeping deeply. Please be quiet."

[1361] Notification sending: The server notifies the user device of the generated behavior prediction and how to deal with it. For example, it sends a notification saying, "The baby is crying. He may be hungry. Please prepare some milk."

[1362] User device (dedicated app):

[1363] Receiving and displaying notifications: The user device receives notifications from the server and displays them on the user interface. The user can check the notifications and take appropriate action. For example, the user can see a message on the smartphone app saying, "Your baby has woken up. Please try to put him / her back to sleep."

[1364] Community function: Users can use the community function to share parenting information with other parents. For example, they can ask questions about how to deal with nighttime crying and receive advice from other parents.

[1365] Specialist consultation function: Users can use the specialist consultation function of the dedicated app to consult with experts online about childcare-related questions. For example, they can send a message to a specialist saying, "My baby wakes up frequently during the night," and receive specific advice.

[1366] Specific examples

[1367] For example, if a baby starts crying in the middle of the night, the system will operate as follows:

[1368] 1. The baby monitor detects the baby's crying and sends the audio data to the server in real time.

[1369] 2. The server receives the crying data and stores it in a database. At the same time, it uses an AI algorithm to analyze the crying data and identify behavioral patterns that indicate "hungry."

[1370] 3. If the server determines that the user is hungry, it generates a solution such as "Please prepare milk" and notifies this information to the user's terminal.

[1371] 4. The user device (dedicated app) receives a notification from the server and displays a message to the parent saying, "The baby is crying. He may be hungry. Please prepare some milk."

[1372] Example input to a generative AI model

[1373] Below are some example prompts to input to the generative AI model:

[1374] User: "My baby started crying in the middle of the night. What should I do?"

[1375] AI: "Let's find out why your baby is crying. First, he may need milk, so get some ready. If he still doesn't stop crying, check to see if he has a wet diaper or is uncomfortable."

[1376] This invention allows parents to monitor their baby's condition in real time and take appropriate measures promptly. In addition, by sharing information with other parents and experts, parents can alleviate concerns and questions about childcare, allowing them to approach childcare with peace of mind.

[1377] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1378] Step 1:

[1379] Data collection from devices (baby monitors, sleep monitors)

[1380] How a baby monitor works: A baby monitor collects a baby's crying sounds in real time. Specifically, the baby monitor's microphone detects the crying sounds and converts the audio data into a digital format. The input is the baby's crying sounds, and the output is digital audio data.

[1381] How a sleep monitor works: A sleep monitor monitors a baby's heart rate, breathing patterns, and rolling over. This data is collected using digital sensors. The input is the baby's physiological movements, and the output is digital physiological data.

[1382] Step 2:

[1383] Sending data from the device to the server

[1384] How the baby monitor works: The baby monitor sends the collected audio data to a server via the Internet. Specifically, the communication module in the device sends the data. The input is digital audio data, and the output is the audio data sent to the server.

[1385] Operation of the sleep monitor: The sleep monitor also sends the collected physiological data to a server. Specifically, a communication module sends the data via the Internet. The input is digital physiological data, and the output is the physiological data sent to the server.

[1386] Step 3:

[1387] Receiving and storing data by the server

[1388] Data reception: The server receives data sent from the baby monitor or sleep monitor. Specifically, the server's communication module receives the data. The input is audio data and physiological data sent via the Internet, and the output is data stored in the server's memory.

[1389] Data Storage Operation: The received audio and physiological data is stored in a database. Specifically, the database management system stores the data in an appropriate format. The input is the received data and the output is the stored database entry.

[1390] Step 4:

[1391] Data analysis by server

[1392] Analysis of audio data: The server analyzes the audio data using an AI algorithm. Specifically, it compares the crying data with specific patterns (hungry, sleepy, etc.). The input is the stored audio data, and the output is the analysis result indicating the cause of the crying.

[1393] Physiological data analysis: The server analyzes the physiological data to assess the baby's sleep cycle and deepness. For example, it analyzes heart rate and breathing patterns to determine whether the baby is sleeping deeply. The input is the stored physiological data, and the output is the analysis result indicating the state of sleep.

[1394] Step 5:

[1395] Server predicts behavior and generates countermeasures

[1396] Generation of behavior prediction: The server predicts the baby's behavior based on the analysis results. For example, if the analysis of a baby's cry indicates that the baby is hungry, a behavior prediction is made based on that information. The input is the analysis results of the voice data and physiological data, and the output is a specific behavior prediction.

[1397] Generation of action plan: The server generates an appropriate action plan based on the behavior prediction. For example, if the predicted result is "I'm hungry," the server generates the action plan "Please prepare some milk." The input is the behavior prediction result, and the output is a specific action plan.

[1398] Step 6:

[1399] Notification from the server to the user device

[1400] Sending notification: The server notifies the user device of the generated behavior prediction and countermeasures. Specifically, the communication module sends the notification data. The input is the behavior prediction and countermeasure data, and the output is a notification to the user device.

[1401] Step 7:

[1402] Receiving and displaying notifications on user devices

[1403] Receiving notifications: The user device receives notifications from the server. The communication module receives the notification data, and the dedicated application uses that data. The input is the notification data sent from the server, and the output is the display on the user interface.

[1404] Displaying a notification: The dedicated app displays a notification on the user interface. Specifically, the app displays a message on the screen saying, "Your baby is crying. He might be hungry. Please prepare some milk." The input is the received notification data, and the output is the notification displayed to the user.

[1405] Step 8:

[1406] Additional functions for user devices (community function, specialist consultation function)

[1407] Community function: Users can share parenting information with other parents through the community function. For example, they can ask questions about specific parenting concerns and receive advice from other parents. The input volume is the questions and information entered by the user, and the output volume is the answers and shared information from other users.

[1408] Specialist consultation function: Users can use the specialist consultation function of the dedicated app to consult with an expert online about questions about childcare. For example, they can send a question such as "My baby wakes up frequently during the night" to a specialist and receive specific advice. The input is the question entered by the user, and the output is advice from the specialist.

[1409] (Application example 1)

[1410] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1411] Modern parents, especially those raising children for the first time, face a great deal of anxiety and stress. While they need to accurately understand their baby's condition and take appropriate measures, choosing the right childcare products for their baby while shopping in a physical store can be a difficult task. Furthermore, when expert advice is needed, there is a lack of means to respond quickly. There is a need to address these issues.

[1412] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1413] In this invention, the server includes means for analyzing the baby's voice data and sleep data to learn the baby's behavioral patterns, means for predicting behavior and generating countermeasures based on the learned behavioral patterns, and means for suggesting childcare products suitable for the baby's condition in a physical store. This allows parents to understand the baby's condition in real time and take appropriate measures, and also makes it possible to suggest childcare products that are optimal for the baby while shopping in the physical store.

[1414] "Baby voice data" is a digital recording of sounds made by a baby, such as crying or vocalizations.

[1415] "Baby sleep data" is a digital record of physiological information related to a baby's sleep, such as their heart rate, breathing patterns, and rolling over.

[1416] "Server" means the central system that receives, analyzes, stores, and sends notifications to user devices.

[1417] A "means for learning behavioral patterns" is an algorithm or method for analyzing collected data, identifying consistencies and changes in a baby's behavior, and learning those patterns.

[1418] The "means for predicting behavior and generating countermeasures" refers to an algorithm and method for predicting future behavior based on learned behavioral patterns and generating appropriate countermeasures for the behavior.

[1419] A "user terminal" is a mobile device such as a smartphone or tablet used by a parent or caregiver, which receives and displays notifications, provides community functions, and provides expert consultation functions.

[1420] "Means for providing community functions" are online bulletin boards and chat functions that allow parenters to share information and receive advice from each other.

[1421] "A way to consult with an expert for free" is a way to ask questions or get advice about child-rearing from an expert online for free.

[1422] The "means for suggesting childcare products in a physical store" refers to a system and method for suggesting appropriate childcare products to parents who are shopping in a store based on the real-time status of their baby.

[1423] A "generative AI model" is a supervised or unsupervised artificial intelligence model used for data analysis and behavior prediction.

[1424] A "prompt" is a textual instruction used as input to an AI model.

[1425] The following describes in detail the embodiments of the present invention. The present invention is a system that manages a baby's behavior in real time and suggests appropriate ways to deal with the baby and childcare products to parents. This system learns and predicts the baby's behavioral patterns by collecting and analyzing the baby's voice data and sleep data, and then suggests ways to deal with the baby and childcare products.

[1426] System configuration

[1427] The system consists of the following main components:

[1428] Devices (baby monitors, sleep monitors):

[1429] Baby monitors collect the baby's cries and vocalizations and send them to a server as audio data. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[1430] The sleep monitor collects the baby's heart rate and movements, and sends the data to a server as sleep data, such as when the baby falls into a deep sleep or wakes up.

[1431] server:

[1432] Data reception and storage: The voice and sleep data sent from the device is received and stored in a database using software such as a common database management system (DBMS) or cloud service.

[1433] Data analysis: The received data is analyzed using a generative AI model to learn the baby's behavioral patterns. The software used is a machine learning library such as TensorFlow or PyTorch.

[1434] Behavior prediction and generation of coping methods: Behavior prediction is made based on learned behavior patterns, and coping methods and childcare product suggestions are generated based on the prediction.

[1435] Sending notification: The generated behavior prediction and how to deal with it are notified to the user terminal.

[1436] User device (dedicated app):

[1437] Receiving and displaying notifications: Receives notifications sent from the server and displays them in the user interface.

[1438] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[1439] Expert consultation function: Provides a function to consult with experts online about questions regarding childcare.

[1440] A function to suggest childcare products in physical stores: Recommends the most suitable childcare products based on the baby's condition in real time.

[1441] Program processing

[1442] The server receives and analyzes the baby's voice and sleep data. For example, if the baby monitor detects the baby crying, the data is sent to the server. The server analyzes the data and determines whether the crying matches a certain pattern (e.g., hungry, sleepy, etc.).

[1443] The server then predicts behavior based on the behavioral patterns and generates suggested solutions and childcare items. This is done using a generative AI model. For example, if the baby is crying because it is hungry, the server predicts the solution, suggesting "prepare milk" and "a baby bottle."

[1444] The generated solutions and suggestions are sent to the user's device, which can be a smartphone, tablet, or other device, and displayed in a dedicated app. Additionally, if the childcare product suggestion function is enabled in the physical store, shelves with suitable childcare products will be displayed in the app, facilitating smooth purchasing for parents.

[1445] Specific examples

[1446] For example, when a baby starts crying in the morning, a baby monitor captures the sound. This data is sent to a server for analysis. If the cry is identified as a sign of hunger, the server sends a notification to the user's device suggesting a baby bottle as an appropriate childcare item, along with a message saying, "Prepare some milk."

[1447] An example of a prompt is, "Generate a notification message for a parent based on baby's crying sound, which indicates possible hunger." This allows users to quickly respond to their baby's needs and easily find and purchase the necessary childcare products.

[1448] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1449] Step 1:

[1450] Collects baby's audio and sleep data. The baby monitor captures the baby's crying in real time, and the sleep monitor collects sleep data such as the baby's heart rate, rolling over, and breathing patterns. The input is the crying audio data and heart rate and movement data, and the output is a digital file containing these data.

[1451] Step 2:

[1452] The device sends collected audio and sleep data to the server. The audio and sleep data sent from the baby monitor and sleep monitor are sent to the server. The input is the collected digital file, and the output is the data stored on the server.

[1453] Step 3:

[1454] The server analyzes the received audio and sleep data and learns the baby's behavioral patterns. The software used is a machine learning library such as TensorFlow or PyTorch. The input is the audio and sleep data stored on the server, and the output is a model that shows the baby's behavioral patterns.

[1455] Step 4:

[1456] The server generates behavior predictions and countermeasures based on the learned behavior patterns. A generative AI model is used for this. The input is a model showing the behavior patterns, and the output is text data containing the behavior predictions and countermeasures. Specific childcare products are also suggested.

[1457] Step 5:

[1458] The server notifies the user device of the generated behavior prediction and countermeasures using a dedicated notification API. The input is text data containing the behavior prediction and countermeasures, and the output is a push notification to the user device.

[1459] Step 6:

[1460] The user terminal displays the notified behavior prediction and countermeasures on the user interface. The input is the notification data sent from the server, and the output is a notification message displayed on the user interface.

[1461] Step 7:

[1462] Through a dedicated app, users are recommended childcare products suited to their baby's condition in a physical store. The input is the store's situation and real-time baby data, and the output is a list of recommended childcare products.

[1463] Step 8:

[1464] The system provides a community function that allows users to share information with others and a function that allows users to consult with experts online for free about child-rearing questions. The input is the user's questions and child-rearing information, and the output is answers and advice from other users and experts.

[1465] This trend will enable parents to understand their baby's condition in real time, take appropriate measures, and quickly select the necessary childcare products in physical stores.

[1466] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1467] A specific embodiment for carrying out the present invention will now be described. The present invention is a system that combines a basic system that manages a baby's behavior in real time with an emotion engine that recognizes the user's emotions and maximizes the effectiveness of childcare support.

[1468] System configuration

[1469] The system consists of the following main components:

[1470] Devices (baby monitors, sleep monitors, user devices):

[1471] The baby monitor collects the baby's cries and vocalizations and sends them to a server as audio data.

[1472] The sleep monitor collects the baby's movements, heart rate, and breathing patterns and sends the data to a server.

[1473] The user device collects the user's voice and facial expressions and sends them to the server as emotional data.

[1474] server:

[1475] Data reception and storage: Receives voice data, sleep data, and emotion data sent from the device and stores them in a database.

[1476] Data analysis: The received data is analyzed using AI algorithms to learn the baby's behavioral patterns and the user's emotional state.

[1477] Behavior prediction and response generation: Generate behavior prediction and response methods based on learned behavioral patterns and emotional states. Adjust the content of notifications and response methods based on the user's emotional state.

[1478] Sending notification: The generated behavior prediction and how to deal with it are notified to the user device.

[1479] User device (dedicated app):

[1480] Receiving and displaying notifications: Receive notifications sent from the server and display them in a user-friendly format. The notification content is adjusted based on the user's emotional state.

[1481] Community features: Provides message boards and chat features for sharing information with other users who have the same child-rearing environment.

[1482] Specialist consultation function: Provides a function to consult with a specialist online about questions regarding childcare.

[1483] Program processing

[1484] The processing performed by the program of this system will be explained below with specific examples.

[1485] Terminals (baby monitors, sleep monitors, user devices)

[1486] A baby monitor detects a baby's crying and sends the audio data to a server. For example, if a baby cries in the middle of the night, the crying data will be sent in real time.

[1487] The sleep monitor detects the baby's heart rate and tossing and turning, and sends sleep data to a server, for example, when the baby goes into deep sleep or wakes up.

[1488] The user device collects the user's voice and facial expressions in real time and transmits the emotional data to the server. For example, if the user is feeling stressed, the facial expression and tone of voice are transmitted as data.

[1489] server

[1490] The server receives data sent from the baby monitor, the sleep monitor, and the user device, such as the baby's crying data, heart rate data, and user emotion data.

[1491] The server stores the received data in a database, where each data item is linked to the baby's profile and the user's profile.

[1492] The server analyzes the stored data using AI algorithms to learn the baby's behavioral patterns and the user's emotional state. For example, by analyzing crying data and the user's emotional data over a certain period of time, it can identify the baby's crying times and crying patterns, as well as the user's stress patterns.

[1493] The server predicts the baby's future behavior based on the learned behavioral patterns and emotional state. For example, it predicts the time of day when the baby will cry based on past data and prepares to notify the user of the prediction.

[1494] The server generates and adjusts appropriate responses to predicted behaviors based on the user's emotional state. For example, if a "crying because they're hungry" pattern is detected, the server generates a message saying "Prepare milk" as a response. If the user is feeling stressed, the server may also include more detailed instructions and tips on how to relax.

[1495] The server then notifies the user device (dedicated app) of the generated behavior predictions and countermeasures. The notification content is sent in real time.

[1496] User device (dedicated app)

[1497] The dedicated app receives notifications from the server and displays them on the user interface. For example, a parent can check a notification on their smartphone saying, "The baby has woken up."

[1498] Through the community function, parents can share parenting information with other parents. For example, they can ask questions about parenting problems and receive advice from other parents in the community.

[1499] You can use the specialist consultation function to consult with a specialist online. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[1500] Specific examples

[1501] For example, if a baby starts crying in the middle of the night, the baby monitor collects audio data and sends it to a server. The server analyzes the audio data and determines that the baby is crying because it is hungry. At the same time, it analyzes emotional data sent from the user's device and recognizes that the user is tired. The server generates a notification that includes instructions such as "Prepare milk" and relaxation techniques such as "Take a slow, deep breath, then prepare milk," and sends it to a dedicated app. The parent can check the notification in the dedicated app, follow the instructions to prepare milk and feed the baby, while also reducing their own stress.

[1502] In this way, the system can not only monitor the baby's condition but also provide comprehensive childcare support that takes into account the parents' emotional state.

[1503] The processing flow will be explained below.

[1504] Step 1:

[1505] The device (baby monitor) collects the baby's cries and vocalizations. For example, if the baby cries, the cry is recorded as audio data.

[1506] Step 2:

[1507] The device (sleep monitor) collects data on the baby's movements, heart rate, and breathing patterns. For example, if the baby moves or their heart rate changes, it will detect and record that data.

[1508] Step 3:

[1509] The terminal (user device) collects the user's voice and facial expression data. For example, when a parent talks to their baby in the living room, the voice and facial expression data are recorded.

[1510] Step 4:

[1511] The devices (baby monitors, sleep monitors, user devices) transmit the collected data to the server in real time using wireless communication methods such as Wi-Fi and Bluetooth.

[1512] Step 5:

[1513] The server receives the voice data, sleep data, and emotion data sent from the device. For example, the server receives data on a baby's crying and heart rate, as well as data on the user's tone of voice and facial expression.

[1514] Step 6:

[1515] The server stores the received data in a database, where each data point is linked to the baby and user profile.

[1516] Step 7:

[1517] The server analyzes the stored data using AI algorithms. For example, it analyzes crying patterns to identify behavioral patterns such as "hungry" or "sleepy." At the same time, it analyzes the user's tone of voice and facial expressions to identify the user's emotional state (e.g., stress).

[1518] Step 8:

[1519] The server predicts the baby's future behavior based on the learned behavioral patterns. For example, it predicts the baby's tendency to wake up in the middle of the night based on past data, and prepares the next response based on the prediction results.

[1520] Step 9:

[1521] The server generates an appropriate response to the predicted behavior. For example, if the baby is crying because it is hungry, it generates a message saying, "Prepare milk." At the same time, it adjusts the response based on the user's emotional state. For example, if the user is feeling stressed, it includes detailed instructions such as, "First, take a deep breath. Then, prepare milk."

[1522] Step 10:

[1523] The server then notifies the user device (dedicated app) of the generated behavior prediction and how to respond. For example, it sends a notification saying, "Your baby has woken up and is hungry. First, take a deep breath, then prepare some milk."

[1524] Step 11:

[1525] The device (dedicated app) receives notifications sent from the server. For example, it receives a notification that "your baby has woken up and is hungry."

[1526] Step 12:

[1527] The device (dedicated app) displays the received notification on the user interface, and the parent checks the notification and decides what to do next.

[1528] Step 13:

[1529] The user (parent) can take appropriate action based on the notification. For example, they can follow the notification and take steps such as "First, take a deep breath, then prepare the milk."

[1530] Step 14:

[1531] The device (dedicated app) provides a community function that allows users to share information with other parents, allowing them to exchange parenting experiences and questions within the community.

[1532] Step 15:

[1533] Users (parents) can consult with a specialist through a dedicated app about questions and concerns about childcare, and based on the specialist's answers, they can take more specific measures.

[1534] This enables the system to perform detailed analysis of the baby's behavioral patterns and the user's emotional state, providing parents with appropriate and tailored parenting support in real time.

[1535] Example 2

[1536] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1537] Conventional childcare support systems focused on monitoring the baby's behavior and condition, but did not take into account the parent's emotional state or stress. As a result, there was a problem that the information and coping methods for baby care were not optimized for the parent's condition. This raised concerns that parents might experience excessive stress from childcare, leading to a decline in the quality of childcare.

[1538] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1539] In this invention, the server includes means for analyzing the voice data, sleep data, and user emotional data to learn the baby's behavioral patterns and the user's emotional state, means for generating baby behavior predictions and countermeasures based on the learned behavioral patterns and emotional state and adjusting them according to the user's emotional state, and means for notifying the user terminal of the generated behavior predictions and countermeasures, thereby making it possible not only to monitor the baby's condition but also to provide optimal support based on the parent's emotional state.

[1540] "Audio data" is digitized sound information such as a baby's vocalizations and cries.

[1541] "Sleep data" is sleep-related information such as a baby's rolling over, heart rate, and breathing patterns that is collected and recorded digitally.

[1542] A "server" is a computer system that receives, stores, and analyzes collected data, generates behavior predictions and countermeasures, and notifies the user terminal.

[1543] A "behavioral pattern" is a specific pattern or tendency extracted from a series of data on a baby's behavior or condition.

[1544] "Emotional state" is data that indicates the user's (parent's) psychological state and stress level, and is collected from voice and facial expressions.

[1545] A "user terminal" is a device that receives notifications from the server and displays them on a user interface, and includes smartphones and tablets.

[1546] The "community function" is a function that allows users to share information and communicate with each other.

[1547] "Means for consulting with experts on childcare" is a function that allows users to consult with experts online about questions about childcare through a dedicated app and receive advice.

[1548] "Behavior prediction" means that the server predicts the baby's future behavior and state in advance.

[1549] "Solutions" are specific steps and advice on caring for the baby that are generated by the server based on behavioral predictions.

[1550] A "user interface" refers to a screen and operating means that allows a user to check and operate notifications and data from a server via a terminal.

[1551] Specific embodiments of the present invention will be described below. The present invention is a system that combines a system for managing a baby's behavior in real time with an emotion engine that recognizes the user's emotions and maximizes the effectiveness of childcare support.

[1552] This system consists of the following main components: terminals (baby monitor, sleep monitor, user device), server, and user terminal (dedicated app).

[1553] Terminals (baby monitors, sleep monitors, user devices)

[1554] The devices include baby monitors that collect the baby's cries and vocalizations, sleep monitors that monitor the baby's movements, heart rate, and breathing patterns, and user devices that collect the user's voice and facial expressions.Specific examples of the use of the following devices are given below.

[1555] Baby monitor: General baby monitor device

[1556] Sleep monitor: General sleep monitor device

[1557] User devices: smartphones and tablets

[1558] Examples:

[1559] For example, if a baby starts crying in the middle of the night, a baby monitor will collect the sound data in real time and send it to a server. A sleep monitor will collect data on the baby's heart rate and how often the baby turns over in bed and send it to a server. A user device will collect the user's stress level from their facial expressions and voice and send it to a server as emotional data.

[1560] server

[1561] The server is a central control unit that receives, stores, and analyzes data sent from the terminals. The server has the following functions:

[1562] 1. Data Reception and Storage: The server receives the baby's voice data, sleep data, and user's emotion data and stores them in a database. The collected data is linked to the baby's and user's profiles.

[1563] 2. Data analysis: The server uses AI algorithms such as Google TensorFlow to analyze the incoming data and learn the baby's behavioral patterns and the user's emotional state.

[1564] 3. Behavior prediction and response generation: Based on the learned behavioral patterns and emotional state, the server generates optimal behavior prediction and response methods. It also adjusts the notification content according to the user's emotional state.

[1565] 4. Notification transmission: The generated behavior prediction and countermeasures are notified to the user terminal in real time.

[1566] Examples:

[1567] For example, when a baby starts crying and audio data is sent to the server, the server determines that the baby is crying because it is hungry. At the same time, emotional data indicating that the user is tired is analyzed and a notification is generated instructing the user to prepare milk. The notification also includes relaxation tips, such as "Take a slow, deep breath, and then prepare the milk."

[1568] User device (dedicated app)

[1569] The user terminal is a dedicated application for receiving notifications from the server and displaying them on the user interface. This application includes the following functions:

[1570] 1. Receiving and displaying notifications: Receives predicted behavior and countermeasures sent from the server and displays them in an easy-to-read format for the user.

[1571] 2. Community function: A function for sharing information with other users. Parenting information can be exchanged through message boards and chats.

[1572] 3. Specialist consultation function: Provides a function that allows users to consult with a specialist online about questions regarding childcare.

[1573] Examples:

[1574] For example, the dedicated app will receive a notification that the baby has woken up, and the parent can check the notification on their smartphone. They can also get advice from other parents through the community function or consult with a specialist.

[1575] Example prompts for generative AI models

[1576] "My baby started crying in the middle of the night. Please suggest appropriate ways to handle this situation. Also, include ways to support the parents if they are feeling stressed."

[1577] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1578] System program processing flow

[1579] Terminals (baby monitors, sleep monitors, user devices)

[1580] Step 1: Collect data

[1581] A baby monitor collects the baby's cries and vocalizations. For example, if a baby starts crying in the middle of the night, the built-in microphone captures the sound. The input is the baby's cry, and the output is audio data.

[1582] A sleep monitor monitors a baby's movements, heart rate, and breathing patterns. For example, a heart rate sensor measures the baby's heart rate in real time. The input is the baby's physical activity, and the output is sleep data.

[1583] The user device detects the user's voice and facial expressions. For example, the built-in camera and microphone collect the user's speaking voice and facial expressions. The input is the user's facial expressions and voice, and the output is emotional data.

[1584] Step 2: Sending data

[1585] The baby monitor transmits the collected audio data to a server, for example, using a Wi-Fi connection to transmit the audio data in real time. The collected audio data is the input, and the transmitted audio data is the output.

[1586] The sleep monitor sends the collected heart rate and sleep movement data to a server. For example, it transfers the data using Bluetooth or Wi-Fi. The collected sleep data is the input, and the sent sleep data is the output.

[1587] The user device sends the collected voice and facial expression data to a server, e.g., uploading emotional data in real time using an internet connection. The collected emotional data is the input, and the sent emotional data is the output.

[1588] server

[1589] Step 3: Receiving the data

[1590] The server receives the voice data, sleep data, and emotion data sent from the device. For example, crying data from a baby monitor, heart rate data from a sleep monitor, and emotion data from the user device are simultaneously received. Various data from the device are input, and a list of received data is output.

[1591] Step 4: Save your data

[1592] The server stores the received data in a database, for example linking it to the baby's profile and storing voice data, sleep data, and emotion data. Data received as input, data records stored as output.

[1593] Step 5: Data analysis

[1594] The server analyzes the stored data using an AI algorithm. The input is the stored data, and the output is the baby's behavioral patterns and the user's emotional state. Using Google TensorFlow, the server learns the baby's behavioral patterns based on crying data and heart rate data. The server also analyzes the user's emotional data to understand the user's emotional state.

[1595] Step 6: Predicting behavior and generating countermeasures

[1596] The server generates behavior predictions and countermeasures based on the learned behavioral patterns and emotional state. The analysis results are input, and the generated behavior predictions and countermeasures are output. For example, it predicts the next time the baby will cry and generates specific countermeasures such as preparing milk. Furthermore, if the user is tired, it generates a notification that includes relaxation techniques.

[1597] Step 7: Sending notifications

[1598] The server notifies the user device of the generated behavior prediction and the countermeasure. The generated behavior prediction and countermeasure are input, and the sent notification is output. For example, a notification that "The baby is crying" and the contents "Please prepare milk" and "Please take slow, deep breaths" are sent to a dedicated app in real time.

[1599] User device (dedicated app)

[1600] Step 8: Receiving and Viewing Notifications

[1601] The user device receives notifications from the server and displays them on the user interface. The notification sent is the input, and the notification content displayed is the output. For example, a parent sees a notification on their smartphone saying "The baby has woken up" and acts accordingly.

[1602] Step 9: Community Features

[1603] The user terminal provides a community function for sharing information with other users. For example, a user can post a question about childcare on a community bulletin board and receive advice from other users. The input is the user's question, and the output is the answer from other users.

[1604] Step 10: Specialist consultation function

[1605] The user device provides a function for online childcare consultation with experts. For example, a user can consult a specialist about a problem such as "my baby is not sleeping well" and receive specific advice. The user's consultation content is input, and the expert's advice is output.

[1606] (Application example 2)

[1607] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1608] For parents who go shopping in physical stores with their babies, it is a significant burden to properly manage their baby's condition while shopping. Furthermore, a lack of support tailored to the baby's condition and the parent's emotional state increases parental stress. Therefore, there is a need for a system that can manage the condition of both the baby and the parent in real time and provide appropriate support.

[1609] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1610] In this invention, the server includes means for collecting baby's voice data, means for collecting baby's sleep data, means for transmitting the collected voice data and sleep data to the server, means for analyzing the voice data and sleep data to learn the baby's behavioral patterns, means for predicting behavior and generating countermeasures based on the learned behavioral patterns, means for collecting parent's voice and facial expressions and transmitting them to the server as emotional data, means for analyzing the emotional data to learn the parent's emotional state, and means for adjusting the behavior predictions and countermeasures based on the learned emotional state. This makes it possible to manage the baby's condition and the parent's emotional state in real time and provide appropriate support and specific countermeasures.

[1611] "Baby voice data" refers to audio information of a baby's cries and vocalizations.

[1612] "Baby sleep data" refers to information about a baby's sleep state, such as the baby's rolling over, heart rate, and breathing patterns.

[1613] "Server" refers to a central processing unit for receiving, storing, and analyzing collected data.

[1614] "Means for learning behavioral patterns" refers to algorithms that analyze collected baby data and identify certain patterns and trends.

[1615] "Means for behavior prediction and countermeasure generation" refers to software or algorithms that predict future behavior and generate appropriate countermeasures based on learned behavioral patterns.

[1616] "User terminal" refers to a device (smartphone, tablet, etc.) used to provide notifications and advice to parents.

[1617] "Means for collecting parent's voice and facial expression" refers to devices or sensors for collecting parent's voice and facial expression data in order to understand the parent's emotional state.

[1618] "Emotional data" refers to information about the parent's emotional state obtained by analyzing their voice and facial expressions.

[1619] "Means for learning emotional states" refers to algorithms that analyze collected emotional data and identify patterns and trends in parents' emotions.

[1620] "Means for adjusting behavioral predictions and coping strategies" refers to software or algorithms that appropriately adjust a baby's behavioral predictions and coping strategies to match the parent's emotional state.

[1621] "Means to link with in-store services" refers to the ability to link with services at physical stores and have employees provide support when parents need it.

[1622] MODE FOR CARRYING OUT THE INVENTION

[1623] The present invention is described in detail below with respect to an embodiment thereof. The present system aims to reduce the burden of child-rearing by managing baby behavior data and parental emotional data in real time and providing appropriate support.

[1624] The system consists of the following main components:

[1625] Terminal

[1626] Baby monitor: A device that collects a baby's cries and vocalizations and sends them to a server as audio data. For example, if a baby starts crying in a store, the device can collect the crying sound.

[1627] Sleep monitor: A device that collects a baby's movements, heart rate, and breathing patterns and sends them to a server as sleep data. For example, it records when a baby moves or wakes up while sleeping.

[1628] User device: A device such as a smartphone or tablet that collects the parent's voice and facial expressions and sends them to a server as emotional data. For example, when a parent is feeling stressed, their facial expression and tone of voice can be recognized.

[1629] server

[1630] The server receives data sent from the baby monitor, sleep monitor, and user device and stores it in a database.

[1631] The received data is analyzed by an AI algorithm to learn the baby's behavioral patterns and the parent's emotional state. Specifically, it uses machine learning models to analyze patterns in the data.

[1632] Based on the stored data, the system predicts the baby's future behavior and generates and adjusts countermeasures based on the parent's emotional state. For example, it suggests appropriate countermeasures based on the "time of day the baby cries" and "pattern of crying."

[1633] The server notifies the user terminal of the generated behavior prediction and the countermeasure, and displays the notified content in an easy-to-read format for the parent.

[1634] User terminal

[1635] Receive notifications sent from the server and display them on the unit or application so that parents can act immediately.

[1636] Through the community function, parents can share parenting information with other parents. They can also exchange opinions using the message board and chat function.

[1637] By using the specialist consultation function, you can consult with a specialist online about your child-rearing questions. For example, you can ask a specialist about your baby's frequent nighttime wake-ups and receive advice.

[1638] It also has the ability to link with in-store services, allowing employees to provide support as needed.

[1639] Software and hardware used

[1640] OpenCV: An image processing library for face recognition and expression analysis.

[1641] Keras: A machine learning library used to implement emotion recognition models.

[1642] TensorFlow: A deep learning framework that acts as a backend for Keras.

[1643] Smartphone: A device used for parental emotion recognition and application integration.

[1644] Baby monitor: A device used to collect audio data from a baby.

[1645] Sleep monitor: A device used to collect data such as a baby's heart rate and how often they turn over.

[1646] Specific examples

[1647] For example, imagine a parent shopping in a physical store with their baby and the baby suddenly starts crying. The baby monitor detects the crying and sends the audio data to the server. At the same time, the smartphone camera analyzes the parent's facial expressions and sends the parent's emotional data to the server. The server analyzes this data, determines that the baby is crying because it is hungry, and generates a countermeasure such as "prepare milk." If the parent is feeling tired or stressed, it may also notify them of relaxation techniques such as "take a slow, deep breath, then prepare milk." The notification content is displayed in real time on the smartphone app, allowing the parent to take appropriate action. Furthermore, if support from an employee in the store is needed, they can request help from an employee through the app.

[1648] Prompt Sentence Examples

[1649] Generate specific instructions based on what kind of support the baby needs when crying, taking into account the parents' emotional data.

[1650] This allows the system to simultaneously monitor the baby's condition and the parent's emotions, providing appropriate childcare support even while shopping in a physical store.

[1651] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1652] Step 1:

[1653] Collect the baby's voice data. In this step, the baby monitor detects the baby's cries and vocalizations and collects them as voice data. Specifically, the microphone captures the sound and converts it into a digital format. The input is the baby's voice, and the output is digital voice data.

[1654] Step 2:

[1655] Collecting the baby's sleep data. In this step, the sleep monitor monitors the baby's movements, heart rate, and breathing patterns and collects this data. Specifically, sensors detect the baby's body movements and biological signals and convert them into data. The input is the baby's body movements and biological signals, and the output is digital data.

[1656] Step 3:

[1657] Collect parent's voice and facial expression data. In this step, the user device (smartphone or tablet) detects the parent's voice and facial expression and collects that data. Specifically, the camera captures the parent's face and the microphone records the voice. The input is the parent's voice and facial expression, and the output is digital emotional data.

[1658] Step 4:

[1659] The collected voice data, sleep data, and emotion data are sent to a server. Specifically, the user device uploads these data to the server via the Internet. The input is the collected data, and the output is the data stored on the server.

[1660] Step 5:

[1661] The server stores the received data in a database. Specifically, the database system classifies and stores the data correctly. The input is the transmitted data, and the output is the data stored in the database.

[1662] Step 6:

[1663] The received data is analyzed using an AI algorithm to learn the baby's behavioral patterns and the parent's emotional state. Specifically, a machine learning model (for example, a model using Keras or TensorFlow) analyzes the data. The input is various stored data, and the output is the learned behavioral patterns and emotional state.

[1664] Step 7:

[1665] The server generates behavior predictions and countermeasures based on the learned behavior patterns. Specifically, the predictive model predicts future behavior and generates appropriate countermeasures. The input is the learned behavior pattern, and the output is the predicted behavior and its countermeasures.

[1666] Step 8:

[1667] The server adjusts behavior predictions and response methods based on the learned emotional state. Specifically, it adjusts the content of response methods and notification methods according to the parent's emotional state. The input is the learned emotional state, behavior predictions, and response methods, and the output is the adjusted response methods.

[1668] Step 9:

[1669] The generated behavior prediction and countermeasures are notified to the user device. Specifically, the notification system sends information to the user's device in real time. The input is the adjusted countermeasures, and the output is a notification to the user device.

[1670] Step 10:

[1671] The notification content is displayed on the user's device to help the parent take appropriate action. Specifically, the notification content is displayed on the smartphone screen. The input is the received notification data, and the output is the content displayed to the parent. ...

Claims

1. a means for collecting baby voice data; A means of collecting your baby's sleep data, means for transmitting the collected voice data and sleep data to a server; means for analyzing the voice data and sleep data to learn the baby's behavioral patterns; a means for generating a behavior prediction and a response method based on the learned behavior pattern; means for notifying a user terminal of the generated behavior prediction and a countermeasure; a means for displaying the notified behavior prediction and response method on a user interface; means for providing a community function for the user to share information with others; A means to consult with experts for free within the dedicated app, A system including:

2. means for analyzing the audio data to identify patterns in the baby's crying; means for estimating the state of the baby, such as whether the baby is hungry or sleepy, based on the identified crying pattern; The system of claim 1 further comprising:

3. The sleep data includes the baby's rolling over, heart rate, and breathing pattern; means for analyzing the sleep data to extract the baby's deep sleep level and sleeping time; a means for providing an appropriate sleep pattern based on the deep sleep level and the sleep duration; The system of claim 1 further comprising:

Images