System

Abstract

A system is provided.SOLUTION: A system comprising: means for recording a voice of an infant; means for analyzing the recorded voice to estimate a state of the infant; means for creating a notification message based on the estimated state of the infant; means for transmitting the created notification message to a terminal of a user; and means for displaying the notification message on the terminal of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Communicating with infants between 0 months and 2 years of age is extremely difficult, making it difficult for parents and caregivers to quickly and accurately understand the infant's condition and needs. In particular, as the burden of childcare increases with the rise of first-time parents, dual-income households, and women's advancement in society, there is a need for a method to appropriately grasp an infant's condition and respond promptly. Conventional technologies do not adequately provide systems that effectively analyze an infant's crying and vital data and provide appropriate notifications to parents. Therefore, the present invention aims to provide a system that uses an infant's voice and vital data to accurately grasp an infant's condition in real time and notify parents and caregivers of the status. [Means for solving the problem]

[0005] The present invention provides a system including a means for recording an infant's voice, a means for analyzing the recorded voice to estimate the infant's condition, a means for creating a notification message based on the estimated infant condition, a means for transmitting the created notification message to a user's device, and a means for displaying the notification message on the user's device (claim 1). The system further includes a means for measuring the infant's vital signs, a means for analyzing the measured vital signs to determine the infant's detailed condition, a means for creating an emergency alert based on the determined condition, a means for transmitting the created emergency alert to the user's device, and a means for displaying the emergency alert on the user's device (claim 2). The system also includes a means for analyzing the voice data and vital signs based on a machine learning model, which is trained to identify the infant's characteristic voice patterns and physiological indicators (claim 3). This allows the system to accurately grasp the infant's condition and needs and provide appropriate notifications to parents and caregivers, thereby reducing the burden of childcare and supporting prompt response.

[0006] "Audio data" refers to digital recordings of sounds such as an infant's crying or laughter.

[0007] "Vital data" refers to data obtained by measuring physiological indicators such as an infant's heart rate and body temperature.

[0008] A "machine learning model" is an algorithm that learns patterns using large amounts of data and makes predictions and classifications for new data.

[0009] "Recording means" refers to devices or functions for recording an infant's voice.

[0010] "Analysis means" refers to algorithms or programs that process recorded audio data and vital data and draw specific conclusions.

[0011] "Estimation means" refers to the process or algorithm for estimating the infant's condition based on the analysis results.

[0012] "Notification Message" means a text or alert message that notifies parents or caregivers of estimated infant status or emergency alerts.

[0013] "Transmission means" refers to a communication protocol or device for sending a notification message to a user's terminal.

[0014] The "display means" refers to an interface or program for displaying the sent notification message on the user's terminal.

[0015] An "urgent alert" is a special notification that quickly notifies parents or caregivers of an abnormal infant condition (e.g., fever or abnormal heartbeat).

[0016] "User device" refers to a digital device such as a smartphone or tablet owned by a parent or caregiver. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] MODE FOR CARRYING OUT THE INVENTION

[0039] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), thereby enabling the user to accurately grasp the infant's condition and take prompt action. The system of the present invention includes the following various means.

[0040] 1. Collection of audio data

[0041] The device records the baby's cries, laughter, and other sounds. For example, the baby's voice can be recorded using a microphone on a smartphone or a dedicated device. Recording starts at regular intervals, and the audio data is automatically acquired.

[0042] 2. Sending audio data

[0043] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[0044] 3. Analysis of audio data

[0045] The server analyzes the received voice data and estimates the infant's condition using a machine learning model based on the infant's crying patterns and voice characteristics.

[0046] 4. State Data Generation

[0047] The server generates data on the baby's condition based on the analysis of the voice data, identifying, for example, the baby's condition such as "needs a diaper change," "sleepy," or "hungry."

[0048] 5. Creating a notification message

[0049] The server generates notification messages for parents and caregivers based on the estimated infant status in a format that is easy for parents to understand.

[0050] 6. Sending Notifications

[0051] The notification message created by the server is sent to the terminal via the Internet.

[0052] 7. Display of notifications

[0053] Displaying notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[0054] 8. Collecting vital data (optional)

[0055] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Vital data is acquired using a smart baby monitor or wearable device.

[0056] 9. Analysis of vital data

[0057] The server analyzes the received vital data and determines whether there are any abnormalities, thereby understanding the infant's detailed physiological condition.

[0058] 10. Creating Emergency Alerts

[0059] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[0060] 11. Sending emergency alerts

[0061] The server sends an emergency alert message to the terminal.

[0062] 12. Displaying emergency alerts

[0063] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[0064] Specific examples

[0065] A specific example of the operation of the system is shown below.

[0066] Example 1: A crying baby

[0067] 1. The device will record your baby's crying for 5 seconds.

[0068] 2. The device sends the recorded data to a server via the Internet.

[0069] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[0070] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[0071] 5. The server creates a notification message saying "Diaper needs changing."

[0072] 6. The server sends a notification message to the terminal.

[0073] 7. The device displays the notification message in the notification bar of the user's smartphone.

[0074] 8. The user checks the notification and changes the baby's diaper.

[0075] Example 2: Vital signs monitoring

[0076] 1. The device measures the baby's temperature using a smart baby monitor.

[0077] 2. The device sends the measurement data to a server via the Internet.

[0078] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[0079] 4. The server creates an emergency alert: "Infant has fever."

[0080] 5. The server sends an emergency alert to the device.

[0081] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[0082] 7. The user checks the emergency alert and takes necessary measures.

[0083] In this way, the system of the present invention can accurately grasp the condition and needs of the infant and assist in prompt response.

[0084] The processing flow will be explained below.

[0085] Program processing steps

[0086] Step 1:

[0087] The device records the baby's voice. The device's microphone is used to record the baby's crying and laughter for a certain period of time (e.g., 5 seconds).

[0088] Step 2:

[0089] The device stores the recorded audio data in a buffer and uploads it to a server via the Internet using a secure communication protocol such as HTTPS.

[0090] Step 3:

[0091] The server decodes the received audio data and begins analyzing it using a machine learning model (e.g., Convolutional Neural Network) to identify patterns and characteristics of crying sounds.

[0092] Step 4:

[0093] The server generates estimated data on the infant's condition based on the results of analyzing the voice data. Specifically, it creates textual data on the infant's condition, such as "needs a diaper change," "sleepy," and "hungry."

[0094] Step 5:

[0095] The server creates notification messages based on the generated status data, and the notification messages are presented in a format that is easy for parents and caregivers to understand.

[0096] Step 6:

[0097] The notification message created by the server is sent to the device via the Internet as a push notification.

[0098] Step 7:

[0099] The device decodes the notification message received and displays it in the notification bar or dedicated app on the smartphone or tablet.

[0100] Step 8:

[0101] The device measures the infant's vital data (e.g., heart rate and body temperature) using a smart baby monitor or wearable device and stores the measured data in a buffer.

[0102] Step 9:

[0103] The vital data measured by the device is uploaded to a server via the Internet.

[0104] Step 10:

[0105] The server analyzes the received vital data and detects abnormal values ​​(e.g., high or low fever) using statistical algorithms and machine learning models.

[0106] Step 11:

[0107] If the server detects abnormal data, it will create an emergency alert. For example, it will generate an alert message saying "The baby has a fever. Please take action immediately."

[0108] Step 12:

[0109] The server creates an emergency alert message and sends it to the device as a push notification via the Internet.

[0110] Step 13:

[0111] The device decodes emergency alert messages received and displays them in the notification bar and in the app with special highlighting, thereby informing the user of the urgency of the message.

[0112] Step 14:

[0113] The user checks notifications and emergency alerts and takes appropriate action (e.g., changing a diaper, checking a temperature).

[0114] Example 1

[0115] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] Rapid and accurate understanding of an infant's health condition and needs is an important challenge for parents and caregivers. However, because infants cannot communicate their condition verbally, they must rely on indirect information such as crying and vital signs. In conventional systems, voice data and vital signs are often collected and analyzed separately, making it difficult to centrally manage and comprehensively analyze this data. In addition, there are limitations to the means of quickly notifying users of analysis results, which can delay response, especially in urgent situations.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0118] In this invention, the server includes means for recording voice data of the infant, means for periodically starting and saving the recorded voice data, means for transmitting the recorded voice data to the server using a secure communication protocol, means for analyzing the received voice data based on a machine learning model to estimate the infant's condition, means for generating estimated infant condition data, means for creating a notification message that is easy for the user to understand, means for transmitting the notification message to the user's terminal using the secure communication protocol, and means for displaying the notification message on the user's terminal, thereby enabling the infant's health condition and needs to be quickly and accurately grasped and appropriate measures to be taken.

[0119] "Means for recording infant sounds" refers to devices and methods for recording sounds such as infant crying and laughter.

[0120] "Means for periodically initiating and saving recorded audio" refers to a device or method for automatically initiating audio recording at set intervals and saving the data.

[0121] "Means for transmitting recorded audio to a server using a secure communication protocol" refers to a device or method for transmitting acquired audio data to a server while maintaining information security.

[0122] "Means for analyzing received voice data based on a machine learning model and estimating the infant's condition" refers to a device or method for analyzing voice data using machine learning technology and estimating a specific infant condition (e.g., hunger, sleepiness, etc.).

[0123] The "means for generating estimated infant condition data" refers to a device or method for converting the infant condition into a data format based on the results of voice analysis.

[0124] The "means for generating user-friendly notification messages" refers to devices or methods for generating notifications based on the estimated infant condition in a format that is easy for parents or caregivers to understand.

[0125] The "means for transmitting a notification message to a user terminal using a secure communication protocol" refers to a device or method for transmitting the generated notification message to a user terminal while maintaining information security.

[0126] The "means for displaying a notification message on a user's terminal" refers to a device or method for visually displaying a received notification message on a user's terminal.

[0127] "Means for measuring infant vital data" refers to devices and methods for measuring physiological indicators such as an infant's heart rate and body temperature.

[0128] "Means for transmitting measured vital data to a server using a secure communication protocol" refers to a device or method for transmitting acquired vital data to a server while maintaining information security.

[0129] The "means for analyzing received vital data and determining whether an abnormal state exists" refers to a device or method for analyzing acquired vital data and determining whether an abnormal state exists.

[0130] The "means for generating an emergency alert when abnormal vital data is detected" refers to a device or method for generating a message to call for urgent attention when abnormal vital data is detected.

[0131] "Means for sending an emergency alert to a user's terminal" refers to a device or method for sending the generated emergency alert message to a user's terminal.

[0132] A "means for specially highlighting emergency alerts on a user's terminal" is a device or method for highlighting emergency alert messages on a user's terminal.

[0133] MODE FOR CARRYING OUT THE INVENTION

[0134] System configuration

[0135] The system of the present invention includes the following means to quickly and accurately grasp an infant's health condition and needs and support appropriate responses. The hardware used includes smartphones, tablets, dedicated devices, smart baby monitors, and wearable devices. The software uses machine learning models and Firebase Cloud Messaging (FCM).

[0136] Audio data collection

[0137] The device records sounds such as the baby's crying and laughter. Specifically, it uses the smartphone's microphone to start recording at regular intervals and automatically saves the audio data.

[0138] Sending audio data

[0139] The audio data recorded by the device is sent to the server via the Internet. The HTTPS protocol is used for transmission to ensure security. The audio data recorded is sent to the API endpoint as a POST request.

[0140] Analysis of audio data

[0141] The server analyzes the received audio data using a machine learning model that has learned patterns of infant crying and laughter, and uses this data to estimate the infant's condition.

[0142] Generate state data

[0143] The server generates data on the baby's condition based on the analysis results. For example, if the baby continues crying, it may assume that the baby needs a diaper change or is hungry. This data is saved in JSON format and the next step is reached.

[0144] Creating a notification message

[0145] The server generates an easy-to-understand notification message for parents or caregivers based on the estimated infant's condition, such as "Your baby is crying and needs a diaper change."

[0146] Sending notifications

[0147] The server creates a notification message and sends it to the device via the Internet using Firebase Cloud Messaging (FCM) or another push notification service.

[0148] Viewing notifications

[0149] Display notification messages received by the device. Displays messages in the notification bar of smartphones and tablets to notify the user.

[0150] Vital data collection (optional)

[0151] The device measures the baby's vital data (heart rate and temperature) and sends it to a server. The data is then collected via Bluetooth using a smart baby monitor or wearable device.

[0152] Analysis of vital data

[0153] The server analyzes the received vital data and determines whether there are any abnormalities. An abnormal data detection algorithm is used to detect an excessively high heart rate or abnormal body temperature.

[0154] Creating an emergency alert

[0155] If the server detects abnormal vital data, it creates an emergency alert message, such as "The baby has a fever," and treats it as a priority message.

[0156] Sending emergency alerts

[0157] The server sends an emergency alert message to the device, using a push notification service such as Firebase Cloud Messaging (FCM) to send the notification immediately.

[0158] Viewing emergency alerts

[0159] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[0160] Specific examples

[0161] Example 1: A crying baby

[0162] 1. The device will record your baby's crying for 5 seconds.

[0163] 2. The device sends the recorded data to a server via the Internet.

[0164] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[0165] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[0166] 5. The server creates a notification message saying "Diaper needs changing."

[0167] 6. The server sends a notification message to the terminal.

[0168] 7. The device displays the notification message in the notification bar of the user's smartphone.

[0169] 8. The user checks the notification and changes the baby's diaper.

[0170] Example 2: Vital signs monitoring

[0171] 1. The device measures the baby's temperature using a smart baby monitor.

[0172] 2. The device sends the measurement data to a server via the Internet.

[0173] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[0174] 4. The server creates an emergency alert: "Infant has fever."

[0175] 5. The server sends an emergency alert to the device.

[0176] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[0177] 7. The user checks the emergency alert and takes necessary measures.

[0178] In this way, the system of the present invention is able to quickly and accurately grasp the health condition and needs of an infant and support appropriate responses.

[0179] Prompt Sentence Examples

[0180] "Please analyze and let me know why my baby is crying."

[0181] "Please take the infant's temperature and issue an alert if there is anything abnormal."

[0182] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0183] System program processing flow

[0184] Step 1: Collecting audio data

[0185] The device records sounds such as the baby's crying and laughter. First, it uses the smartphone's microphone to start recording at regular intervals and saves the audio data. For example, a dedicated app can automatically start a 5-second recording and save the data on the device.

[0186] Input: Trigger to start recording

[0187] Output: Audio file (e.g., wav format)

[0188] Step 2: Sending audio data

[0189] The audio data recorded by the device is sent to the server via the Internet using the HTTPS protocol to ensure data security. Specifically, the audio file is sent as a POST request to the API endpoint.

[0190] Input: Audio file

[0191] Output: Sending status (success / failure)

[0192] Step 3: Analyzing the audio data

[0193] The server analyzes the received audio data using a machine learning model. First, the audio data is input into the machine learning model. This model has learned the patterns of infant crying and laughter and analyzes the audio features. Specifically, it extracts features such as the frequency components and volume of the crying sounds and uses them to estimate the infant's condition.

[0194] Input: Audio data

[0195] Output: Analysis results (e.g. "I want to change your diaper")

[0196] Step 4: Generate state data

[0197] The server generates data on the infant's condition based on the analysis of the voice data. For example, it identifies the infant's condition, such as "needs a diaper change," "hungry," or "sleepy." The inferred results are saved in a structured data format (e.g., JSON).

[0198] Input: Analysis results

[0199] Output: State data (JSON format)

[0200] Step 5: Create a notification message

[0201] The server creates a user-friendly notification message based on the estimated baby's condition, such as "Your baby is crying. Your diaper needs to be changed."

[0202] Input: State data

[0203] Output: Notification message

[0204] Step 6: Sending notifications

[0205] The notification message created by the server is sent to the device via the Internet using a push notification service such as Firebase Cloud Messaging (FCM).

[0206] Input: Notification message

[0207] Output: Sending status (success / failure)

[0208] Step 7: View notifications

[0209] Displaying notification messages received by the device. Visually displaying notifications to the user in the notification bar of a smartphone or tablet or within a dedicated app. For example, displaying a pop-up in the notification bar saying "Diaper needs changing."

[0210] Input: Notification message

[0211] Output: Displayed notification

[0212] Step 8: Collect vital data (optional)

[0213] The device measures the baby's vital data (heart rate and temperature) and sends it to a server. Data is collected using a smart baby monitor or wearable device and stored on the device via Bluetooth.

[0214] Input: Trigger to start measurement

[0215] Output: Vital data

[0216] Step 9: Analyze vital data

[0217] The server analyzes the received vital data and determines whether there are any abnormalities, using an anomaly detection algorithm to detect, for example, an extremely high heart rate or an abnormal body temperature.

[0218] Input: Vital data

[0219] Output: Abnormal condition (e.g. "High fever")

[0220] Step 10: Create an emergency alert

[0221] If the server detects abnormal vital data, it creates an emergency alert message, for example, a message with high urgency such as "The baby has a fever" and marks it as an emergency alert.

[0222] Input: Abnormal condition

[0223] Output: Emergency alert message

[0224] Step 11: Sending emergency alerts

[0225] The server sends an emergency alert message to the device using a push notification service such as Firebase Cloud Messaging (FCM).

[0226] Input: Emergency alert message

[0227] Output: Sending status (success / failure)

[0228] Step 12: View emergency alerts

[0229] The device displays an emergency alert to the user, highlighting it with a special icon or popup in the notification bar of the smartphone or tablet to communicate the urgency to the user. For example, the notification bar will display "Urgent: Baby has a fever!" in red.

[0230] Input: Emergency alert message

[0231] Output: Emergency notification displayed

[0232] (Application example 1)

[0233] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0234] Conventionally, monitoring of an infant's crying and physiological data and notifications based on the results have mainly been done manually by parents or caregivers, which can make it difficult to respond quickly. Furthermore, since the infant's health condition cannot be grasped in real time, there is a risk of missing signs of serious health abnormalities. The objective of this invention is to provide a system that efficiently collects and analyzes an infant's voice and vital data and promptly notifies the user.

[0235] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0236] In this invention, the server includes means for recording the infant's voice, means for analyzing the recorded voice to estimate the infant's condition, means for creating a notification message based on the estimated infant's condition, means for transmitting the created notification message to a user's terminal, means for displaying the notification message on the user's terminal, means for using a wearable device to measure the infant's vital data, means for analyzing the measured vital data to detect abnormalities, means for creating an emergency alert based on the detected abnormality, and means for transmitting the created emergency alert to the user's smart glasses or smartphone and displaying the notification, thereby enabling the infant's health condition to be understood in real time and responding promptly.

[0237] "Infant voice" refers to audio signals such as crying and laughing emitted by an infant.

[0238] "Recording" means recording and storing audio and data.

[0239] "Analysis" is the process of examining collected data and identifying meanings and patterns in it.

[0240] "Condition" refers to a specific situation or condition, such as an infant's health, emotions, or care needs.

[0241] A "notification message" is a message containing information that is sent to a user based on the infant's condition.

[0242] A "user device" is an electronic device used by a parent or caregiver, such as a smartphone or smart glasses.

[0243] "Display" means the visual presentation of information on a screen or display.

[0244] "Vital data" refers to data that indicates an infant's physiological indicators, including heart rate and body temperature.

[0245] A "wearable device" is a device worn by an infant to measure vital data.

[0246] An "abnormality" is a condition that is outside the normal range and may affect health.

[0247] An "urgent alert" is an immediate notification when a serious health or safety issue is detected.

[0248] A "generative AI model" is a machine learning algorithm that analyzes an infant's voice patterns and vital data based on training data.

[0249] A "server" is a central computer system that analyzes and processes data and sends the results to the user's terminal.

[0250] The system of the present invention efficiently collects and analyzes an infant's voice and vital data, and notifies the user to grasp the infant's condition in real time and support prompt response. A specific embodiment of this system is shown below.

[0251] System Configuration

[0252] The system mainly consists of the following components:

[0253] 1. Device: This refers to a smartphone or smart glasses equipped with a microphone and a display, capable of displaying audio data and notifications.

[0254] 2. Server: Plays the central role in data analysis, analyzing voice and vital data using generative AI models.

[0255] 3. Wearable devices: Smart bands that infants can wear to measure vital data such as heart rate and body temperature.

[0256] 4. Generative AI model: A machine learning algorithm that analyzes voice patterns and infant physiological indicators based on training data.

[0257] Hardware and Software Use

[0258] Terminal

[0259] The terminal records voice data and transmits it to the server. It also transmits vital data acquired from the wearable device to the server. It has the function of displaying notification messages and emergency alerts when they are generated.

[0260] server

[0261] The server receives the voice and vital data sent from the device and analyzes them using a generative AI model. Based on the analysis results, it estimates the infant's condition and creates notification messages and emergency alerts.

[0262] Wearable devices

[0263] Wearable devices are used to measure vital data such as the baby's heart rate and body temperature, and the measured data is sent to a terminal via communication means such as Bluetooth.

[0264] Data processing and calculation

[0265] 1. Audio data collection:

[0266] The device records the baby's crying, laughter, and other sounds, for example, five seconds of audio, and saves it as digital data.

[0267] 2. Sending audio data:

[0268] The device sends the recorded audio data to a server via the Internet using the HTTPS protocol for secure data transfer.

[0269] 3. Analysis of audio data:

[0270] The server analyzes the received voice data using a generative AI model, capturing the characteristics of the voice pattern and inferring what the baby needs.

[0271] 4. Create and send a notification message:

[0272] Based on the estimated baby's condition, the server generates a notification message in a format that can be quickly communicated to the user, and the generated message is then sent to the terminal via the Internet.

[0273] 5. Displaying notification messages:

[0274] The device will then display the received notification message to the user: on a smartphone, it will appear in the notification bar, and on the display of smart glasses.

[0275] 6. Vital Data Collection:

[0276] The wearable device measures the infant's heart rate and temperature and transmits the data to a terminal.

[0277] 7. Analysis of vital data:

[0278] The server analyzes the received vital data and creates an emergency alert if it detects any abnormalities.

[0279] 8. Creating and sending emergency alerts:

[0280] When an abnormality is detected, the server quickly creates an emergency alert and sends it to the device.

[0281] 9. Emergency alert display:

[0282] The device will highlight the emergency alert and notify the user.

[0283] Specific examples

[0284] Example 1: A crying baby

[0285] The device records the baby's crying for 5 seconds and sends it to the server.

[0286] The server analyzes the voice data and determines that the diaper needs to be changed.

[0287] A notification message "Diaper change required" is generated and sent to the user's terminal.

[0288] The device will notify the user by displaying a message in the notification bar or on the display of the smart glasses.

[0289] Example 2: Vital signs monitoring

[0290] The wearable device measures the infant's temperature and transmits it to a terminal.

[0291] The server analyzes the body temperature data and detects high fever.

[0292] An emergency alert "Infant has fever" is generated and sent to the user's device.

[0293] The device will highlight emergency alerts and notify the user quickly.

[0294] Example prompt sentence:

[0295] "Analyzing audio data of an infant crying and identifying the cause."

[0296] "Analyze infant heart rate data to detect abnormalities."

[0297] In this way, the system of the present invention can grasp the condition of the infant in real time and promote quick response.

[0298] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0299] Step 1:

[0300] The device records sounds such as the baby's crying and laughter. Specifically, it uses the device's built-in microphone to record audio data for a specified period of time (e.g., 5 seconds). The input here is a real-time audio signal, and the output is digital audio data. The recorded audio data is temporarily stored in a buffer.

[0301] Step 2:

[0302] The device sends the recorded audio data to the server via the Internet. The HTTPS protocol is used for communication to ensure data security. The input here is the recorded audio data, and the output is the data to be sent to the server. Once the transmission is complete, the device receives a confirmation message.

[0303] Step 3:

[0304] The server inputs the received voice data into a generative AI model for analysis. The generative AI model is pre-trained and analyzes the voice features to estimate the infant's state. The input here is the voice data sent to the server, and the output is the infant's estimated state (e.g., "I want to change my diaper" or "I'm hungry"). The AI ​​model analyzes the voice patterns and probabilistically estimates the most appropriate state.

[0305] Step 4:

[0306] The server creates a notification message based on the estimated infant's state. For example, if the AI ​​model estimates that the infant needs a diaper change, it generates a notification message saying, "Your diaper needs to be changed." The input here is the estimated infant's state, and the output is the generated notification message. The server adds this to a message queue.

[0307] Step 5:

[0308] The server then sends the created notification message to the user's terminal, where the input is the notification message and the output is the data sent to the user's terminal. The notification message is securely transmitted over the Internet.

[0309] Step 6:

[0310] The device displays the received notification message. In the case of a smartphone, this is displayed in the notification bar, and in the case of smart glasses, it is visually displayed on the display. The input here is the notification message sent from the server, and the output is the displayed message. The user checks the notification and responds.

[0311] Step 7:

[0312] The wearable device measures the infant's vital data (heart rate and body temperature). The input is the infant's physiological data, and the output is the measured vital data. These data are measured periodically and sent to the terminal.

[0313] Step 8:

[0314] The device sends the received vital data to a server via the Internet. The input here is the measured vital data, and the output is the data to be sent to the server. Data transmission is via Bluetooth or Wi-Fi.

[0315] Step 9:

[0316] The server analyzes the received vital data and detects abnormalities. The generative AI model identifies abnormal patterns and detects abnormalities such as high fever or abnormal heart rate. The input here is the vital data, and the output is the detected abnormality and detailed information about it.

[0317] Step 10:

[0318] The server generates an emergency alert based on the detected anomaly, for example, a message saying "The baby has a fever." Here, the input is the anomaly detection result, and the output is the emergency alert message. The alert is treated as a high-priority message.

[0319] Step 11:

[0320] The server sends the created emergency alert to the user's device. The input here is the emergency alert message, and the output is the data to be sent to the user's device. The server uses a message queue to send the alert immediately.

[0321] Step 12:

[0322] The device highlights the received emergency alert. In the case of a smartphone, it is displayed in the notification bar with an emergency icon, and in the case of smart glasses, it is displayed with a special effect on the display. Here, the input is the emergency alert sent from the server, and the output is a visually highlighted message. The user can check the alert and take necessary measures.

[0323] Example prompt sentence:

[0324] "Analyzing audio data of an infant crying and identifying the cause."

[0325] "Analyze infant heart rate data to detect abnormalities."

[0326] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0327] MODE FOR CARRYING OUT THE INVENTION

[0328] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), enabling the user to accurately grasp the infant's condition and respond promptly. The system of the present invention includes the following means:

[0329] 1. Collection of audio data

[0330] The device records the baby's crying, laughter, and other sounds. For example, the baby's voice can be recorded using a microphone on a smartphone or a dedicated device. Recording begins at regular intervals, and the audio data is automatically acquired.

[0331] 2. Sending audio data

[0332] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[0333] 3. Analysis of audio data

[0334] The server analyzes the received voice data and estimates the infant's condition using a machine learning model based on the infant's crying patterns and voice characteristics.

[0335] 4. State Data Generation

[0336] The server generates data on the baby's condition based on the analysis of the voice data, identifying, for example, whether the baby needs a diaper change, is sleepy, or is hungry.

[0337] 5. Creating a notification message

[0338] The server generates notification messages for parents and caregivers based on the estimated infant status in a format that is easy for parents to understand.

[0339] 6. Sending Notifications

[0340] The notification message created by the server is sent to the terminal via the Internet.

[0341] 7. Display of notifications

[0342] Displaying notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[0343] 8. Collecting vital data (optional)

[0344] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Vital data is acquired using a smart baby monitor or wearable device.

[0345] 9. Analysis of vital data

[0346] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[0347] 10. Creating Emergency Alerts

[0348] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[0349] 11. Sending emergency alerts

[0350] The server sends an emergency alert message to the terminal.

[0351] 12. Displaying emergency alerts

[0352] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[0353] Furthermore, the present invention includes the following features:

[0354] 13. Use of Emotion Engines

[0355] The device uses an emotion engine to recognize the user's emotions, for example by capturing the user's facial expressions with a camera and analyzing the data to identify emotions.

[0356] 14. Transmission of Emotional Data

[0357] The user's emotion data recognized by the device is sent to a server via the Internet.

[0358] 15. Emotional Data Analysis

[0359] The server analyzes the received emotion data and determines the user's emotional state, for example, whether the user is feeling stressed.

[0360] 16. Notification Coordination

[0361] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened.

[0362] Specific examples

[0363] Example 1: A crying baby

[0364] 1. The device will record your baby's crying for 5 seconds.

[0365] 2. The device sends the recorded data to a server via the Internet.

[0366] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[0367] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[0368] 5. The server creates a notification message saying "Diaper needs changing."

[0369] 6. The server sends a notification message to the terminal.

[0370] 7. The device displays the notification message in the notification bar of the user's smartphone.

[0371] 8. The user checks the notification and changes the baby's diaper.

[0372] Example 2: Vital signs monitoring

[0373] 1. The device measures the baby's temperature using a smart baby monitor.

[0374] 2. The device sends the measurement data to a server via the Internet.

[0375] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[0376] 4. The server creates an emergency alert: "Infant has fever."

[0377] 5. The server sends an emergency alert to the device.

[0378] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[0379] 7. The user checks the emergency alert and takes necessary measures.

[0380] Example 3: Using the Emotion Engine

[0381] 1. The device uses a camera to capture the user's facial expressions.

[0382] 2. The device sends the captured image data to a server via the Internet.

[0383] 3. The server uses an emotion engine to analyze the image data and determine the user's emotional state.

[0384] 4. If the server determines that the user is feeling stressed, it creates a notification message with careful consideration of its content and expression.

[0385] 5. The server sends the adjusted notification message to the terminal.

[0386] 6. The device will display adjusted notification messages, making notifications more appropriate for users.

[0387] In this way, the multi-functional system can accurately grasp the infant's condition and provide notifications and alerts that take into account the user's emotional state.

[0388] The processing flow will be explained below.

[0389] Program processing steps

[0390] Baby crying analysis

[0391] Step 1:

[0392] The device records the baby's voice. The device's microphone is used to record the baby's crying and laughter for a certain period of time (e.g., 5 seconds).

[0393] Step 2:

[0394] The device stores the recorded audio data in a buffer and uploads it to a server via the Internet using a secure communication protocol such as HTTPS.

[0395] Step 3:

[0396] The server decodes the received audio data and analyzes it using a machine learning model (e.g., Convolutional Neural Network) to identify patterns and characteristics of crying sounds.

[0397] Step 4:

[0398] The server generates estimated data on the infant's condition based on the results of analyzing the voice data, specifically identifying the infant's condition, such as "needs a diaper change," "sleepy," or "hungry."

[0399] Step 5:

[0400] The server generates notification messages based on the generated status data in a format that is easy for parents and caregivers to understand.

[0401] Step 6:

[0402] The notification message created by the server is sent to the device via the Internet as a push notification.

[0403] Step 7:

[0404] The device decodes the notification message received and displays it in the notification bar or dedicated app on the smartphone or tablet.

[0405] Vital data monitoring and emergency alerts

[0406] Step 8:

[0407] The device measures the infant's vital data (e.g., heart rate and body temperature) using a smart baby monitor or wearable device and stores the measured data in a buffer.

[0408] Step 9:

[0409] The vital data measured by the device is uploaded to a server via the Internet.

[0410] Step 10:

[0411] The server analyzes the received vital data and detects abnormal values ​​(e.g., high or low fever) using statistical algorithms and machine learning models.

[0412] Step 11:

[0413] If the server detects abnormal data, it will create an emergency alert, e.g., "The baby has a fever. Please take action immediately."

[0414] Step 12:

[0415] The server creates an emergency alert message and sends it to the device as a push notification via the Internet.

[0416] Step 13:

[0417] The device decodes emergency alert messages received and displays them in the notification bar and in the app with special highlighting, thereby informing the user of the urgency of the message.

[0418] Use of emotion engine

[0419] Step 14:

[0420] The device captures the user's facial expressions using a camera, for example, the front camera on a smartphone or tablet.

[0421] Step 15:

[0422] The image data captured by the device is uploaded to a server via the Internet.

[0423] Step 16:

[0424] The server uses an emotion engine to analyze the image data and determine the user's emotional state, such as whether they are "stressed" or "relaxed."

[0425] Step 17:

[0426] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. If the user is feeling stressed, the content and wording of the notification will be softened.

[0427] Step 18:

[0428] The server sends the adjusted notification message to the terminal via the Internet.

[0429] Step 19:

[0430] The device decodes the adjusted notification message and displays it in the notification bar or within a dedicated app on the smartphone or tablet, making notifications more appropriate for the user.

[0431] Example 2

[0432] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0433] In infant care, it is necessary to maintain the health and comfort of infants by effectively collecting and analyzing their voice and vital data and notifying parents and caregivers in a timely manner. In particular, it is necessary to improve the accuracy of analyzing infant voices and vital data and issue immediate alerts in the event of an emergency. It is also important to adjust notifications and alerts to take into account the caregiver's emotional state. However, conventional systems have difficulty meeting all of these requirements, making it difficult to accurately monitor infant conditions and respond promptly.

[0434] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for recording the infant's voice, a means for analyzing the recorded voice to estimate the infant's condition, a means for creating a notification message based on the estimated infant's condition, a means for sending the created notification message to a user's terminal, a means for displaying the notification message on the user's terminal, a means for recognizing the user's emotion, and a means for analyzing the recognized user's emotion data and adjusting the content and expression of the notification message. This makes it possible to accurately monitor the infant's health and comfort and notify parents or caregivers at appropriate times. Furthermore, it is possible to provide flexible notification messages according to the user's emotional state, thereby realizing more effective child-rearing support.

[0435] "Means for recording infant sounds" refers to an apparatus or device for recording sounds such as an infant's crying or laughing.

[0436] "Means for estimating an infant's condition by analyzing recorded voice" refers to technologies and algorithms that analyze collected voice data and infer an infant's condition based on voice patterns and characteristics.

[0437] A "means for generating a notification message" is a device or program for generating a message in a format that is easy for a parent or caregiver to understand based on the estimated infant condition.

[0438] The "means for transmitting the created notification message to the user's terminal" refers to a device or technology for transmitting the notification message to the user's terminal via a communication means such as the Internet.

[0439] The "means for displaying a notification message on a user's device" refers to a function or application for displaying a received notification message on a user's device such as a smartphone or tablet.

[0440] "Means for recognizing user emotions" refers to technologies and algorithms that collect a user's facial expressions and other data to identify their emotional state.

[0441] "Means for analyzing the recognized user emotional data and adjusting the content and expression of the notification message" refers to technologies and algorithms for analyzing collected emotional data and appropriately changing the content and expression of the notification message depending on the user's emotional state.

[0442] "Means for measuring infant vital data" refers to equipment or devices for obtaining physiological data such as an infant's heart rate and body temperature.

[0443] "Means for analyzing the measured vital data to determine the detailed condition of the infant" refers to technologies and algorithms for analyzing the collected vital data and determining the infant's health condition or abnormalities based on the results.

[0444] The "means for generating an emergency alert" is a device or program that immediately generates a highly urgent message when an abnormality is detected in the analyzed data.

[0445] The "means for transmitting the created emergency alert to the user's terminal" refers to a technique or device for transmitting the emergency alert to the user's terminal via a communication means such as the Internet.

[0446] "Means for displaying emergency alerts on a user's device" refers to a function or application for highlighting received emergency alerts on a user's device such as a smartphone or tablet.

[0447] A "machine learning model" is an algorithm or program that is trained to learn features from the voice and vital data being analyzed and to classify or predict the infant's condition.

[0448] "Distinctive infant vocal patterns" are vocal characteristics or patterns that indicate a particular state, such as an infant's crying or laughter.

[0449] "Physiological indicators" are physiological data that indicate the health or condition of an infant, such as heart rate, body temperature, and respiratory rate.

[0450] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), enabling the user to accurately grasp the infant's condition and take prompt action. The system of the present invention includes the following means.

[0451] Audio data processing

[0452] Audio data collection:

[0453] The device records sounds such as the baby's crying and laughter. The hardware used includes a microphone on a smartphone or a dedicated device. The device starts recording at regular intervals and automatically acquires audio data. For example, a 5-second recording is performed.

[0454] Sending audio data:

[0455] The device sends the recorded audio data to a server via the internet using a secure communication method such as the HTTPS protocol.

[0456] Analysis of audio data:

[0457] The server analyzes the received voice data using a machine learning model. For example, the voice data is input into a generative AI model, and the model analyzes the baby's crying patterns and voice characteristics using the prompt "Please estimate the baby's condition as indicated by this voice data."

[0458] Generate state data:

[0459] The server generates data on the baby's condition based on the analysis of the voice data, specifically identifying the baby's condition, such as "needs a diaper change," "sleepy," or "hungry."

[0460] Create a notification message:

[0461] The server generates a notification message for parents or caregivers based on the estimated infant condition. The notification message is generated in a format that is easy for parents to understand. For example, it generates a message saying, "Your diaper needs to be changed."

[0462] Send notifications:

[0463] The notification message created by the server is sent to the terminal via the Internet.

[0464] Displaying notifications:

[0465] Display notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[0466] Vital data processing (optional)

[0467] Vital data collection:

[0468] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Smart baby monitors and wearable devices are used to collect vital data.

[0469] Vital data analysis:

[0470] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[0471] Creating an emergency alert:

[0472] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[0473] Sending emergency alerts:

[0474] The server sends an emergency alert message to the terminal.

[0475] Viewing emergency alerts:

[0476] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[0477] Use of emotion engine

[0478] Using the Emotion Engine:

[0479] To recognize the user's emotions, the device captures the user's facial expressions with a camera and analyzes the data to identify the user's emotions.

[0480] Sending Emotion Data:

[0481] The user's emotion data recognized by the device is sent to a server via the Internet.

[0482] Emotional Data Analysis:

[0483] The server analyzes the received emotion data and determines the user's emotional state, for example, whether the user is feeling stressed.

[0484] Notification adjustments:

[0485] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened.

[0486] Specific examples

[0487] Example 1: A crying baby

[0488] 1. The device will record your baby's crying for 5 seconds.

[0489] 2. The device sends the recorded data to a server via the Internet.

[0490] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[0491] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[0492] 5. The server creates a notification message saying "Diaper needs changing."

[0493] 6. The server sends a notification message to the terminal.

[0494] 7. The device displays the notification message in the notification bar of the user's smartphone.

[0495] Example 2: Vital signs monitoring

[0496] 1. The device measures the baby's temperature using a smart baby monitor.

[0497] 2. The device sends the measurement data to a server via the Internet.

[0498] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[0499] 4. The server creates an emergency alert: "Infant has fever."

[0500] 5. The server sends an emergency alert to the device.

[0501] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[0502] 7. The user checks the emergency alert and takes necessary measures.

[0503] Example 3: Using the Emotion Engine

[0504] 1. The device uses a camera to capture the user's facial expressions.

[0505] 2. The device sends the captured image data to a server via the Internet.

[0506] 3. The server uses an emotion engine to analyze the image data and determine the user's emotional state.

[0507] 4. If the server determines that the user is feeling stressed, it creates a notification message with careful consideration of its content and expression.

[0508] 5. The server sends the adjusted notification message to the terminal.

[0509] 6. The terminal displays an adjusted notification message, and the notification to the user is more appropriate.

[0510] Prompt Sentence Examples

[0511] "Analyze a 5-second recording of an infant's cry and use the cry to estimate whether the baby needs a diaper change."

[0512] This multi-functional system allows for accurate understanding of the infant's condition and prompt response, and can also analyze and provide notifications and alerts that take into account the user's emotional state, enabling more effective care.

[0513] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0514] Step 1:

[0515] Audio data collection

[0516] The device records the baby's crying, laughter, and other sounds. Specifically, it uses a smartphone microphone to record 5 seconds of audio. The input is the baby's voice, and the output is the recorded audio data.

[0517] Step 2:

[0518] Sending audio data

[0519] The device sends recorded audio data to the server using the HTTPS protocol. The input is the recorded audio data, and the output is the audio data sent to the server.

[0520] Step 3:

[0521] Analysis of audio data

[0522] The server analyzes the received audio data. Specifically, it uses a generative AI model to analyze the audio data and extract patterns and characteristics of the infant's crying. The analysis is performed using the prompt "Please estimate the infant's condition indicated by this audio data." The input is the audio data sent to the server, and the output is the analysis result of the crying.

[0523] Step 4:

[0524] Generate state data

[0525] The server generates data on the infant's condition based on the analysis of the voice data. Specifically, it identifies the infant's condition, such as "needs a diaper change," "sleepy," or "hungry." The input is the analysis result of the crying, and the output is the infant's condition data.

[0526] Step 5:

[0527] Creating a notification message

[0528] The server generates a notification message for parents or caregivers based on the generated infant status data. Specifically, it generates a message such as "diaper change required." The input is the infant status data, and the output is the notification message.

[0529] Step 6:

[0530] Sending notifications

[0531] The notification message created by the server is sent to the terminal via the Internet. The input is the notification message, and the output is the notification message sent to the terminal.

[0532] Step 7:

[0533] Viewing notifications

[0534] The notification message received by the device is displayed to the user. Specifically, the message is displayed in the notification bar of a smartphone or tablet or within a dedicated app. The input is the notification message sent to the device, and the output is the notification message displayed to the user.

[0535] Step 8:

[0536] Vital data collection (optional)

[0537] The terminal measures the infant's vital data (heart rate and body temperature) and sends it to the server. A smart baby monitor or wearable device is used. The input is the infant's vital data, and the output is the measured vital data.

[0538] Step 9:

[0539] Sending vital data

[0540] The vital data collected by the device is sent to a server via the Internet. The input is the measured vital data, and the output is the vital data sent to the server.

[0541] Step 10:

[0542] Analysis of vital data

[0543] The server analyzes the received vital data and determines whether there is an abnormality. Specifically, it uses statistical algorithms and machine learning models. The input is the vital data sent to the server, and the output is the analysis result of the abnormality.

[0544] Step 11:

[0545] Creating an emergency alert

[0546] If the server detects abnormal vital data, it creates an emergency alert. Specifically, it generates a highly urgent message such as "The baby has a fever." The input is the analysis result of the abnormal condition, and the output is an emergency alert message.

[0547] Step 12:

[0548] Sending emergency alerts

[0549] The server sends an emergency alert message to the terminal, where the input is the emergency alert message and the output is the emergency alert message sent to the terminal.

[0550] Step 13:

[0551] Viewing emergency alerts

[0552] The device displays an emergency alert to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user. The input is the emergency alert message sent to the device, and the output is the emergency alert message displayed to the user.

[0553] Step 14:

[0554] Use of emotion engine

[0555] To recognize the user's emotions, the device captures the user's facial expressions with a camera and analyzes the data to identify the user's emotions. The input is the user's facial expression data, and the output is the recognized emotion data.

[0556] Step 15:

[0557] Sending emotional data

[0558] The device recognizes the user's emotion data and sends it to a server via the Internet. The input is emotion data, and the output is the emotion data sent to the server.

[0559] Step 16:

[0560] Emotional Data Analysis

[0561] The server analyzes the received emotion data and determines the user's emotional state. Specifically, the analysis is performed using an emotion engine. The input is the emotion data sent to the server, and the output is the analysis result of the emotional state.

[0562] Step 17:

[0563] Notification adjustments

[0564] Based on the analysis results, the server creates notification messages and alerts in a format appropriate for the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened. The input is the analysis result of the emotional state, and the output is the adjusted notification message.

[0565] This is the flow of processing for this system. In this way, the condition of the infant can be accurately monitored and notifications and alerts can be provided to the user at the appropriate time.

[0566] (Application example 2)

[0567] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0568] This invention relates to a system that quickly and accurately monitors an infant's condition. In particular, it collects and analyzes an infant's crying and vital signs, and provides appropriate notifications to parents and caregivers, thereby reducing the burden of managing and caring for the infant's health and enabling prompt responses. It also aims to improve the user experience by taking into account the user's emotional state and providing more appropriate notification methods when the user is feeling stressed.

[0569] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0570] In this invention, the server includes means for recording the infant's voice, means for analyzing the recorded voice to estimate the infant's condition, means for creating a notification message based on the estimated infant's condition, means for sending the created notification message to a user's terminal, means for displaying the notification message on the user's terminal, a machine learning model for analyzing the collected infant's voice data, an emotion engine for analyzing the user's emotional state, means for adjusting the content of the notification message based on the analysis results of the emotion engine, means for collecting the infant's vital data and detecting abnormalities, and means for generating and sending an emergency alert when an abnormality is detected. This enables a detailed understanding of the infant's condition based on the infant's voice and vital data, and appropriate notifications corresponding to the user's emotional state.

[0571] "Infant" refers to a young child who is not yet able to use language or clearly express his or her own will.

[0572] "Audio data" refers to information recorded as sound, such as an infant's crying or laughter.

[0573] "Vital data" refers to data that indicates an infant's physiological condition, such as heart rate, body temperature, and respiratory rate.

[0574] "Machine learning model" means a computational model based on algorithms that analyze data and learn patterns and trends.

[0575] The "emotion engine" is a system that analyzes a user's facial expressions captured by a camera and identifies their emotional state.

[0576] "Recording" refers to saving sounds such as an infant's crying or laughter using a recording device.

[0577] "Analysis" refers to processing and analyzing the recorded voice data and vital data, and deriving the infant's condition and characteristics from that information.

[0578] "Estimation" refers to determining the infant's condition based on the recorded data.

[0579] "Notification message" refers to message information sent to parents or caregivers based on the analysis results.

[0580] An "urgent alert" refers to a highly urgent message that is sent to the user immediately when an abnormality is detected in the infant's vital data.

[0581] "Device" refers to a device that receives and displays notification messages, such as a smartphone or tablet.

[0582] "User" refers to the parent or caregiver of the infant.

[0583] "Collection" refers to using a device to collect an infant's voice and vital data.

[0584] "Abnormal" refers to vital data or an infant's condition that is outside the normal range.

[0585] "Capture" refers to using a camera to obtain an image of a user's facial expression.

[0586] "Generation" means creating notification messages or emergency alerts based on the analysis results.

[0587] "Sending" refers to sending the generated notification message or emergency alert to the user's device via the Internet.

[0588] "Display" refers to showing the sent notification message or emergency alert on the device screen.

[0589] The present invention provides a system that collects and analyzes an infant's voice and vital data and notifies the results to users (parents and caregivers). The system of the present invention is implemented using the following hardware and software.

[0590] Hardware and Software Use

[0591] Devices: Smartphones, tablets, smart baby monitors, etc. are used to collect voice and vital data.

[0592] Server: A server for data processing and analysis, where machine learning models and emotion engines are installed.

[0593] Software: TensorFlow (implementation of machine learning models), OpenCV (image processing), requests (HTTP request library), etc.

[0594] Detailed Description of the Invention

[0595] 1. Collection of audio data

[0596] The device records the baby's cries, laughter, and other sounds. The recording device is a microphone on a smartphone or a dedicated device.

[0597] Audio data is automatically recorded at regular intervals.

[0598] 2. Sending audio data

[0599] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[0600] 3. Analysis of audio data

[0601] The server analyzes the received voice data using machine learning models such as TensorFlow.

[0602] The machine learning model is trained to identify characteristic speech patterns of infants.

[0603] 4. Creating and sending notification messages

[0604] The server creates a notification message based on the analysis results.

[0605] The created notification message is sent to the terminal via the Internet.

[0606] The device will display a notification message to the user, either in the notification bar of the smartphone or tablet or within a dedicated app.

[0607] 5. Vital Data Collection and Analysis

[0608] The device collects the baby's vital data (heart rate and temperature), which is collected using a smart baby monitor or wearable device.

[0609] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[0610] 6. Creating and sending emergency alerts

[0611] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[0612] Emergency alerts are sent to your device via the internet.

[0613] The device will highlight the emergency alert to inform the user of the urgency.

[0614] 7. Use of Emotion Engines

[0615] The device uses a camera to capture the user's facial expressions.

[0616] The captured image data is sent to a server via the Internet.

[0617] The server uses an emotion engine to analyze image data and determine the user's emotional state. The analysis uses image processing libraries such as OpenCV.

[0618] If the user is feeling stressed, the content and expression of the notification message are softened and sent.

[0619] Specific examples

[0620] 1. Collection and analysis of audio data

[0621] For example, the device records a baby's crying for five seconds and sends the recording to a server via the internet. The machine learning model analyzes the audio data and generates status data such as "I want to change his / her diaper."

[0622] Example text:

[0623] Please explain the process of recording a baby crying.

[0624] 2. Collection and analysis of vital data

[0625] For example, a device measures an infant's temperature using a smart baby monitor and transmits the measurement data via the Internet to a server, which analyzes the vital data and detects abnormal body temperature (e.g., high fever).

[0626] Example text:

[0627] How can I use a smart baby monitor to collect my newborn's vital data (heart rate, temperature) and notify me if any abnormalities are detected?

[0628] 3. Use of Emotion Engine

[0629] For example, the device uses a camera to capture the user's facial expressions and sends the image data to a server via the Internet. The server then analyzes the image data using an emotion engine, and if it determines that the user is feeling stressed, it softens the tone of the notification message and sends it.

[0630] Example text:

[0631] Describe the process of detecting a parent's emotional state and adjusting notification messages.

[0632] In this way, the multi-functional system can accurately grasp the infant's condition and provide notifications and alerts that take into account the user's emotional state.

[0633] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0634] Step 1:

[0635] The device records the baby's voice data. The baby's cries and laughter are recorded using a microphone on a smartphone or dedicated device and saved as audio data. The input is the baby's cries and laughter, and the output is the saved audio data. Specifically, the device works by pressing the record button to record audio for a set period of time (for example, 5 seconds), and then saving the audio data in the device.

[0636] Step 2:

[0637] The audio data recorded by the device is sent to a server via the Internet. The audio data is uploaded to the server using a secure communication method such as the HTTPS protocol. The input is the audio data stored on the device, and the output is the audio data uploaded to the server. Specifically, after recording is complete, the data transmission module is automatically activated and the audio data is sent to the server.

[0638] Step 3:

[0639] The server analyzes the received voice data. The server processes the voice data using a machine learning model to identify the characteristics and patterns of the infant's voice. The input is the voice data received by the server, and the output is the infant's estimated state (e.g., "I want my diaper changed" or "I'm sleepy"). Specifically, the voice data is preprocessed, input into the machine learning model, and analyzed, and the infant's state is estimated as a result.

[0640] Step 4:

[0641] The server creates a notification message based on the estimated infant condition. Based on the analysis results, the notification message is generated in a format that is easy for parents and caregivers to understand. The input is the estimated infant condition, and the output is the created notification message. Specifically, the server retrieves an appropriate message template from the database, embeds the estimated condition, and generates the notification message.

[0642] Step 5:

[0643] The server sends the created notification message to the terminal via the Internet. The input is the created notification message, and the output is the notification message sent to the terminal. Specifically, the server uses a communication module to send the notification message to the terminal's address.

[0644] Step 6:

[0645] Displays notification messages received by the device. Notification messages are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app. The input is the notification message sent from the server, and the output is the notification message displayed on the device screen. Specifically, the notification is displayed in the notification bar when it is received, and the notification content is displayed in detail when the app is opened.

[0646] Step 7:

[0647] The device collects the infant's vital data. A smart baby monitor or wearable device is used to measure the infant's vital data, such as heart rate and body temperature, and the data is stored in the device. The input is the infant's biological signals, and the output is the measured vital data. Specifically, the device periodically monitors the vital signs of the infant wearing it and collects the data in real time.

[0648] Step 8:

[0649] The vital data collected by the device is sent to a server via the internet. A secure communication protocol is used to upload the vital data to the server. The input is the vital data stored on the device, and the output is the vital data uploaded to the server. Specifically, the device sends the collected vital data to the server at regular intervals.

[0650] Step 9:

[0651] The server analyzes the vital data received and determines whether there is an abnormality. Statistical algorithms and machine learning models are used to identify vital data that deviates from the normal range. The input is the vital data received by the server, and the output is the determination result when an abnormality is detected. Specifically, the vital data is input into the analysis module, and an alert is generated when an abnormal value is detected.

[0652] Step 10:

[0653] The server creates an emergency alert when an anomaly is detected. A highly urgent message is generated based on the detected anomaly. The input is the anomaly judgment result, and the output is the generated emergency alert. Specifically, when an anomaly is detected, an emergency message template is retrieved from the database and an alert is generated using this template.

[0654] Step 11:

[0655] The server sends the generated emergency alert to the terminal. The input is the generated emergency alert, and the output is the emergency alert sent to the terminal. Specifically, the server uses a communication module to send the emergency alert to the terminal's address.

[0656] Step 12:

[0657] Displays emergency alerts received by the device. The alert is notified to the user by highlighting it in the notification bar or within the app. The input is the emergency alert sent from the server, and the output is the emergency alert displayed on the device screen. Specifically, when an emergency alert is received, it is highlighted in the notification bar, and when the app is opened, the details of the alert are displayed.

[0658] Step 13:

[0659] The device uses a camera to capture the user's facial expression. The image data captured by the camera is stored in the device. The input is an image of the user's facial expression, and the output is the stored image data. Specifically, the device starts up the camera, captures the facial expression, and saves the data.

[0660] Step 14:

[0661] The image data captured by the device is sent to the server via the Internet. The image data is uploaded to the server. The input is the image data stored on the device, and the output is the image data uploaded to the server. Specifically, the transmission module automatically sends the image data that has been captured to the server.

[0662] Step 15:

[0663] The server analyzes the received image data and determines the user's emotional state. An emotion engine is used to identify emotions from the user's facial expressions. The input is the image data received by the server, and the output is the emotional state determined by the analysis. Specifically, the received image data is input into the emotion engine, the emotional state is analyzed, and a determination result is obtained.

[0664] Step 16:

[0665] The server adjusts the content of the notification message based on the analysis results. The notification message is created in a format appropriate for the user's emotional state. The input is the determined emotional state, and the output is the adjusted notification message. Specifically, the server adjusts the message template based on the results of the emotion analysis, and generates a notification message that takes emotions into consideration.

[0666] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0667] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0668] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0669] [Second embodiment]

[0670] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0671] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0672] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0673] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0674] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0675] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0676] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0677] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0678] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0679] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0680] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0681] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0682] MODE FOR CARRYING OUT THE INVENTION

[0683] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), thereby enabling the user to accurately grasp the infant's condition and take prompt action. The system of the present invention includes the following various means.

[0684] 1. Collection of audio data

[0685] The device records the baby's cries, laughter, and other sounds. For example, the baby's voice can be recorded using a microphone on a smartphone or a dedicated device. Recording starts at regular intervals, and the audio data is automatically acquired.

[0686] 2. Sending audio data

[0687] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[0688] 3. Analysis of audio data

[0689] The server analyzes the received voice data and estimates the infant's condition using a machine learning model based on the infant's crying patterns and voice characteristics.

[0690] 4. State Data Generation

[0691] The server generates data on the baby's condition based on the analysis of the voice data, identifying, for example, the baby's condition such as "needs a diaper change," "sleepy," or "hungry."

[0692] 5. Creating a notification message

[0693] The server generates notification messages for parents and caregivers based on the estimated infant status in a format that is easy for parents to understand.

[0694] 6. Sending Notifications

[0695] The notification message created by the server is sent to the terminal via the Internet.

[0696] 7. Display of notifications

[0697] Displaying notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[0698] 8. Collecting vital data (optional)

[0699] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Vital data is acquired using a smart baby monitor or wearable device.

[0700] 9. Analysis of vital data

[0701] The server analyzes the received vital data and determines whether there are any abnormalities, thereby understanding the infant's detailed physiological condition.

[0702] 10. Creating Emergency Alerts

[0703] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[0704] 11. Sending emergency alerts

[0705] The server sends an emergency alert message to the terminal.

[0706] 12. Displaying emergency alerts

[0707] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[0708] Specific examples

[0709] A specific example of the operation of the system is shown below.

[0710] Example 1: A crying baby

[0711] 1. The device will record your baby's crying for 5 seconds.

[0712] 2. The device sends the recorded data to a server via the Internet.

[0713] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[0714] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[0715] 5. The server creates a notification message saying "Diaper needs changing."

[0716] 6. The server sends a notification message to the terminal.

[0717] 7. The device displays the notification message in the notification bar of the user's smartphone.

[0718] 8. The user checks the notification and changes the baby's diaper.

[0719] Example 2: Vital signs monitoring

[0720] 1. The device measures the baby's temperature using a smart baby monitor.

[0721] 2. The device sends the measurement data to a server via the Internet.

[0722] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[0723] 4. The server creates an emergency alert: "Infant has fever."

[0724] 5. The server sends an emergency alert to the device.

[0725] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[0726] 7. The user checks the emergency alert and takes necessary measures.

[0727] In this way, the system of the present invention can accurately grasp the condition and needs of the infant and assist in prompt response.

[0728] The processing flow will be explained below.

[0729] Program processing steps

[0730] Step 1:

[0731] The device records the baby's voice. The device's microphone is used to record the baby's crying and laughter for a certain period of time (e.g., 5 seconds).

[0732] Step 2:

[0733] The device stores the recorded audio data in a buffer and uploads it to a server via the Internet using a secure communication protocol such as HTTPS.

[0734] Step 3:

[0735] The server decodes the received audio data and begins analyzing it using a machine learning model (e.g., Convolutional Neural Network) to identify patterns and characteristics of crying sounds.

[0736] Step 4:

[0737] The server generates estimated data on the infant's condition based on the results of analyzing the voice data. Specifically, it creates textual data on the infant's condition, such as "needs a diaper change," "sleepy," and "hungry."

[0738] Step 5:

[0739] The server creates notification messages based on the generated status data, and the notification messages are presented in a format that is easy for parents and caregivers to understand.

[0740] Step 6:

[0741] The notification message created by the server is sent to the device via the Internet as a push notification.

[0742] Step 7:

[0743] The device decodes the notification message received and displays it in the notification bar or dedicated app on the smartphone or tablet.

[0744] Step 8:

[0745] The device measures the infant's vital data (e.g., heart rate and body temperature) using a smart baby monitor or wearable device and stores the measured data in a buffer.

[0746] Step 9:

[0747] The vital data measured by the device is uploaded to a server via the Internet.

[0748] Step 10:

[0749] The server analyzes the received vital data and detects abnormal values ​​(e.g., high or low fever) using statistical algorithms and machine learning models.

[0750] Step 11:

[0751] If the server detects abnormal data, it will create an emergency alert. For example, it will generate an alert message saying "The baby has a fever. Please take action immediately."

[0752] Step 12:

[0753] The server creates an emergency alert message and sends it to the device as a push notification via the Internet.

[0754] Step 13:

[0755] The device decodes emergency alert messages received and displays them in the notification bar and in the app with special highlighting, thereby informing the user of the urgency of the message.

[0756] Step 14:

[0757] The user checks notifications and emergency alerts and takes appropriate action (e.g., changing a diaper, checking a temperature).

[0758] Example 1

[0759] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0760] Rapid and accurate understanding of an infant's health condition and needs is an important challenge for parents and caregivers. However, because infants cannot communicate their condition verbally, they must rely on indirect information such as crying and vital signs. In conventional systems, voice data and vital signs are often collected and analyzed separately, making it difficult to centrally manage and comprehensively analyze this data. In addition, there are limitations to the means of quickly notifying users of analysis results, which can delay response, especially in urgent situations.

[0761] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0762] In this invention, the server includes means for recording voice data of the infant, means for periodically starting and saving the recorded voice data, means for transmitting the recorded voice data to the server using a secure communication protocol, means for analyzing the received voice data based on a machine learning model to estimate the infant's condition, means for generating estimated infant condition data, means for creating a notification message that is easy for the user to understand, means for transmitting the notification message to the user's terminal using the secure communication protocol, and means for displaying the notification message on the user's terminal, thereby enabling the infant's health condition and needs to be quickly and accurately grasped and appropriate measures to be taken.

[0763] "Means for recording infant sounds" refers to devices and methods for recording sounds such as infant crying and laughter.

[0764] "Means for periodically initiating and saving recorded audio" refers to a device or method for automatically initiating audio recording at set intervals and saving the data.

[0765] "Means for transmitting recorded audio to a server using a secure communication protocol" refers to a device or method for transmitting acquired audio data to a server while maintaining information security.

[0766] "Means for analyzing received voice data based on a machine learning model and estimating the infant's condition" refers to a device or method for analyzing voice data using machine learning technology and estimating a specific infant condition (e.g., hunger, sleepiness, etc.).

[0767] The "means for generating estimated infant condition data" refers to a device or method for converting the infant condition into a data format based on the results of voice analysis.

[0768] The "means for generating user-friendly notification messages" refers to devices or methods for generating notifications based on the estimated infant condition in a format that is easy for parents or caregivers to understand.

[0769] The "means for transmitting a notification message to a user terminal using a secure communication protocol" refers to a device or method for transmitting the generated notification message to a user terminal while maintaining information security.

[0770] The "means for displaying a notification message on a user's terminal" refers to a device or method for visually displaying a received notification message on a user's terminal.

[0771] "Means for measuring infant vital data" refers to devices and methods for measuring physiological indicators such as an infant's heart rate and body temperature.

[0772] "Means for transmitting measured vital data to a server using a secure communication protocol" refers to a device or method for transmitting acquired vital data to a server while maintaining information security.

[0773] The "means for analyzing received vital data and determining whether an abnormal state exists" refers to a device or method for analyzing acquired vital data and determining whether an abnormal state exists.

[0774] The "means for generating an emergency alert when abnormal vital data is detected" refers to a device or method for generating a message to call for urgent attention when abnormal vital data is detected.

[0775] "Means for sending an emergency alert to a user's terminal" refers to a device or method for sending the generated emergency alert message to a user's terminal.

[0776] A "means for specially highlighting emergency alerts on a user's terminal" is a device or method for highlighting emergency alert messages on a user's terminal.

[0777] MODE FOR CARRYING OUT THE INVENTION

[0778] System configuration

[0779] The system of the present invention includes the following means to quickly and accurately grasp an infant's health condition and needs and support appropriate responses. The hardware used includes smartphones, tablets, dedicated devices, smart baby monitors, and wearable devices. The software uses machine learning models and Firebase Cloud Messaging (FCM).

[0780] Audio data collection

[0781] The device records sounds such as the baby's crying and laughter. Specifically, it uses the smartphone's microphone to start recording at regular intervals and automatically saves the audio data.

[0782] Sending audio data

[0783] The audio data recorded by the device is sent to the server via the Internet. The HTTPS protocol is used for transmission to ensure security. The audio data recorded is sent to the API endpoint as a POST request.

[0784] Analysis of audio data

[0785] The server analyzes the received audio data using a machine learning model that has learned patterns of infant crying and laughter, and uses this data to estimate the infant's condition.

[0786] Generate state data

[0787] The server generates data on the baby's condition based on the analysis results. For example, if the baby continues crying, it may assume that the baby needs a diaper change or is hungry. This data is saved in JSON format and the next step is reached.

[0788] Creating a notification message

[0789] The server generates an easy-to-understand notification message for parents or caregivers based on the estimated infant's condition, such as "Your baby is crying and needs a diaper change."

[0790] Sending notifications

[0791] The server creates a notification message and sends it to the device via the Internet using Firebase Cloud Messaging (FCM) or another push notification service.

[0792] Viewing notifications

[0793] Display notification messages received by the device. Displays messages in the notification bar of smartphones and tablets to notify the user.

[0794] Vital data collection (optional)

[0795] The device measures the baby's vital data (heart rate and temperature) and sends it to a server. The data is then collected via Bluetooth using a smart baby monitor or wearable device.

[0796] Analysis of vital data

[0797] The server analyzes the received vital data and determines whether there are any abnormalities. An abnormal data detection algorithm is used to detect an excessively high heart rate or abnormal body temperature.

[0798] Creating an emergency alert

[0799] If the server detects abnormal vital data, it creates an emergency alert message, such as "The baby has a fever," and treats it as a priority message.

[0800] Sending emergency alerts

[0801] The server sends an emergency alert message to the device, using a push notification service such as Firebase Cloud Messaging (FCM) to send the notification immediately.

[0802] Viewing emergency alerts

[0803] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[0804] Specific examples

[0805] Example 1: A crying baby

[0806] 1. The device will record your baby's crying for 5 seconds.

[0807] 2. The device sends the recorded data to a server via the Internet.

[0808] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[0809] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[0810] 5. The server creates a notification message saying "Diaper needs changing."

[0811] 6. The server sends a notification message to the terminal.

[0812] 7. The device displays the notification message in the notification bar of the user's smartphone.

[0813] 8. The user checks the notification and changes the baby's diaper.

[0814] Example 2: Vital signs monitoring

[0815] 1. The device measures the baby's temperature using a smart baby monitor.

[0816] 2. The device sends the measurement data to a server via the Internet.

[0817] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[0818] 4. The server creates an emergency alert: "Infant has fever."

[0819] 5. The server sends an emergency alert to the device.

[0820] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[0821] 7. The user checks the emergency alert and takes necessary measures.

[0822] In this way, the system of the present invention is able to quickly and accurately grasp the health condition and needs of an infant and support appropriate responses.

[0823] Prompt Sentence Examples

[0824] "Please analyze and let me know why my baby is crying."

[0825] "Please take the infant's temperature and issue an alert if there is anything abnormal."

[0826] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0827] System program processing flow

[0828] Step 1: Collecting audio data

[0829] The device records sounds such as the baby's crying and laughter. First, it uses the smartphone's microphone to start recording at regular intervals and saves the audio data. For example, a dedicated app can automatically start a 5-second recording and save the data on the device.

[0830] Input: Trigger to start recording

[0831] Output: Audio file (e.g., wav format)

[0832] Step 2: Sending audio data

[0833] The audio data recorded by the device is sent to the server via the Internet using the HTTPS protocol to ensure data security. Specifically, the audio file is sent as a POST request to the API endpoint.

[0834] Input: Audio file

[0835] Output: Sending status (success / failure)

[0836] Step 3: Analyzing the audio data

[0837] The server analyzes the received audio data using a machine learning model. First, the audio data is input into the machine learning model. This model has learned the patterns of infant crying and laughter and analyzes the audio features. Specifically, it extracts features such as the frequency components and volume of the crying sounds and uses them to estimate the infant's condition.

[0838] Input: Audio data

[0839] Output: Analysis results (e.g. "I want to change your diaper")

[0840] Step 4: Generate state data

[0841] The server generates data on the infant's condition based on the analysis of the voice data. For example, it identifies the infant's condition, such as "needs a diaper change," "hungry," or "sleepy." The inferred results are saved in a structured data format (e.g., JSON).

[0842] Input: Analysis results

[0843] Output: State data (JSON format)

[0844] Step 5: Create a notification message

[0845] The server creates a user-friendly notification message based on the estimated baby's condition, such as "Your baby is crying. Your diaper needs to be changed."

[0846] Input: State data

[0847] Output: Notification message

[0848] Step 6: Sending notifications

[0849] The notification message created by the server is sent to the device via the Internet using a push notification service such as Firebase Cloud Messaging (FCM).

[0850] Input: Notification message

[0851] Output: Sending status (success / failure)

[0852] Step 7: View notifications

[0853] Displaying notification messages received by the device. Visually displaying notifications to the user in the notification bar of a smartphone or tablet or within a dedicated app. For example, displaying a pop-up in the notification bar saying "Diaper needs changing."

[0854] Input: Notification message

[0855] Output: Displayed notification

[0856] Step 8: Collect vital data (optional)

[0857] The device measures the baby's vital data (heart rate and temperature) and sends it to a server. Data is collected using a smart baby monitor or wearable device and stored on the device via Bluetooth.

[0858] Input: Trigger to start measurement

[0859] Output: Vital data

[0860] Step 9: Analyze vital data

[0861] The server analyzes the received vital data and determines whether there are any abnormalities, using an anomaly detection algorithm to detect, for example, an extremely high heart rate or an abnormal body temperature.

[0862] Input: Vital data

[0863] Output: Abnormal condition (e.g. "High fever")

[0864] Step 10: Create an emergency alert

[0865] If the server detects abnormal vital data, it creates an emergency alert message, for example, a message with high urgency such as "The baby has a fever" and marks it as an emergency alert.

[0866] Input: Abnormal condition

[0867] Output: Emergency alert message

[0868] Step 11: Sending emergency alerts

[0869] The server sends an emergency alert message to the device using a push notification service such as Firebase Cloud Messaging (FCM).

[0870] Input: Emergency alert message

[0871] Output: Sending status (success / failure)

[0872] Step 12: View emergency alerts

[0873] The device displays an emergency alert to the user, highlighting it with a special icon or popup in the notification bar of the smartphone or tablet to communicate the urgency to the user. For example, the notification bar will display "Urgent: Baby has a fever!" in red.

[0874] Input: Emergency alert message

[0875] Output: Emergency notification displayed

[0876] (Application example 1)

[0877] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0878] Conventionally, monitoring of an infant's crying and physiological data and notifications based on the results have mainly been done manually by parents or caregivers, which can make it difficult to respond quickly. Furthermore, since the infant's health condition cannot be grasped in real time, there is a risk of missing signs of serious health abnormalities. The objective of this invention is to provide a system that efficiently collects and analyzes an infant's voice and vital data and promptly notifies the user.

[0879] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0880] In this invention, the server includes means for recording the infant's voice, means for analyzing the recorded voice to estimate the infant's condition, means for creating a notification message based on the estimated infant's condition, means for transmitting the created notification message to a user's terminal, means for displaying the notification message on the user's terminal, means for using a wearable device to measure the infant's vital data, means for analyzing the measured vital data to detect abnormalities, means for creating an emergency alert based on the detected abnormality, and means for transmitting the created emergency alert to the user's smart glasses or smartphone and displaying the notification, thereby enabling the infant's health condition to be understood in real time and responding promptly.

[0881] "Infant voice" refers to audio signals such as crying and laughing emitted by an infant.

[0882] "Recording" means recording and storing audio and data.

[0883] "Analysis" is the process of examining collected data and identifying meanings and patterns in it.

[0884] "Condition" refers to a specific situation or condition, such as an infant's health, emotions, or care needs.

[0885] A "notification message" is a message containing information that is sent to a user based on the infant's condition.

[0886] A "user device" is an electronic device used by a parent or caregiver, such as a smartphone or smart glasses.

[0887] "Display" means the visual presentation of information on a screen or display.

[0888] "Vital data" refers to data that indicates an infant's physiological indicators, including heart rate and body temperature.

[0889] A "wearable device" is a device worn by an infant to measure vital data.

[0890] An "abnormality" is a condition that is outside the normal range and may affect health.

[0891] An "urgent alert" is an immediate notification when a serious health or safety issue is detected.

[0892] A "generative AI model" is a machine learning algorithm that analyzes an infant's voice patterns and vital data based on training data.

[0893] A "server" is a central computer system that analyzes and processes data and sends the results to the user's terminal.

[0894] The system of the present invention efficiently collects and analyzes an infant's voice and vital data, and notifies the user to grasp the infant's condition in real time and support prompt response. A specific embodiment of this system is shown below.

[0895] System Configuration

[0896] The system mainly consists of the following components:

[0897] 1. Device: This refers to a smartphone or smart glasses equipped with a microphone and a display, capable of displaying audio data and notifications.

[0898] 2. Server: Plays the central role in data analysis, analyzing voice and vital data using generative AI models.

[0899] 3. Wearable devices: Smart bands that infants can wear to measure vital data such as heart rate and body temperature.

[0900] 4. Generative AI model: A machine learning algorithm that analyzes voice patterns and infant physiological indicators based on training data.

[0901] Hardware and Software Use

[0902] Terminal

[0903] The terminal records voice data and transmits it to the server. It also transmits vital data acquired from the wearable device to the server. It has the function of displaying notification messages and emergency alerts when they are generated.

[0904] server

[0905] The server receives the voice and vital data sent from the device and analyzes them using a generative AI model. Based on the analysis results, it estimates the infant's condition and creates notification messages and emergency alerts.

[0906] Wearable devices

[0907] Wearable devices are used to measure vital data such as the baby's heart rate and body temperature, and the measured data is sent to a terminal via communication means such as Bluetooth.

[0908] Data processing and calculation

[0909] 1. Audio data collection:

[0910] The device records the baby's crying, laughter, and other sounds, for example, five seconds of audio, and saves it as digital data.

[0911] 2. Sending audio data:

[0912] The device sends the recorded audio data to a server via the Internet using the HTTPS protocol for secure data transfer.

[0913] 3. Analysis of audio data:

[0914] The server analyzes the received voice data using a generative AI model, capturing the characteristics of the voice pattern and inferring what the baby needs.

[0915] 4. Create and send a notification message:

[0916] Based on the estimated baby's condition, the server generates a notification message in a format that can be quickly communicated to the user, and the generated message is then sent to the terminal via the Internet.

[0917] 5. Displaying notification messages:

[0918] The device will then display the received notification message to the user: on a smartphone, it will appear in the notification bar, and on the display of smart glasses.

[0919] 6. Vital Data Collection:

[0920] The wearable device measures the infant's heart rate and temperature and transmits the data to a terminal.

[0921] 7. Analysis of vital data:

[0922] The server analyzes the received vital data and creates an emergency alert if it detects any abnormalities.

[0923] 8. Creating and sending emergency alerts:

[0924] When an abnormality is detected, the server quickly creates an emergency alert and sends it to the device.

[0925] 9. Emergency alert display:

[0926] The device will highlight the emergency alert and notify the user.

[0927] Specific examples

[0928] Example 1: A crying baby

[0929] The device records the baby's crying for 5 seconds and sends it to the server.

[0930] The server analyzes the voice data and determines that the diaper needs to be changed.

[0931] A notification message "Diaper change required" is generated and sent to the user's terminal.

[0932] The device will notify the user by displaying a message in the notification bar or on the display of the smart glasses.

[0933] Example 2: Vital signs monitoring

[0934] The wearable device measures the infant's temperature and transmits it to a terminal.

[0935] The server analyzes the body temperature data and detects high fever.

[0936] An emergency alert "Infant has fever" is generated and sent to the user's device.

[0937] The device will highlight emergency alerts and notify the user quickly.

[0938] Example prompt sentence:

[0939] "Analyzing audio data of an infant crying and identifying the cause."

[0940] "Analyze infant heart rate data to detect abnormalities."

[0941] In this way, the system of the present invention can grasp the condition of the infant in real time and promote quick response.

[0942] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0943] Step 1:

[0944] The device records sounds such as the baby's crying and laughter. Specifically, it uses the device's built-in microphone to record audio data for a specified period of time (e.g., 5 seconds). The input here is a real-time audio signal, and the output is digital audio data. The recorded audio data is temporarily stored in a buffer.

[0945] Step 2:

[0946] The device sends the recorded audio data to the server via the Internet. The HTTPS protocol is used for communication to ensure data security. The input here is the recorded audio data, and the output is the data to be sent to the server. Once the transmission is complete, the device receives a confirmation message.

[0947] Step 3:

[0948] The server inputs the received voice data into a generative AI model for analysis. The generative AI model is pre-trained and analyzes the voice features to estimate the infant's state. The input here is the voice data sent to the server, and the output is the infant's estimated state (e.g., "I want to change my diaper" or "I'm hungry"). The AI ​​model analyzes the voice patterns and probabilistically estimates the most appropriate state.

[0949] Step 4:

[0950] The server creates a notification message based on the estimated infant's state. For example, if the AI ​​model estimates that the infant needs a diaper change, it generates a notification message saying, "Your diaper needs to be changed." The input here is the estimated infant's state, and the output is the generated notification message. The server adds this to a message queue.

[0951] Step 5:

[0952] The server then sends the created notification message to the user's terminal, where the input is the notification message and the output is the data sent to the user's terminal. The notification message is securely transmitted over the Internet.

[0953] Step 6:

[0954] The device displays the received notification message. In the case of a smartphone, this is displayed in the notification bar, and in the case of smart glasses, it is visually displayed on the display. The input here is the notification message sent from the server, and the output is the displayed message. The user checks the notification and responds.

[0955] Step 7:

[0956] The wearable device measures the infant's vital data (heart rate and body temperature). The input is the infant's physiological data, and the output is the measured vital data. These data are measured periodically and sent to the terminal.

[0957] Step 8:

[0958] The device sends the received vital data to a server via the Internet. The input here is the measured vital data, and the output is the data to be sent to the server. Data transmission is via Bluetooth or Wi-Fi.

[0959] Step 9:

[0960] The server analyzes the received vital data and detects abnormalities. The generative AI model identifies abnormal patterns and detects abnormalities such as high fever or abnormal heart rate. The input here is the vital data, and the output is the detected abnormality and detailed information about it.

[0961] Step 10:

[0962] The server generates an emergency alert based on the detected anomaly, for example, a message saying "The baby has a fever." Here, the input is the anomaly detection result, and the output is the emergency alert message. The alert is treated as a high-priority message.

[0963] Step 11:

[0964] The server sends the created emergency alert to the user's device. The input here is the emergency alert message, and the output is the data to be sent to the user's device. The server uses a message queue to send the alert immediately.

[0965] Step 12:

[0966] The device highlights the received emergency alert. In the case of a smartphone, it is displayed in the notification bar with an emergency icon, and in the case of smart glasses, it is displayed with a special effect on the display. Here, the input is the emergency alert sent from the server, and the output is a visually highlighted message. The user can check the alert and take necessary measures.

[0967] Example prompt sentence:

[0968] "Analyzing audio data of an infant crying and identifying the cause."

[0969] "Analyze infant heart rate data to detect abnormalities."

[0970] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0971] MODE FOR CARRYING OUT THE INVENTION

[0972] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), enabling the user to accurately grasp the infant's condition and respond promptly. The system of the present invention includes the following means:

[0973] 1. Collection of audio data

[0974] The device records the baby's crying, laughter, and other sounds. For example, the baby's voice can be recorded using a microphone on a smartphone or a dedicated device. Recording begins at regular intervals, and the audio data is automatically acquired.

[0975] 2. Sending audio data

[0976] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[0977] 3. Analysis of audio data

[0978] The server analyzes the received voice data and estimates the infant's condition using a machine learning model based on the infant's crying patterns and voice characteristics.

[0979] 4. State Data Generation

[0980] The server generates data on the baby's condition based on the analysis of the voice data, identifying, for example, whether the baby needs a diaper change, is sleepy, or is hungry.

[0981] 5. Creating a notification message

[0982] The server generates notification messages for parents and caregivers based on the estimated infant status in a format that is easy for parents to understand.

[0983] 6. Sending Notifications

[0984] The notification message created by the server is sent to the terminal via the Internet.

[0985] 7. Display of notifications

[0986] Displaying notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[0987] 8. Collecting vital data (optional)

[0988] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Vital data is acquired using a smart baby monitor or wearable device.

[0989] 9. Analysis of vital data

[0990] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[0991] 10. Creating Emergency Alerts

[0992] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[0993] 11. Sending emergency alerts

[0994] The server sends an emergency alert message to the terminal.

[0995] 12. Displaying emergency alerts

[0996] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[0997] Furthermore, the present invention includes the following features:

[0998] 13. Use of Emotion Engines

[0999] The device uses an emotion engine to recognize the user's emotions, for example by capturing the user's facial expressions with a camera and analyzing the data to identify emotions.

[1000] 14. Transmission of Emotional Data

[1001] The user's emotion data recognized by the device is sent to a server via the Internet.

[1002] 15. Emotional Data Analysis

[1003] The server analyzes the received emotion data and determines the user's emotional state, for example, whether the user is feeling stressed.

[1004] 16. Notification Coordination

[1005] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened.

[1006] Specific examples

[1007] Example 1: A crying baby

[1008] 1. The device will record your baby's crying for 5 seconds.

[1009] 2. The device sends the recorded data to a server via the Internet.

[1010] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[1011] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[1012] 5. The server creates a notification message saying "Diaper needs changing."

[1013] 6. The server sends a notification message to the terminal.

[1014] 7. The device displays the notification message in the notification bar of the user's smartphone.

[1015] 8. The user checks the notification and changes the baby's diaper.

[1016] Example 2: Vital signs monitoring

[1017] 1. The device measures the baby's temperature using a smart baby monitor.

[1018] 2. The device sends the measurement data to a server via the Internet.

[1019] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[1020] 4. The server creates an emergency alert: "Infant has fever."

[1021] 5. The server sends an emergency alert to the device.

[1022] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[1023] 7. The user checks the emergency alert and takes necessary measures.

[1024] Example 3: Using the Emotion Engine

[1025] 1. The device uses a camera to capture the user's facial expressions.

[1026] 2. The device sends the captured image data to a server via the Internet.

[1027] 3. The server uses an emotion engine to analyze the image data and determine the user's emotional state.

[1028] 4. If the server determines that the user is feeling stressed, it creates a notification message with careful consideration of its content and expression.

[1029] 5. The server sends the adjusted notification message to the terminal.

[1030] 6. The device will display adjusted notification messages, making notifications more appropriate for users.

[1031] In this way, the multi-functional system can accurately grasp the infant's condition and provide notifications and alerts that take into account the user's emotional state.

[1032] The processing flow will be explained below.

[1033] Program processing steps

[1034] Baby crying analysis

[1035] Step 1:

[1036] The device records the baby's voice. The device's microphone is used to record the baby's crying and laughter for a certain period of time (e.g., 5 seconds).

[1037] Step 2:

[1038] The device stores the recorded audio data in a buffer and uploads it to a server via the Internet using a secure communication protocol such as HTTPS.

[1039] Step 3:

[1040] The server decodes the received audio data and analyzes it using a machine learning model (e.g., Convolutional Neural Network) to identify patterns and characteristics of crying sounds.

[1041] Step 4:

[1042] The server generates estimated data on the infant's condition based on the results of analyzing the voice data, specifically identifying the infant's condition, such as "needs a diaper change," "sleepy," or "hungry."

[1043] Step 5:

[1044] The server generates notification messages based on the generated status data in a format that is easy for parents and caregivers to understand.

[1045] Step 6:

[1046] The notification message created by the server is sent to the device via the Internet as a push notification.

[1047] Step 7:

[1048] The device decodes the notification message received and displays it in the notification bar or dedicated app on the smartphone or tablet.

[1049] Vital data monitoring and emergency alerts

[1050] Step 8:

[1051] The device measures the infant's vital data (e.g., heart rate and body temperature) using a smart baby monitor or wearable device and stores the measured data in a buffer.

[1052] Step 9:

[1053] The vital data measured by the device is uploaded to a server via the Internet.

[1054] Step 10:

[1055] The server analyzes the received vital data and detects abnormal values ​​(e.g., high or low fever) using statistical algorithms and machine learning models.

[1056] Step 11:

[1057] If the server detects abnormal data, it will create an emergency alert, e.g., "The baby has a fever. Please take action immediately."

[1058] Step 12:

[1059] The server creates an emergency alert message and sends it to the device as a push notification via the Internet.

[1060] Step 13:

[1061] The device decodes emergency alert messages received and displays them in the notification bar and in the app with special highlighting, thereby informing the user of the urgency of the message.

[1062] Use of emotion engine

[1063] Step 14:

[1064] The device captures the user's facial expressions using a camera, for example, the front camera on a smartphone or tablet.

[1065] Step 15:

[1066] The image data captured by the device is uploaded to a server via the Internet.

[1067] Step 16:

[1068] The server uses an emotion engine to analyze the image data and determine the user's emotional state, such as whether they are "stressed" or "relaxed."

[1069] Step 17:

[1070] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. If the user is feeling stressed, the content and wording of the notification will be softened.

[1071] Step 18:

[1072] The server sends the adjusted notification message to the terminal via the Internet.

[1073] Step 19:

[1074] The device decodes the adjusted notification message and displays it in the notification bar or within a dedicated app on the smartphone or tablet, making notifications more appropriate for the user.

[1075] Example 2

[1076] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1077] In infant care, it is necessary to maintain the health and comfort of infants by effectively collecting and analyzing their voice and vital data and notifying parents and caregivers in a timely manner. In particular, it is necessary to improve the accuracy of analyzing infant voices and vital data and issue immediate alerts in the event of an emergency. It is also important to adjust notifications and alerts to take into account the caregiver's emotional state. However, conventional systems have difficulty meeting all of these requirements, making it difficult to accurately monitor infant conditions and respond promptly.

[1078] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for recording the infant's voice, a means for analyzing the recorded voice to estimate the infant's condition, a means for creating a notification message based on the estimated infant's condition, a means for sending the created notification message to a user's terminal, a means for displaying the notification message on the user's terminal, a means for recognizing the user's emotion, and a means for analyzing the recognized user's emotion data and adjusting the content and expression of the notification message. This makes it possible to accurately monitor the infant's health and comfort and notify parents or caregivers at appropriate times. Furthermore, it is possible to provide flexible notification messages according to the user's emotional state, thereby realizing more effective child-rearing support.

[1079] "Means for recording infant sounds" refers to an apparatus or device for recording sounds such as an infant's crying or laughing.

[1080] "Means for estimating an infant's condition by analyzing recorded voice" refers to technologies and algorithms that analyze collected voice data and infer an infant's condition based on voice patterns and characteristics.

[1081] A "means for generating a notification message" is a device or program for generating a message in a format that is easy for a parent or caregiver to understand based on the estimated infant condition.

[1082] The "means for transmitting the created notification message to the user's terminal" refers to a device or technology for transmitting the notification message to the user's terminal via a communication means such as the Internet.

[1083] The "means for displaying a notification message on a user's device" refers to a function or application for displaying a received notification message on a user's device such as a smartphone or tablet.

[1084] "Means for recognizing user emotions" refers to technologies and algorithms that collect a user's facial expressions and other data to identify their emotional state.

[1085] "Means for analyzing the recognized user emotional data and adjusting the content and expression of the notification message" refers to technologies and algorithms for analyzing collected emotional data and appropriately changing the content and expression of the notification message depending on the user's emotional state.

[1086] "Means for measuring infant vital data" refers to equipment or devices for obtaining physiological data such as an infant's heart rate and body temperature.

[1087] "Means for analyzing the measured vital data to determine the detailed condition of the infant" refers to technologies and algorithms for analyzing the collected vital data and determining the infant's health condition or abnormalities based on the results.

[1088] The "means for generating an emergency alert" is a device or program that immediately generates a highly urgent message when an abnormality is detected in the analyzed data.

[1089] The "means for transmitting the created emergency alert to the user's terminal" refers to a technique or device for transmitting the emergency alert to the user's terminal via a communication means such as the Internet.

[1090] "Means for displaying emergency alerts on a user's device" refers to a function or application for highlighting received emergency alerts on a user's device such as a smartphone or tablet.

[1091] A "machine learning model" is an algorithm or program that is trained to learn features from the voice and vital data being analyzed and to classify or predict the infant's condition.

[1092] "Distinctive infant vocal patterns" are vocal characteristics or patterns that indicate a particular state, such as an infant's crying or laughter.

[1093] "Physiological indicators" are physiological data that indicate the health or condition of an infant, such as heart rate, body temperature, and respiratory rate.

[1094] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), enabling the user to accurately grasp the infant's condition and take prompt action. The system of the present invention includes the following means.

[1095] Audio data processing

[1096] Audio data collection:

[1097] The device records sounds such as the baby's crying and laughter. The hardware used includes a microphone on a smartphone or a dedicated device. The device starts recording at regular intervals and automatically acquires audio data. For example, a 5-second recording is performed.

[1098] Sending audio data:

[1099] The device sends the recorded audio data to a server via the internet using a secure communication method such as the HTTPS protocol.

[1100] Analysis of audio data:

[1101] The server analyzes the received voice data using a machine learning model. For example, the voice data is input into a generative AI model, and the model analyzes the baby's crying patterns and voice characteristics using the prompt "Please estimate the baby's condition as indicated by this voice data."

[1102] Generate state data:

[1103] The server generates data on the baby's condition based on the analysis of the voice data, specifically identifying the baby's condition, such as "needs a diaper change," "sleepy," or "hungry."

[1104] Create a notification message:

[1105] The server generates a notification message for parents or caregivers based on the estimated infant condition. The notification message is generated in a format that is easy for parents to understand. For example, it generates a message saying, "Your diaper needs to be changed."

[1106] Send notifications:

[1107] The notification message created by the server is sent to the terminal via the Internet.

[1108] Displaying notifications:

[1109] Display notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[1110] Vital data processing (optional)

[1111] Vital data collection:

[1112] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Smart baby monitors and wearable devices are used to collect vital data.

[1113] Vital data analysis:

[1114] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[1115] Creating an emergency alert:

[1116] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[1117] Sending emergency alerts:

[1118] The server sends an emergency alert message to the terminal.

[1119] Viewing emergency alerts:

[1120] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[1121] Use of emotion engine

[1122] Using the Emotion Engine:

[1123] To recognize the user's emotions, the device captures the user's facial expressions with a camera and analyzes the data to identify the user's emotions.

[1124] Sending Emotion Data:

[1125] The user's emotion data recognized by the device is sent to a server via the Internet.

[1126] Emotional Data Analysis:

[1127] The server analyzes the received emotion data and determines the user's emotional state, for example, whether the user is feeling stressed.

[1128] Notification adjustments:

[1129] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened.

[1130] Specific examples

[1131] Example 1: A crying baby

[1132] 1. The device will record your baby's crying for 5 seconds.

[1133] 2. The device sends the recorded data to a server via the Internet.

[1134] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[1135] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[1136] 5. The server creates a notification message saying "Diaper needs changing."

[1137] 6. The server sends a notification message to the terminal.

[1138] 7. The device displays the notification message in the notification bar of the user's smartphone.

[1139] Example 2: Vital signs monitoring

[1140] 1. The device measures the baby's temperature using a smart baby monitor.

[1141] 2. The device sends the measurement data to a server via the Internet.

[1142] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[1143] 4. The server creates an emergency alert: "Infant has fever."

[1144] 5. The server sends an emergency alert to the device.

[1145] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[1146] 7. The user checks the emergency alert and takes necessary measures.

[1147] Example 3: Using the Emotion Engine

[1148] 1. The device uses a camera to capture the user's facial expressions.

[1149] 2. The device sends the captured image data to a server via the Internet.

[1150] 3. The server uses an emotion engine to analyze the image data and determine the user's emotional state.

[1151] 4. If the server determines that the user is feeling stressed, it creates a notification message with careful consideration of its content and expression.

[1152] 5. The server sends the adjusted notification message to the terminal.

[1153] 6. The terminal displays an adjusted notification message, and the notification to the user is more appropriate.

[1154] Prompt Sentence Examples

[1155] "Analyze a 5-second recording of an infant's cry and use the cry to estimate whether the baby needs a diaper change."

[1156] This multi-functional system allows for accurate understanding of the infant's condition and prompt response, and can also analyze and provide notifications and alerts that take into account the user's emotional state, enabling more effective care.

[1157] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1158] Step 1:

[1159] Audio data collection

[1160] The device records the baby's crying, laughter, and other sounds. Specifically, it uses a smartphone microphone to record 5 seconds of audio. The input is the baby's voice, and the output is the recorded audio data.

[1161] Step 2:

[1162] Sending audio data

[1163] The device sends recorded audio data to the server using the HTTPS protocol. The input is the recorded audio data, and the output is the audio data sent to the server.

[1164] Step 3:

[1165] Analysis of audio data

[1166] The server analyzes the received audio data. Specifically, it uses a generative AI model to analyze the audio data and extract patterns and characteristics of the infant's crying. The analysis is performed using the prompt "Please estimate the infant's condition indicated by this audio data." The input is the audio data sent to the server, and the output is the analysis result of the crying.

[1167] Step 4:

[1168] Generate state data

[1169] The server generates data on the infant's condition based on the analysis of the voice data. Specifically, it identifies the infant's condition, such as "needs a diaper change," "sleepy," or "hungry." The input is the analysis result of the crying, and the output is the infant's condition data.

[1170] Step 5:

[1171] Creating a notification message

[1172] The server generates a notification message for parents or caregivers based on the generated infant status data. Specifically, it generates a message such as "diaper change required." The input is the infant status data, and the output is the notification message.

[1173] Step 6:

[1174] Sending notifications

[1175] The notification message created by the server is sent to the terminal via the Internet. The input is the notification message, and the output is the notification message sent to the terminal.

[1176] Step 7:

[1177] Viewing notifications

[1178] The notification message received by the device is displayed to the user. Specifically, the message is displayed in the notification bar of a smartphone or tablet or within a dedicated app. The input is the notification message sent to the device, and the output is the notification message displayed to the user.

[1179] Step 8:

[1180] Vital data collection (optional)

[1181] The terminal measures the infant's vital data (heart rate and body temperature) and sends it to the server. A smart baby monitor or wearable device is used. The input is the infant's vital data, and the output is the measured vital data.

[1182] Step 9:

[1183] Sending vital data

[1184] The vital data collected by the device is sent to a server via the Internet. The input is the measured vital data, and the output is the vital data sent to the server.

[1185] Step 10:

[1186] Analysis of vital data

[1187] The server analyzes the received vital data and determines whether there is an abnormality. Specifically, it uses statistical algorithms and machine learning models. The input is the vital data sent to the server, and the output is the analysis result of the abnormality.

[1188] Step 11:

[1189] Creating an emergency alert

[1190] If the server detects abnormal vital data, it creates an emergency alert. Specifically, it generates a highly urgent message such as "The baby has a fever." The input is the analysis result of the abnormal condition, and the output is an emergency alert message.

[1191] Step 12:

[1192] Sending emergency alerts

[1193] The server sends an emergency alert message to the terminal, where the input is the emergency alert message and the output is the emergency alert message sent to the terminal.

[1194] Step 13:

[1195] Viewing emergency alerts

[1196] The device displays an emergency alert to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user. The input is the emergency alert message sent to the device, and the output is the emergency alert message displayed to the user.

[1197] Step 14:

[1198] Use of emotion engine

[1199] To recognize the user's emotions, the device captures the user's facial expressions with a camera and analyzes the data to identify the user's emotions. The input is the user's facial expression data, and the output is the recognized emotion data.

[1200] Step 15:

[1201] Sending emotional data

[1202] The device recognizes the user's emotion data and sends it to a server via the Internet. The input is emotion data, and the output is the emotion data sent to the server.

[1203] Step 16:

[1204] Emotional Data Analysis

[1205] The server analyzes the received emotion data and determines the user's emotional state. Specifically, the analysis is performed using an emotion engine. The input is the emotion data sent to the server, and the output is the analysis result of the emotional state.

[1206] Step 17:

[1207] Notification adjustments

[1208] Based on the analysis results, the server creates notification messages and alerts in a format appropriate for the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened. The input is the analysis result of the emotional state, and the output is the adjusted notification message.

[1209] This is the flow of processing for this system. In this way, the condition of the infant can be accurately monitored and notifications and alerts can be provided to the user at the appropriate time.

[1210] (Application example 2)

[1211] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1212] This invention relates to a system that quickly and accurately monitors an infant's condition. In particular, it collects and analyzes an infant's crying and vital signs, and provides appropriate notifications to parents and caregivers, thereby reducing the burden of managing and caring for the infant's health and enabling prompt responses. It also aims to improve the user experience by taking into account the user's emotional state and providing more appropriate notification methods when the user is feeling stressed.

[1213] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1214] In this invention, the server includes means for recording the infant's voice, means for analyzing the recorded voice to estimate the infant's condition, means for creating a notification message based on the estimated infant's condition, means for sending the created notification message to a user's terminal, means for displaying the notification message on the user's terminal, a machine learning model for analyzing the collected infant's voice data, an emotion engine for analyzing the user's emotional state, means for adjusting the content of the notification message based on the analysis results of the emotion engine, means for collecting the infant's vital data and detecting abnormalities, and means for generating and sending an emergency alert when an abnormality is detected. This enables a detailed understanding of the infant's condition based on the infant's voice and vital data, and appropriate notifications corresponding to the user's emotional state.

[1215] "Infant" refers to a young child who is not yet able to use language or clearly express his or her own will.

[1216] "Audio data" refers to information recorded as sound, such as an infant's crying or laughter.

[1217] "Vital data" refers to data that indicates an infant's physiological condition, such as heart rate, body temperature, and respiratory rate.

[1218] "Machine learning model" means a computational model based on algorithms that analyze data and learn patterns and trends.

[1219] The "emotion engine" is a system that analyzes a user's facial expressions captured by a camera and identifies their emotional state.

[1220] "Recording" refers to saving sounds such as an infant's crying or laughter using a recording device.

[1221] "Analysis" refers to processing and analyzing the recorded voice data and vital data, and deriving the infant's condition and characteristics from that information.

[1222] "Estimation" refers to determining the infant's condition based on the recorded data.

[1223] "Notification message" refers to message information sent to parents or caregivers based on the analysis results.

[1224] An "urgent alert" refers to a highly urgent message that is sent to the user immediately when an abnormality is detected in the infant's vital data.

[1225] "Device" refers to a device that receives and displays notification messages, such as a smartphone or tablet.

[1226] "User" refers to the parent or caregiver of the infant.

[1227] "Collection" refers to using a device to collect an infant's voice and vital data.

[1228] "Abnormal" refers to vital data or an infant's condition that is outside the normal range.

[1229] "Capture" refers to using a camera to obtain an image of a user's facial expression.

[1230] "Generation" means creating notification messages or emergency alerts based on the analysis results.

[1231] "Sending" refers to sending the generated notification message or emergency alert to the user's device via the Internet.

[1232] "Display" refers to showing the sent notification message or emergency alert on the device screen.

[1233] The present invention provides a system that collects and analyzes an infant's voice and vital data and notifies the results to users (parents and caregivers). The system of the present invention is implemented using the following hardware and software.

[1234] Hardware and Software Use

[1235] Devices: Smartphones, tablets, smart baby monitors, etc. are used to collect voice and vital data.

[1236] Server: A server for data processing and analysis, where machine learning models and emotion engines are installed.

[1237] Software: TensorFlow (implementation of machine learning models), OpenCV (image processing), requests (HTTP request library), etc.

[1238] Detailed Description of the Invention

[1239] 1. Collection of audio data

[1240] The device records the baby's cries, laughter, and other sounds. The recording device is a microphone on a smartphone or a dedicated device.

[1241] Audio data is automatically recorded at regular intervals.

[1242] 2. Sending audio data

[1243] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[1244] 3. Analysis of audio data

[1245] The server analyzes the received voice data using machine learning models such as TensorFlow.

[1246] The machine learning model is trained to identify characteristic speech patterns of infants.

[1247] 4. Creating and sending notification messages

[1248] The server creates a notification message based on the analysis results.

[1249] The created notification message is sent to the terminal via the Internet.

[1250] The device will display a notification message to the user, either in the notification bar of the smartphone or tablet or within a dedicated app.

[1251] 5. Vital Data Collection and Analysis

[1252] The device collects the baby's vital data (heart rate and temperature), which is collected using a smart baby monitor or wearable device.

[1253] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[1254] 6. Creating and sending emergency alerts

[1255] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[1256] Emergency alerts are sent to your device via the internet.

[1257] The device will highlight the emergency alert to inform the user of the urgency.

[1258] 7. Use of Emotion Engines

[1259] The device uses a camera to capture the user's facial expressions.

[1260] The captured image data is sent to a server via the Internet.

[1261] The server uses an emotion engine to analyze image data and determine the user's emotional state. The analysis uses image processing libraries such as OpenCV.

[1262] If the user is feeling stressed, the content and expression of the notification message are softened and sent.

[1263] Specific examples

[1264] 1. Collection and analysis of audio data

[1265] For example, the device records a baby's crying for five seconds and sends the recording to a server via the internet. The machine learning model analyzes the audio data and generates status data such as "I want to change his / her diaper."

[1266] Example text:

[1267] Please explain the process of recording a baby crying.

[1268] 2. Collection and analysis of vital data

[1269] For example, a device measures an infant's temperature using a smart baby monitor and transmits the measurement data via the Internet to a server, which analyzes the vital data and detects abnormal body temperature (e.g., high fever).

[1270] Example text:

[1271] How can I use a smart baby monitor to collect my newborn's vital data (heart rate, temperature) and notify me if any abnormalities are detected?

[1272] 3. Use of Emotion Engine

[1273] For example, the device uses a camera to capture the user's facial expressions and sends the image data to a server via the Internet. The server then analyzes the image data using an emotion engine, and if it determines that the user is feeling stressed, it softens the tone of the notification message and sends it.

[1274] Example text:

[1275] Describe the process of detecting a parent's emotional state and adjusting notification messages.

[1276] In this way, the multi-functional system can accurately grasp the infant's condition and provide notifications and alerts that take into account the user's emotional state.

[1277] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1278] Step 1:

[1279] The device records the baby's voice data. The baby's cries and laughter are recorded using a microphone on a smartphone or dedicated device and saved as audio data. The input is the baby's cries and laughter, and the output is the saved audio data. Specifically, the device works by pressing the record button to record audio for a set period of time (for example, 5 seconds), and then saving the audio data in the device.

[1280] Step 2:

[1281] The audio data recorded by the device is sent to a server via the Internet. The audio data is uploaded to the server using a secure communication method such as the HTTPS protocol. The input is the audio data stored on the device, and the output is the audio data uploaded to the server. Specifically, after recording is complete, the data transmission module is automatically activated and the audio data is sent to the server.

[1282] Step 3:

[1283] The server analyzes the received voice data. The server processes the voice data using a machine learning model to identify the characteristics and patterns of the infant's voice. The input is the voice data received by the server, and the output is the infant's estimated state (e.g., "I want my diaper changed" or "I'm sleepy"). Specifically, the voice data is preprocessed, input into the machine learning model, and analyzed, and the infant's state is estimated as a result.

[1284] Step 4:

[1285] The server creates a notification message based on the estimated infant condition. Based on the analysis results, the notification message is generated in a format that is easy for parents and caregivers to understand. The input is the estimated infant condition, and the output is the created notification message. Specifically, the server retrieves an appropriate message template from the database, embeds the estimated condition, and generates the notification message.

[1286] Step 5:

[1287] The server sends the created notification message to the terminal via the Internet. The input is the created notification message, and the output is the notification message sent to the terminal. Specifically, the server uses a communication module to send the notification message to the terminal's address.

[1288] Step 6:

[1289] Displays notification messages received by the device. Notification messages are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app. The input is the notification message sent from the server, and the output is the notification message displayed on the device screen. Specifically, the notification is displayed in the notification bar when it is received, and the notification content is displayed in detail when the app is opened.

[1290] Step 7:

[1291] The device collects the infant's vital data. A smart baby monitor or wearable device is used to measure the infant's vital data, such as heart rate and body temperature, and the data is stored in the device. The input is the infant's biological signals, and the output is the measured vital data. Specifically, the device periodically monitors the vital signs of the infant wearing it and collects the data in real time.

[1292] Step 8:

[1293] The vital data collected by the device is sent to a server via the internet. A secure communication protocol is used to upload the vital data to the server. The input is the vital data stored on the device, and the output is the vital data uploaded to the server. Specifically, the device sends the collected vital data to the server at regular intervals.

[1294] Step 9:

[1295] The server analyzes the vital data received and determines whether there is an abnormality. Statistical algorithms and machine learning models are used to identify vital data that deviates from the normal range. The input is the vital data received by the server, and the output is the determination result when an abnormality is detected. Specifically, the vital data is input into the analysis module, and an alert is generated when an abnormal value is detected.

[1296] Step 10:

[1297] The server creates an emergency alert when an anomaly is detected. A highly urgent message is generated based on the detected anomaly. The input is the anomaly judgment result, and the output is the generated emergency alert. Specifically, when an anomaly is detected, an emergency message template is retrieved from the database and an alert is generated using this template.

[1298] Step 11:

[1299] The server sends the generated emergency alert to the terminal. The input is the generated emergency alert, and the output is the emergency alert sent to the terminal. Specifically, the server uses a communication module to send the emergency alert to the terminal's address.

[1300] Step 12:

[1301] Displays emergency alerts received by the device. The alert is notified to the user by highlighting it in the notification bar or within the app. The input is the emergency alert sent from the server, and the output is the emergency alert displayed on the device screen. Specifically, when an emergency alert is received, it is highlighted in the notification bar, and when the app is opened, the details of the alert are displayed.

[1302] Step 13:

[1303] The device uses a camera to capture the user's facial expression. The image data captured by the camera is stored in the device. The input is an image of the user's facial expression, and the output is the stored image data. Specifically, the device starts up the camera, captures the facial expression, and saves the data.

[1304] Step 14:

[1305] The image data captured by the device is sent to the server via the Internet. The image data is uploaded to the server. The input is the image data stored on the device, and the output is the image data uploaded to the server. Specifically, the transmission module automatically sends the image data that has been captured to the server.

[1306] Step 15:

[1307] The server analyzes the received image data and determines the user's emotional state. An emotion engine is used to identify emotions from the user's facial expressions. The input is the image data received by the server, and the output is the emotional state determined by the analysis. Specifically, the received image data is input into the emotion engine, the emotional state is analyzed, and a determination result is obtained.

[1308] Step 16:

[1309] The server adjusts the content of the notification message based on the analysis results. The notification message is created in a format appropriate for the user's emotional state. The input is the determined emotional state, and the output is the adjusted notification message. Specifically, the server adjusts the message template based on the results of the emotion analysis, and generates a notification message that takes emotions into consideration.

[1310] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1311] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1312] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1313] [Third embodiment]

[1314] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1315] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1316] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1317] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1318] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1319] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1320] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1321] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1322] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1323] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1324] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1325] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1326] MODE FOR CARRYING OUT THE INVENTION

[1327] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), thereby enabling the user to accurately grasp the infant's condition and take prompt action. The system of the present invention includes the following various means.

[1328] 1. Collection of audio data

[1329] The device records the baby's cries, laughter, and other sounds. For example, the baby's voice can be recorded using a microphone on a smartphone or a dedicated device. Recording starts at regular intervals, and the audio data is automatically acquired.

[1330] 2. Sending audio data

[1331] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[1332] 3. Analysis of audio data

[1333] The server analyzes the received voice data and estimates the infant's condition using a machine learning model based on the infant's crying patterns and voice characteristics.

[1334] 4. State Data Generation

[1335] The server generates data on the baby's condition based on the analysis of the voice data, identifying, for example, the baby's condition such as "needs a diaper change," "sleepy," or "hungry."

[1336] 5. Creating a notification message

[1337] The server generates notification messages for parents and caregivers based on the estimated infant status in a format that is easy for parents to understand.

[1338] 6. Sending Notifications

[1339] The notification message created by the server is sent to the terminal via the Internet.

[1340] 7. Display of notifications

[1341] Displaying notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[1342] 8. Collecting vital data (optional)

[1343] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Vital data is acquired using a smart baby monitor or wearable device.

[1344] 9. Analysis of vital data

[1345] The server analyzes the received vital data and determines whether there are any abnormalities, thereby understanding the infant's detailed physiological condition.

[1346] 10. Creating Emergency Alerts

[1347] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[1348] 11. Sending emergency alerts

[1349] The server sends an emergency alert message to the terminal.

[1350] 12. Displaying emergency alerts

[1351] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[1352] Specific examples

[1353] A specific example of the operation of the system is shown below.

[1354] Example 1: A crying baby

[1355] 1. The device will record your baby's crying for 5 seconds.

[1356] 2. The device sends the recorded data to a server via the Internet.

[1357] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[1358] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[1359] 5. The server creates a notification message saying "Diaper needs changing."

[1360] 6. The server sends a notification message to the terminal.

[1361] 7. The device displays the notification message in the notification bar of the user's smartphone.

[1362] 8. The user checks the notification and changes the baby's diaper.

[1363] Example 2: Vital signs monitoring

[1364] 1. The device measures the baby's temperature using a smart baby monitor.

[1365] 2. The device sends the measurement data to a server via the Internet.

[1366] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[1367] 4. The server creates an emergency alert: "Infant has fever."

[1368] 5. The server sends an emergency alert to the device.

[1369] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[1370] 7. The user checks the emergency alert and takes necessary measures.

[1371] In this way, the system of the present invention can accurately grasp the condition and needs of the infant and assist in prompt response.

[1372] The processing flow will be explained below.

[1373] Program processing steps

[1374] Step 1:

[1375] The device records the baby's voice. The device's microphone is used to record the baby's crying and laughter for a certain period of time (e.g., 5 seconds).

[1376] Step 2:

[1377] The device stores the recorded audio data in a buffer and uploads it to a server via the Internet using a secure communication protocol such as HTTPS.

[1378] Step 3:

[1379] The server decodes the received audio data and begins analyzing it using a machine learning model (e.g., Convolutional Neural Network) to identify patterns and characteristics of crying sounds.

[1380] Step 4:

[1381] The server generates estimated data on the infant's condition based on the results of analyzing the voice data. Specifically, it creates textual data on the infant's condition, such as "needs a diaper change," "sleepy," and "hungry."

[1382] Step 5:

[1383] The server creates notification messages based on the generated status data, and the notification messages are presented in a format that is easy for parents and caregivers to understand.

[1384] Step 6:

[1385] The notification message created by the server is sent to the device via the Internet as a push notification.

[1386] Step 7:

[1387] The device decodes the notification message received and displays it in the notification bar or dedicated app on the smartphone or tablet.

[1388] Step 8:

[1389] The device measures the infant's vital data (e.g., heart rate and body temperature) using a smart baby monitor or wearable device and stores the measured data in a buffer.

[1390] Step 9:

[1391] The vital data measured by the device is uploaded to a server via the Internet.

[1392] Step 10:

[1393] The server analyzes the received vital data and detects abnormal values ​​(e.g., high or low fever) using statistical algorithms and machine learning models.

[1394] Step 11:

[1395] If the server detects abnormal data, it will create an emergency alert. For example, it will generate an alert message saying "The baby has a fever. Please take action immediately."

[1396] Step 12:

[1397] The server creates an emergency alert message and sends it to the device as a push notification via the Internet.

[1398] Step 13:

[1399] The device decodes emergency alert messages received and displays them in the notification bar and in the app with special highlighting, thereby informing the user of the urgency of the message.

[1400] Step 14:

[1401] The user checks notifications and emergency alerts and takes appropriate action (e.g., changing a diaper, checking a temperature).

[1402] Example 1

[1403] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1404] Rapid and accurate understanding of an infant's health condition and needs is an important challenge for parents and caregivers. However, because infants cannot communicate their condition verbally, they must rely on indirect information such as crying and vital signs. In conventional systems, voice data and vital signs are often collected and analyzed separately, making it difficult to centrally manage and comprehensively analyze this data. In addition, there are limitations to the means of quickly notifying users of analysis results, which can delay response, especially in urgent situations.

[1405] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1406] In this invention, the server includes means for recording voice data of the infant, means for periodically starting and saving the recorded voice data, means for transmitting the recorded voice data to the server using a secure communication protocol, means for analyzing the received voice data based on a machine learning model to estimate the infant's condition, means for generating estimated infant condition data, means for creating a notification message that is easy for the user to understand, means for transmitting the notification message to the user's terminal using the secure communication protocol, and means for displaying the notification message on the user's terminal, thereby enabling the infant's health condition and needs to be quickly and accurately grasped and appropriate measures to be taken.

[1407] "Means for recording infant sounds" refers to devices and methods for recording sounds such as infant crying and laughter.

[1408] "Means for periodically initiating and saving recorded audio" refers to a device or method for automatically initiating audio recording at set intervals and saving the data.

[1409] "Means for transmitting recorded audio to a server using a secure communication protocol" refers to a device or method for transmitting acquired audio data to a server while maintaining information security.

[1410] "Means for analyzing received voice data based on a machine learning model and estimating the infant's condition" refers to a device or method for analyzing voice data using machine learning technology and estimating a specific infant condition (e.g., hunger, sleepiness, etc.).

[1411] The "means for generating estimated infant condition data" refers to a device or method for converting the infant condition into a data format based on the results of voice analysis.

[1412] The "means for generating user-friendly notification messages" refers to devices or methods for generating notifications based on the estimated infant condition in a format that is easy for parents or caregivers to understand.

[1413] The "means for transmitting a notification message to a user terminal using a secure communication protocol" refers to a device or method for transmitting the generated notification message to a user terminal while maintaining information security.

[1414] The "means for displaying a notification message on a user's terminal" refers to a device or method for visually displaying a received notification message on a user's terminal.

[1415] "Means for measuring infant vital data" refers to devices and methods for measuring physiological indicators such as an infant's heart rate and body temperature.

[1416] "Means for transmitting measured vital data to a server using a secure communication protocol" refers to a device or method for transmitting acquired vital data to a server while maintaining information security.

[1417] The "means for analyzing received vital data and determining whether an abnormal state exists" refers to a device or method for analyzing acquired vital data and determining whether an abnormal state exists.

[1418] The "means for generating an emergency alert when abnormal vital data is detected" refers to a device or method for generating a message to call for urgent attention when abnormal vital data is detected.

[1419] "Means for sending an emergency alert to a user's terminal" refers to a device or method for sending the generated emergency alert message to a user's terminal.

[1420] A "means for specially highlighting emergency alerts on a user's terminal" is a device or method for highlighting emergency alert messages on a user's terminal.

[1421] MODE FOR CARRYING OUT THE INVENTION

[1422] System configuration

[1423] The system of the present invention includes the following means to quickly and accurately grasp an infant's health condition and needs and support appropriate responses. The hardware used includes smartphones, tablets, dedicated devices, smart baby monitors, and wearable devices. The software uses machine learning models and Firebase Cloud Messaging (FCM).

[1424] Audio data collection

[1425] The device records sounds such as the baby's crying and laughter. Specifically, it uses the smartphone's microphone to start recording at regular intervals and automatically saves the audio data.

[1426] Sending audio data

[1427] The audio data recorded by the device is sent to the server via the Internet. The HTTPS protocol is used for transmission to ensure security. The audio data recorded is sent to the API endpoint as a POST request.

[1428] Analysis of audio data

[1429] The server analyzes the received audio data using a machine learning model that has learned patterns of infant crying and laughter, and uses this data to estimate the infant's condition.

[1430] Generate state data

[1431] The server generates data on the baby's condition based on the analysis results. For example, if the baby continues crying, it may assume that the baby needs a diaper change or is hungry. This data is saved in JSON format and the next step is reached.

[1432] Creating a notification message

[1433] The server generates an easy-to-understand notification message for parents or caregivers based on the estimated infant's condition, such as "Your baby is crying and needs a diaper change."

[1434] Sending notifications

[1435] The server creates a notification message and sends it to the device via the Internet using Firebase Cloud Messaging (FCM) or another push notification service.

[1436] Viewing notifications

[1437] Display notification messages received by the device. Displays messages in the notification bar of smartphones and tablets to notify the user.

[1438] Vital data collection (optional)

[1439] The device measures the baby's vital data (heart rate and temperature) and sends it to a server. The data is then collected via Bluetooth using a smart baby monitor or wearable device.

[1440] Analysis of vital data

[1441] The server analyzes the received vital data and determines whether there are any abnormalities. An abnormal data detection algorithm is used to detect an excessively high heart rate or abnormal body temperature.

[1442] Creating an emergency alert

[1443] If the server detects abnormal vital data, it creates an emergency alert message, such as "The baby has a fever," and treats it as a priority message.

[1444] Sending emergency alerts

[1445] The server sends an emergency alert message to the device, using a push notification service such as Firebase Cloud Messaging (FCM) to send the notification immediately.

[1446] Viewing emergency alerts

[1447] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[1448] Specific examples

[1449] Example 1: A crying baby

[1450] 1. The device will record your baby's crying for 5 seconds.

[1451] 2. The device sends the recorded data to a server via the Internet.

[1452] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[1453] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[1454] 5. The server creates a notification message saying "Diaper needs changing."

[1455] 6. The server sends a notification message to the terminal.

[1456] 7. The device displays the notification message in the notification bar of the user's smartphone.

[1457] 8. The user checks the notification and changes the baby's diaper.

[1458] Example 2: Vital signs monitoring

[1459] 1. The device measures the baby's temperature using a smart baby monitor.

[1460] 2. The device sends the measurement data to a server via the Internet.

[1461] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[1462] 4. The server creates an emergency alert: "Infant has fever."

[1463] 5. The server sends an emergency alert to the device.

[1464] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[1465] 7. The user checks the emergency alert and takes necessary measures.

[1466] In this way, the system of the present invention is able to quickly and accurately grasp the health condition and needs of an infant and support appropriate responses.

[1467] Prompt Sentence Examples

[1468] "Please analyze and let me know why my baby is crying."

[1469] "Please take the infant's temperature and issue an alert if there is anything abnormal."

[1470] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1471] System program processing flow

[1472] Step 1: Collecting audio data

[1473] The device records sounds such as the baby's crying and laughter. First, it uses the smartphone's microphone to start recording at regular intervals and saves the audio data. For example, a dedicated app can automatically start a 5-second recording and save the data on the device.

[1474] Input: Trigger to start recording

[1475] Output: Audio file (e.g., wav format)

[1476] Step 2: Sending audio data

[1477] The audio data recorded by the device is sent to the server via the Internet using the HTTPS protocol to ensure data security. Specifically, the audio file is sent as a POST request to the API endpoint.

[1478] Input: Audio file

[1479] Output: Sending status (success / failure)

[1480] Step 3: Analyzing the audio data

[1481] The server analyzes the received audio data using a machine learning model. First, the audio data is input into the machine learning model. This model has learned the patterns of infant crying and laughter and analyzes the audio features. Specifically, it extracts features such as the frequency components and volume of the crying sounds and uses them to estimate the infant's condition.

[1482] Input: Audio data

[1483] Output: Analysis results (e.g. "I want to change your diaper")

[1484] Step 4: Generate state data

[1485] The server generates data on the infant's condition based on the analysis of the voice data. For example, it identifies the infant's condition, such as "needs a diaper change," "hungry," or "sleepy." The inferred results are saved in a structured data format (e.g., JSON).

[1486] Input: Analysis results

[1487] Output: State data (JSON format)

[1488] Step 5: Create a notification message

[1489] The server creates a user-friendly notification message based on the estimated baby's condition, such as "Your baby is crying. Your diaper needs to be changed."

[1490] Input: State data

[1491] Output: Notification message

[1492] Step 6: Sending notifications

[1493] The notification message created by the server is sent to the device via the Internet using a push notification service such as Firebase Cloud Messaging (FCM).

[1494] Input: Notification message

[1495] Output: Sending status (success / failure)

[1496] Step 7: View notifications

[1497] Displaying notification messages received by the device. Visually displaying notifications to the user in the notification bar of a smartphone or tablet or within a dedicated app. For example, displaying a pop-up in the notification bar saying "Diaper needs changing."

[1498] Input: Notification message

[1499] Output: Displayed notification

[1500] Step 8: Collect vital data (optional)

[1501] The device measures the baby's vital data (heart rate and temperature) and sends it to a server. Data is collected using a smart baby monitor or wearable device and stored on the device via Bluetooth.

[1502] Input: Trigger to start measurement

[1503] Output: Vital data

[1504] Step 9: Analyze vital data

[1505] The server analyzes the received vital data and determines whether there are any abnormalities, using an anomaly detection algorithm to detect, for example, an extremely high heart rate or an abnormal body temperature.

[1506] Input: Vital data

[1507] Output: Abnormal condition (e.g. "High fever")

[1508] Step 10: Create an emergency alert

[1509] If the server detects abnormal vital data, it creates an emergency alert message, for example, a message with high urgency such as "The baby has a fever" and marks it as an emergency alert.

[1510] Input: Abnormal condition

[1511] Output: Emergency alert message

[1512] Step 11: Sending emergency alerts

[1513] The server sends an emergency alert message to the device using a push notification service such as Firebase Cloud Messaging (FCM).

[1514] Input: Emergency alert message

[1515] Output: Sending status (success / failure)

[1516] Step 12: View emergency alerts

[1517] The device displays an emergency alert to the user, highlighting it with a special icon or popup in the notification bar of the smartphone or tablet to communicate the urgency to the user. For example, the notification bar will display "Urgent: Baby has a fever!" in red.

[1518] Input: Emergency alert message

[1519] Output: Emergency notification displayed

[1520] (Application example 1)

[1521] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1522] Conventionally, monitoring of an infant's crying and physiological data and notifications based on the results have mainly been done manually by parents or caregivers, which can make it difficult to respond quickly. Furthermore, since the infant's health condition cannot be grasped in real time, there is a risk of missing signs of serious health abnormalities. The objective of this invention is to provide a system that efficiently collects and analyzes an infant's voice and vital data and promptly notifies the user.

[1523] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1524] In this invention, the server includes means for recording the infant's voice, means for analyzing the recorded voice to estimate the infant's condition, means for creating a notification message based on the estimated infant's condition, means for transmitting the created notification message to a user's terminal, means for displaying the notification message on the user's terminal, means for using a wearable device to measure the infant's vital data, means for analyzing the measured vital data to detect abnormalities, means for creating an emergency alert based on the detected abnormality, and means for transmitting the created emergency alert to the user's smart glasses or smartphone and displaying the notification, thereby enabling the infant's health condition to be understood in real time and responding promptly.

[1525] "Infant voice" refers to audio signals such as crying and laughing emitted by an infant.

[1526] "Recording" means recording and storing audio and data.

[1527] "Analysis" is the process of examining collected data and identifying meanings and patterns in it.

[1528] "Condition" refers to a specific situation or condition, such as an infant's health, emotions, or care needs.

[1529] A "notification message" is a message containing information that is sent to a user based on the infant's condition.

[1530] A "user device" is an electronic device used by a parent or caregiver, such as a smartphone or smart glasses.

[1531] "Display" means the visual presentation of information on a screen or display.

[1532] "Vital data" refers to data that indicates an infant's physiological indicators, including heart rate and body temperature.

[1533] A "wearable device" is a device worn by an infant to measure vital data.

[1534] An "abnormality" is a condition that is outside the normal range and may affect health.

[1535] An "urgent alert" is an immediate notification when a serious health or safety issue is detected.

[1536] A "generative AI model" is a machine learning algorithm that analyzes an infant's voice patterns and vital data based on training data.

[1537] A "server" is a central computer system that analyzes and processes data and sends the results to the user's terminal.

[1538] The system of the present invention efficiently collects and analyzes an infant's voice and vital data, and notifies the user to grasp the infant's condition in real time and support prompt response. A specific embodiment of this system is shown below.

[1539] System Configuration

[1540] The system mainly consists of the following components:

[1541] 1. Device: This refers to a smartphone or smart glasses equipped with a microphone and a display, capable of displaying audio data and notifications.

[1542] 2. Server: Plays the central role in data analysis, analyzing voice and vital data using generative AI models.

[1543] 3. Wearable devices: Smart bands that infants can wear to measure vital data such as heart rate and body temperature.

[1544] 4. Generative AI model: A machine learning algorithm that analyzes voice patterns and infant physiological indicators based on training data.

[1545] Hardware and Software Use

[1546] Terminal

[1547] The terminal records voice data and transmits it to the server. It also transmits vital data acquired from the wearable device to the server. It has the function of displaying notification messages and emergency alerts when they are generated.

[1548] server

[1549] The server receives the voice and vital data sent from the device and analyzes them using a generative AI model. Based on the analysis results, it estimates the infant's condition and creates notification messages and emergency alerts.

[1550] Wearable devices

[1551] Wearable devices are used to measure vital data such as the baby's heart rate and body temperature, and the measured data is sent to a terminal via communication means such as Bluetooth.

[1552] Data processing and calculation

[1553] 1. Audio data collection:

[1554] The device records the baby's crying, laughter, and other sounds, for example, five seconds of audio, and saves it as digital data.

[1555] 2. Sending audio data:

[1556] The device sends the recorded audio data to a server via the Internet using the HTTPS protocol for secure data transfer.

[1557] 3. Analysis of audio data:

[1558] The server analyzes the received voice data using a generative AI model, capturing the characteristics of the voice pattern and inferring what the baby needs.

[1559] 4. Create and send a notification message:

[1560] Based on the estimated baby's condition, the server generates a notification message in a format that can be quickly communicated to the user, and the generated message is then sent to the terminal via the Internet.

[1561] 5. Displaying notification messages:

[1562] The device will then display the received notification message to the user: on a smartphone, it will appear in the notification bar, and on the display of smart glasses.

[1563] 6. Vital Data Collection:

[1564] The wearable device measures the infant's heart rate and temperature and transmits the data to a terminal.

[1565] 7. Analysis of vital data:

[1566] The server analyzes the received vital data and creates an emergency alert if it detects any abnormalities.

[1567] 8. Creating and sending emergency alerts:

[1568] When an abnormality is detected, the server quickly creates an emergency alert and sends it to the device.

[1569] 9. Emergency alert display:

[1570] The device will highlight the emergency alert and notify the user.

[1571] Specific examples

[1572] Example 1: A crying baby

[1573] The device records the baby's crying for 5 seconds and sends it to the server.

[1574] The server analyzes the voice data and determines that the diaper needs to be changed.

[1575] A notification message "Diaper change required" is generated and sent to the user's terminal.

[1576] The device will notify the user by displaying a message in the notification bar or on the display of the smart glasses.

[1577] Example 2: Vital signs monitoring

[1578] The wearable device measures the infant's temperature and transmits it to a terminal.

[1579] The server analyzes the body temperature data and detects high fever.

[1580] An emergency alert "Infant has fever" is generated and sent to the user's device.

[1581] The device will highlight emergency alerts and notify the user quickly.

[1582] Example prompt sentence:

[1583] "Analyzing audio data of an infant crying and identifying the cause."

[1584] "Analyze infant heart rate data to detect abnormalities."

[1585] In this way, the system of the present invention can grasp the condition of the infant in real time and promote quick response.

[1586] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1587] Step 1:

[1588] The device records sounds such as the baby's crying and laughter. Specifically, it uses the device's built-in microphone to record audio data for a specified period of time (e.g., 5 seconds). The input here is a real-time audio signal, and the output is digital audio data. The recorded audio data is temporarily stored in a buffer.

[1589] Step 2:

[1590] The device sends the recorded audio data to the server via the Internet. The HTTPS protocol is used for communication to ensure data security. The input here is the recorded audio data, and the output is the data to be sent to the server. Once the transmission is complete, the device receives a confirmation message.

[1591] Step 3:

[1592] The server inputs the received voice data into a generative AI model for analysis. The generative AI model is pre-trained and analyzes the voice features to estimate the infant's state. The input here is the voice data sent to the server, and the output is the infant's estimated state (e.g., "I want to change my diaper" or "I'm hungry"). The AI ​​model analyzes the voice patterns and probabilistically estimates the most appropriate state.

[1593] Step 4:

[1594] The server creates a notification message based on the estimated infant's state. For example, if the AI ​​model estimates that the infant needs a diaper change, it generates a notification message saying, "Your diaper needs to be changed." The input here is the estimated infant's state, and the output is the generated notification message. The server adds this to a message queue.

[1595] Step 5:

[1596] The server then sends the created notification message to the user's terminal, where the input is the notification message and the output is the data sent to the user's terminal. The notification message is securely transmitted over the Internet.

[1597] Step 6:

[1598] The device displays the received notification message. In the case of a smartphone, this is displayed in the notification bar, and in the case of smart glasses, it is visually displayed on the display. The input here is the notification message sent from the server, and the output is the displayed message. The user checks the notification and responds.

[1599] Step 7:

[1600] The wearable device measures the infant's vital data (heart rate and body temperature). The input is the infant's physiological data, and the output is the measured vital data. These data are measured periodically and sent to the terminal.

[1601] Step 8:

[1602] The device sends the received vital data to a server via the Internet. The input here is the measured vital data, and the output is the data to be sent to the server. Data transmission is via Bluetooth or Wi-Fi.

[1603] Step 9:

[1604] The server analyzes the received vital data and detects abnormalities. The generative AI model identifies abnormal patterns and detects abnormalities such as high fever or abnormal heart rate. The input here is the vital data, and the output is the detected abnormality and detailed information about it.

[1605] Step 10:

[1606] The server generates an emergency alert based on the detected anomaly, for example, a message saying "The baby has a fever." Here, the input is the anomaly detection result, and the output is the emergency alert message. The alert is treated as a high-priority message.

[1607] Step 11:

[1608] The server sends the created emergency alert to the user's device. The input here is the emergency alert message, and the output is the data to be sent to the user's device. The server uses a message queue to send the alert immediately.

[1609] Step 12:

[1610] The device highlights the received emergency alert. In the case of a smartphone, it is displayed in the notification bar with an emergency icon, and in the case of smart glasses, it is displayed with a special effect on the display. Here, the input is the emergency alert sent from the server, and the output is a visually highlighted message. The user can check the alert and take necessary measures.

[1611] Example prompt sentence:

[1612] "Analyzing audio data of an infant crying and identifying the cause."

[1613] "Analyze infant heart rate data to detect abnormalities."

[1614] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1615] MODE FOR CARRYING OUT THE INVENTION

[1616] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), enabling the user to accurately grasp the infant's condition and respond promptly. The system of the present invention includes the following means:

[1617] 1. Collection of audio data

[1618] The device records the baby's crying, laughter, and other sounds. For example, the baby's voice can be recorded using a microphone on a smartphone or a dedicated device. Recording begins at regular intervals, and the audio data is automatically acquired.

[1619] 2. Sending audio data

[1620] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[1621] 3. Analysis of audio data

[1622] The server analyzes the received voice data and estimates the infant's condition using a machine learning model based on the infant's crying patterns and voice characteristics.

[1623] 4. State Data Generation

[1624] The server generates data on the baby's condition based on the analysis of the voice data, identifying, for example, whether the baby needs a diaper change, is sleepy, or is hungry.

[1625] 5. Creating a notification message

[1626] The server generates notification messages for parents and caregivers based on the estimated infant status in a format that is easy for parents to understand.

[1627] 6. Sending Notifications

[1628] The notification message created by the server is sent to the terminal via the Internet.

[1629] 7. Display of notifications

[1630] Displaying notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[1631] 8. Collecting vital data (optional)

[1632] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Vital data is acquired using a smart baby monitor or wearable device.

[1633] 9. Analysis of vital data

[1634] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[1635] 10. Creating Emergency Alerts

[1636] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[1637] 11. Sending emergency alerts

[1638] The server sends an emergency alert message to the terminal.

[1639] 12. Displaying emergency alerts

[1640] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[1641] Furthermore, the present invention includes the following features:

[1642] 13. Use of Emotion Engines

[1643] The device uses an emotion engine to recognize the user's emotions, for example by capturing the user's facial expressions with a camera and analyzing the data to identify emotions.

[1644] 14. Transmission of Emotional Data

[1645] The user's emotion data recognized by the device is sent to a server via the Internet.

[1646] 15. Emotional Data Analysis

[1647] The server analyzes the received emotion data and determines the user's emotional state, for example, whether the user is feeling stressed.

[1648] 16. Notification Coordination

[1649] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened.

[1650] Specific examples

[1651] Example 1: A crying baby

[1652] 1. The device will record your baby's crying for 5 seconds.

[1653] 2. The device sends the recorded data to a server via the Internet.

[1654] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[1655] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[1656] 5. The server creates a notification message saying "Diaper needs changing."

[1657] 6. The server sends a notification message to the terminal.

[1658] 7. The device displays the notification message in the notification bar of the user's smartphone.

[1659] 8. The user checks the notification and changes the baby's diaper.

[1660] Example 2: Vital signs monitoring

[1661] 1. The device measures the baby's temperature using a smart baby monitor.

[1662] 2. The device sends the measurement data to a server via the Internet.

[1663] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[1664] 4. The server creates an emergency alert: "Infant has fever."

[1665] 5. The server sends an emergency alert to the device.

[1666] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[1667] 7. The user checks the emergency alert and takes necessary measures.

[1668] Example 3: Using the Emotion Engine

[1669] 1. The device uses a camera to capture the user's facial expressions.

[1670] 2. The device sends the captured image data to a server via the Internet.

[1671] 3. The server uses an emotion engine to analyze the image data and determine the user's emotional state.

[1672] 4. If the server determines that the user is feeling stressed, it creates a notification message with careful consideration of its content and expression.

[1673] 5. The server sends the adjusted notification message to the terminal.

[1674] 6. The device will display adjusted notification messages, making notifications more appropriate for users.

[1675] In this way, the multi-functional system can accurately grasp the infant's condition and provide notifications and alerts that take into account the user's emotional state.

[1676] The processing flow will be explained below.

[1677] Program processing steps

[1678] Baby crying analysis

[1679] Step 1:

[1680] The device records the baby's voice. The device's microphone is used to record the baby's crying and laughter for a certain period of time (e.g., 5 seconds).

[1681] Step 2:

[1682] The device stores the recorded audio data in a buffer and uploads it to a server via the Internet using a secure communication protocol such as HTTPS.

[1683] Step 3:

[1684] The server decodes the received audio data and analyzes it using a machine learning model (e.g., Convolutional Neural Network) to identify patterns and characteristics of crying sounds.

[1685] Step 4:

[1686] The server generates estimated data on the infant's condition based on the results of analyzing the voice data, specifically identifying the infant's condition, such as "needs a diaper change," "sleepy," or "hungry."

[1687] Step 5:

[1688] The server generates notification messages based on the generated status data in a format that is easy for parents and caregivers to understand.

[1689] Step 6:

[1690] The notification message created by the server is sent to the device via the Internet as a push notification.

[1691] Step 7:

[1692] The device decodes the notification message received and displays it in the notification bar or dedicated app on the smartphone or tablet.

[1693] Vital data monitoring and emergency alerts

[1694] Step 8:

[1695] The device measures the infant's vital data (e.g., heart rate and body temperature) using a smart baby monitor or wearable device and stores the measured data in a buffer.

[1696] Step 9:

[1697] The vital data measured by the device is uploaded to a server via the Internet.

[1698] Step 10:

[1699] The server analyzes the received vital data and detects abnormal values ​​(e.g., high or low fever) using statistical algorithms and machine learning models.

[1700] Step 11:

[1701] If the server detects abnormal data, it will create an emergency alert, e.g., "The baby has a fever. Please take action immediately."

[1702] Step 12:

[1703] The server creates an emergency alert message and sends it to the device as a push notification via the Internet.

[1704] Step 13:

[1705] The device decodes emergency alert messages received and displays them in the notification bar and in the app with special highlighting, thereby informing the user of the urgency of the message.

[1706] Use of emotion engine

[1707] Step 14:

[1708] The device captures the user's facial expressions using a camera, for example, the front camera on a smartphone or tablet.

[1709] Step 15:

[1710] The image data captured by the device is uploaded to a server via the Internet.

[1711] Step 16:

[1712] The server uses an emotion engine to analyze the image data and determine the user's emotional state, such as whether they are "stressed" or "relaxed."

[1713] Step 17:

[1714] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. If the user is feeling stressed, the content and wording of the notification will be softened.

[1715] Step 18:

[1716] The server sends the adjusted notification message to the terminal via the Internet.

[1717] Step 19:

[1718] The device decodes the adjusted notification message and displays it in the notification bar or within a dedicated app on the smartphone or tablet, making notifications more appropriate for the user.

[1719] Example 2

[1720] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1721] In infant care, it is necessary to maintain the health and comfort of infants by effectively collecting and analyzing their voice and vital data and notifying parents and caregivers in a timely manner. In particular, it is necessary to improve the accuracy of analyzing infant voices and vital data and issue immediate alerts in the event of an emergency. It is also important to adjust notifications and alerts to take into account the caregiver's emotional state. However, conventional systems have difficulty meeting all of these requirements, making it difficult to accurately monitor infant conditions and respond promptly.

[1722] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for recording the infant's voice, a means for analyzing the recorded voice to estimate the infant's condition, a means for creating a notification message based on the estimated infant's condition, a means for sending the created notification message to a user's terminal, a means for displaying the notification message on the user's terminal, a means for recognizing the user's emotion, and a means for analyzing the recognized user's emotion data and adjusting the content and expression of the notification message. This makes it possible to accurately monitor the infant's health and comfort and notify parents or caregivers at appropriate times. Furthermore, it is possible to provide flexible notification messages according to the user's emotional state, thereby realizing more effective child-rearing support.

[1723] "Means for recording infant sounds" refers to an apparatus or device for recording sounds such as an infant's crying or laughing.

[1724] "Means for estimating an infant's condition by analyzing recorded voice" refers to technologies and algorithms that analyze collected voice data and infer an infant's condition based on voice patterns and characteristics.

[1725] A "means for generating a notification message" is a device or program for generating a message in a format that is easy for a parent or caregiver to understand based on the estimated infant condition.

[1726] The "means for transmitting the created notification message to the user's terminal" refers to a device or technology for transmitting the notification message to the user's terminal via a communication means such as the Internet.

[1727] The "means for displaying a notification message on a user's device" refers to a function or application for displaying a received notification message on a user's device such as a smartphone or tablet.

[1728] "Means for recognizing user emotions" refers to technologies and algorithms that collect a user's facial expressions and other data to identify their emotional state.

[1729] "Means for analyzing the recognized user emotional data and adjusting the content and expression of the notification message" refers to technologies and algorithms for analyzing collected emotional data and appropriately changing the content and expression of the notification message depending on the user's emotional state.

[1730] "Means for measuring infant vital data" refers to equipment or devices for obtaining physiological data such as an infant's heart rate and body temperature.

[1731] "Means for analyzing the measured vital data to determine the detailed condition of the infant" refers to technologies and algorithms for analyzing the collected vital data and determining the infant's health condition or abnormalities based on the results.

[1732] The "means for generating an emergency alert" is a device or program that immediately generates a highly urgent message when an abnormality is detected in the analyzed data.

[1733] The "means for transmitting the created emergency alert to the user's terminal" refers to a technique or device for transmitting the emergency alert to the user's terminal via a communication means such as the Internet.

[1734] "Means for displaying emergency alerts on a user's device" refers to a function or application for highlighting received emergency alerts on a user's device such as a smartphone or tablet.

[1735] A "machine learning model" is an algorithm or program that is trained to learn features from the voice and vital data being analyzed and to classify or predict the infant's condition.

[1736] "Distinctive infant vocal patterns" are vocal characteristics or patterns that indicate a particular state, such as an infant's crying or laughter.

[1737] "Physiological indicators" are physiological data that indicate the health or condition of an infant, such as heart rate, body temperature, and respiratory rate.

[1738] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), enabling the user to accurately grasp the infant's condition and take prompt action. The system of the present invention includes the following means.

[1739] Audio data processing

[1740] Audio data collection:

[1741] The device records sounds such as the baby's crying and laughter. The hardware used includes a microphone on a smartphone or a dedicated device. The device starts recording at regular intervals and automatically acquires audio data. For example, a 5-second recording is performed.

[1742] Sending audio data:

[1743] The device sends the recorded audio data to a server via the internet using a secure communication method such as the HTTPS protocol.

[1744] Analysis of audio data:

[1745] The server analyzes the received voice data using a machine learning model. For example, the voice data is input into a generative AI model, and the model analyzes the baby's crying patterns and voice characteristics using the prompt "Please estimate the baby's condition as indicated by this voice data."

[1746] Generate state data:

[1747] The server generates data on the baby's condition based on the analysis of the voice data, specifically identifying the baby's condition, such as "needs a diaper change," "sleepy," or "hungry."

[1748] Create a notification message:

[1749] The server generates a notification message for parents or caregivers based on the estimated infant condition. The notification message is generated in a format that is easy for parents to understand. For example, it generates a message saying, "Your diaper needs to be changed."

[1750] Send notifications:

[1751] The notification message created by the server is sent to the terminal via the Internet.

[1752] Displaying notifications:

[1753] Display notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[1754] Vital data processing (optional)

[1755] Vital data collection:

[1756] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Smart baby monitors and wearable devices are used to collect vital data.

[1757] Vital data analysis:

[1758] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[1759] Creating an emergency alert:

[1760] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[1761] Sending emergency alerts:

[1762] The server sends an emergency alert message to the terminal.

[1763] Viewing emergency alerts:

[1764] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[1765] Use of emotion engine

[1766] Using the Emotion Engine:

[1767] To recognize the user's emotions, the device captures the user's facial expressions with a camera and analyzes the data to identify the user's emotions.

[1768] Sending Emotion Data:

[1769] The user's emotion data recognized by the device is sent to a server via the Internet.

[1770] Emotional Data Analysis:

[1771] The server analyzes the received emotion data and determines the user's emotional state, for example, whether the user is feeling stressed.

[1772] Notification adjustments:

[1773] Based on the analysis results, the server creates notification messages and alerts in a format appropriate to the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened.

[1774] Specific examples

[1775] Example 1: A crying baby

[1776] 1. The device will record your baby's crying for 5 seconds.

[1777] 2. The device sends the recorded data to a server via the Internet.

[1778] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[1779] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[1780] 5. The server creates a notification message saying "Diaper needs changing."

[1781] 6. The server sends a notification message to the terminal.

[1782] 7. The device displays the notification message in the notification bar of the user's smartphone.

[1783] Example 2: Vital signs monitoring

[1784] 1. The device measures the baby's temperature using a smart baby monitor.

[1785] 2. The device sends the measurement data to a server via the Internet.

[1786] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[1787] 4. The server creates an emergency alert: "Infant has fever."

[1788] 5. The server sends an emergency alert to the device.

[1789] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[1790] 7. The user checks the emergency alert and takes necessary measures.

[1791] Example 3: Using the Emotion Engine

[1792] 1. The device uses a camera to capture the user's facial expressions.

[1793] 2. The device sends the captured image data to a server via the Internet.

[1794] 3. The server uses an emotion engine to analyze the image data and determine the user's emotional state.

[1795] 4. If the server determines that the user is feeling stressed, it creates a notification message with careful consideration of its content and expression.

[1796] 5. The server sends the adjusted notification message to the terminal.

[1797] 6. The terminal displays an adjusted notification message, and the notification to the user is more appropriate.

[1798] Prompt Sentence Examples

[1799] "Analyze a 5-second recording of an infant's cry and use the cry to estimate whether the baby needs a diaper change."

[1800] This multi-functional system allows for accurate understanding of the infant's condition and prompt response, and can also analyze and provide notifications and alerts that take into account the user's emotional state, enabling more effective care.

[1801] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1802] Step 1:

[1803] Audio data collection

[1804] The device records the baby's crying, laughter, and other sounds. Specifically, it uses a smartphone microphone to record 5 seconds of audio. The input is the baby's voice, and the output is the recorded audio data.

[1805] Step 2:

[1806] Sending audio data

[1807] The device sends recorded audio data to the server using the HTTPS protocol. The input is the recorded audio data, and the output is the audio data sent to the server.

[1808] Step 3:

[1809] Analysis of audio data

[1810] The server analyzes the received audio data. Specifically, it uses a generative AI model to analyze the audio data and extract patterns and characteristics of the infant's crying. The analysis is performed using the prompt "Please estimate the infant's condition indicated by this audio data." The input is the audio data sent to the server, and the output is the analysis result of the crying.

[1811] Step 4:

[1812] Generate state data

[1813] The server generates data on the infant's condition based on the analysis of the voice data. Specifically, it identifies the infant's condition, such as "needs a diaper change," "sleepy," or "hungry." The input is the analysis result of the crying, and the output is the infant's condition data.

[1814] Step 5:

[1815] Creating a notification message

[1816] The server generates a notification message for parents or caregivers based on the generated infant status data. Specifically, it generates a message such as "diaper change required." The input is the infant status data, and the output is the notification message.

[1817] Step 6:

[1818] Sending notifications

[1819] The notification message created by the server is sent to the terminal via the Internet. The input is the notification message, and the output is the notification message sent to the terminal.

[1820] Step 7:

[1821] Viewing notifications

[1822] The notification message received by the device is displayed to the user. Specifically, the message is displayed in the notification bar of a smartphone or tablet or within a dedicated app. The input is the notification message sent to the device, and the output is the notification message displayed to the user.

[1823] Step 8:

[1824] Vital data collection (optional)

[1825] The terminal measures the infant's vital data (heart rate and body temperature) and sends it to the server. A smart baby monitor or wearable device is used. The input is the infant's vital data, and the output is the measured vital data.

[1826] Step 9:

[1827] Sending vital data

[1828] The vital data collected by the device is sent to a server via the Internet. The input is the measured vital data, and the output is the vital data sent to the server.

[1829] Step 10:

[1830] Analysis of vital data

[1831] The server analyzes the received vital data and determines whether there is an abnormality. Specifically, it uses statistical algorithms and machine learning models. The input is the vital data sent to the server, and the output is the analysis result of the abnormality.

[1832] Step 11:

[1833] Creating an emergency alert

[1834] If the server detects abnormal vital data, it creates an emergency alert. Specifically, it generates a highly urgent message such as "The baby has a fever." The input is the analysis result of the abnormal condition, and the output is an emergency alert message.

[1835] Step 12:

[1836] Sending emergency alerts

[1837] The server sends an emergency alert message to the terminal, where the input is the emergency alert message and the output is the emergency alert message sent to the terminal.

[1838] Step 13:

[1839] Viewing emergency alerts

[1840] The device displays an emergency alert to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user. The input is the emergency alert message sent to the device, and the output is the emergency alert message displayed to the user.

[1841] Step 14:

[1842] Use of emotion engine

[1843] To recognize the user's emotions, the device captures the user's facial expressions with a camera and analyzes the data to identify the user's emotions. The input is the user's facial expression data, and the output is the recognized emotion data.

[1844] Step 15:

[1845] Sending emotional data

[1846] The device recognizes the user's emotion data and sends it to a server via the Internet. The input is emotion data, and the output is the emotion data sent to the server.

[1847] Step 16:

[1848] Emotional Data Analysis

[1849] The server analyzes the received emotion data and determines the user's emotional state. Specifically, the analysis is performed using an emotion engine. The input is the emotion data sent to the server, and the output is the analysis result of the emotional state.

[1850] Step 17:

[1851] Notification adjustments

[1852] Based on the analysis results, the server creates notification messages and alerts in a format appropriate for the user's emotional state. For example, if the user is feeling stressed, the content and wording of the notification will be softened. The input is the analysis result of the emotional state, and the output is the adjusted notification message.

[1853] This is the flow of processing for this system. In this way, the condition of the infant can be accurately monitored and notifications and alerts can be provided to the user at the appropriate time.

[1854] (Application example 2)

[1855] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1856] This invention relates to a system that quickly and accurately monitors an infant's condition. In particular, it collects and analyzes an infant's crying and vital signs, and provides appropriate notifications to parents and caregivers, thereby reducing the burden of managing and caring for the infant's health and enabling prompt responses. It also aims to improve the user experience by taking into account the user's emotional state and providing more appropriate notification methods when the user is feeling stressed.

[1857] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1858] In this invention, the server includes means for recording the infant's voice, means for analyzing the recorded voice to estimate the infant's condition, means for creating a notification message based on the estimated infant's condition, means for sending the created notification message to a user's terminal, means for displaying the notification message on the user's terminal, a machine learning model for analyzing the collected infant's voice data, an emotion engine for analyzing the user's emotional state, means for adjusting the content of the notification message based on the analysis results of the emotion engine, means for collecting the infant's vital data and detecting abnormalities, and means for generating and sending an emergency alert when an abnormality is detected. This enables a detailed understanding of the infant's condition based on the infant's voice and vital data, and appropriate notifications corresponding to the user's emotional state.

[1859] "Infant" refers to a young child who is not yet able to use language or clearly express his or her own will.

[1860] "Audio data" refers to information recorded as sound, such as an infant's crying or laughter.

[1861] "Vital data" refers to data that indicates an infant's physiological condition, such as heart rate, body temperature, and respiratory rate.

[1862] "Machine learning model" means a computational model based on algorithms that analyze data and learn patterns and trends.

[1863] The "emotion engine" is a system that analyzes a user's facial expressions captured by a camera and identifies their emotional state.

[1864] "Recording" refers to saving sounds such as an infant's crying or laughter using a recording device.

[1865] "Analysis" refers to processing and analyzing the recorded voice data and vital data, and deriving the infant's condition and characteristics from that information.

[1866] "Estimation" refers to determining the infant's condition based on the recorded data.

[1867] "Notification message" refers to message information sent to parents or caregivers based on the analysis results.

[1868] An "urgent alert" refers to a highly urgent message that is sent to the user immediately when an abnormality is detected in the infant's vital data.

[1869] "Device" refers to a device that receives and displays notification messages, such as a smartphone or tablet.

[1870] "User" refers to the parent or caregiver of the infant.

[1871] "Collection" refers to using a device to collect an infant's voice and vital data.

[1872] "Abnormal" refers to vital data or an infant's condition that is outside the normal range.

[1873] "Capture" refers to using a camera to obtain an image of a user's facial expression.

[1874] "Generation" means creating notification messages or emergency alerts based on the analysis results.

[1875] "Sending" refers to sending the generated notification message or emergency alert to the user's device via the Internet.

[1876] "Display" refers to showing the sent notification message or emergency alert on the device screen.

[1877] The present invention provides a system that collects and analyzes an infant's voice and vital data and notifies the results to users (parents and caregivers). The system of the present invention is implemented using the following hardware and software.

[1878] Hardware and Software Use

[1879] Devices: Smartphones, tablets, smart baby monitors, etc. are used to collect voice and vital data.

[1880] Server: A server for data processing and analysis, where machine learning models and emotion engines are installed.

[1881] Software: TensorFlow (implementation of machine learning models), OpenCV (image processing), requests (HTTP request library), etc.

[1882] Detailed Description of the Invention

[1883] 1. Collection of audio data

[1884] The device records the baby's cries, laughter, and other sounds. The recording device is a microphone on a smartphone or a dedicated device.

[1885] Audio data is automatically recorded at regular intervals.

[1886] 2. Sending audio data

[1887] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[1888] 3. Analysis of audio data

[1889] The server analyzes the received voice data using machine learning models such as TensorFlow.

[1890] The machine learning model is trained to identify characteristic speech patterns of infants.

[1891] 4. Creating and sending notification messages

[1892] The server creates a notification message based on the analysis results.

[1893] The created notification message is sent to the terminal via the Internet.

[1894] The device will display a notification message to the user, either in the notification bar of the smartphone or tablet or within a dedicated app.

[1895] 5. Vital Data Collection and Analysis

[1896] The device collects the baby's vital data (heart rate and temperature), which is collected using a smart baby monitor or wearable device.

[1897] The server analyzes the received vital data and determines whether there are any abnormalities, using statistical algorithms and machine learning models.

[1898] 6. Creating and sending emergency alerts

[1899] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[1900] Emergency alerts are sent to your device via the internet.

[1901] The device will highlight the emergency alert to inform the user of the urgency.

[1902] 7. Use of Emotion Engines

[1903] The device uses a camera to capture the user's facial expressions.

[1904] The captured image data is sent to a server via the Internet.

[1905] The server uses an emotion engine to analyze image data and determine the user's emotional state. The analysis uses image processing libraries such as OpenCV.

[1906] If the user is feeling stressed, the content and expression of the notification message are softened and sent.

[1907] Specific examples

[1908] 1. Collection and analysis of audio data

[1909] For example, the device records a baby's crying for five seconds and sends the recording to a server via the internet. The machine learning model analyzes the audio data and generates status data such as "I want to change his / her diaper."

[1910] Example text:

[1911] Please explain the process of recording a baby crying.

[1912] 2. Collection and analysis of vital data

[1913] For example, a device measures an infant's temperature using a smart baby monitor and transmits the measurement data via the Internet to a server, which analyzes the vital data and detects abnormal body temperature (e.g., high fever).

[1914] Example text:

[1915] How can I use a smart baby monitor to collect my newborn's vital data (heart rate, temperature) and notify me if any abnormalities are detected?

[1916] 3. Use of Emotion Engine

[1917] For example, the device uses a camera to capture the user's facial expressions and sends the image data to a server via the Internet. The server then analyzes the image data using an emotion engine, and if it determines that the user is feeling stressed, it softens the tone of the notification message and sends it.

[1918] Example text:

[1919] Describe the process of detecting a parent's emotional state and adjusting notification messages.

[1920] In this way, the multi-functional system can accurately grasp the infant's condition and provide notifications and alerts that take into account the user's emotional state.

[1921] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1922] Step 1:

[1923] The device records the baby's voice data. The baby's cries and laughter are recorded using a microphone on a smartphone or dedicated device and saved as audio data. The input is the baby's cries and laughter, and the output is the saved audio data. Specifically, the device works by pressing the record button to record audio for a set period of time (for example, 5 seconds), and then saving the audio data in the device.

[1924] Step 2:

[1925] The audio data recorded by the device is sent to a server via the Internet. The audio data is uploaded to the server using a secure communication method such as the HTTPS protocol. The input is the audio data stored on the device, and the output is the audio data uploaded to the server. Specifically, after recording is complete, the data transmission module is automatically activated and the audio data is sent to the server.

[1926] Step 3:

[1927] The server analyzes the received voice data. The server processes the voice data using a machine learning model to identify the characteristics and patterns of the infant's voice. The input is the voice data received by the server, and the output is the infant's estimated state (e.g., "I want my diaper changed" or "I'm sleepy"). Specifically, the voice data is preprocessed, input into the machine learning model, and analyzed, and the infant's state is estimated as a result.

[1928] Step 4:

[1929] The server creates a notification message based on the estimated infant condition. Based on the analysis results, the notification message is generated in a format that is easy for parents and caregivers to understand. The input is the estimated infant condition, and the output is the created notification message. Specifically, the server retrieves an appropriate message template from the database, embeds the estimated condition, and generates the notification message.

[1930] Step 5:

[1931] The server sends the created notification message to the terminal via the Internet. The input is the created notification message, and the output is the notification message sent to the terminal. Specifically, the server uses a communication module to send the notification message to the terminal's address.

[1932] Step 6:

[1933] Displays notification messages received by the device. Notification messages are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app. The input is the notification message sent from the server, and the output is the notification message displayed on the device screen. Specifically, the notification is displayed in the notification bar when it is received, and the notification content is displayed in detail when the app is opened.

[1934] Step 7:

[1935] The device collects the infant's vital data. A smart baby monitor or wearable device is used to measure the infant's vital data, such as heart rate and body temperature, and the data is stored in the device. The input is the infant's biological signals, and the output is the measured vital data. Specifically, the device periodically monitors the vital signs of the infant wearing it and collects the data in real time.

[1936] Step 8:

[1937] The vital data collected by the device is sent to a server via the internet. A secure communication protocol is used to upload the vital data to the server. The input is the vital data stored on the device, and the output is the vital data uploaded to the server. Specifically, the device sends the collected vital data to the server at regular intervals.

[1938] Step 9:

[1939] The server analyzes the vital data received and determines whether there is an abnormality. Statistical algorithms and machine learning models are used to identify vital data that deviates from the normal range. The input is the vital data received by the server, and the output is the determination result when an abnormality is detected. Specifically, the vital data is input into the analysis module, and an alert is generated when an abnormal value is detected.

[1940] Step 10:

[1941] The server creates an emergency alert when an anomaly is detected. A highly urgent message is generated based on the detected anomaly. The input is the anomaly judgment result, and the output is the generated emergency alert. Specifically, when an anomaly is detected, an emergency message template is retrieved from the database and an alert is generated using this template.

[1942] Step 11:

[1943] The server sends the generated emergency alert to the terminal. The input is the generated emergency alert, and the output is the emergency alert sent to the terminal. Specifically, the server uses a communication module to send the emergency alert to the terminal's address.

[1944] Step 12:

[1945] Displays emergency alerts received by the device. The alert is notified to the user by highlighting it in the notification bar or within the app. The input is the emergency alert sent from the server, and the output is the emergency alert displayed on the device screen. Specifically, when an emergency alert is received, it is highlighted in the notification bar, and when the app is opened, the details of the alert are displayed.

[1946] Step 13:

[1947] The device uses a camera to capture the user's facial expression. The image data captured by the camera is stored in the device. The input is an image of the user's facial expression, and the output is the stored image data. Specifically, the device starts up the camera, captures the facial expression, and saves the data.

[1948] Step 14:

[1949] The image data captured by the device is sent to the server via the Internet. The image data is uploaded to the server. The input is the image data stored on the device, and the output is the image data uploaded to the server. Specifically, the transmission module automatically sends the image data that has been captured to the server.

[1950] Step 15:

[1951] The server analyzes the received image data and determines the user's emotional state. An emotion engine is used to identify emotions from the user's facial expressions. The input is the image data received by the server, and the output is the emotional state determined by the analysis. Specifically, the received image data is input into the emotion engine, the emotional state is analyzed, and a determination result is obtained.

[1952] Step 16:

[1953] The server adjusts the content of the notification message based on the analysis results. The notification message is created in a format appropriate for the user's emotional state. The input is the determined emotional state, and the output is the adjusted notification message. Specifically, the server adjusts the message template based on the results of the emotion analysis, and generates a notification message that takes emotions into consideration.

[1954] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1955] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1956] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1957] [Fourth embodiment]

[1958] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1959] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1960] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1961] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1962] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1963] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1964] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1965] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1966] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1967] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1968] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1969] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1970] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1971] MODE FOR CARRYING OUT THE INVENTION

[1972] The system of the present invention collects and analyzes the voice and vital data of an infant and notifies the results to the user (parent or caregiver), thereby enabling the user to accurately grasp the infant's condition and take prompt action. The system of the present invention includes the following various means.

[1973] 1. Collection of audio data

[1974] The device records the baby's cries, laughter, and other sounds. For example, the baby's voice can be recorded using a microphone on a smartphone or a dedicated device. Recording starts at regular intervals, and the audio data is automatically acquired.

[1975] 2. Sending audio data

[1976] The audio data recorded by the device is sent to a server via the internet using a secure communication method such as the HTTPS protocol.

[1977] 3. Analysis of audio data

[1978] The server analyzes the received voice data and estimates the infant's condition using a machine learning model based on the infant's crying patterns and voice characteristics.

[1979] 4. State Data Generation

[1980] The server generates data on the baby's condition based on the analysis of the voice data, identifying, for example, the baby's condition such as "needs a diaper change," "sleepy," or "hungry."

[1981] 5. Creating a notification message

[1982] The server generates notification messages for parents and caregivers based on the estimated infant status in a format that is easy for parents to understand.

[1983] 6. Sending Notifications

[1984] The notification message created by the server is sent to the terminal via the Internet.

[1985] 7. Display of notifications

[1986] Displaying notification messages received by the device. Notifications are displayed to the user in the notification bar of a smartphone or tablet or within a dedicated app.

[1987] 8. Collecting vital data (optional)

[1988] The device measures the infant's vital data (heart rate and body temperature) and sends it to a server. Vital data is acquired using a smart baby monitor or wearable device.

[1989] 9. Analysis of vital data

[1990] The server analyzes the received vital data and determines whether there are any abnormalities, thereby understanding the infant's detailed physiological condition.

[1991] 10. Creating Emergency Alerts

[1992] If the server detects abnormal vital data, it will generate an emergency alert, for example, a message with high urgency such as "Your baby has a fever."

[1993] 11. Sending emergency alerts

[1994] The server sends an emergency alert message to the terminal.

[1995] 12. Displaying emergency alerts

[1996] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[1997] Specific examples

[1998] A specific example of the operation of the system is shown below.

[1999] Example 1: A crying baby

[2000] 1. The device will record your baby's crying for 5 seconds.

[2001] 2. The device sends the recorded data to a server via the Internet.

[2002] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[2003] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[2004] 5. The server creates a notification message saying "Diaper needs changing."

[2005] 6. The server sends a notification message to the terminal.

[2006] 7. The device displays the notification message in the notification bar of the user's smartphone.

[2007] 8. The user checks the notification and changes the baby's diaper.

[2008] Example 2: Vital signs monitoring

[2009] 1. The device measures the baby's temperature using a smart baby monitor.

[2010] 2. The device sends the measurement data to a server via the Internet.

[2011] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[2012] 4. The server creates an emergency alert: "Infant has fever."

[2013] 5. The server sends an emergency alert to the device.

[2014] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[2015] 7. The user checks the emergency alert and takes necessary measures.

[2016] In this way, the system of the present invention can accurately grasp the condition and needs of the infant and assist in prompt response.

[2017] The processing flow will be explained below.

[2018] Program processing steps

[2019] Step 1:

[2020] The device records the baby's voice. The device's microphone is used to record the baby's crying and laughter for a certain period of time (e.g., 5 seconds).

[2021] Step 2:

[2022] The device stores the recorded audio data in a buffer and uploads it to a server via the Internet using a secure communication protocol such as HTTPS.

[2023] Step 3:

[2024] The server decodes the received audio data and begins analyzing it using a machine learning model (e.g., Convolutional Neural Network) to identify patterns and characteristics of crying sounds.

[2025] Step 4:

[2026] The server generates estimated data on the infant's condition based on the results of analyzing the voice data. Specifically, it creates textual data on the infant's condition, such as "needs a diaper change," "sleepy," and "hungry."

[2027] Step 5:

[2028] The server creates notification messages based on the generated status data, and the notification messages are presented in a format that is easy for parents and caregivers to understand.

[2029] Step 6:

[2030] The notification message created by the server is sent to the device via the Internet as a push notification.

[2031] Step 7:

[2032] The device decodes the notification message received and displays it in the notification bar or dedicated app on the smartphone or tablet.

[2033] Step 8:

[2034] The device measures the infant's vital data (e.g., heart rate and body temperature) using a smart baby monitor or wearable device and stores the measured data in a buffer.

[2035] Step 9:

[2036] The vital data measured by the device is uploaded to a server via the Internet.

[2037] Step 10:

[2038] The server analyzes the received vital data and detects abnormal values ​​(e.g., high or low fever) using statistical algorithms and machine learning models.

[2039] Step 11:

[2040] If the server detects abnormal data, it will create an emergency alert. For example, it will generate an alert message saying "The baby has a fever. Please take action immediately."

[2041] Step 12:

[2042] The server creates an emergency alert message and sends it to the device as a push notification via the Internet.

[2043] Step 13:

[2044] The device decodes emergency alert messages received and displays them in the notification bar and in the app with special highlighting, thereby informing the user of the urgency of the message.

[2045] Step 14:

[2046] The user checks notifications and emergency alerts and takes appropriate action (e.g., changing a diaper, checking a temperature).

[2047] Example 1

[2048] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2049] Rapid and accurate understanding of an infant's health condition and needs is an important challenge for parents and caregivers. However, because infants cannot communicate their condition verbally, they must rely on indirect information such as crying and vital signs. In conventional systems, voice data and vital signs are often collected and analyzed separately, making it difficult to centrally manage and comprehensively analyze this data. In addition, there are limitations to the means of quickly notifying users of analysis results, which can delay response, especially in urgent situations.

[2050] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2051] In this invention, the server includes means for recording voice data of the infant, means for periodically starting and saving the recorded voice data, means for transmitting the recorded voice data to the server using a secure communication protocol, means for analyzing the received voice data based on a machine learning model to estimate the infant's condition, means for generating estimated infant condition data, means for creating a notification message that is easy for the user to understand, means for transmitting the notification message to the user's terminal using the secure communication protocol, and means for displaying the notification message on the user's terminal, thereby enabling the infant's health condition and needs to be quickly and accurately grasped and appropriate measures to be taken.

[2052] "Means for recording infant sounds" refers to devices and methods for recording sounds such as infant crying and laughter.

[2053] "Means for periodically initiating and saving recorded audio" refers to a device or method for automatically initiating audio recording at set intervals and saving the data.

[2054] "Means for transmitting recorded audio to a server using a secure communication protocol" refers to a device or method for transmitting acquired audio data to a server while maintaining information security.

[2055] "Means for analyzing received voice data based on a machine learning model and estimating the infant's condition" refers to a device or method for analyzing voice data using machine learning technology and estimating a specific infant condition (e.g., hunger, sleepiness, etc.).

[2056] The "means for generating estimated infant condition data" refers to a device or method for converting the infant condition into a data format based on the results of voice analysis.

[2057] The "means for generating user-friendly notification messages" refers to devices or methods for generating notifications based on the estimated infant condition in a format that is easy for parents or caregivers to understand.

[2058] The "means for transmitting a notification message to a user terminal using a secure communication protocol" refers to a device or method for transmitting the generated notification message to a user terminal while maintaining information security.

[2059] The "means for displaying a notification message on a user's terminal" refers to a device or method for visually displaying a received notification message on a user's terminal.

[2060] "Means for measuring infant vital data" refers to devices and methods for measuring physiological indicators such as an infant's heart rate and body temperature.

[2061] "Means for transmitting measured vital data to a server using a secure communication protocol" refers to a device or method for transmitting acquired vital data to a server while maintaining information security.

[2062] The "means for analyzing received vital data and determining whether an abnormal state exists" refers to a device or method for analyzing acquired vital data and determining whether an abnormal state exists.

[2063] The "means for generating an emergency alert when abnormal vital data is detected" refers to a device or method for generating a message to call for urgent attention when abnormal vital data is detected.

[2064] "Means for sending an emergency alert to a user's terminal" refers to a device or method for sending the generated emergency alert message to a user's terminal.

[2065] A "means for specially highlighting emergency alerts on a user's terminal" is a device or method for highlighting emergency alert messages on a user's terminal.

[2066] MODE FOR CARRYING OUT THE INVENTION

[2067] System configuration

[2068] The system of the present invention includes the following means to quickly and accurately grasp an infant's health condition and needs and support appropriate responses. The hardware used includes smartphones, tablets, dedicated devices, smart baby monitors, and wearable devices. The software uses machine learning models and Firebase Cloud Messaging (FCM).

[2069] Audio data collection

[2070] The device records sounds such as the baby's crying and laughter. Specifically, it uses the smartphone's microphone to start recording at regular intervals and automatically saves the audio data.

[2071] Sending audio data

[2072] The audio data recorded by the device is sent to the server via the Internet. The HTTPS protocol is used for transmission to ensure security. The audio data recorded is sent to the API endpoint as a POST request.

[2073] Analysis of audio data

[2074] The server analyzes the received audio data using a machine learning model that has learned patterns of infant crying and laughter, and uses this data to estimate the infant's condition.

[2075] Generate state data

[2076] The server generates data on the baby's condition based on the analysis results. For example, if the baby continues crying, it may assume that the baby needs a diaper change or is hungry. This data is saved in JSON format and the next step is reached.

[2077] Creating a notification message

[2078] The server generates an easy-to-understand notification message for parents or caregivers based on the estimated infant's condition, such as "Your baby is crying and needs a diaper change."

[2079] Sending notifications

[2080] The server creates a notification message and sends it to the device via the Internet using Firebase Cloud Messaging (FCM) or another push notification service.

[2081] Viewing notifications

[2082] Display notification messages received by the device. Displays messages in the notification bar of smartphones and tablets to notify the user.

[2083] Vital data collection (optional)

[2084] The device measures the baby's vital data (heart rate and temperature) and sends it to a server. The data is then collected via Bluetooth using a smart baby monitor or wearable device.

[2085] Analysis of vital data

[2086] The server analyzes the received vital data and determines whether there are any abnormalities. An abnormal data detection algorithm is used to detect an excessively high heart rate or abnormal body temperature.

[2087] Creating an emergency alert

[2088] If the server detects abnormal vital data, it creates an emergency alert message, such as "The baby has a fever," and treats it as a priority message.

[2089] Sending emergency alerts

[2090] The server sends an emergency alert message to the device, using a push notification service such as Firebase Cloud Messaging (FCM) to send the notification immediately.

[2091] Viewing emergency alerts

[2092] The device will display emergency alerts to the user, either in the notification bar or with special highlighting within the app to communicate the urgency to the user.

[2093] Specific examples

[2094] Example 1: A crying baby

[2095] 1. The device will record your baby's crying for 5 seconds.

[2096] 2. The device sends the recorded data to a server via the Internet.

[2097] 3. The server uses machine learning models to analyze the audio data and identify crying patterns.

[2098] 4. Based on the analysis results, the server generates status data such as "I want to change the diaper."

[2099] 5. The server creates a notification message saying "Diaper needs changing."

[2100] 6. The server sends a notification message to the terminal.

[2101] 7. The device displays the notification message in the notification bar of the user's smartphone.

[2102] 8. The user checks the notification and changes the baby's diaper.

[2103] Example 2: Vital signs monitoring

[2104] 1. The device measures the baby's temperature using a smart baby monitor.

[2105] 2. The device sends the measurement data to a server via the Internet.

[2106] 3. The server analyzes vital data and detects abnormal body temperature (e.g., high fever).

[2107] 4. The server creates an emergency alert: "Infant has fever."

[2108] 5. The server sends an emergency alert to the device.

[2109] 6. The device will highlight the emergency alert in the smartphone's notification bar.

[2110] 7. The user checks the emergency alert and takes necessary measures.

[2111] In this way, the system of the present invention is able to quickly and accurately grasp the health condition and needs of an infant and support appropriate responses.

[2112] Prompt Sentence Examples

[2113] "Please analyze and let me know why my baby is crying."

[2114] "Please take the infant's temperature and issue an alert if there is anything abnormal."

[2115] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2116] System program processing flow

[2117] Step 1: Collecting audio data

[2118] The device records sounds such as the baby's crying and laughter. First, it uses the smartphone's microphone to start recording at regular intervals and saves the audio data. For example, a dedicated app can automatically start a 5-second recording and save the data on the device.

[2119] Input: Trigger to start recording

[2120] Output: Audio file (e.g., wav format)

[2121] Step 2: Sending audio data

[2122] The audio data recorded by the device is sent to the server via the Internet using the HTTPS protocol to ensure data security. Specifically, the audio file is sent as a POST request to the API endpoint.

[2123] Input: Audio file

[2124] Output: Sending status (success / failure)

[2125] Step 3: Analyzing the audio data

[2126] The server analyzes the received audio data using a machine learning model. First, the audio data is input into the machine learning model. This model has learned the patterns of infant crying and laughter and analyzes the audio features. Specifically, it extracts features such as the frequency components and volume of the crying sounds and uses them to estimate the infant's condition.

[2127] Input: Audio data

[2128] Output: Analysis results (e.g. "I want to change your diaper")

[2129] Step 4: Generate state data

[2130] The server generates data on the infant's condition based on the analysis of the voice data. For example, it identifies the infant's condition, such as "needs a diaper change," "hungry," or "sleepy." The inferred results are saved in a structured data format (e.g., JSON).

[2131] Input: Analysis results

[2132] Output: State data (JSON format)

[2133] Step 5: Create a notification message

[2134] The server creates a user-friendly notification message based on the estimated baby's condition, such as "Your baby is crying. Your diaper needs to be changed."

[2135] Input: State data

[2136] Output: Notification message

[2137] Step 6: Sending notifications

[2138] The notification message created by the server is sent to the device via the Internet using a push notification service such as Firebase Cloud Messaging (FCM).

[2139] Input: Notification message

[2140] Output: Sending status (success / failure)

[2141] Step 7: View notifications

[2142] Displaying notification messages received by the device. Visually displaying notifications to the user in the notification bar of a smartphone or tablet or within a dedicated app. For example, displaying a pop-up in the notification bar saying "Diaper needs changing."

[2143] Input: Notification message

[2144] Output: Displayed notification

[2145] Step 8: Collect vital data (optional)

[2146] The device measures the baby's vital data (heart rate and temperature) and sends it to a server. Data is collected using a smart baby monitor or wearable device and stored on the device via Bluetooth.

[2147] Input: Trigger to start measurement

[2148] Output: Vital data

[2149] Step 9: Analyze vital data

[2150] The server analyzes the received vital data and determines whether there are any abnormalities, using an anomaly detection algorithm to detect, for example, an extremely high heart rate or an abnormal body temperature.

[2151] Input: Vital data

[2152] Output: Abnormal condition (e.g. "High fever")

[2153] Step 10: Create an emergency alert

[2154] If the server detects abnormal vital data, it creates an emergency alert message, for example, a message with high urgency such as "The baby has a fever" and marks it as an emergency alert.

[2155] Input: Abnormal condition

[2156] Output: Emergency alert message

[2157] Step 11: Sending emergency alerts

[2158] The server sends an emergency alert message to the device using a push notification service such as Firebase Cloud Messaging (FCM).

[2159] Input: Emergency alert message

[2160] Output: Sending status (success / failure)

[2161] Step 12: View emergency alerts

[2162] The device displays an emergency alert to the user, highlighting it with a special icon or popup in the notification bar of the smartphone or tablet to communicate the urgency to the user. For example, the notification bar will display "Urgent: Baby has a fever!" in red.

[2163] Input: Emergency alert message

[2164] Output: Emergency notification displayed

[2165] (Application example 1)

[2166] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2167] Conventionally, monitoring of an infant's crying and physiological data and notifications based on the results have mainly been done manually by parents or caregivers, which can make it difficult to respond quickly. Furthermore, since the infant's health condition cannot be grasped in real time, there is a risk of missing signs of serious health abnormalities. The objective of this invention is to provide a system that efficiently collects and analyzes an infant's voice and vital data and promptly notifies the user.

[2168] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2169] In this invention, the server includes means for recording the infant's voice, means for analyzing the recorded voice to estimate the infant's condition, means for creating a notification message based on the estimated infant's condition, means for transmitting the created notification message to a user's terminal, means for displaying the notification message on the user's terminal, means for using a wearable device to measure the infant's vital data, means for analyzing the measured vital data to detect abnormalities, means for creating an emergency alert based on the detected abnormality, and means for transmitting the created emergency alert to the user's smart glasses or smartphone and displaying the notification, thereby enabling the infant's health condition to be understood in real time and responding promptly.

[2170] "Infant voice" refers to audio signals such as crying and laughing emitted by an infant.

[2171] "Recording" means recording and storing audio and data.

[2172] "Analysis" is the process of examining collected data and identifying meanings and patterns in it.

[2173] "Condition" refers to a specific situation or condition, such as an infant's health, emotions, or care needs.

[2174] A "notification message" is a message containing information that is sent to a user based on the infant's condition.

[2175] A "user device" is an electronic device used by a parent or caregiver, such as a smartphone or smart glasses.

[2176] "Display" means the visual presentation of information on a screen or display.

[2177] "Vital data" refers to data that indicates an infant's physiological indicators, including heart rate and body temperature.

[2178] A "wearable device" is a device worn by an infant to measure vital data.

[2179] An "abnormality" is a condition that is outside the normal range and may affect health.

[2180] An "urgent alert" is an immediate notification when a serious health or safety issue is detected.

[2181] A "generative AI model" is a machine learning algorithm that analyzes an infant's voice patterns and vital data based on training data.

[2182] A "server" is a central computer system that analyzes and processes data and sends the results to the user's terminal.

[2183] The system of the present invention efficiently collects and analyzes an infant's voice and vital data, and notifies the user to grasp the infant's condition in real time and support prompt response. A specific embodiment of this system is shown below.

[2184] System Configuration

[2185] The system mainly consists of the following components:

[2186] 1. Device: This refers to a smartphone or smart glasses equipped with a microphone and a display, capable of displaying audio data and notifications.

[2187] 2. Server: Plays the central role in data analysis, analyzing voice and vital data using generative AI models.

[2188] 3. Wearable devices: Smart bands that infants can wear to measure vital data such as heart rate and body temperature.

[2189] 4. Generative AI model: A machine learning algorithm that analyzes voice patterns and infant physiological indicators based on training data.

[2190] Hardware and Software Use

[2191] Terminal

[2192] The terminal records voice data and transmits it to the server. It also transmits vital data acquired from the wearable device to the server. It has the function of displaying notification messages and emergency alerts when they are generated.

[2193] server

[2194] The server receives the voice and vital data sent from the device and analyzes them using a generative AI model. Based on the analysis results, it estimates the infant's condition and creates notification messages and emergency alerts.

[2195] Wearable devices

[2196] Wearable devices are used to measure vital data such as the baby's heart rate and body temperature, and the measured data is sent to a terminal via communication means such as Bluetooth.

[2197] Data processing and calculation

[2198] 1. Audio data collection:

[2199] The device records the baby's crying, laughter, and other sounds, for example, five seconds of audio, and saves it as digital data.

[2200] 2. Sending audio data:

[2201] The device sends the recorded audio data to a server via the Internet using the HTTPS protocol for secure data transfer.

[2202] 3. Analysis of audio data:

[2203] The server analyzes the received voice data using a generative AI model, capturing the characteristics of the voice pattern and inferring what the baby needs.

[2204] 4. Create and send a notification message:

[2205] Based on the estimated baby's condition, the server generates a notification message in a format that can be quickly communicated to the user, and the generated message is then sent to the terminal via the Internet.

[2206] 5. Displaying notification messages:

[2207] The device will then display the received notification message to the user: on a smartphone, it will appear in the notification bar, and on the display of smart glasses.

[2208] 6. Vital Data Collection:

[2209] The wearable device measures the infant's heart rate and temperature and transmits the data to a terminal.

[2210] 7. Analysis of vital data:

[2211] The server analyzes the received vital data and creates an emergency alert if it detects any abnormalities.

[2212] 8. Creating and sending emergency alerts:

[2213] When an abnormality is detected, the server quickly creates an emergency alert and sends it to the device.

[2214] 9. Emergency alert display:

[2215] The device will highlight the emergency alert and notify the user.

[2216] Specific examples

[2217] Example 1: A crying baby

[2218] The device records the baby's crying for 5 seconds and sends it to the server.

[2219] The server analyzes the voice data and determines that the diaper needs to be changed.

[2220] A notification message "Diaper change required" is generated and sent to the user's terminal.

[2221] The device will notify the user by displaying a message in the notification bar or on the display of the smart glasses.

[2222] Example 2: Vital signs monitoring

[2223] The wearable device measures the infant's temperature and transmits it to a terminal.

[2224] The server analyzes the body temperature data and detects high fever.

[2225] An emergency alert "Infant has fever" is generated and sent to the user's device.

[2226] The device will highlight emergency alerts and notify the user quickly.

[2227] Example prompt sentence:

[2228] "Analyzing audio data of an infant crying and identifying the cause."

[2229] "Analyze infant heart rate data to detect abnormalities."

[2230] In this way, the system of the present invention can grasp the condition of the infant in real time and promote quick response.

[2231] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2232] Step 1:

[2233] The device records sounds such as the baby's crying and laughter. Specifically, it uses the device's built-in microphone to record audio data for a specified period of time (e.g., 5 seconds). The input here is a real-time audio signal, and the output is digital audio data. The recorded audio data is temporarily stored in a buffer.

[2234] Step 2:

[2235] The device sends the recorded audio data to the server via the Internet. The HTTPS protocol is used for communication to ensure data security. The input here is the recorded audio data, and the output is the data to be sent to the server. Once the transmission is complete, the device receives a confirmation message.

[2236] Step 3:

[2237] The server inputs the received voice data into a generative AI model for analysis. The generative AI model is pre-trained and analyzes the voice features to estimate the infant's state. The input here is the voice data sent to the server, and the output is the infant's estimated state (e.g., "I want to change my diaper" or "I'm hungry"). The AI ​​model analyzes the voice patterns and probabilistically estimates the most appropriate state.

[2238] Step 4: 【223...

Claims

1. a means for recording the infant's voice; A means for estimating the infant's condition by analyzing the recorded voice; means for generating a notification message based on the estimated infant condition; means for transmitting the created notification message to a user's terminal; means for displaying a notification message on the user's terminal; A system including:

2. a means for measuring the vital data of the infant; A means for analyzing the measured vital data to determine the detailed condition of the infant; means for generating an emergency alert based on the determined condition; means for transmitting the created emergency alert to a user's terminal; a means for displaying an emergency alert on a user's device; The system of claim 1 further comprising:

3. means for analyzing the voice data and the vital data based on a machine learning model; The machine learning model is trained to identify characteristic vocal patterns and physiological indicators of infants. The system of claim 1 .

Images