system

Abstract

Provide a system. [Solution] a means for a user to input basic information; means for receiving and storing said basic information in a secure database; A means for accepting user questions and generating answers using a generation AI; means for displaying the generated answer to a user; A means for monitoring the condition of the infant using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert when an abnormality is detected; means for notifying a user of the alert; means for reviewing the user's consultation history and generating and sending follow-up messages; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] ---

[0005] In today's child-rearing environment, parents often raise their children in isolation, which can result in mental and physical strain. Traditional postpartum follow-up care provided by local governments has limited resources, making it difficult for them to care for all parents, and support can be difficult to receive at certain times of the day, especially late at night or early in the morning. This can mean that many parents are unable to receive appropriate support when they need it, which can delay the resolution of their problems. [Means for solving the problem]

[0006] To solve the above problems, the present invention provides the following means. It includes a means for a user to input basic information and a means for receiving the basic information and storing it in a secure database. It also includes a means for accepting a user's questions, generating answers using a generation AI, and displaying the generated answers to the user. It also includes a means for monitoring the infant's condition using a built-in camera and various sensors, and a means for analyzing the monitoring data and generating an alert if an abnormality is detected. It also includes a means for notifying the user of the alert, and a means for checking the user's consultation history and generating and sending a follow-up message. This allows parents to receive appropriate support 24 hours a day without feeling isolated.

[0007] ---

[0008] "User" refers to a parent or guardian who uses a child-rearing support device.

[0009] "Basic information" refers to information for initial setup entered by the user, such as the child's age, gender, and allergy information.

[0010] "Database" refers to a part of a computer system that stores basic information about users, consultation history, etc.

[0011] "Generative AI" refers to an artificial intelligence system that automatically generates answers based on user input.

[0012] "Built-in camera" refers to a camera built into the device to monitor the condition of the infant.

[0013] "Sensors" refers to any sensor device incorporated into the device to monitor the condition of the infant, including, for example, a temperature sensor or an audio sensor.

[0014] "Monitoring Data" refers to data about the infant's condition collected by the built-in cameras and sensors.

[0015] An "alert" refers to warning information that is notified to the user when an abnormal situation is detected.

[0016] A "follow-up message" refers to a message for follow-up support that is generated based on the user's consultation history.

[0017] "Natural language processing technology" refers to technology that enables computers to analyze and understand natural human language.

[0018] "Analysis" refers to the process of examining data to understand its meaning and patterns and extract the information you need. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] ---

[0041] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generation AI, and generating and sending follow-up messages.

[0042] Initial Setup

[0043] 1. Start the device and enter basic information

[0044] First, the user starts up the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[0045] Childcare consultation function

[0046] 2. Enter questions and view answers

[0047] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server, which then uses generative AI and natural language processing technology to analyze the question and generate an appropriate answer. The generated answer is displayed to the user, who can then consult with the server 24 hours a day.

[0048] Examples:

[0049] User: "My baby cries at night, what should I do?"

[0050] Terminal: Sends a question to the server.

[0051] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0052] Terminal: Display the answer to the user.

[0053] Baby monitoring features

[0054] 3. Monitoring and Alerting

[0055] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[0056] Examples:

[0057] Terminal: A camera in the baby's room monitors the crib.

[0058] Server: Detects crying and determines it is abnormal.

[0059] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0060] User: Opens the app and sees live footage from the camera.

[0061] Follow-up function

[0062] 4. Generate and send follow-up messages

[0063] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message provides appropriate follow-up information and advice based on the content of the past consultation. The generated message is sent to the user via the terminal.

[0064] Examples:

[0065] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0066] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[0067] Terminal: Send a follow-up message to the user.

[0068] ---

[0069] As a result, the child-rearing support device of the present invention can provide appropriate support 24 hours a day to families raising children who tend to be isolated.The system of the present invention allows parents to concentrate on raising their children with peace of mind.

[0070] The processing flow will be explained below.

[0071] ---

[0072] Initial Setup

[0073] Step 1: Turn on the device

[0074] The user turns on the device with the power button.

[0075] Step 2: Install the app

[0076] Users install a dedicated app on their smartphone or tablet.

[0077] The terminal notifies the user that the installation is complete.

[0078] Step 3: Enter basic information

[0079] Users launch the app and enter basic information such as their child's age, gender, and allergy information on the profile screen.

[0080] The terminal checks the basic information entered and checks that all required fields have been entered.

[0081] The server receives the basic information sent by the device and stores it in a secure database.

[0082] Childcare consultation function

[0083] Step 1: Start a consultation

[0084] The user presses the "Consult" button within the app.

[0085] Step 2: Enter your question

[0086] The user enters a question by text input or voice input.

[0087] The terminal checks the input and prepares to send it to the server.

[0088] Step 3: Submit your question

[0089] The terminal transmits the entered question to the server.

[0090] Step 4: Question analysis

[0091] The server analyzes the received question and uses natural language processing (NLP) technology to understand the question.

[0092] Step 5: Answer Generation

[0093] The server uses generative AI to generate appropriate answers to questions.

[0094] Step 6: Submit your response

[0095] The server sends the generated response to the terminal.

[0096] Step 7: View Answers

[0097] The terminal displays the answer to the user.

[0098] Examples:

[0099] User: "My baby cries at night, what should I do?"

[0100] Terminal: Sends a question to the server.

[0101] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0102] Terminal: Display the answer to the user.

[0103] Baby monitoring features

[0104] Step 1: Turn on the camera

[0105] The device activates its built-in camera to monitor the baby's room.

[0106] Step 2: Data collection

[0107] The device collects camera footage and audio data in real time.

[0108] Step 3: Send data

[0109] The terminal transmits the collected data to the server.

[0110] Step 4: Data analysis

[0111] The server analyzes the received data and monitors the baby's condition, generating an alert if an abnormality is detected.

[0112] Step 5: Sending an alert

[0113] The server sends an alert to the terminal.

[0114] Step 6: Alert Notifications

[0115] The device will notify the user of the alert via voice or app notification.

[0116] Examples:

[0117] Terminal: A camera in the baby's room monitors the crib.

[0118] Server: Detects the baby's crying and flags it as an anomaly.

[0119] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0120] User: Opens the app and sees live footage from the camera.

[0121] Follow-up function

[0122] Step 1: Follow-up check

[0123] The server periodically checks the user's status and consultation history.

[0124] Step 2: Generate a follow-up message

[0125] The server generates follow-up messages as needed.

[0126] Step 3: Send a follow-up message

[0127] The server sends the generated follow-up message to the terminal.

[0128] Step 4: Display a follow-up message

[0129] The terminal displays the follow-up message to the user.

[0130] Examples:

[0131] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0132] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[0133] Terminal: Send a follow-up message to the user.

[0134] ---

[0135] Example 1

[0136] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0137] With the advancement of information and communication technology, there is a demand for systems that provide efficient and appropriate support to families raising children, who tend to be isolated. However, existing systems have difficulty providing quick and accurate answers to users' questions, continuously monitoring the condition of infants, and immediately notifying them when abnormalities occur. Furthermore, systems that generate follow-up messages based on the user's past consultations are also inadequate. The challenge is to provide a more advanced support system that solves these problems and allows parents to raise their children with peace of mind.

[0138] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0139] In this invention, the server includes: means for a user to input basic information; means for receiving the basic information and storing it in a secure database; means for accepting user questions and generating answers using a generative AI model; means for displaying the generated answers to the user; means for monitoring the infant's condition using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert if an abnormality is detected; means for notifying the user of the alert; means for checking the user's consultation history and generating and sending a follow-up message; and means for transmitting the device's basic information and monitoring data to the server and for the server to analyze the data. This enables 24-hour monitoring of the infant's condition, prompt and accurate answers to questions, and follow-up based on past consultations.

[0140] "Basic information" refers to information about initial settings such as the child's age, sex, and allergy information entered by the user.

[0141] A "generative AI model" refers to an algorithm or program that uses artificial intelligence technology to generate appropriate answers to user questions.

[0142] An "integrated camera" is a camera that is built into a device and is used to monitor real-time footage of an infant.

[0143] A "sensor" is a device built into a device to detect temperature, humidity, sound, etc.

[0144] "Monitoring Data" refers to data about the infant's condition collected by built-in cameras and sensors.

[0145] "Abnormal" is when the baby is crying, not moving, or indicates an unusual situation detected by the sensor.

[0146] An "alert" is a warning message or audio notification that is given to the user when an abnormality is detected.

[0147] A "follow-up message" is a message that includes follow-up information and additional advice that is generated based on the user's past consultation history.

[0148] A "dedicated app" is application software designed for users to install and use on their smartphones or tablets.

[0149] The "server" is a central processing unit that receives and analyzes basic information, questions, and monitoring data entered by users.

[0150] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generative AI model, and generating and sending follow-up messages.

[0151] Initial Setup

[0152] First, the user turns on the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[0153] Childcare consultation function

[0154] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server then analyzes the question using generative AI models and natural language processing technology, and generates an appropriate answer. The generated answer is then sent to the device and displayed to the user. Users can consult at any time, 24 hours a day.

[0155] Examples:

[0156] User: "My baby cries at night, what should I do?"

[0157] Terminal: Sends a question to the server.

[0158] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0159] Terminal: Display the answer to the user.

[0160] Baby monitoring features

[0161] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[0162] Examples:

[0163] Terminal: A camera in the baby's room monitors the crib.

[0164] Server: Detects crying and determines it is abnormal.

[0165] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0166] User: Opens the app and sees live footage from the camera.

[0167] Follow-up function

[0168] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message provides appropriate follow-up information and advice based on the content of the past consultation. The generated message is sent to the user via the terminal.

[0169] Examples:

[0170] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0171] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[0172] Terminal: Send a follow-up message to the user.

[0173] technical operation

[0174] In this system, the server analyzes questions and generates answers using generative AI models (e.g., GPT-3®) and natural language processing technology. The built-in camera and sensors monitor the infant's condition and send the collected data to the server. The server analyzes this data and, if an abnormality is detected, quickly generates an alert and notifies the user. The server also generates appropriate follow-up messages based on past consultation history and sends them to the user.

[0175] Example prompt sentence:

[0176] "My baby cries at night, what should I do?"

[0177] "My baby's constipation won't improve, what should I do?"

[0178] "I don't know when my baby will roll over. Can you tell me?"

[0179] As a result, the child-rearing support device of the present invention can provide 24-hour support to help parents raise their children with peace of mind.

[0180] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0181] Step 1:

[0182] The user starts the device and installs the dedicated app on their smartphone or tablet. When the app is launched, the user is presented with a login screen or new registration screen. The user enters basic information (child's age, gender, allergy information, etc.).

[0183] Input: Device startup, app installation, basic information input

[0184] Output: Display of basic information input screen, basic information entered

[0185] Step 2:

[0186] The device checks the basic information and checks whether all required fields have been entered correctly. If the basic information has been entered correctly, it is sent to the server.

[0187] Input: Basic information

[0188] Data processing: Check required fields

[0189] Output: Send basic information to the server

[0190] Step 3:

[0191] The server stores the received basic information in a secure database.

[0192] Input: Basic information sent from the device

[0193] Data processing: Basic information stored in a database

[0194] Output: Confirmation of successful save

[0195] Step 4:

[0196] The user presses the "Consult" button and enters a question, which can be entered in text or voice format.

[0197] Input: Question (text or voice)

[0198] Output: Question input completed

[0199] Step 5:

[0200] The terminal transmits the entered question to the server.

[0201] Input: User question

[0202] Output: Sending a question to the server

[0203] Step 6:

[0204] The server analyzes the question using generative AI models and natural language processing technology, and generates an appropriate answer based on the analysis results.

[0205] Input: Question

[0206] Data Computing: Parsing questions and generating answers (generative AI models)

[0207] Output: The generated answer

[0208] Step 7:

[0209] The server sends the generated response to the terminal.

[0210] Input: Generated answer

[0211] Output: Sends the answer to the terminal

[0212] Step 8:

[0213] The terminal displays the received answer on the screen, allowing the user to check the answer.

[0214] Input: Generated answer

[0215] Output: Display answer

[0216] Step 9:

[0217] The device uses a built-in camera and various sensors to monitor the baby's room 24 hours a day.

[0218] Input: Camera and sensor activation

[0219] Data processing: Collecting monitoring data

[0220] Output: Real-time monitoring data

[0221] Step 10:

[0222] The terminal transmits the monitoring data to the server in real time.

[0223] Input: Collected monitoring data

[0224] Output: Sending monitoring data to a server

[0225] Step 11:

[0226] The server analyzes the received monitoring data and generates an alert if an abnormality is detected.

[0227] Input: Real-time monitoring data

[0228] Data Calculation: Anomaly Detection and Alert Generation

[0229] Output: The generated alert

[0230] Step 12:

[0231] The server sends the generated alert to the device, which then notifies the user by voice or in-app notification.

[0232] Input: Generated alert

[0233] Output: Alert notification to the user

[0234] Step 13:

[0235] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary.

[0236] Input: Consultation history

[0237] Data calculation: determining whether follow-up is necessary and generating messages

[0238] Output: Generated follow-up message

[0239] Step 14:

[0240] The server sends the generated follow-up message to the terminal, and the terminal notifies the user of the message.

[0241] Input: Generated follow-up message

[0242] Output: Notify user of follow-up message

[0243] (Application example 1)

[0244] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0245] Existing parenting support devices are specialized in monitoring infant status and providing answers to parental questions, but they lack the functionality to provide the support parents need when using food delivery services. This leaves parents, especially those with busy schedules, struggling to prepare meals. Furthermore, the lack of actions that take infant status into consideration often prevents parents from receiving appropriate resources and support.

[0246] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0247] In this invention, the server includes means for a user to input basic information, means for receiving the basic information and storing it in a secure database, means for accepting user questions and generating answers using a generation AI, means for displaying the generated answers to the user, means for monitoring the infant's condition using a built-in camera and various sensors, means for analyzing the monitoring data and generating an alert if an abnormality is detected, means for notifying the user of the alert, means for checking the user's consultation history and generating and sending a follow-up message, means for making individual suggestions taking into account the child-rearing situation, and means for providing appropriate services to the user based on the individual suggestions.This enables busy parents raising children to quickly prepare meals and receive appropriate support taking into account the infant's condition.

[0248] "User" refers to a person who uses the system.

[0249] "Basic Information" refers to basic data about a user that the user enters into the system (e.g., name, age, allergy information, etc.).

[0250] "Generative AI" refers to artificial intelligence technology that generates appropriate answers to user questions.

[0251] "Built-in camera" refers to a camera that is built into a device.

[0252] "Various sensors" refers to devices that detect various conditions such as temperature, humidity, and sound.

[0253] "Surveillance data" refers to data collected from built-in cameras and various sensors.

[0254] An "anomaly" refers to an abnormal situation detected from monitoring data.

[0255] "Alert" refers to a warning generated when an abnormality is detected.

[0256] "Follow-up messages" refer to additional support messages sent periodically based on past consultations.

[0257] "Individual suggestions that take into account the child-rearing situation" refers to individual suggestions provided based on the user's child-rearing situation.

[0258] "Means for providing appropriate services" refers to the function of providing appropriate services according to the needs of the user.

[0259] A "secure database" refers to a system for securely storing and managing user basic information and monitoring data.

[0260] This invention realizes a system to support parents raising children. This system allows users to input basic information, provides answers generated by generative AI, monitors the child's condition, and generates alerts if an abnormality is detected. It also makes individual suggestions and provides appropriate services, providing quick support to busy parents raising children.

[0261] System Configuration

[0262] Hardware

[0263] Built-in camera: A camera installed in the room to monitor the baby's condition in real time, detecting the baby's movements, facial expression, and voice.

[0264] Various sensors: These sensors are used to monitor the environment in the baby's room, including temperature, humidity, and sound.

[0265] Smartphone: A mobile device that users use to operate applications.

[0266] software

[0267] Generative AI: This is an artificial intelligence technology that generates appropriate answers to user questions. Specifically, it uses the OpenAI (registered trademark) API.

[0268] Natural language processing technology: Technology for analyzing questions entered by users.

[0269] Database: A secure database for storing users' basic information and consultation history.

[0270] Operation overview

[0271] 1. Enter basic information

[0272] First, users launch the smartphone app and enter basic information such as their child's name, age, allergy information, etc. This information is received by the server and stored in a secure database.

[0273] 2. Childcare consultation

[0274] Users press the "Consult" button within the app and enter their child-rearing questions either in text or by voice. The device sends the question to the server, where the AI ​​analyzes it and generates an answer. The answer is then displayed on the device for the user to review.

[0275] 3. Baby monitoring

[0276] The built-in camera and sensors monitor the baby's condition 24 hours a day. The monitoring data is sent to a server, and if an abnormality is detected, an alert is generated and a notification is sent to the device. For example, if crying is detected, an alert will be displayed saying, "Your child is crying. Please check."

[0277] 4. Individual proposals

[0278] The system takes into account the child-rearing situation and makes appropriate suggestions such as food delivery services. For example, if a baby is crying at night, the system will suggest, "He might be hungry. Would you like to order some milk, which is easy to prepare?"

[0279] 5. Follow-up

[0280] Based on the user's past consultation history, follow-up messages are periodically generated and sent to the user. For example, if a user consulted about "baby's constipation" a week ago, a follow-up message will be sent asking, "How is the constipation from last week? Has it improved?"

[0281] Specific examples

[0282] Example prompts

[0283] Question: "My baby cries at night, what should I do?"

[0284] Answer: "There are many reasons why your baby may be crying at night, but the first thing to do is to look at your baby's environment. Check the following:

[0285] Is the diaper wet?

[0286] Is the temperature appropriate?

[0287] Are you hungry?

[0288] If these checks don't improve the condition, we recommend consulting your pediatrician.

[0289] As a result, the system of the present invention can solve the various problems faced by parents raising children and provide them with prompt and effective support.

[0290] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0291] Step 1:

[0292] The server provides a means for users to input basic information. This includes an interface for entering the child's name, age, gender, allergy information, etc. into the device through a smartphone app. The device sends the input data to the server, where it is stored in a secure database. The server accepts the basic information provided by the user as input and processes it for secure storage. The output is the stored basic information data.

[0293] Step 2:

[0294] When a user presses the "Consult" button in the app, the device accepts a question from the user. The entered text or voice data is transferred to the server. The server uses a generative AI model to analyze the entered question and generates an appropriate answer using natural language processing technology. The user's question is given as input, and the generative AI model performs data calculations to generate an answer. The output is the generated answer.

[0295] Step 3:

[0296] The server provides a means to display the generated answer to the user. The answer is sent to the device as text or audio, and the device displays it to the user. The device also checks whether the answer has been displayed to the user and reports the display status to the server. The answer from the generative AI model is given as input, and data processing is performed to display it in a form that the user can understand. The output is the answer displayed to the user.

[0297] Step 4:

[0298] The device constantly monitors the baby's condition using a built-in camera and sensors. The monitoring data is sent to a server in real time. The server analyzes this monitoring data and generates an alert if an abnormality is detected. The monitoring data from the camera and sensors is sent to the server as input, and data analysis is performed. The output is alert information when an abnormality is detected.

[0299] Step 5:

[0300] If the server detects an abnormality, it generates an alert and notifies the device, which then displays the details to the user. For example, if crying is detected, an in-app notification displays the message "Your child is crying. Please check." The input is the detected abnormality information, and the server generates an alert message and processes the data for notification. The output is the alert notified to the user.

[0301] Step 6:

[0302] The server periodically checks the user's consultation history and generates follow-up messages as necessary. The messages are sent to the terminal and displayed to the user. For example, if a user consulted about "baby constipation" a week ago, a follow-up message asking "How was last week's constipation?" is displayed. The past consultation history is referenced as input, and the follow-up message is generated through data processing using a generative AI model. The output is the follow-up message displayed to the user.

[0303] Step 7:

[0304] Furthermore, the server has a means for making personalized suggestions that take into account the child-rearing situation. For example, if a baby is crying at night, the server may suggest, "He may be hungry. Would you like to order some milk, which is easy to prepare?" Data on the baby's condition and basic information about the user are used as input, and data calculations are performed to suggest an appropriate response. The output is a personalized suggestion provided to the user.

[0305] Through these processing steps, the system of the present invention provides quick and accurate support to parents raising children.

[0306] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0307] ---

[0308] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generation AI, generating and sending follow-up messages, and combining them with an emotion engine that recognizes the user's emotions.

[0309] Initial Setup

[0310] 1. Start the device and enter basic information

[0311] First, the user starts up the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[0312] Childcare consultation function

[0313] 2. Enter questions and view answers

[0314] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server uses natural language processing technology to analyze the question and uses generative AI to generate an appropriate answer. The generated answer is evaluated using an emotion engine to assess the user's emotional state and is presented in a way that takes the user's emotions into consideration. This allows users to consult at any time, 24 hours a day.

[0315] Examples:

[0316] User: "My baby cries at night, what should I do?"

[0317] Terminal: Sends a question to the server.

[0318] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0319] Server: Recognizes the user's emotions using an emotion engine and provides answers in a gentle tone.

[0320] Terminal: Display the answer to the user.

[0321] Baby monitoring features

[0322] 3. Monitoring and Alerting

[0323] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[0324] Examples:

[0325] Terminal: A camera in the baby's room monitors the crib.

[0326] Server: Detects crying and determines it is abnormal.

[0327] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0328] User: Opens the app and sees live footage from the camera.

[0329] Follow-up function

[0330] 4. Generate and send follow-up messages

[0331] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the user's emotional state using an emotion engine and providing appropriate follow-up information and advice. The generated message is then sent to the user via their device.

[0332] Examples:

[0333] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0334] Server: Generate a follow-up message: "I was concerned about your constipation last week. Has it improved?"

[0335] Server: Recognizes the user's emotions using an emotion engine and presents follow-up messages in appropriate words.

[0336] Terminal: Send a follow-up message to the user.

[0337] ---

[0338] As a result, the child-rearing support device of the present invention can provide appropriate support 24 hours a day, taking into consideration the emotional needs of families raising children, who tend to be isolated.The system of the present invention allows parents to concentrate on raising their children with peace of mind.

[0339] The processing flow will be explained below.

[0340] Initial Setup

[0341] Step 1: Turn on the device

[0342] The user turns on the device with the power button.

[0343] Step 2: Install the app

[0344] Users install a dedicated app on their smartphone or tablet.

[0345] The terminal notifies the user that the installation is complete.

[0346] Step 3: Enter basic information

[0347] Users launch the app and enter basic information such as their child's age, gender, and allergy information on the profile screen.

[0348] The terminal checks the basic information entered and checks that all required fields have been entered.

[0349] The server receives the basic information sent by the device and stores it in a secure database.

[0350] Childcare consultation function

[0351] Step 1: Start a consultation

[0352] The user presses the "Consult" button within the app.

[0353] Step 2: Enter your question

[0354] The user enters a question by text input or voice input.

[0355] The terminal checks the input and prepares to send it to the server.

[0356] Step 3: Submit your question

[0357] The terminal transmits the entered question to the server.

[0358] Step 4: Emotion Recognition

[0359] The server passes the received question to an emotion engine to recognize the user's emotional state.

[0360] Step 5: Question Analysis

[0361] The server uses natural language processing technology to analyze the question and understand its content.

[0362] Step 6: Answer Generation

[0363] The server uses generative AI to generate appropriate answers to questions.

[0364] Step 7: Emotional Consideration

[0365] The server passes the generated answer to the emotion engine and adjusts the answer content in a way that takes the user's emotions into consideration.

[0366] Step 8: Submit your response

[0367] The server sends the adjusted response to the terminal.

[0368] Step 9: View Answers

[0369] The terminal displays the adjusted answer to the user.

[0370] Examples:

[0371] User: "My baby cries at night, what should I do?"

[0372] Terminal: Sends a question to the server.

[0373] Server: The emotion engine recognizes the user's emotions, analyzes the question, and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment." The answer is adjusted taking into account the user's emotional state.

[0374] Terminal: Display the adjusted answer to the user.

[0375] Baby monitoring features

[0376] Step 1: Turn on the camera

[0377] The device activates its built-in camera to monitor the baby's room.

[0378] Step 2: Data collection

[0379] The device collects camera footage and audio data in real time.

[0380] Step 3: Send data

[0381] The terminal transmits the collected data to the server.

[0382] Step 4: Data analysis

[0383] The server analyzes the received data and monitors the baby's condition, generating an alert if an abnormality is detected.

[0384] Step 5: Sending an alert

[0385] The server sends an alert to the terminal.

[0386] Step 6: Alert Notifications

[0387] The device will notify the user of the alert via voice or app notification.

[0388] Examples:

[0389] Terminal: A camera in the baby's room monitors the crib.

[0390] Server: Detects crying and generates an alert as an anomaly.

[0391] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0392] User: Opens the app and sees live footage from the camera.

[0393] Follow-up function

[0394] Step 1: Follow-up check

[0395] The server periodically checks the user's status and consultation history.

[0396] Step 2: Emotion Recognition

[0397] The server evaluates the user's emotional state using an emotion engine.

[0398] Step 3: Generate a follow-up message

[0399] The server optionally generates follow-up messages based on the user's emotional state as assessed by the emotion engine.

[0400] Step 4: Send a follow-up message

[0401] The server sends the generated follow-up message to the terminal.

[0402] Step 5: Display a follow-up message

[0403] The terminal displays the follow-up message to the user.

[0404] Examples:

[0405] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0406] Server: The emotion engine recognizes the user's emotions and generates a follow-up message such as, "I'm worried about your constipation last week. Has it improved?"

[0407] Terminal: Send a follow-up message to the user.

[0408] Example 2

[0409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0410] Conventional child-rearing support systems have difficulty providing comprehensive support in environments where parents tend to feel isolated. They also lack the functionality to monitor the infant's condition and respond immediately if an abnormality occurs, or the follow-up function to consider the parents' feelings. As a result, parents are unable to concentrate on raising their children with peace of mind, and this places a heavy mental burden on them.

[0411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0412] In this invention, the server includes a means for a user to input basic information, a means for receiving the basic information and storing it in a secure database, and a means for evaluating the user's emotional state and generating appropriately tailored responses and follow-up messages based on the evaluation. This allows parents to receive appropriate parenting support 24 hours a day, allowing them to focus on raising their children with peace of mind.

[0413] "Users" refer to people who use the system, and are primarily parents and guardians who are raising children.

[0414] "Basic information" is information required for initial setup, such as the child's age, sex, and allergy information, entered by the user.

[0415] A "database" is a storage device for safely storing data such as a user's basic information and consultation history.

[0416] "Generative AI" is an artificial intelligence technology that generates appropriate answers to questions from users.

[0417] An "integrated camera" is a camera that is built into the device and is used to monitor the condition of the infant.

[0418] A "sensor" is a device used to detect an infant's environment and condition, detecting temperature, humidity, sound, etc.

[0419] "Natural language processing technology" is a technology for analyzing the content of a user's question and understanding its meaning.

[0420] An "emotion engine" is a technology that assesses a user's emotional state and adjusts the answers and messages it provides based on that.

[0421] A "follow-up message" is a tracking message generated based on the user's past consultation history and is used to provide additional advice or confirmation.

[0422] The present invention relates to a child-rearing support system that enables parents to raise their children without feeling isolated. Specific embodiments of this system will be described in detail below.

[0423] The system aims to provide comprehensive parenting support by allowing users to input basic information and then responding in various ways based on that information. Specifically, the device, which has multiple functions, monitors the infant's condition using built-in cameras and sensors, and generates an alert when an abnormality is detected. It also accepts questions from users, provides answers using generative AI, and generates and sends follow-up messages. The system also combines an emotion engine that recognizes the user's emotions, allowing it to provide more personalized support.

[0424] Initial Setup

[0425] First, the user turns on the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user enters basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and ensures that all required fields have been entered correctly. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[0426] Childcare consultation function

[0427] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server uses natural language processing technology to analyze the question and uses generative AI to generate an appropriate answer. The generated answer is evaluated using an emotion engine to assess the user's emotional state and is presented in a way that takes the user's emotions into consideration. This allows users to consult at any time, 24 hours a day.

[0428] Specific examples

[0429] User: "My baby cries at night, what should I do?"

[0430] Terminal: Sends a question to the server.

[0431] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0432] Server: Recognizes the user's emotions using an emotion engine and provides answers in a gentle tone.

[0433] Terminal: Display the answer to the user.

[0434] Baby monitoring features

[0435] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[0436] Specific examples

[0437] Terminal: A camera in the baby's room monitors the crib.

[0438] Server: Detects crying and determines it is abnormal.

[0439] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0440] User: Opens the app and sees live footage from the camera.

[0441] Follow-up function

[0442] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the user's emotional state using an emotion engine and providing appropriate follow-up information and advice. The generated message is then sent to the user via their device.

[0443] Specific examples

[0444] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0445] Server: Generate a follow-up message: "I was concerned about your constipation last week. Has it improved?"

[0446] Server: Recognizes the user's emotions using an emotion engine and presents follow-up messages in appropriate words.

[0447] Terminal: Send a follow-up message to the user.

[0448] As a result, the child-rearing support system of this invention can provide appropriate support 24 hours a day, taking into consideration the emotional needs of families raising children, who tend to be isolated. This allows parents to concentrate on raising their children with peace of mind, and is expected to reduce stress related to child-rearing.

[0449] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0450] Step 1:

[0451] Booting the device and entering basic information

[0452] User: Turn on your smartphone or tablet and open the dedicated app.

[0453] User: Download and install the dedicated app from the device store.

[0454] User: Launch the app and enter basic information such as the child's age, gender, and allergies on the profile screen that appears.

[0455] Input: Basic information entered by the user (e.g., child's age, gender, allergy information).

[0456] Terminal: Once the user has completed their input, save the data and check that all required fields have been entered.

[0457] Server: Receives basic information sent from the device and stores it in a secure database.

[0458] Output: Basic information stored in the database.

[0459] Step 2:

[0460] Enter and submit your question

[0461] User: Press the "Consult" button in the app and enter a question via text or voice.

[0462] Input: A question typed by the user (e.g., "My baby cries at night. What should I do?")

[0463] Terminal: Sends the question entered by the user to the server.

[0464] Step 3:

[0465] Parsing questions and generating answers

[0466] Server: Receives the question and automatically analyzes it using natural language processing technology. For example, it extracts keywords such as "night crying," "baby," and "measures" from the question.

[0467] Input: User question data

[0468] Server: Based on the analysis results, the generative AI model generates an appropriate answer. For example, it generates an answer such as, "There are various causes of nighttime crying, but first, try reconsidering your baby's environment."

[0469] Output: Generated answer (e.g. "There are many causes of night crying, but first try looking at your baby's environment.")

[0470] Step 4:

[0471] Ratings and answers displayed by the sentiment engine

[0472] Server: Uses an emotion engine to assess the user's emotional state and adjusts the generated response appropriately, for example, by changing the tone of voice to a gentler one.

[0473] Input: Generated answers and user emotional state assessment data

[0474] Output: Final, emotion-sensitive answer

[0475] Device: Display the adjusted answer on the user's device.

[0476] Step 5:

[0477] Baby monitoring and alert notifications

[0478] Device: Monitors the baby's condition 24 hours a day using a built-in camera and various sensors.

[0479] Input: Camera footage and sensor data

[0480] Terminal: Sends monitoring data to the server in real time.

[0481] Server: Analyzes the received data and generates an alert if an abnormality is detected, such as abnormally continuous crying.

[0482] Output: Alert message (e.g. "Your child is crying. Please check on him / her.")

[0483] On device: The alert message is communicated to the user via audio notification or in-app notification.

[0484] Step 6:

[0485] Generate and send follow-up messages

[0486] Server: Periodically checks the user's past consultation history and generates a message if it determines that follow-up is necessary.

[0487] Input: Consultation history data

[0488] Server: Generates follow-up messages and uses the emotion engine to tailor them with appropriate language. For example, "I'm concerned about your constipation issue last week. Has it improved?"

[0489] Output: Follow-up message (e.g., "I was concerned about your constipation issue last week. Has it improved?")

[0490] Device: Send a follow-up message to the user's device.

[0491] (Application example 2)

[0492] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0493] While existing parenting support devices provide support for parenting situations where parents tend to feel isolated, there has also been a lack of means to provide appropriate support for customers' worries and anxieties in the field of customer service in brick-and-mortar stores. In particular, brick-and-mortar stores need to be able to grasp customers' emotions in real time and respond accordingly, but the current situation is that such systems are not fully developed. Furthermore, building ongoing relationships with customers is important, and follow-up functions based on past consultations are also needed.

[0494] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to input basic information; means for receiving the basic information and storing it in a secure database; means for accepting a user's question and generating an answer using a generation AI; means for displaying the generated answer to the user; means for monitoring the status of an object using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert when an abnormality is detected; means for notifying the user of the alert; means for checking the user's consultation history and generating and sending a follow-up message; means for evaluating the user's emotions in real time using an emotion recognition engine; and means for providing the user with the optimal response based on the evaluation results. This enables real-time responses and continuous follow-up that take customer emotions into consideration in customer service in physical stores.

[0495] "Means for users to enter basic information" refers to means by which users enter profile information and other required data, including dedicated applications and web interfaces.

[0496] "Means for receiving the basic information and storing it in a secure database" refers to means for receiving information entered by the user and storing it in a secure database, and includes encryption technology and access control.

[0497] "Means for accepting user questions and generating answers using generative AI" refers to means for accepting questions from users and using generative AI (for example, a model using natural language processing technology) to analyze the questions and generate appropriate answers.

[0498] The "means for displaying the generated answer to the user" refers to a means for displaying the generated answer on the user's device (smartphone, tablet, etc.), and includes a user interface.

[0499] "Means for monitoring the condition of an object using a built-in camera and various sensors" means means for continuously monitoring the condition of an object (e.g., an infant or a customer) using a built-in camera and various sensors.

[0500] The "means for analyzing the monitoring data and generating an alert when an abnormality is detected" refers to a means for analyzing the monitoring data and generating a warning when an abnormal condition is detected, such as by sending a voice notification or a message.

[0501] The "means for notifying the user of the alert" refers to a means for notifying the user's device of the generated alert, and includes push notification and email transmission.

[0502] The "means for checking the user's consultation history and generating and sending a follow-up message" refers to a means for checking the past consultation history, generating a follow-up message based on the history, and sending the message to the user at an appropriate time.

[0503] The "means for evaluating a user's emotions in real time using an emotion recognition engine" is a means for using an emotion recognition engine to evaluate a user's facial expression and tone of voice in real time and grasp their emotional state.

[0504] The "means for providing the user with the most appropriate response based on the evaluation results" is a means for providing the user with the most appropriate response method or information based on the evaluation results of the emotion recognition engine.

[0505] The present invention provides a "smart customer service device" to improve customer service in brick-and-mortar stores. The system receives basic information entered by customers, stores it securely, and then takes various actions based on that information.

[0506] Initial Setup

[0507] 1. Enter basic information

[0508] The user (store staff) activates the device and installs a dedicated app on their smartphone or smart glasses. Once the app is activated, the user enters customer profile information, including the customer's name, contact details, and past purchase history. The server receives this basic information and stores it in a secure database.

[0509] Customer service functions

[0510] 2. Question acceptance and answer generation

[0511] When a user (customer) inputs a question into the device, the server accepts it. The question can be entered as text or voice. The server analyzes the received question and generates an appropriate answer using a generative AI model (e.g., GPT-3). The generated answer is displayed on the user's device.

[0512] Examples:

[0513] User (customer): "I'd like to know more about this product?"

[0514] Server: Parses the question and generates an answer using a generative AI model.

[0515] Server: "This product is manufactured using the latest technology and has excellent performance."

[0516] Emotion recognition function

[0517] 3. Emotion recognition and response guidance

[0518] The system uses cameras and sensors inside the building to analyze customers' facial expressions and tone of voice, and evaluates them in real time using an emotion recognition engine. Based on the evaluation results, the system provides the user (customer) with the most appropriate response method.

[0519] Examples:

[0520] Terminal: Sends the customer's facial expression analysis data to the server.

[0521] Server: Detects "anxiety" using an emotion recognition engine.

[0522] Server: "You seem concerned. What are you worried about?" I suggest.

[0523] Alert notification function

[0524] 4. Anomaly detection and alert notification

[0525] The device is equipped with a built-in camera and various sensors that monitor the customer's condition. The monitoring data is analyzed on the server, and an alert is generated if an abnormality is detected. The alert is communicated to the user (store staff) as a voice or notification.

[0526] Examples:

[0527] Device: In-store camera detects abnormal behavior.

[0528] Server: Determines that something is abnormal and generates an alert.

[0529] Terminal: Notifies, "Abnormal behavior has been detected in Area A."

[0530] Follow-up function

[0531] 5. Generate and send follow-up messages

[0532] The server periodically checks the customer's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the customer's emotional state using an emotion recognition engine, and is sent to the user (customer) to provide appropriate follow-up information and advice.

[0533] Examples:

[0534] Server: Check the history of customers who consulted about "how to use the product" one week ago.

[0535] Server: Generates and sends a follow-up message: "Regarding the product issue from last week, has the issue been resolved since then?"

[0536] Prompt Sentence Examples

[0537] A customer asks the following question: I'd like to know more about this product? Emotion recognition indicates that the customer is interested. Generate an appropriate answer for this situation.

[0538] This system can improve customer satisfaction in physical stores by understanding customer sentiment in real time and providing optimal responses and continuous follow-up.

[0539] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0540] Step 1:

[0541] The user (store staff) installs the dedicated app on their smartphone or smart glasses and launches the app. When the app is launched, the user enters customer profile information (such as name, contact information, and past purchase history). This information is sent from the device to the server and stored in a secure database.

[0542] Input: Customer profile information

[0543] Output: Customer information stored in a secure database

[0544] Step 2:

[0545] The user (customer) enters a question into the device via text or voice. The device sends this question to the server. The server receives the question and analyzes it using natural language processing technology. It uses a generative AI model (e.g., GPT-3) to generate an appropriate answer and sends it to the device.

[0546] Input: User question

[0547] Output: Answer from the generative AI model

[0548] Step 3:

[0549] The terminal displays the generated answer on the user's (customer's) device, and the user (customer) can check the answer to resolve their doubts and ask additional questions as needed.

[0550] Input: Generated answer

[0551] Output: Display the answer to the user (customer)

[0552] Step 4:

[0553] The device uses a built-in camera and sensors to monitor customer facial expressions and tone of voice in real time, and the monitoring data is sent from the device to a server, which then uses an emotion recognition engine to analyze the data and evaluate the customer's emotional state.

[0554] Input: Surveillance data (facial expressions, tone of voice)

[0555] Output: Customer emotional state assessment

[0556] Step 5:

[0557] The server proposes the optimal response based on the evaluation results. It generates a prompt using the generative AI model along with the evaluation results, presents an appropriate response plan, and sends it to the user (store staff).

[0558] Input: Customer sentiment evaluation results

[0559] Output: Suggested action

[0560] Step 6:

[0561] The device continuously monitors the situation inside the store using a built-in camera and various sensors. The monitoring data is sent from the terminal to a server, and if the server detects an abnormality, it generates an alert and notifies the user (store staff).

[0562] Input: In-store surveillance data

[0563] Output: Alert notification in case of abnormality

[0564] Step 7:

[0565] The server periodically checks the customer's past consultation and purchase history, and if it determines that follow-up is necessary, it uses an emotion recognition engine to generate an appropriate follow-up message, which is then sent to the user (customer) via their device.

[0566] Input: Consultation history, purchase history

[0567] Output: Follow-up message

[0568] Step 8:

[0569] The user (customer) receives the follow-up message and replies as needed. The reply is also sent back to the server, enabling continuous support.

[0570] Input: Follow-up message

[0571] Output: Reply from the user (customer)

[0572] Through this series of steps, customer service in brick-and-mortar stores can be significantly improved by utilizing emotion recognition and generative AI models, leading to increased customer satisfaction.

[0573] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0574] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0575] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0576] [Second embodiment]

[0577] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0578] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0579] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0580] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0581] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0582] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0583] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0584] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0585] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0586] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0587] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0588] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0589] ---

[0590] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generation AI, and generating and sending follow-up messages.

[0591] Initial Setup

[0592] 1. Start the device and enter basic information

[0593] First, the user starts up the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[0594] Childcare consultation function

[0595] 2. Enter questions and view answers

[0596] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server, which then uses generative AI and natural language processing technology to analyze the question and generate an appropriate answer. The generated answer is displayed to the user, who can then consult with the server 24 hours a day.

[0597] Examples:

[0598] User: "My baby cries at night, what should I do?"

[0599] Terminal: Sends a question to the server.

[0600] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0601] Terminal: Display the answer to the user.

[0602] Baby monitoring features

[0603] 3. Monitoring and Alerting

[0604] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[0605] Examples:

[0606] Terminal: A camera in the baby's room monitors the crib.

[0607] Server: Detects crying and determines it is abnormal.

[0608] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0609] User: Opens the app and sees live footage from the camera.

[0610] Follow-up function

[0611] 4. Generate and send follow-up messages

[0612] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message provides appropriate follow-up information and advice based on the content of the past consultation. The generated message is sent to the user via the terminal.

[0613] Examples:

[0614] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0615] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[0616] Terminal: Send a follow-up message to the user.

[0617] ---

[0618] As a result, the child-rearing support device of the present invention can provide appropriate support 24 hours a day to families raising children who tend to be isolated.The system of the present invention allows parents to concentrate on raising their children with peace of mind.

[0619] The processing flow will be explained below.

[0620] ---

[0621] Initial Setup

[0622] Step 1: Turn on the device

[0623] The user turns on the device with the power button.

[0624] Step 2: Install the app

[0625] Users install a dedicated app on their smartphone or tablet.

[0626] The terminal notifies the user that the installation is complete.

[0627] Step 3: Enter basic information

[0628] Users launch the app and enter basic information such as their child's age, gender, and allergy information on the profile screen.

[0629] The terminal checks the basic information entered and checks that all required fields have been entered.

[0630] The server receives the basic information sent by the device and stores it in a secure database.

[0631] Childcare consultation function

[0632] Step 1: Start a consultation

[0633] The user presses the "Consult" button within the app.

[0634] Step 2: Enter your question

[0635] The user enters a question by text input or voice input.

[0636] The terminal checks the input and prepares to send it to the server.

[0637] Step 3: Submit your question

[0638] The terminal transmits the entered question to the server.

[0639] Step 4: Question analysis

[0640] The server analyzes the received question and uses natural language processing (NLP) technology to understand the question.

[0641] Step 5: Answer Generation

[0642] The server uses generative AI to generate appropriate answers to questions.

[0643] Step 6: Submit your response

[0644] The server sends the generated response to the terminal.

[0645] Step 7: View Answers

[0646] The terminal displays the answer to the user.

[0647] Examples:

[0648] User: "My baby cries at night, what should I do?"

[0649] Terminal: Sends a question to the server.

[0650] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0651] Terminal: Display the answer to the user.

[0652] Baby monitoring features

[0653] Step 1: Turn on the camera

[0654] The device activates its built-in camera to monitor the baby's room.

[0655] Step 2: Data collection

[0656] The device collects camera footage and audio data in real time.

[0657] Step 3: Send data

[0658] The terminal transmits the collected data to the server.

[0659] Step 4: Data analysis

[0660] The server analyzes the received data and monitors the baby's condition, generating an alert if an abnormality is detected.

[0661] Step 5: Sending an alert

[0662] The server sends an alert to the terminal.

[0663] Step 6: Alert Notifications

[0664] The device will notify the user of the alert via voice or app notification.

[0665] Examples:

[0666] Terminal: A camera in the baby's room monitors the crib.

[0667] Server: Detects the baby's crying and flags it as an anomaly.

[0668] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0669] User: Opens the app and sees live footage from the camera.

[0670] Follow-up function

[0671] Step 1: Follow-up check

[0672] The server periodically checks the user's status and consultation history.

[0673] Step 2: Generate a follow-up message

[0674] The server generates follow-up messages as needed.

[0675] Step 3: Send a follow-up message

[0676] The server sends the generated follow-up message to the terminal.

[0677] Step 4: Display a follow-up message

[0678] The terminal displays the follow-up message to the user.

[0679] Examples:

[0680] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0681] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[0682] Terminal: Send a follow-up message to the user.

[0683] ---

[0684] Example 1

[0685] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0686] With the advancement of information and communication technology, there is a demand for systems that provide efficient and appropriate support to families raising children, who tend to be isolated. However, existing systems have difficulty providing quick and accurate answers to users' questions, continuously monitoring the condition of infants, and immediately notifying them when abnormalities occur. Furthermore, systems that generate follow-up messages based on the user's past consultations are also inadequate. The challenge is to provide a more advanced support system that solves these problems and allows parents to raise their children with peace of mind.

[0687] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0688] In this invention, the server includes: means for a user to input basic information; means for receiving the basic information and storing it in a secure database; means for accepting user questions and generating answers using a generative AI model; means for displaying the generated answers to the user; means for monitoring the infant's condition using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert if an abnormality is detected; means for notifying the user of the alert; means for checking the user's consultation history and generating and sending a follow-up message; and means for transmitting the device's basic information and monitoring data to the server and for the server to analyze the data. This enables 24-hour monitoring of the infant's condition, prompt and accurate answers to questions, and follow-up based on past consultations.

[0689] "Basic information" refers to information about initial settings such as the child's age, sex, and allergy information entered by the user.

[0690] A "generative AI model" refers to an algorithm or program that uses artificial intelligence technology to generate appropriate answers to user questions.

[0691] An "integrated camera" is a camera that is built into a device and is used to monitor real-time footage of an infant.

[0692] A "sensor" is a device built into a device to detect temperature, humidity, sound, etc.

[0693] "Monitoring Data" refers to data about the infant's condition collected by built-in cameras and sensors.

[0694] "Abnormal" is when the baby is crying, not moving, or indicates an unusual situation detected by the sensor.

[0695] An "alert" is a warning message or audio notification that is given to the user when an abnormality is detected.

[0696] A "follow-up message" is a message that includes follow-up information and additional advice that is generated based on the user's past consultation history.

[0697] A "dedicated app" is application software designed for users to install and use on their smartphones or tablets.

[0698] The "server" is a central processing unit that receives and analyzes basic information, questions, and monitoring data entered by users.

[0699] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generative AI model, and generating and sending follow-up messages.

[0700] Initial Setup

[0701] First, the user turns on the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[0702] Childcare consultation function

[0703] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server then analyzes the question using generative AI models and natural language processing technology, and generates an appropriate answer. The generated answer is then sent to the device and displayed to the user. Users can consult at any time, 24 hours a day.

[0704] Examples:

[0705] User: "My baby cries at night, what should I do?"

[0706] Terminal: Sends a question to the server.

[0707] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0708] Terminal: Display the answer to the user.

[0709] Baby monitoring features

[0710] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[0711] Examples:

[0712] Terminal: A camera in the baby's room monitors the crib.

[0713] Server: Detects crying and determines it is abnormal.

[0714] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0715] User: Opens the app and sees live footage from the camera.

[0716] Follow-up function

[0717] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message provides appropriate follow-up information and advice based on the content of the past consultation. The generated message is sent to the user via the terminal.

[0718] Examples:

[0719] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0720] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[0721] Terminal: Send a follow-up message to the user.

[0722] technical operation

[0723] In this system, the server analyzes questions and generates answers using generative AI models (e.g., GPT-3) and natural language processing technology. The built-in camera and sensors monitor the infant's condition and send the collected data to the server. The server analyzes this data and, if an abnormality is detected, quickly generates an alert and notifies the user. The server also generates appropriate follow-up messages based on past consultation history and sends them to the user.

[0724] Example prompt sentence:

[0725] "My baby cries at night, what should I do?"

[0726] "My baby's constipation won't improve, what should I do?"

[0727] "I don't know when my baby will roll over. Can you tell me?"

[0728] As a result, the child-rearing support device of the present invention can provide 24-hour support to help parents raise their children with peace of mind.

[0729] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0730] Step 1:

[0731] The user starts the device and installs the dedicated app on their smartphone or tablet. When the app is launched, the user is presented with a login screen or new registration screen. The user enters basic information (child's age, gender, allergy information, etc.).

[0732] Input: Device startup, app installation, basic information input

[0733] Output: Display of basic information input screen, basic information entered

[0734] Step 2:

[0735] The device checks the basic information and checks whether all required fields have been entered correctly. If the basic information has been entered correctly, it is sent to the server.

[0736] Input: Basic information

[0737] Data processing: Check required fields

[0738] Output: Send basic information to the server

[0739] Step 3:

[0740] The server stores the received basic information in a secure database.

[0741] Input: Basic information sent from the device

[0742] Data processing: Basic information stored in a database

[0743] Output: Confirmation of successful save

[0744] Step 4:

[0745] The user presses the "Consult" button and enters a question, which can be entered in text or voice format.

[0746] Input: Question (text or voice)

[0747] Output: Question input completed

[0748] Step 5:

[0749] The terminal transmits the entered question to the server.

[0750] Input: User question

[0751] Output: Sending a question to the server

[0752] Step 6:

[0753] The server analyzes the question using generative AI models and natural language processing technology, and generates an appropriate answer based on the analysis results.

[0754] Input: Question

[0755] Data Computing: Parsing questions and generating answers (generative AI models)

[0756] Output: The generated answer

[0757] Step 7:

[0758] The server sends the generated response to the terminal.

[0759] Input: Generated answer

[0760] Output: Sends the answer to the terminal

[0761] Step 8:

[0762] The terminal displays the received answer on the screen, allowing the user to check the answer.

[0763] Input: Generated answer

[0764] Output: Display answer

[0765] Step 9:

[0766] The device uses a built-in camera and various sensors to monitor the baby's room 24 hours a day.

[0767] Input: Camera and sensor activation

[0768] Data processing: Collecting monitoring data

[0769] Output: Real-time monitoring data

[0770] Step 10:

[0771] The terminal transmits the monitoring data to the server in real time.

[0772] Input: Collected monitoring data

[0773] Output: Sending monitoring data to a server

[0774] Step 11:

[0775] The server analyzes the received monitoring data and generates an alert if an abnormality is detected.

[0776] Input: Real-time monitoring data

[0777] Data Calculation: Anomaly Detection and Alert Generation

[0778] Output: The generated alert

[0779] Step 12:

[0780] The server sends the generated alert to the device, which then notifies the user by voice or in-app notification.

[0781] Input: Generated alert

[0782] Output: Alert notification to the user

[0783] Step 13:

[0784] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary.

[0785] Input: Consultation history

[0786] Data calculation: determining whether follow-up is necessary and generating messages

[0787] Output: Generated follow-up message

[0788] Step 14:

[0789] The server sends the generated follow-up message to the terminal, and the terminal notifies the user of the message.

[0790] Input: Generated follow-up message

[0791] Output: Notify user of follow-up message

[0792] (Application example 1)

[0793] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0794] Existing parenting support devices are specialized in monitoring infant status and providing answers to parental questions, but they lack the functionality to provide the support parents need when using food delivery services. This leaves parents, especially those with busy schedules, struggling to prepare meals. Furthermore, the lack of actions that take infant status into consideration often prevents parents from receiving appropriate resources and support.

[0795] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0796] In this invention, the server includes means for a user to input basic information, means for receiving the basic information and storing it in a secure database, means for accepting user questions and generating answers using a generation AI, means for displaying the generated answers to the user, means for monitoring the infant's condition using a built-in camera and various sensors, means for analyzing the monitoring data and generating an alert if an abnormality is detected, means for notifying the user of the alert, means for checking the user's consultation history and generating and sending a follow-up message, means for making individual suggestions taking into account the child-rearing situation, and means for providing appropriate services to the user based on the individual suggestions.This enables busy parents raising children to quickly prepare meals and receive appropriate support taking into account the infant's condition.

[0797] "User" refers to a person who uses the system.

[0798] "Basic Information" refers to basic data about a user that the user enters into the system (e.g., name, age, allergy information, etc.).

[0799] "Generative AI" refers to artificial intelligence technology that generates appropriate answers to user questions.

[0800] "Built-in camera" refers to a camera that is built into a device.

[0801] "Various sensors" refers to devices that detect various conditions such as temperature, humidity, and sound.

[0802] "Surveillance data" refers to data collected from built-in cameras and various sensors.

[0803] An "anomaly" refers to an abnormal situation detected from monitoring data.

[0804] "Alert" refers to a warning generated when an abnormality is detected.

[0805] "Follow-up messages" refer to additional support messages sent periodically based on past consultations.

[0806] "Individual suggestions that take into account the child-rearing situation" refers to individual suggestions provided based on the user's child-rearing situation.

[0807] "Means for providing appropriate services" refers to the function of providing appropriate services according to the needs of the user.

[0808] A "secure database" refers to a system for securely storing and managing user basic information and monitoring data.

[0809] This invention realizes a system to support parents raising children. This system allows users to input basic information, provides answers generated by generative AI, monitors the child's condition, and generates alerts if an abnormality is detected. It also makes individual suggestions and provides appropriate services, providing quick support to busy parents raising children.

[0810] System Configuration

[0811] Hardware

[0812] Built-in camera: A camera installed in the room to monitor the baby's condition in real time, detecting the baby's movements, facial expression, and voice.

[0813] Various sensors: These sensors are used to monitor the environment in the baby's room, including temperature, humidity, and sound.

[0814] Smartphone: A mobile device that users use to operate applications.

[0815] software

[0816] Generative AI: This is an artificial intelligence technology that generates appropriate answers to user questions. Specifically, it uses the OpenAI API.

[0817] Natural language processing technology: Technology for analyzing questions entered by users.

[0818] Database: A secure database for storing users' basic information and consultation history.

[0819] Operation overview

[0820] 1. Enter basic information

[0821] First, users launch the smartphone app and enter basic information such as their child's name, age, allergy information, etc. This information is received by the server and stored in a secure database.

[0822] 2. Childcare consultation

[0823] Users press the "Consult" button within the app and enter their child-rearing questions either in text or by voice. The device sends the question to the server, where the AI ​​analyzes it and generates an answer. The answer is then displayed on the device for the user to review.

[0824] 3. Baby monitoring

[0825] The built-in camera and sensors monitor the baby's condition 24 hours a day. The monitoring data is sent to a server, and if an abnormality is detected, an alert is generated and a notification is sent to the device. For example, if crying is detected, an alert will be displayed saying, "Your child is crying. Please check."

[0826] 4. Individual proposals

[0827] The system takes into account the child-rearing situation and makes appropriate suggestions such as food delivery services. For example, if a baby is crying at night, the system will suggest, "He might be hungry. Would you like to order some milk, which is easy to prepare?"

[0828] 5. Follow-up

[0829] Based on the user's past consultation history, follow-up messages are periodically generated and sent to the user. For example, if a user consulted about "baby's constipation" a week ago, a follow-up message will be sent asking, "How is the constipation from last week? Has it improved?"

[0830] Specific examples

[0831] Example prompts

[0832] Question: "My baby cries at night, what should I do?"

[0833] Answer: "There are many reasons why your baby may be crying at night, but the first thing to do is to look at your baby's environment. Check the following:

[0834] Is the diaper wet?

[0835] Is the temperature appropriate?

[0836] Are you hungry?

[0837] If these checks don't improve the condition, we recommend consulting your pediatrician.

[0838] As a result, the system of the present invention can solve the various problems faced by parents raising children and provide them with prompt and effective support.

[0839] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0840] Step 1:

[0841] The server provides a means for users to input basic information. This includes an interface for entering the child's name, age, gender, allergy information, etc. into the device through a smartphone app. The device sends the input data to the server, where it is stored in a secure database. The server accepts the basic information provided by the user as input and processes it for secure storage. The output is the stored basic information data.

[0842] Step 2:

[0843] When a user presses the "Consult" button in the app, the device accepts a question from the user. The entered text or voice data is transferred to the server. The server uses a generative AI model to analyze the entered question and generates an appropriate answer using natural language processing technology. The user's question is given as input, and the generative AI model performs data calculations to generate an answer. The output is the generated answer.

[0844] Step 3:

[0845] The server provides a means to display the generated answer to the user. The answer is sent to the device as text or audio, and the device displays it to the user. The device also checks whether the answer has been displayed to the user and reports the display status to the server. The answer from the generative AI model is given as input, and data processing is performed to display it in a form that the user can understand. The output is the answer displayed to the user.

[0846] Step 4:

[0847] The device constantly monitors the baby's condition using a built-in camera and sensors. The monitoring data is sent to a server in real time. The server analyzes this monitoring data and generates an alert if an abnormality is detected. The monitoring data from the camera and sensors is sent to the server as input, and data analysis is performed. The output is alert information when an abnormality is detected.

[0848] Step 5:

[0849] If the server detects an abnormality, it generates an alert and notifies the device, which then displays the details to the user. For example, if crying is detected, an in-app notification displays the message "Your child is crying. Please check." The input is the detected abnormality information, and the server generates an alert message and processes the data for notification. The output is the alert notified to the user.

[0850] Step 6:

[0851] The server periodically checks the user's consultation history and generates follow-up messages as necessary. The messages are sent to the terminal and displayed to the user. For example, if a user consulted about "baby constipation" a week ago, a follow-up message asking "How was last week's constipation?" is displayed. The past consultation history is referenced as input, and the follow-up message is generated through data processing using a generative AI model. The output is the follow-up message displayed to the user.

[0852] Step 7:

[0853] Furthermore, the server has a means for making personalized suggestions that take into account the child-rearing situation. For example, if a baby is crying at night, the server may suggest, "He may be hungry. Would you like to order some milk, which is easy to prepare?" Data on the baby's condition and basic information about the user are used as input, and data calculations are performed to suggest an appropriate response. The output is a personalized suggestion provided to the user.

[0854] Through these processing steps, the system of the present invention provides quick and accurate support to parents raising children.

[0855] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0856] ---

[0857] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generation AI, generating and sending follow-up messages, and combining them with an emotion engine that recognizes the user's emotions.

[0858] Initial Setup

[0859] 1. Start the device and enter basic information

[0860] First, the user starts up the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[0861] Childcare consultation function

[0862] 2. Enter questions and view answers

[0863] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server uses natural language processing technology to analyze the question and uses generative AI to generate an appropriate answer. The generated answer is evaluated using an emotion engine to assess the user's emotional state and is presented in a way that takes the user's emotions into consideration. This allows users to consult at any time, 24 hours a day.

[0864] Examples:

[0865] User: "My baby cries at night, what should I do?"

[0866] Terminal: Sends a question to the server.

[0867] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0868] Server: Recognizes the user's emotions using an emotion engine and provides answers in a gentle tone.

[0869] Terminal: Display the answer to the user.

[0870] Baby monitoring features

[0871] 3. Monitoring and Alerting

[0872] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[0873] Examples:

[0874] Terminal: A camera in the baby's room monitors the crib.

[0875] Server: Detects crying and determines it is abnormal.

[0876] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0877] User: Opens the app and sees live footage from the camera.

[0878] Follow-up function

[0879] 4. Generate and send follow-up messages

[0880] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the user's emotional state using an emotion engine and providing appropriate follow-up information and advice. The generated message is then sent to the user via their device.

[0881] Examples:

[0882] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0883] Server: Generate a follow-up message: "I was concerned about your constipation last week. Has it improved?"

[0884] Server: Recognizes the user's emotions using an emotion engine and presents follow-up messages in appropriate words.

[0885] Terminal: Send a follow-up message to the user.

[0886] ---

[0887] As a result, the child-rearing support device of the present invention can provide appropriate support 24 hours a day, taking into consideration the emotional needs of families raising children, who tend to be isolated.The system of the present invention allows parents to concentrate on raising their children with peace of mind.

[0888] The processing flow will be explained below.

[0889] Initial Setup

[0890] Step 1: Turn on the device

[0891] The user turns on the device with the power button.

[0892] Step 2: Install the app

[0893] Users install a dedicated app on their smartphone or tablet.

[0894] The terminal notifies the user that the installation is complete.

[0895] Step 3: Enter basic information

[0896] Users launch the app and enter basic information such as their child's age, gender, and allergy information on the profile screen.

[0897] The terminal checks the basic information entered and checks that all required fields have been entered.

[0898] The server receives the basic information sent by the device and stores it in a secure database.

[0899] Childcare consultation function

[0900] Step 1: Start a consultation

[0901] The user presses the "Consult" button within the app.

[0902] Step 2: Enter your question

[0903] The user enters a question by text input or voice input.

[0904] The terminal checks the input and prepares to send it to the server.

[0905] Step 3: Submit your question

[0906] The terminal transmits the entered question to the server.

[0907] Step 4: Emotion Recognition

[0908] The server passes the received question to an emotion engine to recognize the user's emotional state.

[0909] Step 5: Question Analysis

[0910] The server uses natural language processing technology to analyze the question and understand its content.

[0911] Step 6: Answer Generation

[0912] The server uses generative AI to generate appropriate answers to questions.

[0913] Step 7: Emotional Consideration

[0914] The server passes the generated answer to the emotion engine and adjusts the answer content in a way that takes the user's emotions into consideration.

[0915] Step 8: Submit your response

[0916] The server sends the adjusted response to the terminal.

[0917] Step 9: View Answers

[0918] The terminal displays the adjusted answer to the user.

[0919] Examples:

[0920] User: "My baby cries at night, what should I do?"

[0921] Terminal: Sends a question to the server.

[0922] Server: The emotion engine recognizes the user's emotions, analyzes the question, and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment." The answer is adjusted taking into account the user's emotional state.

[0923] Terminal: Display the adjusted answer to the user.

[0924] Baby monitoring features

[0925] Step 1: Turn on the camera

[0926] The device activates its built-in camera to monitor the baby's room.

[0927] Step 2: Data collection

[0928] The device collects camera footage and audio data in real time.

[0929] Step 3: Send data

[0930] The terminal transmits the collected data to the server.

[0931] Step 4: Data analysis

[0932] The server analyzes the received data and monitors the baby's condition, generating an alert if an abnormality is detected.

[0933] Step 5: Sending an alert

[0934] The server sends an alert to the terminal.

[0935] Step 6: Alert Notifications

[0936] The device will notify the user of the alert via voice or app notification.

[0937] Examples:

[0938] Terminal: A camera in the baby's room monitors the crib.

[0939] Server: Detects crying and generates an alert as an anomaly.

[0940] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0941] User: Opens the app and sees live footage from the camera.

[0942] Follow-up function

[0943] Step 1: Follow-up check

[0944] The server periodically checks the user's status and consultation history.

[0945] Step 2: Emotion Recognition

[0946] The server evaluates the user's emotional state using an emotion engine.

[0947] Step 3: Generate a follow-up message

[0948] The server optionally generates follow-up messages based on the user's emotional state as assessed by the emotion engine.

[0949] Step 4: Send a follow-up message

[0950] The server sends the generated follow-up message to the terminal.

[0951] Step 5: Display a follow-up message

[0952] The terminal displays the follow-up message to the user.

[0953] Examples:

[0954] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0955] Server: The emotion engine recognizes the user's emotions and generates a follow-up message such as, "I'm worried about your constipation last week. Has it improved?"

[0956] Terminal: Send a follow-up message to the user.

[0957] Example 2

[0958] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0959] Conventional child-rearing support systems have difficulty providing comprehensive support in environments where parents tend to feel isolated. They also lack the functionality to monitor the infant's condition and respond immediately if an abnormality occurs, or the follow-up function to consider the parents' feelings. As a result, parents are unable to concentrate on raising their children with peace of mind, and this places a heavy mental burden on them.

[0960] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0961] In this invention, the server includes a means for a user to input basic information, a means for receiving the basic information and storing it in a secure database, and a means for evaluating the user's emotional state and generating appropriately tailored responses and follow-up messages based on the evaluation. This allows parents to receive appropriate parenting support 24 hours a day, allowing them to focus on raising their children with peace of mind.

[0962] "Users" refer to people who use the system, and are primarily parents and guardians who are raising children.

[0963] "Basic information" is information required for initial setup, such as the child's age, sex, and allergy information, entered by the user.

[0964] A "database" is a storage device for safely storing data such as a user's basic information and consultation history.

[0965] "Generative AI" is an artificial intelligence technology that generates appropriate answers to questions from users.

[0966] An "integrated camera" is a camera that is built into the device and is used to monitor the condition of the infant.

[0967] A "sensor" is a device used to detect an infant's environment and condition, detecting temperature, humidity, sound, etc.

[0968] "Natural language processing technology" is a technology for analyzing the content of a user's question and understanding its meaning.

[0969] An "emotion engine" is a technology that assesses a user's emotional state and adjusts the answers and messages it provides based on that.

[0970] A "follow-up message" is a tracking message generated based on the user's past consultation history and is used to provide additional advice or confirmation.

[0971] The present invention relates to a child-rearing support system that enables parents to raise their children without feeling isolated. Specific embodiments of this system will be described in detail below.

[0972] The system aims to provide comprehensive parenting support by allowing users to input basic information and then responding in various ways based on that information. Specifically, the device, which has multiple functions, monitors the infant's condition using built-in cameras and sensors, and generates an alert when an abnormality is detected. It also accepts questions from users, provides answers using generative AI, and generates and sends follow-up messages. The system also combines an emotion engine that recognizes the user's emotions, allowing it to provide more personalized support.

[0973] Initial Setup

[0974] First, the user turns on the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user enters basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and ensures that all required fields have been entered correctly. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[0975] Childcare consultation function

[0976] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server uses natural language processing technology to analyze the question and uses generative AI to generate an appropriate answer. The generated answer is evaluated using an emotion engine to assess the user's emotional state and is presented in a way that takes the user's emotions into consideration. This allows users to consult at any time, 24 hours a day.

[0977] Specific examples

[0978] User: "My baby cries at night, what should I do?"

[0979] Terminal: Sends a question to the server.

[0980] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[0981] Server: Recognizes the user's emotions using an emotion engine and provides answers in a gentle tone.

[0982] Terminal: Display the answer to the user.

[0983] Baby monitoring features

[0984] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[0985] Specific examples

[0986] Terminal: A camera in the baby's room monitors the crib.

[0987] Server: Detects crying and determines it is abnormal.

[0988] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[0989] User: Opens the app and sees live footage from the camera.

[0990] Follow-up function

[0991] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the user's emotional state using an emotion engine and providing appropriate follow-up information and advice. The generated message is then sent to the user via their device.

[0992] Specific examples

[0993] Server: Check the history of users who consulted about "baby constipation" one week ago.

[0994] Server: Generate a follow-up message: "I was concerned about your constipation last week. Has it improved?"

[0995] Server: Recognizes the user's emotions using an emotion engine and presents follow-up messages in appropriate words.

[0996] Terminal: Send a follow-up message to the user.

[0997] As a result, the child-rearing support system of this invention can provide appropriate support 24 hours a day, taking into consideration the emotional needs of families raising children, who tend to be isolated. This allows parents to concentrate on raising their children with peace of mind, and is expected to reduce stress related to child-rearing.

[0998] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0999] Step 1:

[1000] Booting the device and entering basic information

[1001] User: Turn on your smartphone or tablet and open the dedicated app.

[1002] User: Download and install the dedicated app from the device store.

[1003] User: Launch the app and enter basic information such as the child's age, gender, and allergies on the profile screen that appears.

[1004] Input: Basic information entered by the user (e.g., child's age, gender, allergy information).

[1005] Terminal: Once the user has completed their input, save the data and check that all required fields have been entered.

[1006] Server: Receives basic information sent from the device and stores it in a secure database.

[1007] Output: Basic information stored in the database.

[1008] Step 2:

[1009] Enter and submit your question

[1010] User: Press the "Consult" button in the app and enter a question via text or voice.

[1011] Input: A question typed by the user (e.g., "My baby cries at night. What should I do?")

[1012] Terminal: Sends the question entered by the user to the server.

[1013] Step 3:

[1014] Parsing questions and generating answers

[1015] Server: Receives the question and automatically analyzes it using natural language processing technology. For example, it extracts keywords such as "night crying," "baby," and "measures" from the question.

[1016] Input: User question data

[1017] Server: Based on the analysis results, the generative AI model generates an appropriate answer. For example, it generates an answer such as, "There are various causes of nighttime crying, but first, try reconsidering your baby's environment."

[1018] Output: Generated answer (e.g. "There are many causes of night crying, but first try looking at your baby's environment.")

[1019] Step 4:

[1020] Ratings and answers displayed by the sentiment engine

[1021] Server: Uses an emotion engine to assess the user's emotional state and adjusts the generated response appropriately, for example, by changing the tone of voice to a gentler one.

[1022] Input: Generated answers and user emotional state assessment data

[1023] Output: Final, emotion-sensitive answer

[1024] Device: Display the adjusted answer on the user's device.

[1025] Step 5:

[1026] Baby monitoring and alert notifications

[1027] Device: Monitors the baby's condition 24 hours a day using a built-in camera and various sensors.

[1028] Input: Camera footage and sensor data

[1029] Terminal: Sends monitoring data to the server in real time.

[1030] Server: Analyzes the received data and generates an alert if an abnormality is detected, such as abnormally continuous crying.

[1031] Output: Alert message (e.g. "Your child is crying. Please check on him / her.")

[1032] On device: The alert message is communicated to the user via audio notification or in-app notification.

[1033] Step 6:

[1034] Generate and send follow-up messages

[1035] Server: Periodically checks the user's past consultation history and generates a message if it determines that follow-up is necessary.

[1036] Input: Consultation history data

[1037] Server: Generates follow-up messages and uses the emotion engine to tailor them with appropriate language. For example, "I'm concerned about your constipation issue last week. Has it improved?"

[1038] Output: Follow-up message (e.g., "I was concerned about your constipation issue last week. Has it improved?")

[1039] Device: Send a follow-up message to the user's device.

[1040] (Application example 2)

[1041] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1042] While existing parenting support devices provide support for parenting situations where parents tend to feel isolated, there has also been a lack of means to provide appropriate support for customers' worries and anxieties in the field of customer service in brick-and-mortar stores. In particular, brick-and-mortar stores need to be able to grasp customers' emotions in real time and respond accordingly, but the current situation is that such systems are not fully developed. Furthermore, building ongoing relationships with customers is important, and follow-up functions based on past consultations are also needed.

[1043] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to input basic information; means for receiving the basic information and storing it in a secure database; means for accepting a user's question and generating an answer using a generation AI; means for displaying the generated answer to the user; means for monitoring the status of an object using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert when an abnormality is detected; means for notifying the user of the alert; means for checking the user's consultation history and generating and sending a follow-up message; means for evaluating the user's emotions in real time using an emotion recognition engine; and means for providing the user with the optimal response based on the evaluation results. This enables real-time responses and continuous follow-up that take customer emotions into consideration in customer service in physical stores.

[1044] "Means for users to enter basic information" refers to means by which users enter profile information and other required data, including dedicated applications and web interfaces.

[1045] "Means for receiving the basic information and storing it in a secure database" refers to means for receiving information entered by the user and storing it in a secure database, and includes encryption technology and access control.

[1046] "Means for accepting user questions and generating answers using generative AI" refers to means for accepting questions from users and using generative AI (for example, a model using natural language processing technology) to analyze the questions and generate appropriate answers.

[1047] The "means for displaying the generated answer to the user" refers to a means for displaying the generated answer on the user's device (smartphone, tablet, etc.), and includes a user interface.

[1048] "Means for monitoring the condition of an object using a built-in camera and various sensors" means means for continuously monitoring the condition of an object (e.g., an infant or a customer) using a built-in camera and various sensors.

[1049] The "means for analyzing the monitoring data and generating an alert when an abnormality is detected" refers to a means for analyzing the monitoring data and generating a warning when an abnormal condition is detected, such as by sending a voice notification or a message.

[1050] The "means for notifying the user of the alert" refers to a means for notifying the user's device of the generated alert, and includes push notification and email transmission.

[1051] The "means for checking the user's consultation history and generating and sending a follow-up message" refers to a means for checking the past consultation history, generating a follow-up message based on the history, and sending the message to the user at an appropriate time.

[1052] The "means for evaluating a user's emotions in real time using an emotion recognition engine" is a means for using an emotion recognition engine to evaluate a user's facial expression and tone of voice in real time and grasp their emotional state.

[1053] The "means for providing the user with the most appropriate response based on the evaluation results" is a means for providing the user with the most appropriate response method or information based on the evaluation results of the emotion recognition engine.

[1054] The present invention provides a "smart customer service device" to improve customer service in brick-and-mortar stores. The system receives basic information entered by customers, stores it securely, and then takes various actions based on that information.

[1055] Initial Setup

[1056] 1. Enter basic information

[1057] The user (store staff) activates the device and installs a dedicated app on their smartphone or smart glasses. Once the app is activated, the user enters customer profile information, including the customer's name, contact details, and past purchase history. The server receives this basic information and stores it in a secure database.

[1058] Customer service functions

[1059] 2. Question acceptance and answer generation

[1060] When a user (customer) inputs a question into the device, the server accepts it. The question can be entered as text or voice. The server analyzes the received question and generates an appropriate answer using a generative AI model (e.g., GPT-3). The generated answer is displayed on the user's device.

[1061] Examples:

[1062] User (customer): "I'd like to know more about this product?"

[1063] Server: Parses the question and generates an answer using a generative AI model.

[1064] Server: "This product is manufactured using the latest technology and has excellent performance."

[1065] Emotion recognition function

[1066] 3. Emotion recognition and response guidance

[1067] The system uses cameras and sensors inside the building to analyze customers' facial expressions and tone of voice, and evaluates them in real time using an emotion recognition engine. Based on the evaluation results, the system provides the user (customer) with the most appropriate response method.

[1068] Examples:

[1069] Terminal: Sends the customer's facial expression analysis data to the server.

[1070] Server: Detects "anxiety" using an emotion recognition engine.

[1071] Server: "You seem concerned. What are you worried about?" I suggest.

[1072] Alert notification function

[1073] 4. Anomaly detection and alert notification

[1074] The device is equipped with a built-in camera and various sensors that monitor the customer's condition. The monitoring data is analyzed on the server, and an alert is generated if an abnormality is detected. The alert is communicated to the user (store staff) as a voice or notification.

[1075] Examples:

[1076] Device: In-store camera detects abnormal behavior.

[1077] Server: Determines that something is abnormal and generates an alert.

[1078] Terminal: Notifies, "Abnormal behavior has been detected in Area A."

[1079] Follow-up function

[1080] 5. Generate and send follow-up messages

[1081] The server periodically checks the customer's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the customer's emotional state using an emotion recognition engine, and is sent to the user (customer) to provide appropriate follow-up information and advice.

[1082] Examples:

[1083] Server: Check the history of customers who consulted about "how to use the product" one week ago.

[1084] Server: Generates and sends a follow-up message: "Regarding the product issue from last week, has the issue been resolved since then?"

[1085] Prompt Sentence Examples

[1086] A customer asks the following question: I'd like to know more about this product? Emotion recognition indicates that the customer is interested. Generate an appropriate answer for this situation.

[1087] This system can improve customer satisfaction in physical stores by understanding customer sentiment in real time and providing optimal responses and continuous follow-up.

[1088] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1089] Step 1:

[1090] The user (store staff) installs the dedicated app on their smartphone or smart glasses and launches the app. When the app is launched, the user enters customer profile information (such as name, contact information, and past purchase history). This information is sent from the device to the server and stored in a secure database.

[1091] Input: Customer profile information

[1092] Output: Customer information stored in a secure database

[1093] Step 2:

[1094] The user (customer) enters a question into the device via text or voice. The device sends this question to the server. The server receives the question and analyzes it using natural language processing technology. It uses a generative AI model (e.g., GPT-3) to generate an appropriate answer and sends it to the device.

[1095] Input: User question

[1096] Output: Answer from the generative AI model

[1097] Step 3:

[1098] The terminal displays the generated answer on the user's (customer's) device, and the user (customer) can check the answer to resolve their doubts and ask additional questions as needed.

[1099] Input: Generated answer

[1100] Output: Display the answer to the user (customer)

[1101] Step 4:

[1102] The device uses a built-in camera and sensors to monitor customer facial expressions and tone of voice in real time, and the monitoring data is sent from the device to a server, which then uses an emotion recognition engine to analyze the data and evaluate the customer's emotional state.

[1103] Input: Surveillance data (facial expressions, tone of voice)

[1104] Output: Customer emotional state assessment

[1105] Step 5:

[1106] The server proposes the optimal response based on the evaluation results. It generates a prompt using the generative AI model along with the evaluation results, presents an appropriate response plan, and sends it to the user (store staff).

[1107] Input: Customer sentiment evaluation results

[1108] Output: Suggested action

[1109] Step 6:

[1110] The device continuously monitors the situation inside the store using a built-in camera and various sensors. The monitoring data is sent from the terminal to a server, and if the server detects an abnormality, it generates an alert and notifies the user (store staff).

[1111] Input: In-store surveillance data

[1112] Output: Alert notification in case of abnormality

[1113] Step 7:

[1114] The server periodically checks the customer's past consultation and purchase history, and if it determines that follow-up is necessary, it uses an emotion recognition engine to generate an appropriate follow-up message, which is then sent to the user (customer) via their device.

[1115] Input: Consultation history, purchase history

[1116] Output: Follow-up message

[1117] Step 8:

[1118] The user (customer) receives the follow-up message and replies as needed. The reply is also sent back to the server, enabling continuous support.

[1119] Input: Follow-up message

[1120] Output: Reply from the user (customer)

[1121] Through this series of steps, customer service in brick-and-mortar stores can be significantly improved by utilizing emotion recognition and generative AI models, leading to increased customer satisfaction.

[1122] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1123] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1124] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1125] [Third embodiment]

[1126] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1127] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1128] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1129] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1130] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1131] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1132] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1133] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1134] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1135] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1136] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1137] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1138] ---

[1139] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generation AI, and generating and sending follow-up messages.

[1140] Initial Setup

[1141] 1. Start the device and enter basic information

[1142] First, the user starts up the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[1143] Childcare consultation function

[1144] 2. Enter questions and view answers

[1145] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server, which then uses generative AI and natural language processing technology to analyze the question and generate an appropriate answer. The generated answer is displayed to the user, who can then consult with the server 24 hours a day.

[1146] Examples:

[1147] User: "My baby cries at night, what should I do?"

[1148] Terminal: Sends a question to the server.

[1149] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1150] Terminal: Display the answer to the user.

[1151] Baby monitoring features

[1152] 3. Monitoring and Alerting

[1153] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[1154] Examples:

[1155] Terminal: A camera in the baby's room monitors the crib.

[1156] Server: Detects crying and determines it is abnormal.

[1157] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1158] User: Opens the app and sees live footage from the camera.

[1159] Follow-up function

[1160] 4. Generate and send follow-up messages

[1161] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message provides appropriate follow-up information and advice based on the content of the past consultation. The generated message is sent to the user via the terminal.

[1162] Examples:

[1163] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1164] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[1165] Terminal: Send a follow-up message to the user.

[1166] ---

[1167] As a result, the child-rearing support device of the present invention can provide appropriate support 24 hours a day to families raising children who tend to be isolated.The system of the present invention allows parents to concentrate on raising their children with peace of mind.

[1168] The processing flow will be explained below.

[1169] ---

[1170] Initial Setup

[1171] Step 1: Turn on the device

[1172] The user turns on the device with the power button.

[1173] Step 2: Install the app

[1174] Users install a dedicated app on their smartphone or tablet.

[1175] The terminal notifies the user that the installation is complete.

[1176] Step 3: Enter basic information

[1177] Users launch the app and enter basic information such as their child's age, gender, and allergy information on the profile screen.

[1178] The terminal checks the basic information entered and checks that all required fields have been entered.

[1179] The server receives the basic information sent by the device and stores it in a secure database.

[1180] Childcare consultation function

[1181] Step 1: Start a consultation

[1182] The user presses the "Consult" button within the app.

[1183] Step 2: Enter your question

[1184] The user enters a question by text input or voice input.

[1185] The terminal checks the input and prepares to send it to the server.

[1186] Step 3: Submit your question

[1187] The terminal transmits the entered question to the server.

[1188] Step 4: Question analysis

[1189] The server analyzes the received question and uses natural language processing (NLP) technology to understand the question.

[1190] Step 5: Answer Generation

[1191] The server uses generative AI to generate appropriate answers to questions.

[1192] Step 6: Submit your response

[1193] The server sends the generated response to the terminal.

[1194] Step 7: View Answers

[1195] The terminal displays the answer to the user.

[1196] Examples:

[1197] User: "My baby cries at night, what should I do?"

[1198] Terminal: Sends a question to the server.

[1199] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1200] Terminal: Display the answer to the user.

[1201] Baby monitoring features

[1202] Step 1: Turn on the camera

[1203] The device activates its built-in camera to monitor the baby's room.

[1204] Step 2: Data collection

[1205] The device collects camera footage and audio data in real time.

[1206] Step 3: Send data

[1207] The terminal transmits the collected data to the server.

[1208] Step 4: Data analysis

[1209] The server analyzes the received data and monitors the baby's condition, generating an alert if an abnormality is detected.

[1210] Step 5: Sending an alert

[1211] The server sends an alert to the terminal.

[1212] Step 6: Alert Notifications

[1213] The device will notify the user of the alert via voice or app notification.

[1214] Examples:

[1215] Terminal: A camera in the baby's room monitors the crib.

[1216] Server: Detects the baby's crying and flags it as an anomaly.

[1217] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1218] User: Opens the app and sees live footage from the camera.

[1219] Follow-up function

[1220] Step 1: Follow-up check

[1221] The server periodically checks the user's status and consultation history.

[1222] Step 2: Generate a follow-up message

[1223] The server generates follow-up messages as needed.

[1224] Step 3: Send a follow-up message

[1225] The server sends the generated follow-up message to the terminal.

[1226] Step 4: Display a follow-up message

[1227] The terminal displays the follow-up message to the user.

[1228] Examples:

[1229] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1230] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[1231] Terminal: Send a follow-up message to the user.

[1232] ---

[1233] Example 1

[1234] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1235] With the advancement of information and communication technology, there is a demand for systems that provide efficient and appropriate support to families raising children, who tend to be isolated. However, existing systems have difficulty providing quick and accurate answers to users' questions, continuously monitoring the condition of infants, and immediately notifying them when abnormalities occur. Furthermore, systems that generate follow-up messages based on the user's past consultations are also inadequate. The challenge is to provide a more advanced support system that solves these problems and allows parents to raise their children with peace of mind.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1237] In this invention, the server includes: means for a user to input basic information; means for receiving the basic information and storing it in a secure database; means for accepting user questions and generating answers using a generative AI model; means for displaying the generated answers to the user; means for monitoring the infant's condition using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert if an abnormality is detected; means for notifying the user of the alert; means for checking the user's consultation history and generating and sending a follow-up message; and means for transmitting the device's basic information and monitoring data to the server and for the server to analyze the data. This enables 24-hour monitoring of the infant's condition, prompt and accurate answers to questions, and follow-up based on past consultations.

[1238] "Basic information" refers to information about initial settings such as the child's age, sex, and allergy information entered by the user.

[1239] A "generative AI model" refers to an algorithm or program that uses artificial intelligence technology to generate appropriate answers to user questions.

[1240] An "integrated camera" is a camera that is built into a device and is used to monitor real-time footage of an infant.

[1241] A "sensor" is a device built into a device to detect temperature, humidity, sound, etc.

[1242] "Monitoring Data" refers to data about the infant's condition collected by built-in cameras and sensors.

[1243] "Abnormal" is when the baby is crying, not moving, or indicates an unusual situation detected by the sensor.

[1244] An "alert" is a warning message or audio notification that is given to the user when an abnormality is detected.

[1245] A "follow-up message" is a message that includes follow-up information and additional advice that is generated based on the user's past consultation history.

[1246] A "dedicated app" is application software designed for users to install and use on their smartphones or tablets.

[1247] The "server" is a central processing unit that receives and analyzes basic information, questions, and monitoring data entered by users.

[1248] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generative AI model, and generating and sending follow-up messages.

[1249] Initial Setup

[1250] First, the user turns on the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[1251] Childcare consultation function

[1252] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server then analyzes the question using generative AI models and natural language processing technology, and generates an appropriate answer. The generated answer is then sent to the device and displayed to the user. Users can consult at any time, 24 hours a day.

[1253] Examples:

[1254] User: "My baby cries at night, what should I do?"

[1255] Terminal: Sends a question to the server.

[1256] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1257] Terminal: Display the answer to the user.

[1258] Baby monitoring features

[1259] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[1260] Examples:

[1261] Terminal: A camera in the baby's room monitors the crib.

[1262] Server: Detects crying and determines it is abnormal.

[1263] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1264] User: Opens the app and sees live footage from the camera.

[1265] Follow-up function

[1266] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message provides appropriate follow-up information and advice based on the content of the past consultation. The generated message is sent to the user via the terminal.

[1267] Examples:

[1268] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1269] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[1270] Terminal: Send a follow-up message to the user.

[1271] technical operation

[1272] In this system, the server analyzes questions and generates answers using generative AI models (e.g., GPT-3) and natural language processing technology. The built-in camera and sensors monitor the infant's condition and send the collected data to the server. The server analyzes this data and, if an abnormality is detected, quickly generates an alert and notifies the user. The server also generates appropriate follow-up messages based on past consultation history and sends them to the user.

[1273] Example prompt sentence:

[1274] "My baby cries at night, what should I do?"

[1275] "My baby's constipation won't improve, what should I do?"

[1276] "I don't know when my baby will roll over. Can you tell me?"

[1277] As a result, the child-rearing support device of the present invention can provide 24-hour support to help parents raise their children with peace of mind.

[1278] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1279] Step 1:

[1280] The user starts the device and installs the dedicated app on their smartphone or tablet. When the app is launched, the user is presented with a login screen or new registration screen. The user enters basic information (child's age, gender, allergy information, etc.).

[1281] Input: Device startup, app installation, basic information input

[1282] Output: Display of basic information input screen, basic information entered

[1283] Step 2:

[1284] The device checks the basic information and checks whether all required fields have been entered correctly. If the basic information has been entered correctly, it is sent to the server.

[1285] Input: Basic information

[1286] Data processing: Check required fields

[1287] Output: Send basic information to the server

[1288] Step 3:

[1289] The server stores the received basic information in a secure database.

[1290] Input: Basic information sent from the device

[1291] Data processing: Basic information stored in a database

[1292] Output: Confirmation of successful save

[1293] Step 4:

[1294] The user presses the "Consult" button and enters a question, which can be entered in text or voice format.

[1295] Input: Question (text or voice)

[1296] Output: Question input completed

[1297] Step 5:

[1298] The terminal transmits the entered question to the server.

[1299] Input: User question

[1300] Output: Sending a question to the server

[1301] Step 6:

[1302] The server analyzes the question using generative AI models and natural language processing technology, and generates an appropriate answer based on the analysis results.

[1303] Input: Question

[1304] Data Computing: Parsing questions and generating answers (generative AI models)

[1305] Output: The generated answer

[1306] Step 7:

[1307] The server sends the generated response to the terminal.

[1308] Input: Generated answer

[1309] Output: Sends the answer to the terminal

[1310] Step 8:

[1311] The terminal displays the received answer on the screen, allowing the user to check the answer.

[1312] Input: Generated answer

[1313] Output: Display answer

[1314] Step 9:

[1315] The device uses a built-in camera and various sensors to monitor the baby's room 24 hours a day.

[1316] Input: Camera and sensor activation

[1317] Data processing: Collecting monitoring data

[1318] Output: Real-time monitoring data

[1319] Step 10:

[1320] The terminal transmits the monitoring data to the server in real time.

[1321] Input: Collected monitoring data

[1322] Output: Sending monitoring data to a server

[1323] Step 11:

[1324] The server analyzes the received monitoring data and generates an alert if an abnormality is detected.

[1325] Input: Real-time monitoring data

[1326] Data Calculation: Anomaly Detection and Alert Generation

[1327] Output: The generated alert

[1328] Step 12:

[1329] The server sends the generated alert to the device, which then notifies the user by voice or in-app notification.

[1330] Input: Generated alert

[1331] Output: Alert notification to the user

[1332] Step 13:

[1333] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary.

[1334] Input: Consultation history

[1335] Data calculation: determining whether follow-up is necessary and generating messages

[1336] Output: Generated follow-up message

[1337] Step 14:

[1338] The server sends the generated follow-up message to the terminal, and the terminal notifies the user of the message.

[1339] Input: Generated follow-up message

[1340] Output: Notify user of follow-up message

[1341] (Application example 1)

[1342] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1343] Existing parenting support devices are specialized in monitoring infant status and providing answers to parental questions, but they lack the functionality to provide the support parents need when using food delivery services. This leaves parents, especially those with busy schedules, struggling to prepare meals. Furthermore, the lack of actions that take infant status into consideration often prevents parents from receiving appropriate resources and support.

[1344] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1345] In this invention, the server includes means for a user to input basic information, means for receiving the basic information and storing it in a secure database, means for accepting user questions and generating answers using a generation AI, means for displaying the generated answers to the user, means for monitoring the infant's condition using a built-in camera and various sensors, means for analyzing the monitoring data and generating an alert if an abnormality is detected, means for notifying the user of the alert, means for checking the user's consultation history and generating and sending a follow-up message, means for making individual suggestions taking into account the child-rearing situation, and means for providing appropriate services to the user based on the individual suggestions.This enables busy parents raising children to quickly prepare meals and receive appropriate support taking into account the infant's condition.

[1346] "User" refers to a person who uses the system.

[1347] "Basic Information" refers to basic data about a user that the user enters into the system (e.g., name, age, allergy information, etc.).

[1348] "Generative AI" refers to artificial intelligence technology that generates appropriate answers to user questions.

[1349] "Built-in camera" refers to a camera that is built into a device.

[1350] "Various sensors" refers to devices that detect various conditions such as temperature, humidity, and sound.

[1351] "Surveillance data" refers to data collected from built-in cameras and various sensors.

[1352] An "anomaly" refers to an abnormal situation detected from monitoring data.

[1353] "Alert" refers to a warning generated when an abnormality is detected.

[1354] "Follow-up messages" refer to additional support messages sent periodically based on past consultations.

[1355] "Individual suggestions that take into account the child-rearing situation" refers to individual suggestions provided based on the user's child-rearing situation.

[1356] "Means for providing appropriate services" refers to the function of providing appropriate services according to the needs of the user.

[1357] A "secure database" refers to a system for securely storing and managing user basic information and monitoring data.

[1358] This invention realizes a system to support parents raising children. This system allows users to input basic information, provides answers generated by generative AI, monitors the child's condition, and generates alerts if an abnormality is detected. It also makes individual suggestions and provides appropriate services, providing quick support to busy parents raising children.

[1359] System Configuration

[1360] Hardware

[1361] Built-in camera: A camera installed in the room to monitor the baby's condition in real time, detecting the baby's movements, facial expression, and voice.

[1362] Various sensors: These sensors are used to monitor the environment in the baby's room, including temperature, humidity, and sound.

[1363] Smartphone: A mobile device that users use to operate applications.

[1364] software

[1365] Generative AI: This is an artificial intelligence technology that generates appropriate answers to user questions. Specifically, it uses the OpenAI API.

[1366] Natural language processing technology: Technology for analyzing questions entered by users.

[1367] Database: A secure database for storing users' basic information and consultation history.

[1368] Operation overview

[1369] 1. Enter basic information

[1370] First, users launch the smartphone app and enter basic information such as their child's name, age, allergy information, etc. This information is received by the server and stored in a secure database.

[1371] 2. Childcare consultation

[1372] Users press the "Consult" button within the app and enter their child-rearing questions either in text or by voice. The device sends the question to the server, where the AI ​​analyzes it and generates an answer. The answer is then displayed on the device for the user to review.

[1373] 3. Baby monitoring

[1374] The built-in camera and sensors monitor the baby's condition 24 hours a day. The monitoring data is sent to a server, and if an abnormality is detected, an alert is generated and a notification is sent to the device. For example, if crying is detected, an alert will be displayed saying, "Your child is crying. Please check."

[1375] 4. Individual proposals

[1376] The system takes into account the child-rearing situation and makes appropriate suggestions such as food delivery services. For example, if a baby is crying at night, the system will suggest, "He might be hungry. Would you like to order some milk, which is easy to prepare?"

[1377] 5. Follow-up

[1378] Based on the user's past consultation history, follow-up messages are periodically generated and sent to the user. For example, if a user consulted about "baby's constipation" a week ago, a follow-up message will be sent asking, "How is the constipation from last week? Has it improved?"

[1379] Specific examples

[1380] Example prompts

[1381] Question: "My baby cries at night, what should I do?"

[1382] Answer: "There are many reasons why your baby may be crying at night, but the first thing to do is to look at your baby's environment. Check the following:

[1383] Is the diaper wet?

[1384] Is the temperature appropriate?

[1385] Are you hungry?

[1386] If these checks don't improve the condition, we recommend consulting your pediatrician.

[1387] As a result, the system of the present invention can solve the various problems faced by parents raising children and provide them with prompt and effective support.

[1388] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1389] Step 1:

[1390] The server provides a means for users to input basic information. This includes an interface for entering the child's name, age, gender, allergy information, etc. into the device through a smartphone app. The device sends the input data to the server, where it is stored in a secure database. The server accepts the basic information provided by the user as input and processes it for secure storage. The output is the stored basic information data.

[1391] Step 2:

[1392] When a user presses the "Consult" button in the app, the device accepts a question from the user. The entered text or voice data is transferred to the server. The server uses a generative AI model to analyze the entered question and generates an appropriate answer using natural language processing technology. The user's question is given as input, and the generative AI model performs data calculations to generate an answer. The output is the generated answer.

[1393] Step 3:

[1394] The server provides a means to display the generated answer to the user. The answer is sent to the device as text or audio, and the device displays it to the user. The device also checks whether the answer has been displayed to the user and reports the display status to the server. The answer from the generative AI model is given as input, and data processing is performed to display it in a form that the user can understand. The output is the answer displayed to the user.

[1395] Step 4:

[1396] The device constantly monitors the baby's condition using a built-in camera and sensors. The monitoring data is sent to a server in real time. The server analyzes this monitoring data and generates an alert if an abnormality is detected. The monitoring data from the camera and sensors is sent to the server as input, and data analysis is performed. The output is alert information when an abnormality is detected.

[1397] Step 5:

[1398] If the server detects an abnormality, it generates an alert and notifies the device, which then displays the details to the user. For example, if crying is detected, an in-app notification displays the message "Your child is crying. Please check." The input is the detected abnormality information, and the server generates an alert message and processes the data for notification. The output is the alert notified to the user.

[1399] Step 6:

[1400] The server periodically checks the user's consultation history and generates follow-up messages as necessary. The messages are sent to the terminal and displayed to the user. For example, if a user consulted about "baby constipation" a week ago, a follow-up message asking "How was last week's constipation?" is displayed. The past consultation history is referenced as input, and the follow-up message is generated through data processing using a generative AI model. The output is the follow-up message displayed to the user.

[1401] Step 7:

[1402] Furthermore, the server has a means for making personalized suggestions that take into account the child-rearing situation. For example, if a baby is crying at night, the server may suggest, "He may be hungry. Would you like to order some milk, which is easy to prepare?" Data on the baby's condition and basic information about the user are used as input, and data calculations are performed to suggest an appropriate response. The output is a personalized suggestion provided to the user.

[1403] Through these processing steps, the system of the present invention provides quick and accurate support to parents raising children.

[1404] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1405] ---

[1406] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generation AI, generating and sending follow-up messages, and combining them with an emotion engine that recognizes the user's emotions.

[1407] Initial Setup

[1408] 1. Start the device and enter basic information

[1409] First, the user starts up the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[1410] Childcare consultation function

[1411] 2. Enter questions and view answers

[1412] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server uses natural language processing technology to analyze the question and uses generative AI to generate an appropriate answer. The generated answer is evaluated using an emotion engine to assess the user's emotional state and is presented in a way that takes the user's emotions into consideration. This allows users to consult at any time, 24 hours a day.

[1413] Examples:

[1414] User: "My baby cries at night, what should I do?"

[1415] Terminal: Sends a question to the server.

[1416] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1417] Server: Recognizes the user's emotions using an emotion engine and provides answers in a gentle tone.

[1418] Terminal: Display the answer to the user.

[1419] Baby monitoring features

[1420] 3. Monitoring and Alerting

[1421] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[1422] Examples:

[1423] Terminal: A camera in the baby's room monitors the crib.

[1424] Server: Detects crying and determines it is abnormal.

[1425] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1426] User: Opens the app and sees live footage from the camera.

[1427] Follow-up function

[1428] 4. Generate and send follow-up messages

[1429] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the user's emotional state using an emotion engine and providing appropriate follow-up information and advice. The generated message is then sent to the user via their device.

[1430] Examples:

[1431] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1432] Server: Generate a follow-up message: "I was concerned about your constipation last week. Has it improved?"

[1433] Server: Recognizes the user's emotions using an emotion engine and presents follow-up messages in appropriate words.

[1434] Terminal: Send a follow-up message to the user.

[1435] ---

[1436] As a result, the child-rearing support device of the present invention can provide appropriate support 24 hours a day, taking into consideration the emotional needs of families raising children, who tend to be isolated.The system of the present invention allows parents to concentrate on raising their children with peace of mind.

[1437] The processing flow will be explained below.

[1438] Initial Setup

[1439] Step 1: Turn on the device

[1440] The user turns on the device with the power button.

[1441] Step 2: Install the app

[1442] Users install a dedicated app on their smartphone or tablet.

[1443] The terminal notifies the user that the installation is complete.

[1444] Step 3: Enter basic information

[1445] Users launch the app and enter basic information such as their child's age, gender, and allergy information on the profile screen.

[1446] The terminal checks the basic information entered and checks that all required fields have been entered.

[1447] The server receives the basic information sent by the device and stores it in a secure database.

[1448] Childcare consultation function

[1449] Step 1: Start a consultation

[1450] The user presses the "Consult" button within the app.

[1451] Step 2: Enter your question

[1452] The user enters a question by text input or voice input.

[1453] The terminal checks the input and prepares to send it to the server.

[1454] Step 3: Submit your question

[1455] The terminal transmits the entered question to the server.

[1456] Step 4: Emotion Recognition

[1457] The server passes the received question to an emotion engine to recognize the user's emotional state.

[1458] Step 5: Question Analysis

[1459] The server uses natural language processing technology to analyze the question and understand its content.

[1460] Step 6: Answer Generation

[1461] The server uses generative AI to generate appropriate answers to questions.

[1462] Step 7: Emotional Consideration

[1463] The server passes the generated answer to the emotion engine and adjusts the answer content in a way that takes the user's emotions into consideration.

[1464] Step 8: Submit your response

[1465] The server sends the adjusted response to the terminal.

[1466] Step 9: View Answers

[1467] The terminal displays the adjusted answer to the user.

[1468] Examples:

[1469] User: "My baby cries at night, what should I do?"

[1470] Terminal: Sends a question to the server.

[1471] Server: The emotion engine recognizes the user's emotions, analyzes the question, and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment." The answer is adjusted taking into account the user's emotional state.

[1472] Terminal: Display the adjusted answer to the user.

[1473] Baby monitoring features

[1474] Step 1: Turn on the camera

[1475] The device activates its built-in camera to monitor the baby's room.

[1476] Step 2: Data collection

[1477] The device collects camera footage and audio data in real time.

[1478] Step 3: Send data

[1479] The terminal transmits the collected data to the server.

[1480] Step 4: Data analysis

[1481] The server analyzes the received data and monitors the baby's condition, generating an alert if an abnormality is detected.

[1482] Step 5: Sending an alert

[1483] The server sends an alert to the terminal.

[1484] Step 6: Alert Notifications

[1485] The device will notify the user of the alert via voice or app notification.

[1486] Examples:

[1487] Terminal: A camera in the baby's room monitors the crib.

[1488] Server: Detects crying and generates an alert as an anomaly.

[1489] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1490] User: Opens the app and sees live footage from the camera.

[1491] Follow-up function

[1492] Step 1: Follow-up check

[1493] The server periodically checks the user's status and consultation history.

[1494] Step 2: Emotion Recognition

[1495] The server evaluates the user's emotional state using an emotion engine.

[1496] Step 3: Generate a follow-up message

[1497] The server optionally generates follow-up messages based on the user's emotional state as assessed by the emotion engine.

[1498] Step 4: Send a follow-up message

[1499] The server sends the generated follow-up message to the terminal.

[1500] Step 5: Display a follow-up message

[1501] The terminal displays the follow-up message to the user.

[1502] Examples:

[1503] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1504] Server: The emotion engine recognizes the user's emotions and generates a follow-up message such as, "I'm worried about your constipation last week. Has it improved?"

[1505] Terminal: Send a follow-up message to the user.

[1506] Example 2

[1507] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1508] Conventional child-rearing support systems have difficulty providing comprehensive support in environments where parents tend to feel isolated. They also lack the functionality to monitor the infant's condition and respond immediately if an abnormality occurs, or the follow-up function to consider the parents' feelings. As a result, parents are unable to concentrate on raising their children with peace of mind, and this places a heavy mental burden on them.

[1509] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1510] In this invention, the server includes a means for a user to input basic information, a means for receiving the basic information and storing it in a secure database, and a means for evaluating the user's emotional state and generating appropriately tailored responses and follow-up messages based on the evaluation. This allows parents to receive appropriate parenting support 24 hours a day, allowing them to focus on raising their children with peace of mind.

[1511] "Users" refer to people who use the system, and are primarily parents and guardians who are raising children.

[1512] "Basic information" is information required for initial setup, such as the child's age, sex, and allergy information, entered by the user.

[1513] A "database" is a storage device for safely storing data such as a user's basic information and consultation history.

[1514] "Generative AI" is an artificial intelligence technology that generates appropriate answers to questions from users.

[1515] An "integrated camera" is a camera that is built into the device and is used to monitor the condition of the infant.

[1516] A "sensor" is a device used to detect an infant's environment and condition, detecting temperature, humidity, sound, etc.

[1517] "Natural language processing technology" is a technology for analyzing the content of a user's question and understanding its meaning.

[1518] An "emotion engine" is a technology that assesses a user's emotional state and adjusts the answers and messages it provides based on that.

[1519] A "follow-up message" is a tracking message generated based on the user's past consultation history and is used to provide additional advice or confirmation.

[1520] The present invention relates to a child-rearing support system that enables parents to raise their children without feeling isolated. Specific embodiments of this system will be described in detail below.

[1521] The system aims to provide comprehensive parenting support by allowing users to input basic information and then responding in various ways based on that information. Specifically, the device, which has multiple functions, monitors the infant's condition using built-in cameras and sensors, and generates an alert when an abnormality is detected. It also accepts questions from users, provides answers using generative AI, and generates and sends follow-up messages. The system also combines an emotion engine that recognizes the user's emotions, allowing it to provide more personalized support.

[1522] Initial Setup

[1523] First, the user turns on the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user enters basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and ensures that all required fields have been entered correctly. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[1524] Childcare consultation function

[1525] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server uses natural language processing technology to analyze the question and uses generative AI to generate an appropriate answer. The generated answer is evaluated using an emotion engine to assess the user's emotional state and is presented in a way that takes the user's emotions into consideration. This allows users to consult at any time, 24 hours a day.

[1526] Specific examples

[1527] User: "My baby cries at night, what should I do?"

[1528] Terminal: Sends a question to the server.

[1529] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1530] Server: Recognizes the user's emotions using an emotion engine and provides answers in a gentle tone.

[1531] Terminal: Display the answer to the user.

[1532] Baby monitoring features

[1533] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[1534] Specific examples

[1535] Terminal: A camera in the baby's room monitors the crib.

[1536] Server: Detects crying and determines it is abnormal.

[1537] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1538] User: Opens the app and sees live footage from the camera.

[1539] Follow-up function

[1540] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the user's emotional state using an emotion engine and providing appropriate follow-up information and advice. The generated message is then sent to the user via their device.

[1541] Specific examples

[1542] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1543] Server: Generate a follow-up message: "I was concerned about your constipation last week. Has it improved?"

[1544] Server: Recognizes the user's emotions using an emotion engine and presents follow-up messages in appropriate words.

[1545] Terminal: Send a follow-up message to the user.

[1546] As a result, the child-rearing support system of this invention can provide appropriate support 24 hours a day, taking into consideration the emotional needs of families raising children, who tend to be isolated. This allows parents to concentrate on raising their children with peace of mind, and is expected to reduce stress related to child-rearing.

[1547] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1548] Step 1:

[1549] Booting the device and entering basic information

[1550] User: Turn on your smartphone or tablet and open the dedicated app.

[1551] User: Download and install the dedicated app from the device store.

[1552] User: Launch the app and enter basic information such as the child's age, gender, and allergies on the profile screen that appears.

[1553] Input: Basic information entered by the user (e.g., child's age, gender, allergy information).

[1554] Terminal: Once the user has completed their input, save the data and check that all required fields have been entered.

[1555] Server: Receives basic information sent from the device and stores it in a secure database.

[1556] Output: Basic information stored in the database.

[1557] Step 2:

[1558] Enter and submit your question

[1559] User: Press the "Consult" button in the app and enter a question via text or voice.

[1560] Input: A question typed by the user (e.g., "My baby cries at night. What should I do?")

[1561] Terminal: Sends the question entered by the user to the server.

[1562] Step 3:

[1563] Parsing questions and generating answers

[1564] Server: Receives the question and automatically analyzes it using natural language processing technology. For example, it extracts keywords such as "night crying," "baby," and "measures" from the question.

[1565] Input: User question data

[1566] Server: Based on the analysis results, the generative AI model generates an appropriate answer. For example, it generates an answer such as, "There are various causes of nighttime crying, but first, try reconsidering your baby's environment."

[1567] Output: Generated answer (e.g. "There are many causes of night crying, but first try looking at your baby's environment.")

[1568] Step 4:

[1569] Ratings and answers displayed by the sentiment engine

[1570] Server: Uses an emotion engine to assess the user's emotional state and adjusts the generated response appropriately, for example, by changing the tone of voice to a gentler one.

[1571] Input: Generated answers and user emotional state assessment data

[1572] Output: Final, emotion-sensitive answer

[1573] Device: Display the adjusted answer on the user's device.

[1574] Step 5:

[1575] Baby monitoring and alert notifications

[1576] Device: Monitors the baby's condition 24 hours a day using a built-in camera and various sensors.

[1577] Input: Camera footage and sensor data

[1578] Terminal: Sends monitoring data to the server in real time.

[1579] Server: Analyzes the received data and generates an alert if an abnormality is detected, such as abnormally continuous crying.

[1580] Output: Alert message (e.g. "Your child is crying. Please check on him / her.")

[1581] On device: The alert message is communicated to the user via audio notification or in-app notification.

[1582] Step 6:

[1583] Generate and send follow-up messages

[1584] Server: Periodically checks the user's past consultation history and generates a message if it determines that follow-up is necessary.

[1585] Input: Consultation history data

[1586] Server: Generates follow-up messages and uses the emotion engine to tailor them with appropriate language. For example, "I'm concerned about your constipation issue last week. Has it improved?"

[1587] Output: Follow-up message (e.g., "I was concerned about your constipation issue last week. Has it improved?")

[1588] Device: Send a follow-up message to the user's device.

[1589] (Application example 2)

[1590] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1591] While existing parenting support devices provide support for parenting situations where parents tend to feel isolated, there has also been a lack of means to provide appropriate support for customers' worries and anxieties in the field of customer service in brick-and-mortar stores. In particular, brick-and-mortar stores need to be able to grasp customers' emotions in real time and respond accordingly, but the current situation is that such systems are not fully developed. Furthermore, building ongoing relationships with customers is important, and follow-up functions based on past consultations are also needed.

[1592] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to input basic information; means for receiving the basic information and storing it in a secure database; means for accepting a user's question and generating an answer using a generation AI; means for displaying the generated answer to the user; means for monitoring the status of an object using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert when an abnormality is detected; means for notifying the user of the alert; means for checking the user's consultation history and generating and sending a follow-up message; means for evaluating the user's emotions in real time using an emotion recognition engine; and means for providing the user with the optimal response based on the evaluation results. This enables real-time responses and continuous follow-up that take customer emotions into consideration in customer service in physical stores.

[1593] "Means for users to enter basic information" refers to means by which users enter profile information and other required data, including dedicated applications and web interfaces.

[1594] "Means for receiving the basic information and storing it in a secure database" refers to means for receiving information entered by the user and storing it in a secure database, and includes encryption technology and access control.

[1595] "Means for accepting user questions and generating answers using generative AI" refers to means for accepting questions from users and using generative AI (for example, a model using natural language processing technology) to analyze the questions and generate appropriate answers.

[1596] The "means for displaying the generated answer to the user" refers to a means for displaying the generated answer on the user's device (smartphone, tablet, etc.), and includes a user interface.

[1597] "Means for monitoring the condition of an object using a built-in camera and various sensors" means means for continuously monitoring the condition of an object (e.g., an infant or a customer) using a built-in camera and various sensors.

[1598] The "means for analyzing the monitoring data and generating an alert when an abnormality is detected" refers to a means for analyzing the monitoring data and generating a warning when an abnormal condition is detected, such as by sending a voice notification or a message.

[1599] The "means for notifying the user of the alert" refers to a means for notifying the user's device of the generated alert, and includes push notification and email transmission.

[1600] The "means for checking the user's consultation history and generating and sending a follow-up message" refers to a means for checking the past consultation history, generating a follow-up message based on the history, and sending the message to the user at an appropriate time.

[1601] The "means for evaluating a user's emotions in real time using an emotion recognition engine" is a means for using an emotion recognition engine to evaluate a user's facial expression and tone of voice in real time and grasp their emotional state.

[1602] The "means for providing the user with the most appropriate response based on the evaluation results" is a means for providing the user with the most appropriate response method or information based on the evaluation results of the emotion recognition engine.

[1603] The present invention provides a "smart customer service device" to improve customer service in brick-and-mortar stores. The system receives basic information entered by customers, stores it securely, and then takes various actions based on that information.

[1604] Initial Setup

[1605] 1. Enter basic information

[1606] The user (store staff) activates the device and installs a dedicated app on their smartphone or smart glasses. Once the app is activated, the user enters customer profile information, including the customer's name, contact details, and past purchase history. The server receives this basic information and stores it in a secure database.

[1607] Customer service functions

[1608] 2. Question acceptance and answer generation

[1609] When a user (customer) inputs a question into the device, the server accepts it. The question can be entered as text or voice. The server analyzes the received question and generates an appropriate answer using a generative AI model (e.g., GPT-3). The generated answer is displayed on the user's device.

[1610] Examples:

[1611] User (customer): "I'd like to know more about this product?"

[1612] Server: Parses the question and generates an answer using a generative AI model.

[1613] Server: "This product is manufactured using the latest technology and has excellent performance."

[1614] Emotion recognition function

[1615] 3. Emotion recognition and response guidance

[1616] The system uses cameras and sensors inside the building to analyze customers' facial expressions and tone of voice, and evaluates them in real time using an emotion recognition engine. Based on the evaluation results, the system provides the user (customer) with the most appropriate response method.

[1617] Examples:

[1618] Terminal: Sends the customer's facial expression analysis data to the server.

[1619] Server: Detects "anxiety" using an emotion recognition engine.

[1620] Server: "You seem concerned. What are you worried about?" I suggest.

[1621] Alert notification function

[1622] 4. Anomaly detection and alert notification

[1623] The device is equipped with a built-in camera and various sensors that monitor the customer's condition. The monitoring data is analyzed on the server, and an alert is generated if an abnormality is detected. The alert is communicated to the user (store staff) as a voice or notification.

[1624] Examples:

[1625] Device: In-store camera detects abnormal behavior.

[1626] Server: Determines that something is abnormal and generates an alert.

[1627] Terminal: Notifies, "Abnormal behavior has been detected in Area A."

[1628] Follow-up function

[1629] 5. Generate and send follow-up messages

[1630] The server periodically checks the customer's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the customer's emotional state using an emotion recognition engine, and is sent to the user (customer) to provide appropriate follow-up information and advice.

[1631] Examples:

[1632] Server: Check the history of customers who consulted about "how to use the product" one week ago.

[1633] Server: Generates and sends a follow-up message: "Regarding the product issue from last week, has the issue been resolved since then?"

[1634] Prompt Sentence Examples

[1635] A customer asks the following question: I'd like to know more about this product? Emotion recognition indicates that the customer is interested. Generate an appropriate answer for this situation.

[1636] This system can improve customer satisfaction in physical stores by understanding customer sentiment in real time and providing optimal responses and continuous follow-up.

[1637] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1638] Step 1:

[1639] The user (store staff) installs the dedicated app on their smartphone or smart glasses and launches the app. When the app is launched, the user enters customer profile information (such as name, contact information, and past purchase history). This information is sent from the device to the server and stored in a secure database.

[1640] Input: Customer profile information

[1641] Output: Customer information stored in a secure database

[1642] Step 2:

[1643] The user (customer) enters a question into the device via text or voice. The device sends this question to the server. The server receives the question and analyzes it using natural language processing technology. It uses a generative AI model (e.g., GPT-3) to generate an appropriate answer and sends it to the device.

[1644] Input: User question

[1645] Output: Answer from the generative AI model

[1646] Step 3:

[1647] The terminal displays the generated answer on the user's (customer's) device, and the user (customer) can check the answer to resolve their doubts and ask additional questions as needed.

[1648] Input: Generated answer

[1649] Output: Display the answer to the user (customer)

[1650] Step 4:

[1651] The device uses a built-in camera and sensors to monitor customer facial expressions and tone of voice in real time, and the monitoring data is sent from the device to a server, which then uses an emotion recognition engine to analyze the data and evaluate the customer's emotional state.

[1652] Input: Surveillance data (facial expressions, tone of voice)

[1653] Output: Customer emotional state assessment

[1654] Step 5:

[1655] The server proposes the optimal response based on the evaluation results. It generates a prompt using the generative AI model along with the evaluation results, presents an appropriate response plan, and sends it to the user (store staff).

[1656] Input: Customer sentiment evaluation results

[1657] Output: Suggested action

[1658] Step 6:

[1659] The device continuously monitors the situation inside the store using a built-in camera and various sensors. The monitoring data is sent from the terminal to a server, and if the server detects an abnormality, it generates an alert and notifies the user (store staff).

[1660] Input: In-store surveillance data

[1661] Output: Alert notification in case of abnormality

[1662] Step 7:

[1663] The server periodically checks the customer's past consultation and purchase history, and if it determines that follow-up is necessary, it uses an emotion recognition engine to generate an appropriate follow-up message, which is then sent to the user (customer) via their device.

[1664] Input: Consultation history, purchase history

[1665] Output: Follow-up message

[1666] Step 8:

[1667] The user (customer) receives the follow-up message and replies as needed. The reply is also sent back to the server, enabling continuous support.

[1668] Input: Follow-up message

[1669] Output: Reply from the user (customer)

[1670] Through this series of steps, customer service in brick-and-mortar stores can be significantly improved by utilizing emotion recognition and generative AI models, leading to increased customer satisfaction.

[1671] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1672] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1673] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1674] [Fourth embodiment]

[1675] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1676] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1677] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1678] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1679] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1680] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1681] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1682] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1683] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1684] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1685] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1686] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1687] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1688] ---

[1689] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generation AI, and generating and sending follow-up messages.

[1690] Initial Setup

[1691] 1. Start the device and enter basic information

[1692] First, the user starts up the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[1693] Childcare consultation function

[1694] 2. Enter questions and view answers

[1695] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server, which then uses generative AI and natural language processing technology to analyze the question and generate an appropriate answer. The generated answer is displayed to the user, who can then consult with the server 24 hours a day.

[1696] Examples:

[1697] User: "My baby cries at night, what should I do?"

[1698] Terminal: Sends a question to the server.

[1699] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1700] Terminal: Display the answer to the user.

[1701] Baby monitoring features

[1702] 3. Monitoring and Alerting

[1703] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[1704] Examples:

[1705] Terminal: A camera in the baby's room monitors the crib.

[1706] Server: Detects crying and determines it is abnormal.

[1707] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1708] User: Opens the app and sees live footage from the camera.

[1709] Follow-up function

[1710] 4. Generate and send follow-up messages

[1711] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message provides appropriate follow-up information and advice based on the content of the past consultation. The generated message is sent to the user via the terminal.

[1712] Examples:

[1713] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1714] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[1715] Terminal: Send a follow-up message to the user.

[1716] ---

[1717] As a result, the child-rearing support device of the present invention can provide appropriate support 24 hours a day to families raising children who tend to be isolated.The system of the present invention allows parents to concentrate on raising their children with peace of mind.

[1718] The processing flow will be explained below.

[1719] ---

[1720] Initial Setup

[1721] Step 1: Turn on the device

[1722] The user turns on the device with the power button.

[1723] Step 2: Install the app

[1724] Users install a dedicated app on their smartphone or tablet.

[1725] The terminal notifies the user that the installation is complete.

[1726] Step 3: Enter basic information

[1727] Users launch the app and enter basic information such as their child's age, gender, and allergy information on the profile screen.

[1728] The terminal checks the basic information entered and checks that all required fields have been entered.

[1729] The server receives the basic information sent by the device and stores it in a secure database.

[1730] Childcare consultation function

[1731] Step 1: Start a consultation

[1732] The user presses the "Consult" button within the app.

[1733] Step 2: Enter your question

[1734] The user enters a question by text input or voice input.

[1735] The terminal checks the input and prepares to send it to the server.

[1736] Step 3: Submit your question

[1737] The terminal transmits the entered question to the server.

[1738] Step 4: Question analysis

[1739] The server analyzes the received question and uses natural language processing (NLP) technology to understand the question.

[1740] Step 5: Answer Generation

[1741] The server uses generative AI to generate appropriate answers to questions.

[1742] Step 6: Submit your response

[1743] The server sends the generated response to the terminal.

[1744] Step 7: View Answers

[1745] The terminal displays the answer to the user.

[1746] Examples:

[1747] User: "My baby cries at night, what should I do?"

[1748] Terminal: Sends a question to the server.

[1749] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1750] Terminal: Display the answer to the user.

[1751] Baby monitoring features

[1752] Step 1: Turn on the camera

[1753] The device activates its built-in camera to monitor the baby's room.

[1754] Step 2: Data collection

[1755] The device collects camera footage and audio data in real time.

[1756] Step 3: Send data

[1757] The terminal transmits the collected data to the server.

[1758] Step 4: Data analysis

[1759] The server analyzes the received data and monitors the baby's condition, generating an alert if an abnormality is detected.

[1760] Step 5: Sending an alert

[1761] The server sends an alert to the terminal.

[1762] Step 6: Alert Notifications

[1763] The device will notify the user of the alert via voice or app notification.

[1764] Examples:

[1765] Terminal: A camera in the baby's room monitors the crib.

[1766] Server: Detects the baby's crying and flags it as an anomaly.

[1767] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1768] User: Opens the app and sees live footage from the camera.

[1769] Follow-up function

[1770] Step 1: Follow-up check

[1771] The server periodically checks the user's status and consultation history.

[1772] Step 2: Generate a follow-up message

[1773] The server generates follow-up messages as needed.

[1774] Step 3: Send a follow-up message

[1775] The server sends the generated follow-up message to the terminal.

[1776] Step 4: Display a follow-up message

[1777] The terminal displays the follow-up message to the user.

[1778] Examples:

[1779] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1780] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[1781] Terminal: Send a follow-up message to the user.

[1782] ---

[1783] Example 1

[1784] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1785] With the advancement of information and communication technology, there is a demand for systems that provide efficient and appropriate support to families raising children, who tend to be isolated. However, existing systems have difficulty providing quick and accurate answers to users' questions, continuously monitoring the condition of infants, and immediately notifying them when abnormalities occur. Furthermore, systems that generate follow-up messages based on the user's past consultations are also inadequate. The challenge is to provide a more advanced support system that solves these problems and allows parents to raise their children with peace of mind.

[1786] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1787] In this invention, the server includes: means for a user to input basic information; means for receiving the basic information and storing it in a secure database; means for accepting user questions and generating answers using a generative AI model; means for displaying the generated answers to the user; means for monitoring the infant's condition using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert if an abnormality is detected; means for notifying the user of the alert; means for checking the user's consultation history and generating and sending a follow-up message; and means for transmitting the device's basic information and monitoring data to the server and for the server to analyze the data. This enables 24-hour monitoring of the infant's condition, prompt and accurate answers to questions, and follow-up based on past consultations.

[1788] "Basic information" refers to information about initial settings such as the child's age, sex, and allergy information entered by the user.

[1789] A "generative AI model" refers to an algorithm or program that uses artificial intelligence technology to generate appropriate answers to user questions.

[1790] An "integrated camera" is a camera that is built into a device and is used to monitor real-time footage of an infant.

[1791] A "sensor" is a device built into a device to detect temperature, humidity, sound, etc.

[1792] "Monitoring Data" refers to data about the infant's condition collected by built-in cameras and sensors.

[1793] "Abnormal" is when the baby is crying, not moving, or indicates an unusual situation detected by the sensor.

[1794] An "alert" is a warning message or audio notification that is given to the user when an abnormality is detected.

[1795] A "follow-up message" is a message that includes follow-up information and additional advice that is generated based on the user's past consultation history.

[1796] A "dedicated app" is application software designed for users to install and use on their smartphones or tablets.

[1797] The "server" is a central processing unit that receives and analyzes basic information, questions, and monitoring data entered by users.

[1798] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generative AI model, and generating and sending follow-up messages.

[1799] Initial Setup

[1800] First, the user turns on the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[1801] Childcare consultation function

[1802] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server then analyzes the question using generative AI models and natural language processing technology, and generates an appropriate answer. The generated answer is then sent to the device and displayed to the user. Users can consult at any time, 24 hours a day.

[1803] Examples:

[1804] User: "My baby cries at night, what should I do?"

[1805] Terminal: Sends a question to the server.

[1806] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1807] Terminal: Display the answer to the user.

[1808] Baby monitoring features

[1809] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[1810] Examples:

[1811] Terminal: A camera in the baby's room monitors the crib.

[1812] Server: Detects crying and determines it is abnormal.

[1813] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1814] User: Opens the app and sees live footage from the camera.

[1815] Follow-up function

[1816] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message provides appropriate follow-up information and advice based on the content of the past consultation. The generated message is sent to the user via the terminal.

[1817] Examples:

[1818] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1819] Server: Generate a follow-up message: "How's the constipation from last week going? Has it improved?"

[1820] Terminal: Send a follow-up message to the user.

[1821] technical operation

[1822] In this system, the server analyzes questions and generates answers using generative AI models (e.g., GPT-3) and natural language processing technology. The built-in camera and sensors monitor the infant's condition and send the collected data to the server. The server analyzes this data and, if an abnormality is detected, quickly generates an alert and notifies the user. The server also generates appropriate follow-up messages based on past consultation history and sends them to the user.

[1823] Example prompt sentence:

[1824] "My baby cries at night, what should I do?"

[1825] "My baby's constipation won't improve, what should I do?"

[1826] "I don't know when my baby will roll over. Can you tell me?"

[1827] As a result, the child-rearing support device of the present invention can provide 24-hour support to help parents raise their children with peace of mind.

[1828] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1829] Step 1:

[1830] The user starts the device and installs the dedicated app on their smartphone or tablet. When the app is launched, the user is presented with a login screen or new registration screen. The user enters basic information (child's age, gender, allergy information, etc.).

[1831] Input: Device startup, app installation, basic information input

[1832] Output: Display of basic information input screen, basic information entered

[1833] Step 2:

[1834] The device checks the basic information and checks whether all required fields have been entered correctly. If the basic information has been entered correctly, it is sent to the server.

[1835] Input: Basic information

[1836] Data processing: Check required fields

[1837] Output: Send basic information to the server

[1838] Step 3:

[1839] The server stores the received basic information in a secure database.

[1840] Input: Basic information sent from the device

[1841] Data processing: Basic information stored in a database

[1842] Output: Confirmation of successful save

[1843] Step 4:

[1844] The user presses the "Consult" button and enters a question, which can be entered in text or voice format.

[1845] Input: Question (text or voice)

[1846] Output: Question input completed

[1847] Step 5:

[1848] The terminal transmits the entered question to the server.

[1849] Input: User question

[1850] Output: Sending a question to the server

[1851] Step 6:

[1852] The server analyzes the question using generative AI models and natural language processing technology, and generates an appropriate answer based on the analysis results.

[1853] Input: Question

[1854] Data Computing: Parsing questions and generating answers (generative AI models)

[1855] Output: The generated answer

[1856] Step 7:

[1857] The server sends the generated response to the terminal.

[1858] Input: Generated answer

[1859] Output: Sends the answer to the terminal

[1860] Step 8:

[1861] The terminal displays the received answer on the screen, allowing the user to check the answer.

[1862] Input: Generated answer

[1863] Output: Display answer

[1864] Step 9:

[1865] The device uses a built-in camera and various sensors to monitor the baby's room 24 hours a day.

[1866] Input: Camera and sensor activation

[1867] Data processing: Collecting monitoring data

[1868] Output: Real-time monitoring data

[1869] Step 10:

[1870] The terminal transmits the monitoring data to the server in real time.

[1871] Input: Collected monitoring data

[1872] Output: Sending monitoring data to a server

[1873] Step 11:

[1874] The server analyzes the received monitoring data and generates an alert if an abnormality is detected.

[1875] Input: Real-time monitoring data

[1876] Data Calculation: Anomaly Detection and Alert Generation

[1877] Output: The generated alert

[1878] Step 12:

[1879] The server sends the generated alert to the device, which then notifies the user by voice or in-app notification.

[1880] Input: Generated alert

[1881] Output: Alert notification to the user

[1882] Step 13:

[1883] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary.

[1884] Input: Consultation history

[1885] Data calculation: determining whether follow-up is necessary and generating messages

[1886] Output: Generated follow-up message

[1887] Step 14:

[1888] The server sends the generated follow-up message to the terminal, and the terminal notifies the user of the message.

[1889] Input: Generated follow-up message

[1890] Output: Notify user of follow-up message

[1891] (Application example 1)

[1892] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1893] Existing parenting support devices are specialized in monitoring infant status and providing answers to parental questions, but they lack the functionality to provide the support parents need when using food delivery services. This leaves parents, especially those with busy schedules, struggling to prepare meals. Furthermore, the lack of actions that take infant status into consideration often prevents parents from receiving appropriate resources and support.

[1894] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1895] In this invention, the server includes means for a user to input basic information, means for receiving the basic information and storing it in a secure database, means for accepting user questions and generating answers using a generation AI, means for displaying the generated answers to the user, means for monitoring the infant's condition using a built-in camera and various sensors, means for analyzing the monitoring data and generating an alert if an abnormality is detected, means for notifying the user of the alert, means for checking the user's consultation history and generating and sending a follow-up message, means for making individual suggestions taking into account the child-rearing situation, and means for providing appropriate services to the user based on the individual suggestions.This enables busy parents raising children to quickly prepare meals and receive appropriate support taking into account the infant's condition.

[1896] "User" refers to a person who uses the system.

[1897] "Basic Information" refers to basic data about a user that the user enters into the system (e.g., name, age, allergy information, etc.).

[1898] "Generative AI" refers to artificial intelligence technology that generates appropriate answers to user questions.

[1899] "Built-in camera" refers to a camera that is built into a device.

[1900] "Various sensors" refers to devices that detect various conditions such as temperature, humidity, and sound.

[1901] "Surveillance data" refers to data collected from built-in cameras and various sensors.

[1902] An "anomaly" refers to an abnormal situation detected from monitoring data.

[1903] "Alert" refers to a warning generated when an abnormality is detected.

[1904] "Follow-up messages" refer to additional support messages sent periodically based on past consultations.

[1905] "Individual suggestions that take into account the child-rearing situation" refers to individual suggestions provided based on the user's child-rearing situation.

[1906] "Means for providing appropriate services" refers to the function of providing appropriate services according to the needs of the user.

[1907] A "secure database" refers to a system for securely storing and managing user basic information and monitoring data.

[1908] This invention realizes a system to support parents raising children. This system allows users to input basic information, provides answers generated by generative AI, monitors the child's condition, and generates alerts if an abnormality is detected. It also makes individual suggestions and provides appropriate services, providing quick support to busy parents raising children.

[1909] System Configuration

[1910] Hardware

[1911] Built-in camera: A camera installed in the room to monitor the baby's condition in real time, detecting the baby's movements, facial expression, and voice.

[1912] Various sensors: These sensors are used to monitor the environment in the baby's room, including temperature, humidity, and sound.

[1913] Smartphone: A mobile device that users use to operate applications.

[1914] software

[1915] Generative AI: This is an artificial intelligence technology that generates appropriate answers to user questions. Specifically, it uses the OpenAI API.

[1916] Natural language processing technology: Technology for analyzing questions entered by users.

[1917] Database: A secure database for storing users' basic information and consultation history.

[1918] Operation overview

[1919] 1. Enter basic information

[1920] First, users launch the smartphone app and enter basic information such as their child's name, age, allergy information, etc. This information is received by the server and stored in a secure database.

[1921] 2. Childcare consultation

[1922] Users press the "Consult" button within the app and enter their child-rearing questions either in text or by voice. The device sends the question to the server, where the AI ​​analyzes it and generates an answer. The answer is then displayed on the device for the user to review.

[1923] 3. Baby monitoring

[1924] The built-in camera and sensors monitor the baby's condition 24 hours a day. The monitoring data is sent to a server, and if an abnormality is detected, an alert is generated and a notification is sent to the device. For example, if crying is detected, an alert will be displayed saying, "Your child is crying. Please check."

[1925] 4. Individual proposals

[1926] The system takes into account the child-rearing situation and makes appropriate suggestions such as food delivery services. For example, if a baby is crying at night, the system will suggest, "He might be hungry. Would you like to order some milk, which is easy to prepare?"

[1927] 5. Follow-up

[1928] Based on the user's past consultation history, follow-up messages are periodically generated and sent to the user. For example, if a user consulted about "baby's constipation" a week ago, a follow-up message will be sent asking, "How is the constipation from last week? Has it improved?"

[1929] Specific examples

[1930] Example prompts

[1931] Question: "My baby cries at night, what should I do?"

[1932] Answer: "There are many reasons why your baby may be crying at night, but the first thing to do is to look at your baby's environment. Check the following:

[1933] Is the diaper wet?

[1934] Is the temperature appropriate?

[1935] Are you hungry?

[1936] If these checks don't improve the condition, we recommend consulting your pediatrician.

[1937] As a result, the system of the present invention can solve the various problems faced by parents raising children and provide them with prompt and effective support.

[1938] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1939] Step 1:

[1940] The server provides a means for users to input basic information. This includes an interface for entering the child's name, age, gender, allergy information, etc. into the device through a smartphone app. The device sends the input data to the server, where it is stored in a secure database. The server accepts the basic information provided by the user as input and processes it for secure storage. The output is the stored basic information data.

[1941] Step 2:

[1942] When a user presses the "Consult" button in the app, the device accepts a question from the user. The entered text or voice data is transferred to the server. The server uses a generative AI model to analyze the entered question and generates an appropriate answer using natural language processing technology. The user's question is given as input, and the generative AI model performs data calculations to generate an answer. The output is the generated answer.

[1943] Step 3:

[1944] The server provides a means to display the generated answer to the user. The answer is sent to the device as text or audio, and the device displays it to the user. The device also checks whether the answer has been displayed to the user and reports the display status to the server. The answer from the generative AI model is given as input, and data processing is performed to display it in a form that the user can understand. The output is the answer displayed to the user.

[1945] Step 4:

[1946] The device constantly monitors the baby's condition using a built-in camera and sensors. The monitoring data is sent to a server in real time. The server analyzes this monitoring data and generates an alert if an abnormality is detected. The monitoring data from the camera and sensors is sent to the server as input, and data analysis is performed. The output is alert information when an abnormality is detected.

[1947] Step 5:

[1948] If the server detects an abnormality, it generates an alert and notifies the device, which then displays the details to the user. For example, if crying is detected, an in-app notification displays the message "Your child is crying. Please check." The input is the detected abnormality information, and the server generates an alert message and processes the data for notification. The output is the alert notified to the user.

[1949] Step 6:

[1950] The server periodically checks the user's consultation history and generates follow-up messages as necessary. The messages are sent to the terminal and displayed to the user. For example, if a user consulted about "baby constipation" a week ago, a follow-up message asking "How was last week's constipation?" is displayed. The past consultation history is referenced as input, and the follow-up message is generated through data processing using a generative AI model. The output is the follow-up message displayed to the user.

[1951] Step 7:

[1952] Furthermore, the server has a means for making personalized suggestions that take into account the child-rearing situation. For example, if a baby is crying at night, the server may suggest, "He may be hungry. Would you like to order some milk, which is easy to prepare?" Data on the baby's condition and basic information about the user are used as input, and data calculations are performed to suggest an appropriate response. The output is a personalized suggestion provided to the user.

[1953] Through these processing steps, the system of the present invention provides quick and accurate support to parents raising children.

[1954] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1955] ---

[1956] The present invention relates to a parenting support device that allows parents to raise their children without feeling isolated. Specifically, it is a system in which users input basic information and respond to various situations based on that information. The device of the present invention has the functions of monitoring the condition of an infant using a built-in camera and sensors, generating and notifying alerts when an abnormality is detected, accepting user questions and providing answers using a generation AI, generating and sending follow-up messages, and combining them with an emotion engine that recognizes the user's emotions.

[1957] Initial Setup

[1958] 1. Start the device and enter basic information

[1959] First, the user starts up the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user is prompted to enter basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and checks that all required fields have been entered. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[1960] Childcare consultation function

[1961] 2. Enter questions and view answers

[1962] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server uses natural language processing technology to analyze the question and uses generative AI to generate an appropriate answer. The generated answer is evaluated using an emotion engine to assess the user's emotional state and is presented in a way that takes the user's emotions into consideration. This allows users to consult at any time, 24 hours a day.

[1963] Examples:

[1964] User: "My baby cries at night, what should I do?"

[1965] Terminal: Sends a question to the server.

[1966] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[1967] Server: Recognizes the user's emotions using an emotion engine and provides answers in a gentle tone.

[1968] Terminal: Display the answer to the user.

[1969] Baby monitoring features

[1970] 3. Monitoring and Alerting

[1971] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[1972] Examples:

[1973] Terminal: A camera in the baby's room monitors the crib.

[1974] Server: Detects crying and determines it is abnormal.

[1975] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[1976] User: Opens the app and sees live footage from the camera.

[1977] Follow-up function

[1978] 4. Generate and send follow-up messages

[1979] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the user's emotional state using an emotion engine and providing appropriate follow-up information and advice. The generated message is then sent to the user via their device.

[1980] Examples:

[1981] Server: Check the history of users who consulted about "baby constipation" one week ago.

[1982] Server: Generate a follow-up message: "I was concerned about your constipation last week. Has it improved?"

[1983] Server: Recognizes the user's emotions using an emotion engine and presents follow-up messages in appropriate words.

[1984] Terminal: Send a follow-up message to the user.

[1985] ---

[1986] As a result, the child-rearing support device of the present invention can provide appropriate support 24 hours a day, taking into consideration the emotional needs of families raising children, who tend to be isolated.The system of the present invention allows parents to concentrate on raising their children with peace of mind.

[1987] The processing flow will be explained below.

[1988] Initial Setup

[1989] Step 1: Turn on the device

[1990] The user turns on the device with the power button.

[1991] Step 2: Install the app

[1992] Users install a dedicated app on their smartphone or tablet.

[1993] The terminal notifies the user that the installation is complete.

[1994] Step 3: Enter basic information

[1995] Users launch the app and enter basic information such as their child's age, gender, and allergy information on the profile screen.

[1996] The terminal checks the basic information entered and checks that all required fields have been entered.

[1997] The server receives the basic information sent by the device and stores it in a secure database.

[1998] Childcare consultation function

[1999] Step 1: Start a consultation

[2000] The user presses the "Consult" button within the app.

[2001] Step 2: Enter your question

[2002] The user enters a question by text input or voice input.

[2003] The terminal checks the input and prepares to send it to the server.

[2004] Step 3: Submit your question

[2005] The terminal transmits the entered question to the server.

[2006] Step 4: Emotion Recognition

[2007] The server passes the received question to an emotion engine to recognize the user's emotional state.

[2008] Step 5: Question Analysis

[2009] The server uses natural language processing technology to analyze the question and understand its content.

[2010] Step 6: Answer Generation

[2011] The server uses generative AI to generate appropriate answers to questions.

[2012] Step 7: Emotional Consideration

[2013] The server passes the generated answer to the emotion engine and adjusts the answer content in a way that takes the user's emotions into consideration.

[2014] Step 8: Submit your response

[2015] The server sends the adjusted response to the terminal.

[2016] Step 9: View Answers

[2017] The terminal displays the adjusted answer to the user.

[2018] Examples:

[2019] User: "My baby cries at night, what should I do?"

[2020] Terminal: Sends a question to the server.

[2021] Server: The emotion engine recognizes the user's emotions, analyzes the question, and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment." The answer is adjusted taking into account the user's emotional state.

[2022] Terminal: Display the adjusted answer to the user.

[2023] Baby monitoring features

[2024] Step 1: Turn on the camera

[2025] The device activates its built-in camera to monitor the baby's room.

[2026] Step 2: Data collection

[2027] The device collects camera footage and audio data in real time.

[2028] Step 3: Send data

[2029] The terminal transmits the collected data to the server.

[2030] Step 4: Data analysis

[2031] The server analyzes the received data and monitors the baby's condition, generating an alert if an abnormality is detected.

[2032] Step 5: Sending an alert

[2033] The server sends an alert to the terminal.

[2034] Step 6: Alert Notifications

[2035] The device will notify the user of the alert via voice or app notification.

[2036] Examples:

[2037] Terminal: A camera in the baby's room monitors the crib.

[2038] Server: Detects crying and generates an alert as an anomaly.

[2039] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[2040] User: Opens the app and sees live footage from the camera.

[2041] Follow-up function

[2042] Step 1: Follow-up check

[2043] The server periodically checks the user's status and consultation history.

[2044] Step 2: Emotion Recognition

[2045] The server evaluates the user's emotional state using an emotion engine.

[2046] Step 3: Generate a follow-up message

[2047] The server optionally generates follow-up messages based on the user's emotional state as assessed by the emotion engine.

[2048] Step 4: Send a follow-up message

[2049] The server sends the generated follow-up message to the terminal.

[2050] Step 5: Display a follow-up message

[2051] The terminal displays the follow-up message to the user.

[2052] Examples:

[2053] Server: Check the history of users who consulted about "baby constipation" one week ago.

[2054] Server: The emotion engine recognizes the user's emotions and generates a follow-up message such as, "I'm worried about your constipation last week. Has it improved?"

[2055] Terminal: Send a follow-up message to the user.

[2056] Example 2

[2057] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2058] Conventional child-rearing support systems have difficulty providing comprehensive support in environments where parents tend to feel isolated. They also lack the functionality to monitor the infant's condition and respond immediately if an abnormality occurs, or the follow-up function to consider the parents' feelings. As a result, parents are unable to concentrate on raising their children with peace of mind, and this places a heavy mental burden on them.

[2059] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2060] In this invention, the server includes a means for a user to input basic information, a means for receiving the basic information and storing it in a secure database, and a means for evaluating the user's emotional state and generating appropriately tailored responses and follow-up messages based on the evaluation. This allows parents to receive appropriate parenting support 24 hours a day, allowing them to focus on raising their children with peace of mind.

[2061] "Users" refer to people who use the system, and are primarily parents and guardians who are raising children.

[2062] "Basic information" is information required for initial setup, such as the child's age, sex, and allergy information, entered by the user.

[2063] A "database" is a storage device for safely storing data such as a user's basic information and consultation history.

[2064] "Generative AI" is an artificial intelligence technology that generates appropriate answers to questions from users.

[2065] An "integrated camera" is a camera that is built into the device and is used to monitor the condition of the infant.

[2066] A "sensor" is a device used to detect an infant's environment and condition, detecting temperature, humidity, sound, etc.

[2067] "Natural language processing technology" is a technology for analyzing the content of a user's question and understanding its meaning.

[2068] An "emotion engine" is a technology that assesses a user's emotional state and adjusts the answers and messages it provides based on that.

[2069] A "follow-up message" is a tracking message generated based on the user's past consultation history and is used to provide additional advice or confirmation.

[2070] The present invention relates to a child-rearing support system that enables parents to raise their children without feeling isolated. Specific embodiments of this system will be described in detail below.

[2071] The system aims to provide comprehensive parenting support by allowing users to input basic information and then responding in various ways based on that information. Specifically, the device, which has multiple functions, monitors the infant's condition using built-in cameras and sensors, and generates an alert when an abnormality is detected. It also accepts questions from users, provides answers using generative AI, and generates and sends follow-up messages. The system also combines an emotion engine that recognizes the user's emotions, allowing it to provide more personalized support.

[2072] Initial Setup

[2073] First, the user turns on the device and installs the dedicated app on their smartphone or tablet. Once the app is launched, the user enters basic information such as the child's age, gender, and allergy information on the profile screen. The device then verifies this information and ensures that all required fields have been entered correctly. Once the basic information is entered correctly, the server receives the information and stores it in a secure database.

[2074] Childcare consultation function

[2075] When a user presses the "Consult" button in the app, they can enter a question via text or voice. The device then sends the entered question to the server. The server uses natural language processing technology to analyze the question and uses generative AI to generate an appropriate answer. The generated answer is evaluated using an emotion engine to assess the user's emotional state and is presented in a way that takes the user's emotions into consideration. This allows users to consult at any time, 24 hours a day.

[2076] Specific examples

[2077] User: "My baby cries at night, what should I do?"

[2078] Terminal: Sends a question to the server.

[2079] Server: Analyzes the question and generates an answer such as, "There are various causes of nighttime crying, but first, try looking at your baby's environment."

[2080] Server: Recognizes the user's emotions using an emotion engine and provides answers in a gentle tone.

[2081] Terminal: Display the answer to the user.

[2082] Baby monitoring features

[2083] The device is equipped with a built-in camera and various sensors, allowing it to monitor the baby's condition 24 hours a day. Data sent from the device is analyzed by a server, and if an abnormality is detected, an alert is generated. The alert is communicated to the user via voice or in-app notification.

[2084] Specific examples

[2085] Terminal: A camera in the baby's room monitors the crib.

[2086] Server: Detects crying and determines it is abnormal.

[2087] Device: Sends a notification to the user saying, "Your child is crying. Please check on them."

[2088] User: Opens the app and sees live footage from the camera.

[2089] Follow-up function

[2090] The server periodically checks the user's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the user's emotional state using an emotion engine and providing appropriate follow-up information and advice. The generated message is then sent to the user via their device.

[2091] Specific examples

[2092] Server: Check the history of users who consulted about "baby constipation" one week ago.

[2093] Server: Generate a follow-up message: "I was concerned about your constipation last week. Has it improved?"

[2094] Server: Recognizes the user's emotions using an emotion engine and presents follow-up messages in appropriate words.

[2095] Terminal: Send a follow-up message to the user.

[2096] As a result, the child-rearing support system of this invention can provide appropriate support 24 hours a day, taking into consideration the emotional needs of families raising children, who tend to be isolated. This allows parents to concentrate on raising their children with peace of mind, and is expected to reduce stress related to child-rearing.

[2097] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2098] Step 1:

[2099] Booting the device and entering basic information

[2100] User: Turn on your smartphone or tablet and open the dedicated app.

[2101] User: Download and install the dedicated app from the device store.

[2102] User: Launch the app and enter basic information such as the child's age, gender, and allergies on the profile screen that appears.

[2103] Input: Basic information entered by the user (e.g., child's age, gender, allergy information).

[2104] Terminal: Once the user has completed their input, save the data and check that all required fields have been entered.

[2105] Server: Receives basic information sent from the device and stores it in a secure database.

[2106] Output: Basic information stored in the database.

[2107] Step 2:

[2108] Enter and submit your question

[2109] User: Press the "Consult" button in the app and enter a question via text or voice.

[2110] Input: A question typed by the user (e.g., "My baby cries at night. What should I do?")

[2111] Terminal: Sends the question entered by the user to the server.

[2112] Step 3:

[2113] Parsing questions and generating answers

[2114] Server: Receives the question and automatically analyzes it using natural language processing technology. For example, it extracts keywords such as "night crying," "baby," and "measures" from the question.

[2115] Input: User question data

[2116] Server: Based on the analysis results, the generative AI model generates an appropriate answer. For example, it generates an answer such as, "There are various causes of nighttime crying, but first, try reconsidering your baby's environment."

[2117] Output: Generated answer (e.g. "There are many causes of night crying, but first try looking at your baby's environment.")

[2118] Step 4:

[2119] Ratings and answers displayed by the sentiment engine

[2120] Server: Uses an emotion engine to assess the user's emotional state and adjusts the generated response appropriately, for example, by changing the tone of voice to a gentler one.

[2121] Input: Generated answers and user emotional state assessment data

[2122] Output: Final, emotion-sensitive answer

[2123] Device: Display the adjusted answer on the user's device.

[2124] Step 5:

[2125] Baby monitoring and alert notifications

[2126] Device: Monitors the baby's condition 24 hours a day using a built-in camera and various sensors.

[2127] Input: Camera footage and sensor data

[2128] Terminal: Sends monitoring data to the server in real time.

[2129] Server: Analyzes the received data and generates an alert if an abnormality is detected, such as abnormally continuous crying.

[2130] Output: Alert message (e.g. "Your child is crying. Please check on him / her.")

[2131] On device: The alert message is communicated to the user via audio notification or in-app notification.

[2132] Step 6:

[2133] Generate and send follow-up messages

[2134] Server: Periodically checks the user's past consultation history and generates a message if it determines that follow-up is necessary.

[2135] Input: Consultation history data

[2136] Server: Generates follow-up messages and uses the emotion engine to tailor them with appropriate language. For example, "I'm concerned about your constipation issue last week. Has it improved?"

[2137] Output: Follow-up message (e.g., "I was concerned about your constipation issue last week. Has it improved?")

[2138] Device: Send a follow-up message to the user's device.

[2139] (Application example 2)

[2140] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2141] While existing parenting support devices provide support for parenting situations where parents tend to feel isolated, there has also been a lack of means to provide appropriate support for customers' worries and anxieties in the field of customer service in brick-and-mortar stores. In particular, brick-and-mortar stores need to be able to grasp customers' emotions in real time and respond accordingly, but the current situation is that such systems are not fully developed. Furthermore, building ongoing relationships with customers is important, and follow-up functions based on past consultations are also needed.

[2142] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to input basic information; means for receiving the basic information and storing it in a secure database; means for accepting a user's question and generating an answer using a generation AI; means for displaying the generated answer to the user; means for monitoring the status of an object using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert when an abnormality is detected; means for notifying the user of the alert; means for checking the user's consultation history and generating and sending a follow-up message; means for evaluating the user's emotions in real time using an emotion recognition engine; and means for providing the user with the optimal response based on the evaluation results. This enables real-time responses and continuous follow-up that take customer emotions into consideration in customer service in physical stores.

[2143] "Means for users to enter basic information" refers to means by which users enter profile information and other required data, including dedicated applications and web interfaces.

[2144] "Means for receiving the basic information and storing it in a secure database" refers to means for receiving information entered by the user and storing it in a secure database, and includes encryption technology and access control.

[2145] "Means for accepting user questions and generating answers using generative AI" refers to means for accepting questions from users and using generative AI (for example, a model using natural language processing technology) to analyze the questions and generate appropriate answers.

[2146] The "means for displaying the generated answer to the user" refers to a means for displaying the generated answer on the user's device (smartphone, tablet, etc.), and includes a user interface.

[2147] "Means for monitoring the condition of an object using a built-in camera and various sensors" means means for continuously monitoring the condition of an object (e.g., an infant or a customer) using a built-in camera and various sensors.

[2148] The "means for analyzing the monitoring data and generating an alert when an abnormality is detected" refers to a means for analyzing the monitoring data and generating a warning when an abnormal condition is detected, such as by sending a voice notification or a message.

[2149] The "means for notifying the user of the alert" refers to a means for notifying the user's device of the generated alert, and includes push notification and email transmission.

[2150] The "means for checking the user's consultation history and generating and sending a follow-up message" refers to a means for checking the past consultation history, generating a follow-up message based on the history, and sending the message to the user at an appropriate time.

[2151] The "means for evaluating a user's emotions in real time using an emotion recognition engine" is a means for using an emotion recognition engine to evaluate a user's facial expression and tone of voice in real time and grasp their emotional state.

[2152] The "means for providing the user with the most appropriate response based on the evaluation results" is a means for providing the user with the most appropriate response method or information based on the evaluation results of the emotion recognition engine.

[2153] The present invention provides a "smart customer service device" to improve customer service in brick-and-mortar stores. The system receives basic information entered by customers, stores it securely, and then takes various actions based on that information.

[2154] Initial Setup

[2155] 1. Enter basic information

[2156] The user (store staff) activates the device and installs a dedicated app on their smartphone or smart glasses. Once the app is activated, the user enters customer profile information, including the customer's name, contact details, and past purchase history. The server receives this basic information and stores it in a secure database.

[2157] Customer service functions

[2158] 2. Question acceptance and answer generation

[2159] When a user (customer) inputs a question into the device, the server accepts it. The question can be entered as text or voice. The server analyzes the received question and generates an appropriate answer using a generative AI model (e.g., GPT-3). The generated answer is displayed on the user's device.

[2160] Examples:

[2161] User (customer): "I'd like to know more about this product?"

[2162] Server: Parses the question and generates an answer using a generative AI model.

[2163] Server: "This product is manufactured using the latest technology and has excellent performance."

[2164] Emotion recognition function

[2165] 3. Emotion recognition and response guidance

[2166] The system uses cameras and sensors inside the building to analyze customers' facial expressions and tone of voice, and evaluates them in real time using an emotion recognition engine. Based on the evaluation results, the system provides the user (customer) with the most appropriate response method.

[2167] Examples:

[2168] Terminal: Sends the customer's facial expression analysis data to the server.

[2169] Server: Detects "anxiety" using an emotion recognition engine.

[2170] Server: "You seem concerned. What are you worried about?" I suggest.

[2171] Alert notification function

[2172] 4. Anomaly detection and alert notification

[2173] The device is equipped with a built-in camera and various sensors that monitor the customer's condition. The monitoring data is analyzed on the server, and an alert is generated if an abnormality is detected. The alert is communicated to the user (store staff) as a voice or notification.

[2174] Examples:

[2175] Device: In-store camera detects abnormal behavior.

[2176] Server: Determines that something is abnormal and generates an alert.

[2177] Terminal: Notifies, "Abnormal behavior has been detected in Area A."

[2178] Follow-up function

[2179] 5. Generate and send follow-up messages

[2180] The server periodically checks the customer's consultation history and generates a message if it determines that follow-up is necessary. The follow-up message is generated by evaluating the customer's emotional state using an emotion recognition engine, and is sent to the user (customer) to provide appropriate follow-up information and advice.

[2181] Examples:

[2182] Server: Check the history of customers who consulted about "how to use the product" one week ago.

[2183] Server: Generates and sends a follow-up message: "Regarding the product issue from last week, has the issue been resolved since then?"

[2184] Prompt Sentence Examples

[2185] A customer asks the following question: I'd like to know more about this product? Emotion recognition indicates that the customer is interested. Generate an appropriate answer for this situation.

[2186] This system can improve customer satisfaction in physical stores by understanding customer sentiment in real time and providing optimal responses and continuous follow-up.

[2187] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2188] Step 1:

[2189] The user (store staff) installs the dedicated app on their smartphone or smart glasses and launches the app. When the app is launched, the user enters customer profile information (such as name, contact information, and past purchase history). This information is sent from the device to the server and stored in a secure database.

[2190] Input: Customer profile information

[2191] Output: Customer information stored in a secure database

[2192] Step 2:

[2193] The user (customer) enters a question into the device via text or voice. The device sends this question to the server. The server receives the question and analyzes it using natural language processing technology. It uses a generative AI model (e.g., GPT-3) to generate an appropriate answer and sends it to the device.

[2194] Input: User question

[2195] Output: Answer from the generative AI model

[2196] Step 3:

[2197] The terminal displays the generated answer on the user's (customer's) device, and the user (customer) can check the answer to resolve their doubts and ask additional questions as needed.

[2198] Input: Generated answer

[2199] Output: Display the answer to the user (customer)

[2200] Step 4:

[2201] The device uses a built-in camera and sensors to monitor customer facial expressions and tone of voice in real time, and the monitoring data is sent from the device to a server, which then uses an emotion recognition engine to analyze the data and evaluate the customer's emotional state.

[2202] Input: Surveillance data (facial expressions, tone of voice)

[2203] Output: Customer emotional state assessment

[2204] Step 5:

[2205] The server proposes the optimal response based on the evaluation results. It generates a prompt using the generative AI model along with the evaluation results, presents an appropriate response plan, and sends it to the user (store staff).

[2206] Input: Customer sentiment evaluation results

[2207] Output: Suggested action

[2208] Step 6:

[2209] The device continuously monitors the situation inside the store using a built-in camera and various sensors. The monitoring data is sent from the terminal to a server, and if the server detects an abnormality, it generates an alert and notifies the user (store staff).

[2210] Input: In-store surveillance data

[2211] Output: Alert notification in case of abnormality

[2212] Step 7:

[2213] The server periodically checks the customer's past consultation and purchase history, and if it determines that follow-up is necessary, it uses an emotion recognition engine to generate an appropriate follow-up message, which is then sent to the user (customer) via their device.

[2214] Input: Consultation history, purchase history

[2215] Output: Follow-up message

[2216] Step 8:

[2217] The user (customer) receives the follow-up message and replies as needed. The reply is also sent back to the server, enabling continuous support.

[2218] Input: Follow-up message

[2219] Output: Reply from the user (customer)

[2220] Through this series of steps, customer service in brick-and-mortar stores can be significantly improved by utilizing emotion recognition and generative AI models, leading to increased customer satisfaction.

[2221] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2222] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2223] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2224] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2225] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2226] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2227] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2228] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2229] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2230] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2231] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2232] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2233] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2234] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2235] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2236] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2237] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2238] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2239] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2240] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2241] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2242] The following is further disclosed regarding the above embodiment.

[2243] ---

[2244] (Claim 1)

[2245] a means for a user to input basic information;

[2246] means for receiving and storing said basic information in a secure database;

[2247] A means for accepting user questions and generating answers using a generation AI;

[2248] means for displaying the generated answer to a user;

[2249] A means for monitoring the condition of the infant using a built-in camera and various sensors;

[2250] means for analyzing the monitoring data and generating an alert when an abnormality is detected;

[2251] means for notifying a user of the alert;

[2252] means for reviewing the user's consultation history and generating and sending follow-up messages;

[2253] A system including:

[2254] (Claim 2)

[2255] 10. The system of claim 1, further comprising means for parsing the user's question using natural language processing techniques.

[2256] (Claim 3)

[2257] The system of claim 1 , wherein the follow-up message is generated based on the user's past consultations.

[2258] "Example 1"

[2259] (Claim 1)

[2260] a means for a user to input basic information;

[2261] means for receiving and storing said basic information in a secure database;

[2262] A means for accepting a user's question and generating an answer using a generative AI model;

[2263] means for displaying the generated answer to a user;

[2264] A means for monitoring the condition of the infant using a built-in camera and various sensors;

[2265] means for analyzing the monitoring data and generating an alert when an abnormality is detected;

[2266] means for notifying a user of the alert;

[2267] means for reviewing the user's consultation history and generating and sending follow-up messages;

[2268] means for transmitting basic information and monitoring data of the device to a server, and for the server to analyze the data;

[2269] A system including:

[2270] (Claim 2)

[2271] 10. The system of claim 1, further comprising means for parsing the user's question using natural language processing techniques.

[2272] (Claim 3)

[2273] The system of claim 1 , wherein the follow-up message is generated based on the user's past consultations.

[2274] "Application Example 1"

[2275] (Claim 1)

[2276] a means for a user to input basic information;

[2277] means for receiving and storing said basic information in a secure database;

[2278] A means for accepting user questions and generating answers using a generation AI;

[2279] means for displaying the generated answer to a user;

[2280] A means for monitoring the condition of the infant using a built-in camera and various sensors;

[2281] means for analyzing the monitoring data and generating an alert when an abnormality is detected;

[2282] means for notifying a user of the alert;

[2283] means for reviewing the user's consultation history and generating and sending follow-up messages;

[2284] A means of making individual proposals that take into account the child-rearing situation, and

[2285] A means for providing an appropriate service to a user based on the individual proposal;

[2286] A system including:

[2287] (Claim 2)

[2288] 10. The system of claim 1, further comprising means for parsing the user's question using natural language processing techniques.

[2289] (Claim 3)

[2290] The system of claim 1 , wherein the follow-up message is generated based on the user's past consultations.

[2291] "Example 2: Combining Emotion Engines"

[2292] (Claim 1)

[2293] a means for a user to input basic information;

[2294] means for receiving and storing said basic information in a secure database;

[2295] A means for accepting user questions and generating answers using a generation AI;

[2296] means for displaying the generated answer to a user;

[2297] A means for monitoring the condition of the infant using a built-in camera and various sensors;

[2298] means for analyzing the monitoring data and generating an alert when an abnormality is detected;

[2299] means for notifying a user of the alert;

[2300] means for reviewing the user's consultation history and generating and sending follow-up messages;

[2301] means for assessin...

Claims

1. a means for a user to input basic information; means for receiving and storing said basic information in a secure database; A means for accepting user questions and generating answers using a generation AI; means for displaying the generated answer to a user; A means for monitoring the condition of the infant using a built-in camera and various sensors; means for analyzing the monitoring data and generating an alert when an abnormality is detected; means for notifying a user of the alert; means for reviewing the user's consultation history and generating and sending follow-up messages; A system including:

2. The system of claim 1 , further comprising: means for analyzing the user's question using natural language processing techniques.

3. The system of claim 1 , wherein the follow-up message is generated based on the user's past consultations.

Images