system

Abstract

Provide a system. A means for setting a wake-up time; means for sending an alarm at a set wake-up time; means for ascertaining a user's response to the alarm; a means for setting the time for daytime activity; a means for sending reminders at set activity times; a means for confirming a user's response to the reminder; a means for setting a bedtime; A means for sending a notification at a set bedtime prompting the user to get ready for bed; A means to shut down devices in the home at bedtime; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many people today lead busy lives, making it difficult to maintain a healthy lifestyle. Excessive use of electronic devices and the internet often has adverse effects on health. In particular, people often fail to establish regular wake-up and bedtime routines, resulting in sleep deprivation and disrupted daily rhythms. To solve these problems, proper time management and enforced control are necessary at every stage of daily life. However, self-management is difficult for humans, and people often forget or become lazy. [Means for solving the problem]

[0005] The present invention proposes a system that provides a means for users to set their wake-up time, activity time, and bedtime, and automatically sends alarms, reminders, and notifications based on those settings. In this invention, a server sends alarms and reminders to a terminal at the corresponding times according to the user's settings and confirms the user's response. The system also includes a means for automatically shutting down home electronic devices at bedtime. This allows users to wake up at an appropriate time, be efficient with their daytime activities, and get a good night's rest. Furthermore, adding a shutdown monitoring means confirms that devices have been shut down properly, providing further peace of mind. This enables semi-mandatory management to maintain a healthy lifestyle cycle.

[0006] "Wake-up time" refers to the time that a user sets to wake up each morning.

[0007] An "alarm" is a signal or sound that notifies the user at a set time and encourages them to wake up.

[0008] "Active time" refers to a time period that a user sets for a specific purpose (such as work or study).

[0009] A "reminder" is a message or signal that notifies a user at a set activity time and prompts them to begin a specific activity.

[0010] "Bedtime" refers to the time that a user sets to go to bed each night.

[0011] A "notification" is a message or signal sent to a user to prompt them to take a particular action.

[0012] "Shutdown" refers to the operation of turning off the power to an electronic device and causing it to stop operating.

[0013] "Server" refers to the computer system responsible for storing user settings and sending alarms, reminders, notifications, and shutdown signals at corresponding times.

[0014] A "terminal" is a device that displays alarms, reminders, and notifications to a user and receives responses from the user.

[0015] "User response" refers to the action or reaction (e.g., pressing a button) a user takes in response to an alarm, reminder, or notification. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a home appliance management system for leading a healthy lifestyle.

[0038] System configuration

[0039] The system of the present invention consists of three main components: a server, a terminal, and a user.

[0040] The server manages the data of the user's set wake-up time, activity time, and bedtime, and is responsible for sending corresponding notifications to the device based on this data.

[0041] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server.

[0042] The user sets his or her own schedule on the server and responds to notifications from the terminal based on that setting.

[0043] Program processing

[0044] The processing of the program of the present invention will be explained in natural language below.

[0045] 1. Manage your wake-up time

[0046] The user sets the wake-up time each morning on the server.

[0047] The server sends an alarm notification to the terminal at the set time.

[0048] The device will sound an alarm and continue to notify you until the user presses the wake-up button.

[0049] When the user presses the button, the terminal sends wake-up confirmation data to the server.

[0050] 2. Daytime activity management

[0051] The user sets the daytime activity times on the server (for example, 9:00 AM to 12:00 PM).

[0052] The server sends a reminder to the device when it is time to start the activity.

[0053] The device will display a reminder and continue to notify you until you press the confirmation button.

[0054] When the user presses the confirmation button, the terminal transmits confirmation data for the start of the activity to the server.

[0055] 3. Bedtime notifications

[0056] The user sets the time they go to bed each night on the server.

[0057] The server sends a notification to the device to prepare for bed 30 minutes before going to bed.

[0058] The device will display a notification to remind the user to get ready for bed.

[0059] 4. Shut down the device

[0060] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[0061] The server sends a shutdown signal to these devices at the set time.

[0062] The terminal checks whether the device has been shut down normally and reports the result to the server.

[0063] Specific examples

[0064] 1. Manage your wake-up time

[0065] For example, a user configures the server to wake up at 7am every morning.

[0066] The server sends an alarm notification to the device at exactly 7:00, and the device sounds an alarm.

[0067] When the user presses the wake-up button, the terminal transmits the information to the server to confirm the wake-up.

[0068] 2. Daytime activity management

[0069] For example, a user may set his working hours from 9:00 AM to 12:00 PM.

[0070] At 9:00 AM, the server sends a reminder to the device saying, "It's time to get to work."

[0071] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[0072] 3. Bedtime notifications

[0073] For example, a user sets the time to go to bed at 10 o'clock every night.

[0074] At 9:30 p.m., the server sends a notification to the device saying, "It's bedtime in 30 minutes."

[0075] The device will display a notification to remind the user to get ready for bed.

[0076] 4. Shut down the device

[0077] For example, a user can set the TV and Wi-Fi router in their home to shut down at 10:00.

[0078] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[0079] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[0080] As described above, the system of the present invention manages the user's life cycle according to a set time and provides a specific means for supporting a healthy lifestyle.

[0081] The processing flow will be explained below.

[0082] Wake-up time management

[0083] Step 1:

[0084] The user logs into the server and sets the time to wake up each morning.

[0085] Step 2:

[0086] The server stores the wake-up time entered by the user in a database.

[0087] Step 3:

[0088] Five minutes before the set wake-up time, the server prepares for the alarm notification.

[0089] Step 4:

[0090] When the set wake-up time arrives, the server sends an alarm notification to the terminal.

[0091] Step 5:

[0092] The device will sound an alarm and display a wake-up notification on the screen.

[0093] Step 6:

[0094] When the user presses the wake-up button on the terminal, the terminal reports the operation to the server.

[0095] Step 7:

[0096] The server confirms that the user has pressed the wake-up button and records the wake-up confirmation information in the database.

[0097] Daytime activity management

[0098] Step 1:

[0099] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[0100] Step 2:

[0101] The server stores the activity time entered by the user in a database.

[0102] Step 3:

[0103] Five minutes before the activity start time, the server prepares the reminder notification.

[0104] Step 4:

[0105] When the activity start time arrives, the server sends a reminder notification to the device.

[0106] Step 5:

[0107] The device will display a reminder on the screen, telling the user, "It's time to start work."

[0108] Step 6:

[0109] When the user presses the confirmation button on the terminal, the terminal reports the operation to the server.

[0110] Step 7:

[0111] The server confirms that the user has pressed the confirmation button and records the confirmation information for the start of the activity in the database.

[0112] Bedtime notifications

[0113] Step 1:

[0114] The user sets the time they go to bed each night on the server.

[0115] Step 2:

[0116] The server stores the bedtime entered by the user in a database.

[0117] Step 3:

[0118] Thirty minutes before bedtime, the server prepares the bedtime notification.

[0119] Step 4:

[0120] Thirty minutes before bedtime, the server sends a notification to the device to prepare for bed.

[0121] Step 5:

[0122] The device displays the message "Bedtime in 30 minutes" on the screen, encouraging the user to get ready for bed.

[0123] Step 6:

[0124] When the user presses the prepare button, the terminal reports the operation to the server.

[0125] Step 7:

[0126] The server confirms that the user has pressed the prepare button and records the confirmation of preparation in the database.

[0127] Equipment shutdown

[0128] Step 1:

[0129] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[0130] Step 2:

[0131] The server stores the shutdown time entered by the user in a database.

[0132] Step 3:

[0133] Five minutes before the shutdown time, the server prepares for the shutdown notification.

[0134] Step 4:

[0135] When the set shutdown time arrives, the server sends a shutdown signal to the target device.

[0136] Step 5:

[0137] The terminal receives the shutdown signal and initiates shutdown of the target device.

[0138] Step 6:

[0139] The terminal checks whether the shutdown of the target device has been completed successfully.

[0140] Step 7:

[0141] The terminal reports that the shutdown is complete to the server, and the server records the information in a database.

[0142] Example 1

[0143] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0144] Maintaining a regular lifestyle is important for improving health and productivity. However, in today's busy lives, daily time management is becoming increasingly difficult. Conventional methods require users to manually manage their time, including waking up, daily activities, going to bed, and managing home devices, which can be cumbersome. This creates a particularly difficult problem for maintaining a healthy lifestyle.

[0145] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0146] In this invention, the server includes means for a user to set a wake-up time, means for sending an alarm when the set wake-up time arrives, means for the terminal to sound the alarm and confirm the user's response, means for a user to set a daytime activity time, means for sending a reminder at the set activity time, means for the terminal to display the reminder and confirm the user's response, means for a user to set a bedtime, means for sending a notification urging the user to get ready for bed 30 minutes before the set bedtime, means for a user to set a shutdown time for home appliances, and means for sending a shutdown signal to the home appliances at the set shutdown time. This makes it possible to automatically manage a user's lifestyle via the server.

[0147] "User" refers to an individual who uses this system.

[0148] "Server" refers to a computer system that records and manages user settings and transmits each piece of information to the terminal.

[0149] "Terminal" refers to a device that displays notifications and reminders from the server to the user and receives the user's responses.

[0150] "Means for setting a wake-up time" refers to the specific functions and methods by which a user inputs and sets the wake-up time each morning.

[0151] "Means for sending an alarm" refers to a specific function or method for sending an alarm notification from the server to the terminal at the set wake-up time.

[0152] The term "means for the terminal to sound an alarm and confirm the user's response" refers to a function or method for the terminal to sound an alarm and receive a user's wake-up confirmation operation.

[0153] The "means for setting daytime activity times" refers to a specific function or method for a user to set a specific time period as a daytime activity time.

[0154] "Means for sending reminders" refers to specific functions and methods for sending reminder notifications from the server to the device when the set activity time arrives.

[0155] The "means for the terminal to display the reminder and confirm the user's response" refers to a function or method for the terminal to display the reminder and receive a confirmation operation from the user.

[0156] "Means for setting bedtime" refers to the specific functions and methods by which a user can input and set their nightly bedtime.

[0157] "Means for sending a notification encouraging users to get ready for bed" refers to the specific functions and methods for sending a notification from the server to the device prompting users to get ready for bed 30 minutes before the set bedtime.

[0158] "Means for setting the shutdown time of devices in the home" refers to specific functions and methods that allow a user to input and set the shutdown time of a specific device in the home.

[0159] "Means for sending a shutdown signal to a device in the home" refers to a specific function or method for sending a shutdown signal from the server to a specific device in the home when the set shutdown time arrives.

[0160] "Means for confirming that home devices have been shut down normally" refers to specific functions and methods that allow a terminal to check the shutdown status of home devices and report the results to a server.

[0161] This invention relates to a home appliance management system for promoting a healthy lifestyle. This system consists of three main components: a server, a terminal, and a user. Below, we will explain in detail how each component operates, including the hardware and software used.

[0162] Server Roles

[0163] The server manages various schedules set by the user (wake-up time, activity time, bedtime, shutdown time of home devices) and sends appropriate notifications to the device based on those schedules. The server records the user's setting data in a database (e.g., MySQL (registered trademark)) and generates corresponding notifications at specific times and sends them to the device. For example, the server sends an alarm notification to the device when the set wake-up time (e.g., 7:00 AM) arrives.

[0164] Device Role

[0165] The device is responsible for displaying notifications sent from the server (such as alarms, reminders, and bedtime notifications) to the user and receiving the user's responses. The device also has the function of reporting the user's responses (such as pressing a button) to the server. For example, if the device sounds an alarm and the user presses the "wake-up button," that information is sent to the server.

[0166] User Roles

[0167] Users set various schedules on the server using a smartphone app, etc. This allows the server to generate and manage notifications that fit the user's lifestyle. For example, if a user sets the server as their active time from 9:00 AM to 12:00 PM, the server will send a reminder to the device at 9:00 AM.

[0168] Specific examples

[0169] Consider a scenario where a user wakes up at 7am every morning, sets their work hours from 9am to 12pm, goes to bed at 10pm every night, and shuts down the TV and Wi-Fi router in their home at the same time.

[0170] 1. Manage your wake-up time:

[0171] The user opens the smartphone app and sets the wake-up time to 7:00 AM every morning. The server receives the wake-up time data (7:00 AM) from the user and records it in a database. When the set time of 7:00 AM arrives, the server sends an alarm notification to the device, which then sounds the alarm continuously. When the user taps the "wake-up button," the device sends "wake-up confirmation data" to the server, which records it in the database.

[0172] 2. Daytime activity management:

[0173] The user sets the working hours from 9:00 AM to 12:00 PM. The server records the received activity time data (9:00 AM - 12:00 PM) in the database. At 9:00 AM, the server sends a reminder notification to the device saying "It's time to start work." The device displays the reminder, and when the user taps the "confirm button," "activity start confirmation data" is sent to the server, which records it in the database.

[0174] 3. Bedtime notifications:

[0175] The user sets the bedtime to 10 PM every night. The server receives the bedtime data (10:00 PM) and records it in the database. At 9:30 PM, the server sends a notification to the device saying, "It's 30 minutes until bedtime." The device continues to display the notification, encouraging the user to get ready for bed.

[0176] 4. Shutdown the equipment:

[0177] A user sets the shutdown time for specific devices in the home (TV and Wi-Fi router). The server records the received shutdown time data (10:00 PM) in a database. At 10 PM, the server sends a shutdown signal to these devices. The terminal checks whether these devices have been shut down successfully and reports the results to the server. The server records this information in a database.

[0178] Prompt Sentence Examples

[0179] Here are some examples of prompts:

[0180] 1. Explain specific examples of managing your wake-up time:

[0181] Let's say a user configures the system to wake up at 7:00 every morning. The server sends a notification at exactly 7:00, and the device continues to ring the alarm. When the user presses the wake-up button, that information is sent to the server.

[0182] 2. Describe specific examples of daytime activity management:

[0183] If a user sets their active time from 9:00 AM to 12:00 PM, the server will send a reminder at 9:00 AM and the device will display the reminder. When the user presses the confirmation button, the information is sent to the server.

[0184] 3. Explain specific examples of bedtime notifications:

[0185] For users who have set their bedtime schedule to 10 p.m. every night, the server will send a notification at 9:30 p.m. and the device will display the message "Bedtime in 30 minutes."

[0186] 4. Describe an example of equipment shutdown:

[0187] If the TV and Wi-Fi router in the home are set to shut down at 10 PM, the server will send a shutdown signal to these devices at 10 PM. The device will check to see if the devices have shut down and report the results to the server.

[0188] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0189] Wake-up time management

[0190] Step 1:

[0191] The user uses a smartphone app to set the time they wake up each morning.

[0192] Input: Wake-up time (e.g. 7:00 AM)

[0193] Output: Configuration data sent to the server

[0194] Specific operation: The user enters the wake-up time on the app's settings screen and presses the Set button.

[0195] Step 2:

[0196] The server records the received wake-up time data in a database.

[0197] Input: Wake-up time data sent by the user

[0198] Output: Wake-up time data recorded in a database

[0199] Specific operation: The server executes a query to store the wake-up time in a database (e.g. MySQL).

[0200] Step 3:

[0201] When the server reaches the set wake-up time, it sends an alarm notification to the terminal.

[0202] Input: Current time and wake-up time data recorded in the database

[0203] Output: Alarm notification is sent to the device

[0204] Specific operation: The server detects that a specific time has arrived, generates an alarm notification, and sends a push notification to the device.

[0205] Step 4:

[0206] The device will sound an alarm and continue to notify you.

[0207] Input: Alarm notification sent from the server

[0208] Output: The device continues to ring

[0209] Specific behavior: The device will start playing a sound file and display a "wake-up button" on the screen.

[0210] Step 5:

[0211] The user presses the wake-up button.

[0212] Input: User presses a button

[0213] Output: Wake-up confirmation data is generated

[0214] Specific action: The user taps the "wake-up button" on the device.

[0215] Step 6:

[0216] The terminal transmits wake-up confirmation data to the server.

[0217] Input: User button press information

[0218] Output: Wake-up confirmation data is sent to the server and recorded in the database

[0219] Specific operation: The device sends an HTTP request to the server, and the server stores the wake-up confirmation data in a database.

[0220] Daytime activity management

[0221] Step 1:

[0222] The user sets the time for daytime activity using a smartphone app.

[0223] Input: Activity time (e.g. 9:00 AM - 12:00 PM)

[0224] Output: Configuration data sent to the server

[0225] Specific operation: The user enters the activity time on the app's settings screen and presses the settings button.

[0226] Step 2:

[0227] The server records the received activity time data in a database.

[0228] Input: Activity time data sent by the user

[0229] Output: Activity time data recorded in a database

[0230] Specific operation: The server executes a query to store the activity time in the database.

[0231] Step 3:

[0232] The server sends a reminder to the device at the start time of the activity.

[0233] Input: Current time and activity time data recorded in the database

[0234] Output: Reminder notification sent to device

[0235] Specific operation: The server detects when a specific time has arrived, generates a reminder notification, and sends it as a push notification to the device.

[0236] Step 4:

[0237] Your device will display reminders and keep you informed.

[0238] Input: Reminder notification sent from the server

[0239] Output: The device continues to display the reminder

[0240] Specific behavior: The device will display a reminder notification and a "Confirm" button on the screen.

[0241] Step 5:

[0242] The user presses the confirm button.

[0243] Input: User presses a button

[0244] Output: Activity start confirmation data is generated

[0245] Specific action: The user taps the "confirmation button" on the device.

[0246] Step 6:

[0247] The terminal transmits activity start confirmation data to the server.

[0248] Input: User button press information

[0249] Output: Activity start confirmation data is sent to the server and recorded in the database

[0250] Specific operation: The terminal sends an HTTP request to the server, and the server stores the activity start confirmation data in the database.

[0251] Bedtime notifications

[0252] Step 1:

[0253] The user uses a smartphone app to set their bedtime each night.

[0254] Input: Bedtime (e.g. 10:00 PM)

[0255] Output: Configuration data sent to the server

[0256] Specific behavior: The user enters bedtime on the app's settings screen and presses the Set button.

[0257] Step 2:

[0258] The server records the received bedtime data in a database.

[0259] Input: Bedtime data submitted by the user

[0260] Output: Bedtime data recorded in a database

[0261] What happens: The server runs a query to store the bedtime in the database.

[0262] Step 3:

[0263] The server sends a bedtime preparation notification to the device 30 minutes before bedtime.

[0264] Input: Current time and bedtime data recorded in the database

[0265] Output: Bedtime notification sent to device

[0266] Specific operation: The server detects when a specific time has arrived, generates a bedtime preparation notification, and sends it as a push notification to the device.

[0267] Step 4:

[0268] The device displays a notification to remind the user to get ready for bed.

[0269] Input: Bedtime notification sent from the server

[0270] Output: The device keeps displaying the bedtime notification

[0271] What it does: Your device will display a bedtime notification and display "Start getting ready for bed" on the screen.

[0272] Equipment shutdown

[0273] Step 1:

[0274] A user uses a smartphone app to set shutdown times for specific devices in the home.

[0275] Input: Shutdown time (e.g. 10:00 PM)

[0276] Output: Configuration data sent to the server

[0277] Specific behavior: The user enters the shutdown time on the app's settings screen and presses the Set button.

[0278] Step 2:

[0279] The shutdown time data received by the server is recorded in a database.

[0280] Input: Shutdown time data sent by the user

[0281] Output: Shutdown time data recorded in the database

[0282] What happens: The server runs a query to store the shutdown time in the database.

[0283] Step 3:

[0284] The server transmits a shutdown signal to a specific device in the home at a specified time.

[0285] Input: Current time and shutdown time data recorded in the database

[0286] Output: A shutdown signal is sent to the device

[0287] Specific operation: The server detects that a specific time has arrived, generates a shutdown signal, and sends it to the home appliance control system (e.g., IoT device).

[0288] Step 4:

[0289] The terminal checks whether the device has been shut down normally and reports the result to the server.

[0290] Input: Device response data to shutdown signal

[0291] Output: Shutdown confirmation data is sent to the server and recorded in the database.

[0292] Specific operation: The terminal checks the device status and reports to the server that the shutdown is complete.

[0293] (Application example 1)

[0294] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0295] Conventional lifestyle management in homes and cars is often performed separately, lacking integrated management. In particular, in autonomous vehicles, time management for users while driving or napping is difficult, and methods for realizing a comfortable and efficient lifestyle are required. Appropriate shutdown management of in-car entertainment systems and lighting is also required.

[0296] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0297] In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time, means for confirming a user's response to the alarm, means for setting a daytime activity time, means for sending a reminder at the set activity time, means for confirming a user's response to the reminder, means for setting a bedtime, means for sending a notification encouraging the user to get ready for bed at the set bedtime, means for shutting down home appliances at bedtime, means for sending specified reminders and notifications for lifestyle cycle management in the autonomous vehicle, and means for shutting down the entertainment system and lighting in the autonomous vehicle at set times, thereby enabling integrated and efficient lifestyle cycle management at home and in the autonomous vehicle.

[0298] A "wake-up time" is a specific time that a user sets to wake up.

[0299] An "alarm" is a means of sending a notification to the user at a set wake-up time to prompt them to wake up.

[0300] The "user's response to the alarm" is an action that the user takes in response to the alarm notification, such as pressing a button or using voice recognition to confirm that the user is awake.

[0301] The "daytime activity time" is a set time indicating the activities (for example, work or exercise) that the user should perform within a certain period of time.

[0302] A "reminder" is a notification means that prompts the user to start an activity at a set time.

[0303] A "user's response to a reminder" is a confirmation action that a user takes in response to a reminder notification.

[0304] A "bedtime" is a specific time that a user sets for going to bed.

[0305] The "notification to urge the user to get ready for bed" is a notification means for urging the user to get ready for bed prior to the set bedtime.

[0306] The "means for shutting down devices" is a means for automatically shutting down specific devices in a home or car at a set time.

[0307] "In-vehicle entertainment system" refers to devices and software that allow users to enjoy entertainment in an autonomous vehicle.

[0308] "Lighting" refers to a lighting device for providing brightness inside an autonomous vehicle.

[0309] This invention is a system for managing a life cycle in a home and an autonomous vehicle to lead a healthy life cycle. This system consists of three main components: a server, a terminal, and a user.

[0310] The server manages the data of the wake-up time, activity time, and bedtime set by the user, and sends corresponding notifications to the device based on this. For example, if the user sets the server to wake up at 7:00 every morning, the server will send an alarm notification to the device at exactly 7:00, and the device will ring the alarm.

[0311] The device is responsible for displaying notifications to the user, receiving the user's response (such as pressing a button), and reporting it to the server. It can be implemented as a smartphone, tablet, or a dedicated device in an autonomous vehicle. For example, the device might display a reminder saying "It's time to start work" at 9:00 AM, and when the user presses the confirmation button, the response is reported to the server.

[0312] Users can set their own schedules on the server and respond to notifications from their devices based on those settings, making it easier for users to manage their daily activity cycles.

[0313] When used in an autonomous vehicle, for example, if a user sets a wake-up time during a nap, a wake-up notification will be sent 10 minutes before the scheduled arrival time. The entertainment system and lighting will also be shut down at the set time, allowing for a comfortable in-car experience.

[0314] Hardware and software used

[0315] To implement this system, a cloud server (e.g., AWS (registered trademark), Google (registered trademark) Cloud Platform) is used on the server side, and a smartphone (e.g., iPhone (registered trademark), Android (registered trademark), tablet, or in-car device (e.g., in-car infotainment system) is used on the terminal side. Such devices are connected to the Internet and have an application installed to communicate with the server.

[0316] Data calculation and processing

[0317] The server receives the life cycle data set by the user and creates a notification schedule based on this. For example, when it receives wake-up time data, it stores that data and generates a notification when the alarm time arrives. It also receives response data from the device and manages the user's behavior history. A database (e.g., MySQL, PostgreSQL) is used to process this data.

[0318] Examples of concrete examples and prompts

[0319] As a concrete example, consider the case where a user sets the device to wake up at 7:00 every morning. At 7:00, the server sends an alarm notification to the device, and the device sounds the alarm. When the user checks the alarm and presses a button, that information is sent to the server.

[0320] An example prompt might look like this:

[0321] "Generate Python code to send a nap notification so that the user can wake up 10 minutes before their scheduled arrival time."

[0322] "Generate Python code to manage work time in the car and display a reminder 10 minutes before a meeting."

[0323] This approach enables integrated and efficient life cycle management within the home and in autonomous vehicles, enabling users to maintain a healthy and comfortable lifestyle.

[0324] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0325] Step 1:

[0326] (Server) Setting user lifecycle data

[0327] Input: The user sets the wake-up time, daytime activity times, bedtime, and device shutdown time to the server.

[0328] Processing: The server receives this data and stores it in a database (e.g., MySQL, PostgreSQL).

[0329] Output: The server is ready to generate a notification schedule based on the configured data.

[0330] Step 2:

[0331] (Server) Wake-up time management

[0332] Input: The server retrieves the user's wake-up time information from the database.

[0333] Processing: When the set wake-up time approaches, the server generates an alarm notification and sends it to the device.

[0334] Output: The server sends an alarm notification to the terminal.

[0335] Step 3:

[0336] (Device) Display alarm notifications

[0337] Input: Receives alarm notifications sent from the server.

[0338] Action: The device will sound an alarm and display a wake-up notification on the screen until the user presses the wake-up button.

[0339] Output: When the user presses the button, the device sends the response data to the server.

[0340] Step 4:

[0341] (Server) Managing daytime activity times

[0342] Input: Retrieve information about the user's daily activity times from the database.

[0343] Processing: When the set activity start time approaches, the server generates a reminder notification and sends it to the device.

[0344] Output: The server sends a reminder notification to the device.

[0345] Step 5:

[0346] (Device) Displaying reminder notifications

[0347] Input: Receive reminder notifications sent from the server.

[0348] Processing: The device displays a reminder to prompt the user to start the activity. The notification continues until the user presses the confirmation button.

[0349] Output: When the user presses the button, the device sends the response data to the server.

[0350] Step 6:

[0351] (Server) Bedtime management

[0352] Input: Get the bedtime information set by the user from the database.

[0353] Processing: 30 minutes before the set bedtime, the server generates a notification to prompt the user to get ready for bed and sends it to the device.

[0354] Output: The server sends a notification to the device to prepare for bed.

[0355] Step 7:

[0356] (Device) Display bedtime notification

[0357] Input: Receive bedtime notifications sent from the server.

[0358] Processing: The device displays a notification to prompt the user to prepare for bed. The notification continues until the user presses the confirmation button.

[0359] Output: When the user presses the button, the device sends the response data to the server.

[0360] Step 8:

[0361] (Server) Device Shutdown

[0362] Input: Retrieve information about the device shutdown time set by the user from the database.

[0363] Processing: When the scheduled shutdown time arrives, the server sends a shutdown signal to the target device, and also sends a similar signal to the entertainment system and lighting in the autonomous vehicle.

[0364] Output: The server checks for shutdown and records the result.

[0365] These steps will enable integrated and efficient life cycle management within homes and autonomous vehicles.

[0366] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0367] The present invention relates to a system that provides more personalized support by combining an emotion engine that recognizes the user's emotions with a home appliance management system for living a healthy lifestyle.

[0368] System configuration

[0369] The system of the present invention consists of four main components: a server, a terminal, an emotion engine, and a user.

[0370] The server manages the data of the user's set wake-up time, activity time, and bedtime, and is responsible for sending corresponding notifications to the device based on this data.

[0371] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server.

[0372] The emotion engine is responsible for recognizing the user's emotions in real time and providing the analysis results to the server.

[0373] The user sets his or her own schedule on the server and responds to notifications from the terminal based on that setting.

[0374] Program processing

[0375] The processing of the program of the present invention will be explained in natural language below.

[0376] 1. Manage your wake-up time

[0377] The user sets the wake-up time each morning on the server.

[0378] When the set time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device.

[0379] The device will sound an alarm and display a wake-up notification on the screen.

[0380] When the user presses the wake-up button, the terminal reports the operation to the server.

[0381] The emotion engine analyzes the user's emotional state in real time when they wake up and provides the results to the server.

[0382] 2. Daytime activity management

[0383] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[0384] When the time to start the activity arrives, the server adjusts the content of the reminder message based on the analysis results of the emotion engine and sends a reminder notification to the device.

[0385] For example, if the user's emotional state is low, the device will display a reminder with an encouraging message.

[0386] When the user presses the confirmation button, the terminal reports the operation to the server.

[0387] The emotion engine analyzes the user's emotional state in real time while they are active and provides information to the server as needed.

[0388] 3. Bedtime notifications

[0389] The user sets the time they go to bed each night on the server.

[0390] Thirty minutes before bedtime, the server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device.

[0391] For example, if the user is relaxing, the notification may be displayed with soft music in the background.

[0392] The device will display a notification to remind the user to get ready for bed.

[0393] When the user presses the prepare button, the terminal reports the operation to the server.

[0394] 4. Shut down the device

[0395] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[0396] When the shutdown time arrives, the server sends a shutdown signal to these devices.

[0397] The terminal receives the shutdown signal and initiates shutdown of the target device.

[0398] The terminal checks whether the device shutdown has been completed successfully and reports the result to the server.

[0399] Specific examples

[0400] 1. Manage your wake-up time

[0401] For example, a user configures the server to wake up at 7am every morning.

[0402] The server sends an alarm notification to the device at exactly 7:00, and the device plays an appropriate alarm sound based on the results of the emotion engine.

[0403] When the user presses the wake-up button, the device sends the information to the server, and the emotion engine analyzes and reports the user's emotions.

[0404] 2. Daytime activity management

[0405] For example, a user may set his working hours from 9:00 AM to 12:00 PM.

[0406] At 9 a.m., the server uses the emotion engine's analysis to send a reminder to the device, including the encouraging message, "It's time to get to work."

[0407] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[0408] 3. Bedtime notifications

[0409] For example, a user sets the time to go to bed at 10 o'clock every night.

[0410] At 9:30 p.m., the server uses the emotion engine's analysis results to send a bedtime notification to the device, including a relaxing message.

[0411] The device will display a notification to remind the user to get ready for bed.

[0412] 4. Shut down the device

[0413] For example, a user can set the TV and Wi-Fi router in their home to shut down at 10:00.

[0414] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[0415] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[0416] As described above, the system of the present invention not only manages household appliances but also provides more personalized lifestyle management by taking into account the user's emotions.

[0417] The processing flow will be explained below.

[0418] Wake-up time management

[0419] Step 1:

[0420] The user logs into the server and sets the time to wake up each morning.

[0421] Step 2:

[0422] The server stores the wake-up time entered by the user in a database.

[0423] Step 3:

[0424] Five minutes before the set wake-up time, the server prepares for the alarm notification.

[0425] Step 4:

[0426] The emotion engine analyzes the user's current emotional state.

[0427] Step 5:

[0428] When the set wake-up time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device.

[0429] Step 6:

[0430] The device will sound an alarm and display a wake-up notification on the screen.

[0431] Step 7:

[0432] When the user presses the wake-up button on the terminal, the terminal reports the operation to the server.

[0433] Step 8:

[0434] The server confirms that the user has pressed the wake-up button and records the wake-up confirmation information in the database.

[0435] Daytime activity management

[0436] Step 1:

[0437] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[0438] Step 2:

[0439] The server stores the activity time entered by the user in a database.

[0440] Step 3:

[0441] Five minutes before the activity start time, the server prepares the reminder notification.

[0442] Step 4:

[0443] The emotion engine analyzes the user's current emotional state.

[0444] Step 5:

[0445] When the time to start the activity arrives, the server adjusts the content of the reminder message based on the analysis results of the emotion engine and sends a reminder notification to the device.

[0446] Step 6:

[0447] The device will display a reminder on the screen, telling the user, "It's time to start work."

[0448] Step 7:

[0449] When the user presses the confirmation button on the terminal, the terminal reports the operation to the server.

[0450] Step 8:

[0451] The server confirms that the user has pressed the confirmation button and records the confirmation information for the start of the activity in the database.

[0452] Bedtime notifications

[0453] Step 1:

[0454] The user sets the time they go to bed each night on the server.

[0455] Step 2:

[0456] The server stores the bedtime entered by the user in a database.

[0457] Step 3:

[0458] Thirty minutes before bedtime, the server prepares the bedtime notification.

[0459] Step 4:

[0460] The emotion engine analyzes the user's current emotional state.

[0461] Step 5:

[0462] The server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device.

[0463] Step 6:

[0464] The device displays the message "Bedtime in 30 minutes" on the screen, encouraging the user to get ready for bed.

[0465] Step 7:

[0466] When the user presses the prepare button, the terminal reports the operation to the server.

[0467] Step 8:

[0468] The server confirms that the user has pressed the prepare button and records the confirmation of preparation in the database.

[0469] Equipment shutdown

[0470] Step 1:

[0471] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[0472] Step 2:

[0473] The server stores the shutdown time entered by the user in a database.

[0474] Step 3:

[0475] Five minutes before the shutdown time, the server prepares for the shutdown notification.

[0476] Step 4:

[0477] When the shutdown time arrives, the server sends a shutdown signal to the target device.

[0478] Step 5:

[0479] The terminal receives the shutdown signal and initiates shutdown of the target device.

[0480] Step 6:

[0481] The terminal checks whether the shutdown of the target device has been completed successfully.

[0482] Step 7:

[0483] The terminal reports that the shutdown is complete to the server, and the server records the information in a database.

[0484] The above is a specific flow of processing steps in the embodiment of the present invention. By having the server, terminal, emotion engine, and user fulfill their respective roles, it becomes possible to maintain a healthy lifestyle.

[0485] Example 2

[0486] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0487] Conventional home device management systems have the ability to send notifications and reminders based on the user's schedule, but they have the problem of not being able to provide a personalized experience because they do not take the user's emotional state into consideration.In addition, they cannot appropriately adjust the content of notifications or alarm sounds depending on the user's emotional state, resulting in situations where users do not receive sufficient support or reminders.

[0488] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0489] In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time, means for checking the user's response to the alarm, means for setting a daytime activity time, means for sending a reminder at the set activity time, means for checking the user's response to the reminder, means for setting a bedtime, means for sending a notification urging the user to get ready for bed at the set bedtime, means for shutting down home appliances at bedtime, means for analyzing the user's emotional state in real time, and means for adjusting the notification content and alarm volume based on the user's emotional state. This makes it possible to provide personalized notifications and alarms according to the user's emotional state, thereby enabling the user's lifestyle to be managed more healthily and efficiently.

[0490] The "means for setting the wake-up time" is a function that allows the user to specify the wake-up time each morning and register it in the server.

[0491] The "means for sending an alarm at a set wake-up time" is a function that sends an alarm notification from the server to the terminal at a set time.

[0492] The "means for checking the user's response to the alarm" is a function that detects whether the user has responded to the alarm (for example, by pressing the wake-up button) and reports that information to the server.

[0493] The "means for setting daytime activity times" is a function that allows the user to specify specific activity times during the day and register them on the server.

[0494] The "means for sending a reminder at a set activity time" is a function for sending a reminder notification from the server to the terminal when the set activity time arrives.

[0495] The "means for confirming the user's response to the reminder" is a function that detects whether the user has responded to the reminder notification by pressing a confirmation button, and reports that information to the server.

[0496] The "means for setting bedtime" is a function that allows the user to specify the time they go to bed each night and register it in the server.

[0497] The "means for sending a notification prompting the user to get ready for bed at a set bedtime" is a function for sending a notification prompting the user to get ready for bed from the server to the terminal before the set bedtime.

[0498] The "means for shutting down devices in the home at bedtime" is a function that sends a shutdown signal from the server to specific devices in the home when the set bedtime arrives.

[0499] "Means for analyzing the user's emotional state in real time" is a function that analyzes the user's facial expressions, voice, etc. using emotion recognition technology to grasp the user's emotional state in real time.

[0500] "Means for adjusting notification content and alarm volume based on the user's emotional state" is a function for optimizing the content and volume of notifications and alarms based on the analysis of the user's emotional state.

[0501] The present invention relates to a home appliance management system that supports users in maintaining a healthy lifestyle. The system recognizes the user's emotional state in real time and adjusts notification content and alarm volume based on that information to provide more personalized support.

[0502] System configuration

[0503] The system of the present invention consists of four main components: a server, a terminal, an emotion engine, and a user.

[0504] The server manages the data of the user's set wake-up time, daytime activity time, and bedtime, and sends corresponding notifications to the device based on this data. Specifically, a cloud server such as Amazon EC2 or Google Cloud can be used.

[0505] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server. The terminal can be an iOS or Android device.

[0506] The emotion engine recognizes the user's emotions in real time and provides the analysis results to the server. Specifically, it uses emotion recognition APIs from IBM Watson (registered trademark) and Microsoft (registered trademark) Azure (registered trademark).

[0507] The user sets his or her own schedule in the server and responds to notifications from the terminal based on that schedule.

[0508] Wake-up time management

[0509] The user sets the wake-up time each morning on the server through a dedicated application. When the set time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device. The device then plays the set music and displays "It's time to wake up" on the screen. When the user presses the wake-up button, the device sends that information to the server, and the emotion engine analyzes and reports the user's emotions.

[0510] Examples:

[0511] For example, a user sets the server to wake up at 7:00 every morning. The server sends an alarm notification to the device at exactly 7:00, and the device sounds an appropriate alarm sound based on the results of the emotion engine. When the user presses the wake-up button, the device sends that information to the server, and the emotion engine analyzes and reports the user's emotions. An example of a prompt sentence is, "Write a program that sets the user to wake up at 7:00, and adjusts the alarm volume based on the analysis results of the emotion engine."

[0512] Daytime activity management

[0513] The user sets the time of daytime activities on the server through a dedicated app. For example, they can set "work hours from 9:00 AM to 12:00 PM." When it's time to start the activity, the server adjusts the reminder message based on the analysis results of the emotion engine and sends it to the device. The device displays the reminder, and when the user presses the confirmation button, it reports the operation to the server.

[0514] Examples:

[0515] For example, a user sets their working hours from 9:00 AM to 12:00 PM. At 9:00 AM, the server reflects the analysis results of the emotion engine and sends a reminder to the device containing an encouraging message saying, "It's time to start work." The device displays the reminder, and when the user presses the confirmation button, the information is sent to the server. An example of a prompt sentence is, "Write a program that sets the user's daytime activity hours as 9:00 AM to 12:00 PM and adjusts the reminder content based on the results of the emotion engine."

[0516] Bedtime notifications

[0517] The user sets their nightly bedtime on the server via a dedicated app. 30 minutes before their set bedtime, the server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device. The device displays the notification, and when the user presses the bedtime preparation button, the information is sent to the server.

[0518] Examples:

[0519] For example, a user sets the bedtime to 10 o'clock every night. At 9:30 p.m., the server reflects the analysis results of the emotion engine and sends a bedtime notification with a relaxing message to the device. The device displays the notification, and when the user presses the ready button, the information is sent to the server. An example of a prompt sentence is, "Write a program that sets the user's bedtime to 10 o'clock and plays calm music based on the results of the emotion engine."

[0520] Shut down household devices

[0521] The user sets the shutdown time for a specific device in the home on the server through a dedicated application. When the shutdown time arrives, the server sends a shutdown signal to the target device, and the device receives the signal and starts shutting down the device. The device confirms that the device has been shut down successfully and reports the status to the server.

[0522] Examples:

[0523] For example, a user sets the TV and Wi-Fi router in their home to shut down at 10 o'clock. At 10 pm, the server sends a shutdown signal to the TV and Wi-Fi router. The device confirms that these devices have shut down successfully and reports the results to the server. An example of a prompt sentence is, "Write a program that shuts down the TV and Wi-Fi router in your home every night at 10 o'clock. Please also add a function to report the results to the server."

[0524] As described above, the system of the present invention not only manages household appliances but also provides more personalized lifestyle support by taking into account the emotional state of the user.

[0525] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0526] Program processing steps

[0527] Wake-up time management

[0528] Step 1:

[0529] The user sets the wake-up time on the server.

[0530] Input: User's wake-up time (e.g. 7:00)

[0531] Specific operation: The user enters the wake-up time in a dedicated application and registers it on the server.

[0532] Output: The wake-up time data is saved on the server.

[0533] Step 2:

[0534] The server prepares the notification based on the configured wake-up time.

[0535] Input: Wake-up time stored on the server

[0536] Specific operation: The server prepares an alarm notification based on the set wake-up time (e.g., 7:00).

[0537] Output: Alarm notification is ready.

[0538] Step 3:

[0539] An emotion engine analyzes the user's emotional state.

[0540] Input: Real-time user data (face recognition, voice data, etc.)

[0541] Specific operation: The emotion engine analyzes the user's face and voice through the camera and microphone to analyze their emotional state.

[0542] Output: Emotional state data (e.g., relaxed state)

[0543] Step 4:

[0544] The server determines the optimal alarm volume and tone and sends a notification to the device.

[0545] Input: Emotional state data, saved wake-up time

[0546] Specific operation: Based on the results of the emotion engine, the server selects the optimal alarm volume and tone and sends an alarm notification to the device.

[0547] Output: An alarm notification is sent to the device.

[0548] Step 5:

[0549] Your device will sound an alarm and display a wake-up notification.

[0550] Input: Alarm notification

[0551] Specific actions: The device receives a notification, plays the set music, and displays a wake-up time notification on the screen.

[0552] Output: An alarm will sound and a wake-up notification will appear on the screen.

[0553] Step 6:

[0554] The user presses the wake-up button.

[0555] Input: User button presses

[0556] Specific operation: The user presses the alarm stop button on the terminal.

[0557] Output: The alarm stops and operation data is generated on the terminal.

[0558] Step 7:

[0559] The terminal reports the user's operations to the server.

[0560] Input: Operation data (alarm stop button pressed)

[0561] Specific operation: The terminal transmits the user's button operation information to the server.

[0562] Output: Operation information is reported to the server.

[0563] Step 8:

[0564] The emotion engine analyzes the user's emotional state when they wake up in real time and provides the information to the server.

[0565] Input: Real-time user data (face recognition, voice data, etc.)

[0566] Specific operation: The emotion engine re-analyzes the user's emotions upon waking up and sends the results to the server.

[0567] Output: The emotional state data is provided to the server.

[0568] Daytime activity management

[0569] Step 1:

[0570] The user sets the daytime activity times on the server.

[0571] Input: User activity time (e.g. 9:00 to 12:00)

[0572] Specific operation: The user inputs the activity time in a dedicated application and registers it on the server.

[0573] Output: Activity time data is saved on the server.

[0574] Step 2:

[0575] The server prepares notifications based on configured active times.

[0576] Input: Activity time stored on the server

[0577] Specific behavior: The server prepares a reminder notification based on the set activity time (e.g. 9:00).

[0578] Output: Reminder notification is ready.

[0579] Step 3:

[0580] The emotion engine analyzes the user's emotional state in real time during activity.

[0581] Input: Real-time user data (face recognition, voice data, etc.)

[0582] Specific operation: The emotion engine analyzes the user's facial expressions and voice to analyze the user's emotional state in real time during an activity.

[0583] Output: Emotional state data during the activity

[0584] Step 4:

[0585] The server determines the optimal message content and sends a notification to the device.

[0586] Input: Emotional state data during activity, saved activity time

[0587] Specific operation: Based on the results of the emotion engine, the server selects a message, such as "Good luck," and sends a reminder notification to the device.

[0588] Output: A reminder notification is sent to your device.

[0589] Step 5:

[0590] Your device will display the reminder.

[0591] Input: Reminder

[0592] Specific behavior: The device receives a notification and displays something like, "It's 9 o'clock, time to get to work."

[0593] Output: The reminder will be displayed on your device.

[0594] Step 6:

[0595] The user presses the confirm button.

[0596] Input: User confirmation button operation

[0597] Specific operation: The user presses the confirmation button in response to the reminder notification.

[0598] Output: Operation data is generated when the confirmation button is pressed.

[0599] Step 7:

[0600] The terminal reports the user's operations to the server.

[0601] Input: Operation data (Confirm button pressed)

[0602] Specific operation: The terminal transmits the user's confirmation button operation information to the server.

[0603] Output: Operation information is reported to the server.

[0604] Bedtime notifications

[0605] Step 1:

[0606] The user sets the bedtime on the server.

[0607] Input: User's bedtime (e.g. 10:00)

[0608] Specific operation: The user enters the bedtime in a dedicated application and registers it on the server.

[0609] Output: Bedtime data is saved on the server.

[0610] Step 2:

[0611] The server prepares a notification before bedtime.

[0612] Input: Bedtime stored on the server

[0613] Specific operation: The server prepares a bedtime notification based on the set time 30 minutes before bedtime (e.g. 9:30).

[0614] Output: Bedtime notification is ready.

[0615] Step 3:

[0616] An emotion engine analyzes the user's emotional state.

[0617] Input: Real-time user data (face recognition, voice data, etc.)

[0618] Specific operation: The emotion engine analyzes the user's face and voice to analyze their emotional state before going to sleep.

[0619] Output: Emotional state data before going to bed

[0620] Step 4:

[0621] The server determines the optimal notification content and sends it to the device.

[0622] Input: emotional state data before bedtime, saved bedtime

[0623] Specific operation: Based on the results of the emotion engine, the server selects calm music if the user is in a relaxed state and sends a notification to the device to prepare for bed.

[0624] Output: A bedtime notification is sent to the device.

[0625] Step 5:

[0626] Your device will display a bedtime notification.

[0627] Input: Bedtime notification

[0628] Specific behavior: The device receives a notification and displays "Start getting ready for bed."

[0629] Output: A bedtime notification will appear on your device.

[0630] Step 6:

[0631] The user presses the ready for bed button.

[0632] Input: User confirmation button operation

[0633] Specific Action: The user presses the Ready for Bed button.

[0634] Output: Operation data is generated when the ready button is pressed.

[0635] Step 7:

[0636] The terminal reports the user's operations to the server.

[0637] Input: Operation data (Ready button pressed)

[0638] Specific operation: The terminal sends the user's preparation completion button operation information to the server.

[0639] Output: Operation information is reported to the server.

[0640] Shut down household devices

[0641] Step 1:

[0642] The user sets the shutdown time of the home appliances in the server.

[0643] Input: User shutdown time (e.g. 10:00)

[0644] Specific operation: The user enters the shutdown time in a dedicated application and registers it on the server.

[0645] Output: Shutdown time data is saved on the server.

[0646] Step 2:

[0647] The server prepares a notification when it is time to shut down.

[0648] Input: Shutdown time stored on the server

[0649] Specific operation: The server prepares a shutdown notification based on the configured shutdown time (e.g., 10:00).

[0650] Output: Shutdown notification is ready.

[0651] Step 3:

[0652] The server sends a shutdown signal to the home appliance.

[0653] Input: Shutdown notification

[0654] Specific operation: When the set shutdown time arrives, the server sends a shutdown signal to the specified home devices (TVs and Wi-Fi routers).

[0655] Output: A shutdown signal is sent to the home appliance.

[0656] Step 4:

[0657] The terminal initiates a shutdown of the target device.

[0658] Input: Shutdown signal

[0659] Specific operation: The terminal receives the shutdown signal and starts the shutdown process of the specified device.

[0660] Output: Equipment shutdown is initiated.

[0661] Step 5:

[0662] The terminal confirms that the device has been shut down and reports this to the server.

[0663] Input: Device shutdown state

[0664] Specific operation: The device checks whether the devices have been shut down successfully and reports the result to the server, for example, notifying the server that "The TV and Wi-Fi router have been shut down successfully."

[0665] Output: The shutdown state is reported to the server.

[0666] (Application example 2)

[0667] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0668] Conventional home appliance management systems send uniform notifications and alarms without considering the user's emotions, resulting in a lack of personalization tailored to the user's psychological state. Furthermore, in-store shopping experiences are also challenged by the inability to provide support tailored to the individual user's emotional state, which can easily lead to dissatisfaction, especially when the user's emotions are unstable.

[0669] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time and adjusting the alarm sound based on the emotion analysis result, and means for checking the user's response to the alarm and sending the emotion analysis result to the server. This makes it possible to provide personalized alarm notifications and daytime activity reminders according to the user's emotional state.

[0670] The system also includes a means for providing personalized navigation and product recommendations based on the user's emotional state in a physical store, thereby enabling optimal support according to the user's emotions even in the physical store, and providing a more satisfying shopping experience.

[0671] The "means for setting a wake-up time" is a function that allows the user to set in advance the wake-up time each morning.

[0672] "Means for sending alarms" is a function that notifies you by voice or visually at a set time.

[0673] "Emotion analysis results" are the results of analyzing the user's emotional state in real time from their face and behavior, and providing the results as data.

[0674] "Means for adjusting alarm sound" is a function that selects the optimal volume and tone based on the results of emotion analysis.

[0675] The "means for checking the user's response to the alarm" is a function that detects the operation the user has performed on the alarm (such as snooze or stop) and collects that information.

[0676] The "means for setting daytime activity times" is a function that allows the user to set specific activity times (e.g., work time or exercise time).

[0677] The "means for sending a reminder" is a function for sending a reminder notification at a set activity time and notifying the user of the same.

[0678] The "means for adjusting reminder content" is a function that optimizes the content of reminder messages and notifications based on the user's emotional state.

[0679] The "means for setting bedtime" is a function that allows the user to set the time they go to bed each night.

[0680] The "means for sending a notification to urge the user to get ready for bed" is a function that sends a notification to urge the user to get ready for bed when the set bedtime approaches.

[0681] "Means for shutting down household devices" refers to a function that turns off household devices such as televisions and Wi-Fi routers at bedtime.

[0682] The "emotion analysis means" is a function for recognizing and analyzing the user's emotional state in real time and providing the results to the system.

[0683] "Means for personalized navigation and product recommendations" refers to a function that recommends optimal routes and products based on the user's emotional state within a physical store.

[0684] The present invention is a system for personalizing user experiences in home devices and brick-and-mortar stores using emotion analysis. The system consists of four main components: a server, a terminal, an emotion analysis means, and a user.

[0685] System configuration

[0686] 1. Server

[0687] The server manages schedule data set by the user, such as wake-up time, daytime activity times, and bedtime, and also collects emotional data provided by an emotion analysis tool, optimizing various notifications and recommendations based on this data.

[0688] 2. Terminal

[0689] The device can be a smartphone, smart glasses, or a head-mounted display (HMD), and it displays notifications to the user, collects the user's responses, and sends them to a server. It also provides an interface for performing actions such as shutting down home appliances.

[0690] 3. Emotion analysis method

[0691] Emotion analyzers are software that uses cameras and sensors to capture a user's face and behavior, and analyzes them to identify their emotional state, including the Microsoft Azure Emotion API and Affectiva SDK.

[0692] 4. Users

[0693] Users set their own schedules on the server and receive notifications from their devices based on those schedules. In physical stores, users can also use navigation and product recommendations based on their emotional state.

[0694] Program processing

[0695] Hardware / Software used

[0696] Smartphone (iOS or Android)

[0697] Emotion analysis software (Microsoft Azure Emotion API, Affectiva SDK)

[0698] Server (AWS Lambda + DynamoDB)

[0699] UI framework (React Native)

[0700] Data processing and calculation

[0701] The server collects and manages user settings and emotion analysis data, and determines optimized alarm sounds and reminder content based on this data.

[0702] The emotion analysis means analyzes the emotions of the user from their face and actions in real time, and sends the results to the server as data.

[0703] The device receives and displays notifications from the server, reports user responses, and, in physical stores, displays data for navigation and product recommendations.

[0704] Specific examples

[0705] 1. Manage your wake-up time

[0706] The user sets the time to wake up at 7am every morning.

[0707] At 7:00, the server determines the optimal alarm sound based on the emotion analysis results and sends a notification to the device.

[0708] The terminal sounds an alarm, and when the user presses the wake-up button, the information is sent to the server, and the emotion analysis means analyzes the emotion and provides the result to the server.

[0709] 2. Daytime activity management

[0710] A user sets their work hours as 9:00 AM to 12:00 PM.

[0711] At 9 a.m., the server uses the results of emotion analysis to send a reminder to the device, including the encouraging message, "It's time to get to work."

[0712] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[0713] 3. Bedtime notifications

[0714] The user sets the bedtime to 10 o'clock every night.

[0715] At 9:30 p.m., the server uses the results of the emotion analysis to send a bedtime notification to the device, including a relaxing message.

[0716] The device will display a notification to remind the user to get ready for bed.

[0717] 4. Shut down the device

[0718] A user sets the TV and Wi-Fi router in their home to shut down at 10:00.

[0719] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[0720] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[0721] Example prompts to input to the generative AI model

[0722] "The user's current emotional state is joy. If the emotion of joy is high, what is the next recommended product in the store?"

[0723] "If a user's emotional state is low, what message would be effective in encouraging them?"

[0724] By utilizing sentiment analysis, this system significantly personalizes users' lifestyles and in-store experiences, helping them live richer lives.

[0725] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0726] Step 1:

[0727] The user sets the wake-up time using a smartphone device. The input is the wake-up time entered by the user into the app, and the output is the wake-up time data stored on the server. The wake-up time set on the smartphone app is sent to the server and stored in a database.

[0728] Step 2:

[0729] When the set wake-up time arrives, the server requests the emotion analysis means to capture the user's face and behavior. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analysis means captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[0730] Step 3:

[0731] The server determines the optimal alarm volume and tone based on the acquired emotional data. The input is the emotional analysis data, and the output is the alarm sound setting. The server adjusts the alarm sound according to the emotional state and sends the setting to the device.

[0732] Step 4:

[0733] The device plays the adjusted alarm sound to prompt the user to wake up. The input is the alarm sound setting sent from the server, and the output is the alarm sound and wake-up notification. The device waits for the user to press the wake-up button.

[0734] Step 5:

[0735] When a user presses the wake-up button, the device reports the operation to the server. The input is the user pressing the wake-up button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with the emotion data at the time of waking up.

[0736] Step 6:

[0737] The user sets the activity time during the day (e.g., work hours). The input is the activity time entered by the user into the app, and the output is the activity time data stored on the server. The activity time set on the smartphone app is sent to the server and stored in the database.

[0738] Step 7:

[0739] When the set activity time arrives, the server requests the user's emotional data from the emotion analysis means. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analysis means captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[0740] Step 8:

[0741] The server generates an appropriate reminder message based on the acquired emotional data. The input is the emotion analysis data, and the output is the reminder message. The server optimizes the reminder content according to the emotional state and sends the message to the device.

[0742] Step 9:

[0743] The terminal displays the generated reminder message and prompts the user to start the activity. The input is the reminder message sent from the server, and the output is the reminder notification. The terminal waits for the user to press the confirmation button.

[0744] Step 10:

[0745] When the user presses the confirmation button, the device reports the operation to the server. The input is the user pressing the confirmation button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with the emotion data during the activity.

[0746] Step 11:

[0747] The user sets a bedtime. The input is the bedtime entered by the user into the app, and the output is the bedtime data stored on the server. The bedtime set on the smartphone app is sent to the server and stored in a database.

[0748] Step 12:

[0749] Thirty minutes before bedtime, the server requests the user's emotional data from the emotion analyzer. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analyzer captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[0750] Step 13:

[0751] The server generates an appropriate bedtime notification based on the acquired emotional data. The input is the emotion analysis data, and the output is the bedtime notification. The server optimizes the notification content according to the emotional state and sends the message to the device.

[0752] Step 14:

[0753] The terminal displays the generated bedtime notification and prompts the user to get ready for bed. The input is the bedtime notification sent from the server, and the output is the bedtime notification. The terminal waits for the user to press the bedtime button.

[0754] Step 15:

[0755] When a user presses the "Get ready for bed" button, the device reports the operation to the server. The input is the user pressing the "Get ready for bed" button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with emotional data at bedtime.

[0756] Step 16:

[0757] When the set bedtime arrives, the server sends a shutdown signal to home devices. The input is the set shutdown time, and the output is the shutdown signal. The server sends a signal to home devices such as TVs and Wi-Fi routers to initiate shutdown.

[0758] Step 17:

[0759] The terminal confirms receipt of the shutdown signal and checks whether the shutdown of the home device has been completed successfully. The input is the shutdown signal sent from the server, and the output is the shutdown confirmation result. Once the terminal confirms that the shutdown has been completed, it reports the result to the server.

[0760] This series of steps will enable the realization of a system that provides personalized support according to the user's emotional state, thereby enriching daily life and shopping experiences.

[0761] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0762] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0763] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0764] [Second embodiment]

[0765] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0766] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0767] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0768] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0769] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0770] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0771] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0772] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0773] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0774] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0775] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0776] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0777] The present invention relates to a home appliance management system for leading a healthy lifestyle.

[0778] System configuration

[0779] The system of the present invention consists of three main components: a server, a terminal, and a user.

[0780] The server manages the data of the user's set wake-up time, activity time, and bedtime, and is responsible for sending corresponding notifications to the device based on this data.

[0781] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server.

[0782] The user sets his or her own schedule on the server and responds to notifications from the terminal based on that setting.

[0783] Program processing

[0784] The processing of the program of the present invention will be explained in natural language below.

[0785] 1. Manage your wake-up time

[0786] The user sets the wake-up time each morning on the server.

[0787] The server sends an alarm notification to the terminal at the set time.

[0788] The device will sound an alarm and continue to notify you until the user presses the wake-up button.

[0789] When the user presses the button, the terminal sends wake-up confirmation data to the server.

[0790] 2. Daytime activity management

[0791] The user sets the daytime activity times on the server (for example, 9:00 AM to 12:00 PM).

[0792] The server sends a reminder to the device when it is time to start the activity.

[0793] The device will display a reminder and continue to notify you until you press the confirmation button.

[0794] When the user presses the confirmation button, the terminal transmits confirmation data for the start of the activity to the server.

[0795] 3. Bedtime notifications

[0796] The user sets the time they go to bed each night on the server.

[0797] The server sends a notification to the device to prepare for bed 30 minutes before going to bed.

[0798] The device will display a notification to remind the user to get ready for bed.

[0799] 4. Shut down the device

[0800] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[0801] The server sends a shutdown signal to these devices at the set time.

[0802] The terminal checks whether the device has been shut down normally and reports the result to the server.

[0803] Specific examples

[0804] 1. Manage your wake-up time

[0805] For example, a user configures the server to wake up at 7am every morning.

[0806] The server sends an alarm notification to the device at exactly 7:00, and the device sounds an alarm.

[0807] When the user presses the wake-up button, the terminal transmits the information to the server to confirm the wake-up.

[0808] 2. Daytime activity management

[0809] For example, a user may set his working hours from 9:00 AM to 12:00 PM.

[0810] At 9:00 AM, the server sends a reminder to the device saying, "It's time to get to work."

[0811] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[0812] 3. Bedtime notifications

[0813] For example, a user sets the time to go to bed at 10 o'clock every night.

[0814] At 9:30 p.m., the server sends a notification to the device saying, "It's bedtime in 30 minutes."

[0815] The device will display a notification to remind the user to get ready for bed.

[0816] 4. Shut down the device

[0817] For example, a user can set the TV and Wi-Fi router in their home to shut down at 10:00.

[0818] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[0819] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[0820] As described above, the system of the present invention manages the user's life cycle according to a set time and provides a specific means for supporting a healthy lifestyle.

[0821] The processing flow will be explained below.

[0822] Wake-up time management

[0823] Step 1:

[0824] The user logs into the server and sets the time to wake up each morning.

[0825] Step 2:

[0826] The server stores the wake-up time entered by the user in a database.

[0827] Step 3:

[0828] Five minutes before the set wake-up time, the server prepares for the alarm notification.

[0829] Step 4:

[0830] When the set wake-up time arrives, the server sends an alarm notification to the terminal.

[0831] Step 5:

[0832] The device will sound an alarm and display a wake-up notification on the screen.

[0833] Step 6:

[0834] When the user presses the wake-up button on the terminal, the terminal reports the operation to the server.

[0835] Step 7:

[0836] The server confirms that the user has pressed the wake-up button and records the wake-up confirmation information in the database.

[0837] Daytime activity management

[0838] Step 1:

[0839] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[0840] Step 2:

[0841] The server stores the activity time entered by the user in a database.

[0842] Step 3:

[0843] Five minutes before the activity start time, the server prepares the reminder notification.

[0844] Step 4:

[0845] When the activity start time arrives, the server sends a reminder notification to the device.

[0846] Step 5:

[0847] The device will display a reminder on the screen, telling the user, "It's time to start work."

[0848] Step 6:

[0849] When the user presses the confirmation button on the terminal, the terminal reports the operation to the server.

[0850] Step 7:

[0851] The server confirms that the user has pressed the confirmation button and records the confirmation information for the start of the activity in the database.

[0852] Bedtime notifications

[0853] Step 1:

[0854] The user sets the time they go to bed each night on the server.

[0855] Step 2:

[0856] The server stores the bedtime entered by the user in a database.

[0857] Step 3:

[0858] Thirty minutes before bedtime, the server prepares the bedtime notification.

[0859] Step 4:

[0860] Thirty minutes before bedtime, the server sends a notification to the device to prepare for bed.

[0861] Step 5:

[0862] The device displays the message "Bedtime in 30 minutes" on the screen, encouraging the user to get ready for bed.

[0863] Step 6:

[0864] When the user presses the prepare button, the terminal reports the operation to the server.

[0865] Step 7:

[0866] The server confirms that the user has pressed the prepare button and records the confirmation of preparation in the database.

[0867] Equipment shutdown

[0868] Step 1:

[0869] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[0870] Step 2:

[0871] The server stores the shutdown time entered by the user in a database.

[0872] Step 3:

[0873] Five minutes before the shutdown time, the server prepares for the shutdown notification.

[0874] Step 4:

[0875] When the set shutdown time arrives, the server sends a shutdown signal to the target device.

[0876] Step 5:

[0877] The terminal receives the shutdown signal and initiates shutdown of the target device.

[0878] Step 6:

[0879] The terminal checks whether the shutdown of the target device has been completed successfully.

[0880] Step 7:

[0881] The terminal reports that the shutdown is complete to the server, and the server records the information in a database.

[0882] Example 1

[0883] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0884] Maintaining a regular lifestyle is important for improving health and productivity. However, in today's busy lives, daily time management is becoming increasingly difficult. Conventional methods require users to manually manage their time, including waking up, daily activities, going to bed, and managing home devices, which can be cumbersome. This creates a particularly difficult problem for maintaining a healthy lifestyle.

[0885] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0886] In this invention, the server includes means for a user to set a wake-up time, means for sending an alarm when the set wake-up time arrives, means for the terminal to sound the alarm and confirm the user's response, means for a user to set a daytime activity time, means for sending a reminder at the set activity time, means for the terminal to display the reminder and confirm the user's response, means for a user to set a bedtime, means for sending a notification urging the user to get ready for bed 30 minutes before the set bedtime, means for a user to set a shutdown time for home appliances, and means for sending a shutdown signal to the home appliances at the set shutdown time. This makes it possible to automatically manage a user's lifestyle via the server.

[0887] "User" refers to an individual who uses this system.

[0888] "Server" refers to a computer system that records and manages user settings and transmits each piece of information to the terminal.

[0889] "Terminal" refers to a device that displays notifications and reminders from the server to the user and receives the user's responses.

[0890] "Means for setting a wake-up time" refers to the specific functions and methods by which a user inputs and sets the wake-up time each morning.

[0891] "Means for sending an alarm" refers to a specific function or method for sending an alarm notification from the server to the terminal at the set wake-up time.

[0892] The term "means for the terminal to sound an alarm and confirm the user's response" refers to a function or method for the terminal to sound an alarm and receive a user's wake-up confirmation operation.

[0893] The "means for setting daytime activity times" refers to a specific function or method for a user to set a specific time period as a daytime activity time.

[0894] "Means for sending reminders" refers to specific functions and methods for sending reminder notifications from the server to the device when the set activity time arrives.

[0895] The "means for the terminal to display the reminder and confirm the user's response" refers to a function or method for the terminal to display the reminder and receive a confirmation operation from the user.

[0896] "Means for setting bedtime" refers to the specific functions and methods by which a user can input and set their nightly bedtime.

[0897] "Means for sending a notification encouraging users to get ready for bed" refers to the specific functions and methods for sending a notification from the server to the device prompting users to get ready for bed 30 minutes before the set bedtime.

[0898] "Means for setting the shutdown time of devices in the home" refers to specific functions and methods that allow a user to input and set the shutdown time of a specific device in the home.

[0899] "Means for sending a shutdown signal to a device in the home" refers to a specific function or method for sending a shutdown signal from the server to a specific device in the home when the set shutdown time arrives.

[0900] "Means for confirming that home devices have been shut down normally" refers to specific functions and methods that allow a terminal to check the shutdown status of home devices and report the results to a server.

[0901] This invention relates to a home appliance management system for promoting a healthy lifestyle. This system consists of three main components: a server, a terminal, and a user. Below, we will explain in detail how each component operates, including the hardware and software used.

[0902] Server Roles

[0903] The server manages various schedules set by the user (wake-up time, activity time, bedtime, shutdown time of home devices) and sends appropriate notifications to the device based on those schedules. The server records the user's setting data in a database (e.g., MySQL), and generates corresponding notifications at specific times and sends them to the device. For example, the server sends an alarm notification to the device when the set wake-up time (e.g., 7:00 AM) arrives.

[0904] Device Role

[0905] The device is responsible for displaying notifications sent from the server (such as alarms, reminders, and bedtime notifications) to the user and receiving the user's responses. The device also has the function of reporting the user's responses (such as pressing a button) to the server. For example, if the device sounds an alarm and the user presses the "wake-up button," that information is sent to the server.

[0906] User Roles

[0907] Users set various schedules on the server using a smartphone app, etc. This allows the server to generate and manage notifications that fit the user's lifestyle. For example, if a user sets the server as their active time from 9:00 AM to 12:00 PM, the server will send a reminder to the device at 9:00 AM.

[0908] Specific examples

[0909] Consider a scenario where a user wakes up at 7am every morning, sets their work hours from 9am to 12pm, goes to bed at 10pm every night, and shuts down the TV and Wi-Fi router in their home at the same time.

[0910] 1. Manage your wake-up time:

[0911] The user opens the smartphone app and sets the wake-up time to 7:00 AM every morning. The server receives the wake-up time data (7:00 AM) from the user and records it in a database. When the set time of 7:00 AM arrives, the server sends an alarm notification to the device, which then sounds the alarm continuously. When the user taps the "wake-up button," the device sends "wake-up confirmation data" to the server, which records it in the database.

[0912] 2. Daytime activity management:

[0913] The user sets the working hours from 9:00 AM to 12:00 PM. The server records the received activity time data (9:00 AM - 12:00 PM) in the database. At 9:00 AM, the server sends a reminder notification to the device saying "It's time to start work." The device displays the reminder, and when the user taps the "confirm button," "activity start confirmation data" is sent to the server, which records it in the database.

[0914] 3. Bedtime notifications:

[0915] The user sets the bedtime to 10 PM every night. The server receives the bedtime data (10:00 PM) and records it in the database. At 9:30 PM, the server sends a notification to the device saying, "It's 30 minutes until bedtime." The device continues to display the notification, encouraging the user to get ready for bed.

[0916] 4. Shutdown the equipment:

[0917] A user sets the shutdown time for specific devices in the home (TV and Wi-Fi router). The server records the received shutdown time data (10:00 PM) in a database. At 10 PM, the server sends a shutdown signal to these devices. The terminal checks whether these devices have been shut down successfully and reports the results to the server. The server records this information in a database.

[0918] Prompt Sentence Examples

[0919] Here are some examples of prompts:

[0920] 1. Explain specific examples of managing your wake-up time:

[0921] Let's say a user configures the system to wake up at 7:00 every morning. The server sends a notification at exactly 7:00, and the device continues to ring the alarm. When the user presses the wake-up button, that information is sent to the server.

[0922] 2. Describe specific examples of daytime activity management:

[0923] If a user sets their active time from 9:00 AM to 12:00 PM, the server will send a reminder at 9:00 AM and the device will display the reminder. When the user presses the confirmation button, the information is sent to the server.

[0924] 3. Explain specific examples of bedtime notifications:

[0925] For users who have set their bedtime schedule to 10 p.m. every night, the server will send a notification at 9:30 p.m. and the device will display the message "Bedtime in 30 minutes."

[0926] 4. Describe an example of equipment shutdown:

[0927] If the TV and Wi-Fi router in the home are set to shut down at 10 PM, the server will send a shutdown signal to these devices at 10 PM. The device will check to see if the devices have shut down and report the results to the server.

[0928] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0929] Wake-up time management

[0930] Step 1:

[0931] The user uses a smartphone app to set the time they wake up each morning.

[0932] Input: Wake-up time (e.g. 7:00 AM)

[0933] Output: Configuration data sent to the server

[0934] Specific operation: The user enters the wake-up time on the app's settings screen and presses the Set button.

[0935] Step 2:

[0936] The server records the received wake-up time data in a database.

[0937] Input: Wake-up time data sent by the user

[0938] Output: Wake-up time data recorded in a database

[0939] Specific operation: The server executes a query to store the wake-up time in a database (e.g. MySQL).

[0940] Step 3:

[0941] When the server reaches the set wake-up time, it sends an alarm notification to the terminal.

[0942] Input: Current time and wake-up time data recorded in the database

[0943] Output: Alarm notification is sent to the device

[0944] Specific operation: The server detects that a specific time has arrived, generates an alarm notification, and sends a push notification to the device.

[0945] Step 4:

[0946] The device will sound an alarm and continue to notify you.

[0947] Input: Alarm notification sent from the server

[0948] Output: The device continues to ring

[0949] Specific behavior: The device will start playing a sound file and display a "wake-up button" on the screen.

[0950] Step 5:

[0951] The user presses the wake-up button.

[0952] Input: User presses a button

[0953] Output: Wake-up confirmation data is generated

[0954] Specific action: The user taps the "wake-up button" on the device.

[0955] Step 6:

[0956] The terminal transmits wake-up confirmation data to the server.

[0957] Input: User button press information

[0958] Output: Wake-up confirmation data is sent to the server and recorded in the database

[0959] Specific operation: The device sends an HTTP request to the server, and the server stores the wake-up confirmation data in a database.

[0960] Daytime activity management

[0961] Step 1:

[0962] The user sets the time for daytime activity using a smartphone app.

[0963] Input: Activity time (e.g. 9:00 AM - 12:00 PM)

[0964] Output: Configuration data sent to the server

[0965] Specific operation: The user enters the activity time on the app's settings screen and presses the settings button.

[0966] Step 2:

[0967] The server records the received activity time data in a database.

[0968] Input: Activity time data sent by the user

[0969] Output: Activity time data recorded in a database

[0970] Specific operation: The server executes a query to store the activity time in the database.

[0971] Step 3:

[0972] The server sends a reminder to the device at the start time of the activity.

[0973] Input: Current time and activity time data recorded in the database

[0974] Output: Reminder notification sent to device

[0975] Specific operation: The server detects when a specific time has arrived, generates a reminder notification, and sends it as a push notification to the device.

[0976] Step 4:

[0977] Your device will display reminders and keep you informed.

[0978] Input: Reminder notification sent from the server

[0979] Output: The device continues to display the reminder

[0980] Specific behavior: The device will display a reminder notification and a "Confirm" button on the screen.

[0981] Step 5:

[0982] The user presses the confirm button.

[0983] Input: User presses a button

[0984] Output: Activity start confirmation data is generated

[0985] Specific action: The user taps the "confirmation button" on the device.

[0986] Step 6:

[0987] The terminal transmits activity start confirmation data to the server.

[0988] Input: User button press information

[0989] Output: Activity start confirmation data is sent to the server and recorded in the database

[0990] Specific operation: The terminal sends an HTTP request to the server, and the server stores the activity start confirmation data in the database.

[0991] Bedtime notifications

[0992] Step 1:

[0993] The user uses a smartphone app to set their bedtime each night.

[0994] Input: Bedtime (e.g. 10:00 PM)

[0995] Output: Configuration data sent to the server

[0996] Specific behavior: The user enters bedtime on the app's settings screen and presses the Set button.

[0997] Step 2:

[0998] The server records the received bedtime data in a database.

[0999] Input: Bedtime data submitted by the user

[1000] Output: Bedtime data recorded in a database

[1001] What happens: The server runs a query to store the bedtime in the database.

[1002] Step 3:

[1003] The server sends a bedtime preparation notification to the device 30 minutes before bedtime.

[1004] Input: Current time and bedtime data recorded in the database

[1005] Output: Bedtime notification sent to device

[1006] Specific operation: The server detects when a specific time has arrived, generates a bedtime preparation notification, and sends it as a push notification to the device.

[1007] Step 4:

[1008] The device displays a notification to remind the user to get ready for bed.

[1009] Input: Bedtime notification sent from the server

[1010] Output: The device keeps displaying the bedtime notification

[1011] What it does: Your device will display a bedtime notification and display "Start getting ready for bed" on the screen.

[1012] Equipment shutdown

[1013] Step 1:

[1014] A user uses a smartphone app to set shutdown times for specific devices in the home.

[1015] Input: Shutdown time (e.g. 10:00 PM)

[1016] Output: Configuration data sent to the server

[1017] Specific behavior: The user enters the shutdown time on the app's settings screen and presses the Set button.

[1018] Step 2:

[1019] The shutdown time data received by the server is recorded in a database.

[1020] Input: Shutdown time data sent by the user

[1021] Output: Shutdown time data recorded in the database

[1022] What happens: The server runs a query to store the shutdown time in the database.

[1023] Step 3:

[1024] The server transmits a shutdown signal to a specific device in the home at a specified time.

[1025] Input: Current time and shutdown time data recorded in the database

[1026] Output: A shutdown signal is sent to the device

[1027] Specific operation: The server detects that a specific time has arrived, generates a shutdown signal, and sends it to the home appliance control system (e.g., IoT device).

[1028] Step 4:

[1029] The terminal checks whether the device has been shut down normally and reports the result to the server.

[1030] Input: Device response data to shutdown signal

[1031] Output: Shutdown confirmation data is sent to the server and recorded in the database.

[1032] Specific operation: The terminal checks the device status and reports to the server that the shutdown is complete.

[1033] (Application example 1)

[1034] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1035] Conventional lifestyle management in homes and cars is often performed separately, lacking integrated management. In particular, in autonomous vehicles, time management for users while driving or napping is difficult, and methods for realizing a comfortable and efficient lifestyle are required. Appropriate shutdown management of in-car entertainment systems and lighting is also required.

[1036] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1037] In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time, means for confirming a user's response to the alarm, means for setting a daytime activity time, means for sending a reminder at the set activity time, means for confirming a user's response to the reminder, means for setting a bedtime, means for sending a notification encouraging the user to get ready for bed at the set bedtime, means for shutting down home appliances at bedtime, means for sending specified reminders and notifications for lifestyle cycle management in the autonomous vehicle, and means for shutting down the entertainment system and lighting in the autonomous vehicle at set times, thereby enabling integrated and efficient lifestyle cycle management at home and in the autonomous vehicle.

[1038] A "wake-up time" is a specific time that a user sets to wake up.

[1039] An "alarm" is a means of sending a notification to the user at a set wake-up time to prompt them to wake up.

[1040] The "user's response to the alarm" is an action that the user takes in response to the alarm notification, such as pressing a button or using voice recognition to confirm that the user is awake.

[1041] The "daytime activity time" is a set time indicating the activities (for example, work or exercise) that the user should perform within a certain period of time.

[1042] A "reminder" is a notification means that prompts the user to start an activity at a set time.

[1043] A "user's response to a reminder" is a confirmation action that a user takes in response to a reminder notification.

[1044] A "bedtime" is a specific time that a user sets for going to bed.

[1045] The "notification to urge the user to get ready for bed" is a notification means for urging the user to get ready for bed prior to the set bedtime.

[1046] The "means for shutting down devices" is a means for automatically shutting down specific devices in a home or car at a set time.

[1047] "In-vehicle entertainment system" refers to devices and software that allow users to enjoy entertainment in an autonomous vehicle.

[1048] "Lighting" refers to a lighting device for providing brightness inside an autonomous vehicle.

[1049] This invention is a system for managing a life cycle in a home and an autonomous vehicle to lead a healthy life cycle. This system consists of three main components: a server, a terminal, and a user.

[1050] The server manages the data of the wake-up time, activity time, and bedtime set by the user, and sends corresponding notifications to the device based on this. For example, if the user sets the server to wake up at 7:00 every morning, the server will send an alarm notification to the device at exactly 7:00, and the device will ring the alarm.

[1051] The device is responsible for displaying notifications to the user, receiving the user's response (such as pressing a button), and reporting it to the server. It can be implemented as a smartphone, tablet, or a dedicated device in an autonomous vehicle. For example, the device might display a reminder saying "It's time to start work" at 9:00 AM, and when the user presses the confirmation button, the response is reported to the server.

[1052] Users can set their own schedules on the server and respond to notifications from their devices based on those settings, making it easier for users to manage their daily activity cycles.

[1053] When used in an autonomous vehicle, for example, if a user sets a wake-up time during a nap, a wake-up notification will be sent 10 minutes before the scheduled arrival time. The entertainment system and lighting will also be shut down at the set time, allowing for a comfortable in-car experience.

[1054] Hardware and software used

[1055] To implement this system, a cloud server (e.g., AWS, Google Cloud Platform) is used on the server side, and a smartphone (e.g., iPhone, Android), tablet, or in-car device (e.g., in-car infotainment system) is used on the terminal side. Such devices are connected to the Internet and have an application installed to communicate with the server.

[1056] Data calculation and processing

[1057] The server receives the life cycle data set by the user and creates a notification schedule based on this. For example, when it receives wake-up time data, it stores that data and generates a notification when the alarm time arrives. It also receives response data from the device and manages the user's behavior history. A database (e.g., MySQL, PostgreSQL) is used to process this data.

[1058] Examples of concrete examples and prompts

[1059] As a concrete example, consider the case where a user sets the device to wake up at 7:00 every morning. At 7:00, the server sends an alarm notification to the device, and the device sounds the alarm. When the user checks the alarm and presses a button, that information is sent to the server.

[1060] An example prompt might look like this:

[1061] "Generate Python code to send a nap notification so that the user can wake up 10 minutes before their scheduled arrival time."

[1062] "Generate Python code to manage work time in the car and display a reminder 10 minutes before a meeting."

[1063] This approach enables integrated and efficient life cycle management within the home and in autonomous vehicles, enabling users to maintain a healthy and comfortable lifestyle.

[1064] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1065] Step 1:

[1066] (Server) Setting user lifecycle data

[1067] Input: The user sets the wake-up time, daytime activity times, bedtime, and device shutdown time to the server.

[1068] Processing: The server receives this data and stores it in a database (e.g., MySQL, PostgreSQL).

[1069] Output: The server is ready to generate a notification schedule based on the configured data.

[1070] Step 2:

[1071] (Server) Wake-up time management

[1072] Input: The server retrieves the user's wake-up time information from the database.

[1073] Processing: When the set wake-up time approaches, the server generates an alarm notification and sends it to the device.

[1074] Output: The server sends an alarm notification to the terminal.

[1075] Step 3:

[1076] (Device) Display alarm notifications

[1077] Input: Receives alarm notifications sent from the server.

[1078] Action: The device will sound an alarm and display a wake-up notification on the screen until the user presses the wake-up button.

[1079] Output: When the user presses the button, the device sends the response data to the server.

[1080] Step 4:

[1081] (Server) Managing daytime activity times

[1082] Input: Retrieve information about the user's daily activity times from the database.

[1083] Processing: When the set activity start time approaches, the server generates a reminder notification and sends it to the device.

[1084] Output: The server sends a reminder notification to the device.

[1085] Step 5:

[1086] (Device) Displaying reminder notifications

[1087] Input: Receive reminder notifications sent from the server.

[1088] Processing: The device displays a reminder to prompt the user to start the activity. The notification continues until the user presses the confirmation button.

[1089] Output: When the user presses the button, the device sends the response data to the server.

[1090] Step 6:

[1091] (Server) Bedtime management

[1092] Input: Get the bedtime information set by the user from the database.

[1093] Processing: 30 minutes before the set bedtime, the server generates a notification to prompt the user to get ready for bed and sends it to the device.

[1094] Output: The server sends a notification to the device to prepare for bed.

[1095] Step 7:

[1096] (Device) Display bedtime notification

[1097] Input: Receive bedtime notifications sent from the server.

[1098] Processing: The device displays a notification to prompt the user to prepare for bed. The notification continues until the user presses the confirmation button.

[1099] Output: When the user presses the button, the device sends the response data to the server.

[1100] Step 8:

[1101] (Server) Device Shutdown

[1102] Input: Retrieve information about the device shutdown time set by the user from the database.

[1103] Processing: When the scheduled shutdown time arrives, the server sends a shutdown signal to the target device, and also sends a similar signal to the entertainment system and lighting in the autonomous vehicle.

[1104] Output: The server checks for shutdown and records the result.

[1105] These steps will enable integrated and efficient life cycle management within homes and autonomous vehicles.

[1106] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1107] The present invention relates to a system that provides more personalized support by combining an emotion engine that recognizes the user's emotions with a home appliance management system for living a healthy lifestyle.

[1108] System configuration

[1109] The system of the present invention consists of four main components: a server, a terminal, an emotion engine, and a user.

[1110] The server manages the data of the user's set wake-up time, activity time, and bedtime, and is responsible for sending corresponding notifications to the device based on this data.

[1111] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server.

[1112] The emotion engine is responsible for recognizing the user's emotions in real time and providing the analysis results to the server.

[1113] The user sets his or her own schedule on the server and responds to notifications from the terminal based on that setting.

[1114] Program processing

[1115] The processing of the program of the present invention will be explained in natural language below.

[1116] 1. Manage your wake-up time

[1117] The user sets the wake-up time each morning on the server.

[1118] When the set time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device.

[1119] The device will sound an alarm and display a wake-up notification on the screen.

[1120] When the user presses the wake-up button, the terminal reports the operation to the server.

[1121] The emotion engine analyzes the user's emotional state in real time when they wake up and provides the results to the server.

[1122] 2. Daytime activity management

[1123] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[1124] When the time to start the activity arrives, the server adjusts the content of the reminder message based on the analysis results of the emotion engine and sends a reminder notification to the device.

[1125] For example, if the user's emotional state is low, the device will display a reminder with an encouraging message.

[1126] When the user presses the confirmation button, the terminal reports the operation to the server.

[1127] The emotion engine analyzes the user's emotional state in real time while they are active and provides information to the server as needed.

[1128] 3. Bedtime notifications

[1129] The user sets the time they go to bed each night on the server.

[1130] Thirty minutes before bedtime, the server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device.

[1131] For example, if the user is relaxing, the notification may be displayed with soft music in the background.

[1132] The device will display a notification to remind the user to get ready for bed.

[1133] When the user presses the prepare button, the terminal reports the operation to the server.

[1134] 4. Shut down the device

[1135] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[1136] When the shutdown time arrives, the server sends a shutdown signal to these devices.

[1137] The terminal receives the shutdown signal and initiates shutdown of the target device.

[1138] The terminal checks whether the device shutdown has been completed successfully and reports the result to the server.

[1139] Specific examples

[1140] 1. Manage your wake-up time

[1141] For example, a user configures the server to wake up at 7am every morning.

[1142] The server sends an alarm notification to the device at exactly 7:00, and the device plays an appropriate alarm sound based on the results of the emotion engine.

[1143] When the user presses the wake-up button, the device sends the information to the server, and the emotion engine analyzes and reports the user's emotions.

[1144] 2. Daytime activity management

[1145] For example, a user may set his working hours from 9:00 AM to 12:00 PM.

[1146] At 9 a.m., the server uses the emotion engine's analysis to send a reminder to the device, including the encouraging message, "It's time to get to work."

[1147] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[1148] 3. Bedtime notifications

[1149] For example, a user sets the time to go to bed at 10 o'clock every night.

[1150] At 9:30 p.m., the server uses the emotion engine's analysis results to send a bedtime notification to the device, including a relaxing message.

[1151] The device will display a notification to remind the user to get ready for bed.

[1152] 4. Shut down the device

[1153] For example, a user can set the TV and Wi-Fi router in their home to shut down at 10:00.

[1154] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[1155] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[1156] As described above, the system of the present invention not only manages household appliances but also provides more personalized lifestyle management by taking into account the user's emotions.

[1157] The processing flow will be explained below.

[1158] Wake-up time management

[1159] Step 1:

[1160] The user logs into the server and sets the time to wake up each morning.

[1161] Step 2:

[1162] The server stores the wake-up time entered by the user in a database.

[1163] Step 3:

[1164] Five minutes before the set wake-up time, the server prepares for the alarm notification.

[1165] Step 4:

[1166] The emotion engine analyzes the user's current emotional state.

[1167] Step 5:

[1168] When the set wake-up time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device.

[1169] Step 6:

[1170] The device will sound an alarm and display a wake-up notification on the screen.

[1171] Step 7:

[1172] When the user presses the wake-up button on the terminal, the terminal reports the operation to the server.

[1173] Step 8:

[1174] The server confirms that the user has pressed the wake-up button and records the wake-up confirmation information in the database.

[1175] Daytime activity management

[1176] Step 1:

[1177] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[1178] Step 2:

[1179] The server stores the activity time entered by the user in a database.

[1180] Step 3:

[1181] Five minutes before the activity start time, the server prepares the reminder notification.

[1182] Step 4:

[1183] The emotion engine analyzes the user's current emotional state.

[1184] Step 5:

[1185] When the time to start the activity arrives, the server adjusts the content of the reminder message based on the analysis results of the emotion engine and sends a reminder notification to the device.

[1186] Step 6:

[1187] The device will display a reminder on the screen, telling the user, "It's time to start work."

[1188] Step 7:

[1189] When the user presses the confirmation button on the terminal, the terminal reports the operation to the server.

[1190] Step 8:

[1191] The server confirms that the user has pressed the confirmation button and records the confirmation information for the start of the activity in the database.

[1192] Bedtime notifications

[1193] Step 1:

[1194] The user sets the time they go to bed each night on the server.

[1195] Step 2:

[1196] The server stores the bedtime entered by the user in a database.

[1197] Step 3:

[1198] Thirty minutes before bedtime, the server prepares the bedtime notification.

[1199] Step 4:

[1200] The emotion engine analyzes the user's current emotional state.

[1201] Step 5:

[1202] The server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device.

[1203] Step 6:

[1204] The device displays the message "Bedtime in 30 minutes" on the screen, encouraging the user to get ready for bed.

[1205] Step 7:

[1206] When the user presses the prepare button, the terminal reports the operation to the server.

[1207] Step 8:

[1208] The server confirms that the user has pressed the prepare button and records the confirmation of preparation in the database.

[1209] Equipment shutdown

[1210] Step 1:

[1211] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[1212] Step 2:

[1213] The server stores the shutdown time entered by the user in a database.

[1214] Step 3:

[1215] Five minutes before the shutdown time, the server prepares for the shutdown notification.

[1216] Step 4:

[1217] When the shutdown time arrives, the server sends a shutdown signal to the target device.

[1218] Step 5:

[1219] The terminal receives the shutdown signal and initiates shutdown of the target device.

[1220] Step 6:

[1221] The terminal checks whether the shutdown of the target device has been completed successfully.

[1222] Step 7:

[1223] The terminal reports that the shutdown is complete to the server, and the server records the information in a database.

[1224] The above is a specific flow of processing steps in the embodiment of the present invention. By having the server, terminal, emotion engine, and user fulfill their respective roles, it becomes possible to maintain a healthy lifestyle.

[1225] Example 2

[1226] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1227] Conventional home device management systems have the ability to send notifications and reminders based on the user's schedule, but they have the problem of not being able to provide a personalized experience because they do not take the user's emotional state into consideration.In addition, they cannot appropriately adjust the content of notifications or alarm sounds depending on the user's emotional state, resulting in situations where users do not receive sufficient support or reminders.

[1228] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1229] In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time, means for checking the user's response to the alarm, means for setting a daytime activity time, means for sending a reminder at the set activity time, means for checking the user's response to the reminder, means for setting a bedtime, means for sending a notification urging the user to get ready for bed at the set bedtime, means for shutting down home appliances at bedtime, means for analyzing the user's emotional state in real time, and means for adjusting the notification content and alarm volume based on the user's emotional state. This makes it possible to provide personalized notifications and alarms according to the user's emotional state, thereby enabling the user's lifestyle to be managed more healthily and efficiently.

[1230] The "means for setting the wake-up time" is a function that allows the user to specify the wake-up time each morning and register it in the server.

[1231] The "means for sending an alarm at a set wake-up time" is a function that sends an alarm notification from the server to the terminal at a set time.

[1232] The "means for checking the user's response to the alarm" is a function that detects whether the user has responded to the alarm (for example, by pressing the wake-up button) and reports that information to the server.

[1233] The "means for setting daytime activity times" is a function that allows the user to specify specific activity times during the day and register them on the server.

[1234] The "means for sending a reminder at a set activity time" is a function for sending a reminder notification from the server to the terminal when the set activity time arrives.

[1235] The "means for confirming the user's response to the reminder" is a function that detects whether the user has responded to the reminder notification by pressing a confirmation button, and reports that information to the server.

[1236] The "means for setting bedtime" is a function that allows the user to specify the time they go to bed each night and register it in the server.

[1237] The "means for sending a notification prompting the user to get ready for bed at a set bedtime" is a function for sending a notification prompting the user to get ready for bed from the server to the terminal before the set bedtime.

[1238] The "means for shutting down devices in the home at bedtime" is a function that sends a shutdown signal from the server to specific devices in the home when the set bedtime arrives.

[1239] "Means for analyzing the user's emotional state in real time" is a function that analyzes the user's facial expressions, voice, etc. using emotion recognition technology to grasp the user's emotional state in real time.

[1240] "Means for adjusting notification content and alarm volume based on the user's emotional state" is a function for optimizing the content and volume of notifications and alarms based on the analysis of the user's emotional state.

[1241] The present invention relates to a home appliance management system that supports users in maintaining a healthy lifestyle. The system recognizes the user's emotional state in real time and adjusts notification content and alarm volume based on that information to provide more personalized support.

[1242] System configuration

[1243] The system of the present invention consists of four main components: a server, a terminal, an emotion engine, and a user.

[1244] The server manages the data of the user's set wake-up time, daytime activity time, and bedtime, and sends corresponding notifications to the device based on this data. Specifically, a cloud server such as Amazon EC2 or Google Cloud can be used.

[1245] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server. The terminal can be an iOS or Android device.

[1246] The emotion engine recognizes the user's emotions in real time and provides the analysis results to the server. Specifically, it uses emotion recognition APIs from IBM Watson and Microsoft Azure.

[1247] The user sets his or her own schedule in the server and responds to notifications from the terminal based on that schedule.

[1248] Wake-up time management

[1249] The user sets the wake-up time each morning on the server through a dedicated application. When the set time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device. The device then plays the set music and displays "It's time to wake up" on the screen. When the user presses the wake-up button, the device sends that information to the server, and the emotion engine analyzes and reports the user's emotions.

[1250] Examples:

[1251] For example, a user sets the server to wake up at 7:00 every morning. The server sends an alarm notification to the device at exactly 7:00, and the device sounds an appropriate alarm sound based on the results of the emotion engine. When the user presses the wake-up button, the device sends that information to the server, and the emotion engine analyzes and reports the user's emotions. An example of a prompt sentence is, "Write a program that sets the user to wake up at 7:00, and adjusts the alarm volume based on the analysis results of the emotion engine."

[1252] Daytime activity management

[1253] The user sets the time of daytime activities on the server through a dedicated app. For example, they can set "work hours from 9:00 AM to 12:00 PM." When it's time to start the activity, the server adjusts the reminder message based on the analysis results of the emotion engine and sends it to the device. The device displays the reminder, and when the user presses the confirmation button, it reports the operation to the server.

[1254] Examples:

[1255] For example, a user sets their working hours from 9:00 AM to 12:00 PM. At 9:00 AM, the server reflects the analysis results of the emotion engine and sends a reminder to the device containing an encouraging message saying, "It's time to start work." The device displays the reminder, and when the user presses the confirmation button, the information is sent to the server. An example of a prompt sentence is, "Write a program that sets the user's daytime activity hours as 9:00 AM to 12:00 PM and adjusts the reminder content based on the results of the emotion engine."

[1256] Bedtime notifications

[1257] The user sets their nightly bedtime on the server via a dedicated app. 30 minutes before their set bedtime, the server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device. The device displays the notification, and when the user presses the bedtime preparation button, the information is sent to the server.

[1258] Examples:

[1259] For example, a user sets the bedtime to 10 o'clock every night. At 9:30 p.m., the server reflects the analysis results of the emotion engine and sends a bedtime notification with a relaxing message to the device. The device displays the notification, and when the user presses the ready button, the information is sent to the server. An example of a prompt sentence is, "Write a program that sets the user's bedtime to 10 o'clock and plays calm music based on the results of the emotion engine."

[1260] Shut down household devices

[1261] The user sets the shutdown time for a specific device in the home on the server through a dedicated application. When the shutdown time arrives, the server sends a shutdown signal to the target device, and the device receives the signal and starts shutting down the device. The device confirms that the device has been shut down successfully and reports the status to the server.

[1262] Examples:

[1263] For example, a user sets the TV and Wi-Fi router in their home to shut down at 10 o'clock. At 10 pm, the server sends a shutdown signal to the TV and Wi-Fi router. The device confirms that these devices have shut down successfully and reports the results to the server. An example of a prompt sentence is, "Write a program that shuts down the TV and Wi-Fi router in your home every night at 10 o'clock. Please also add a function to report the results to the server."

[1264] As described above, the system of the present invention not only manages household appliances but also provides more personalized lifestyle support by taking into account the emotional state of the user.

[1265] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1266] Program processing steps

[1267] Wake-up time management

[1268] Step 1:

[1269] The user sets the wake-up time on the server.

[1270] Input: User's wake-up time (e.g. 7:00)

[1271] Specific operation: The user enters the wake-up time in a dedicated application and registers it on the server.

[1272] Output: The wake-up time data is saved on the server.

[1273] Step 2:

[1274] The server prepares the notification based on the configured wake-up time.

[1275] Input: Wake-up time stored on the server

[1276] Specific operation: The server prepares an alarm notification based on the set wake-up time (e.g., 7:00).

[1277] Output: Alarm notification is ready.

[1278] Step 3:

[1279] An emotion engine analyzes the user's emotional state.

[1280] Input: Real-time user data (face recognition, voice data, etc.)

[1281] Specific operation: The emotion engine analyzes the user's face and voice through the camera and microphone to analyze their emotional state.

[1282] Output: Emotional state data (e.g., relaxed state)

[1283] Step 4:

[1284] The server determines the optimal alarm volume and tone and sends a notification to the device.

[1285] Input: Emotional state data, saved wake-up time

[1286] Specific operation: Based on the results of the emotion engine, the server selects the optimal alarm volume and tone and sends an alarm notification to the device.

[1287] Output: An alarm notification is sent to the device.

[1288] Step 5:

[1289] Your device will sound an alarm and display a wake-up notification.

[1290] Input: Alarm notification

[1291] Specific actions: The device receives a notification, plays the set music, and displays a wake-up time notification on the screen.

[1292] Output: An alarm will sound and a wake-up notification will appear on the screen.

[1293] Step 6:

[1294] The user presses the wake-up button.

[1295] Input: User button presses

[1296] Specific operation: The user presses the alarm stop button on the terminal.

[1297] Output: The alarm stops and operation data is generated on the terminal.

[1298] Step 7:

[1299] The terminal reports the user's operations to the server.

[1300] Input: Operation data (alarm stop button pressed)

[1301] Specific operation: The terminal transmits the user's button operation information to the server.

[1302] Output: Operation information is reported to the server.

[1303] Step 8:

[1304] The emotion engine analyzes the user's emotional state when they wake up in real time and provides the information to the server.

[1305] Input: Real-time user data (face recognition, voice data, etc.)

[1306] Specific operation: The emotion engine re-analyzes the user's emotions upon waking up and sends the results to the server.

[1307] Output: The emotional state data is provided to the server.

[1308] Daytime activity management

[1309] Step 1:

[1310] The user sets the daytime activity times on the server.

[1311] Input: User activity time (e.g. 9:00 to 12:00)

[1312] Specific operation: The user inputs the activity time in a dedicated application and registers it on the server.

[1313] Output: Activity time data is saved on the server.

[1314] Step 2:

[1315] The server prepares notifications based on configured active times.

[1316] Input: Activity time stored on the server

[1317] Specific behavior: The server prepares a reminder notification based on the set activity time (e.g. 9:00).

[1318] Output: Reminder notification is ready.

[1319] Step 3:

[1320] The emotion engine analyzes the user's emotional state in real time during activity.

[1321] Input: Real-time user data (face recognition, voice data, etc.)

[1322] Specific operation: The emotion engine analyzes the user's facial expressions and voice to analyze the user's emotional state in real time during an activity.

[1323] Output: Emotional state data during the activity

[1324] Step 4:

[1325] The server determines the optimal message content and sends a notification to the device.

[1326] Input: Emotional state data during activity, saved activity time

[1327] Specific operation: Based on the results of the emotion engine, the server selects a message, such as "Good luck," and sends a reminder notification to the device.

[1328] Output: A reminder notification is sent to your device.

[1329] Step 5:

[1330] Your device will display the reminder.

[1331] Input: Reminder

[1332] Specific behavior: The device receives a notification and displays something like, "It's 9 o'clock, time to get to work."

[1333] Output: The reminder will be displayed on your device.

[1334] Step 6:

[1335] The user presses the confirm button.

[1336] Input: User confirmation button operation

[1337] Specific operation: The user presses the confirmation button in response to the reminder notification.

[1338] Output: Operation data is generated when the confirmation button is pressed.

[1339] Step 7:

[1340] The terminal reports the user's operations to the server.

[1341] Input: Operation data (Confirm button pressed)

[1342] Specific operation: The terminal transmits the user's confirmation button operation information to the server.

[1343] Output: Operation information is reported to the server.

[1344] Bedtime notifications

[1345] Step 1:

[1346] The user sets the bedtime on the server.

[1347] Input: User's bedtime (e.g. 10:00)

[1348] Specific operation: The user enters the bedtime in a dedicated application and registers it on the server.

[1349] Output: Bedtime data is saved on the server.

[1350] Step 2:

[1351] The server prepares a notification before bedtime.

[1352] Input: Bedtime stored on the server

[1353] Specific operation: The server prepares a bedtime notification based on the set time 30 minutes before bedtime (e.g. 9:30).

[1354] Output: Bedtime notification is ready.

[1355] Step 3:

[1356] An emotion engine analyzes the user's emotional state.

[1357] Input: Real-time user data (face recognition, voice data, etc.)

[1358] Specific operation: The emotion engine analyzes the user's face and voice to analyze their emotional state before going to sleep.

[1359] Output: Emotional state data before going to bed

[1360] Step 4:

[1361] The server determines the optimal notification content and sends it to the device.

[1362] Input: emotional state data before bedtime, saved bedtime

[1363] Specific operation: Based on the results of the emotion engine, the server selects calm music if the user is in a relaxed state and sends a notification to the device to prepare for bed.

[1364] Output: A bedtime notification is sent to the device.

[1365] Step 5:

[1366] Your device will display a bedtime notification.

[1367] Input: Bedtime notification

[1368] Specific behavior: The device receives a notification and displays "Start getting ready for bed."

[1369] Output: A bedtime notification will appear on your device.

[1370] Step 6:

[1371] The user presses the ready for bed button.

[1372] Input: User confirmation button operation

[1373] Specific Action: The user presses the Ready for Bed button.

[1374] Output: Operation data is generated when the ready button is pressed.

[1375] Step 7:

[1376] The terminal reports the user's operations to the server.

[1377] Input: Operation data (Ready button pressed)

[1378] Specific operation: The terminal sends the user's preparation completion button operation information to the server.

[1379] Output: Operation information is reported to the server.

[1380] Shut down household devices

[1381] Step 1:

[1382] The user sets the shutdown time of the home appliances in the server.

[1383] Input: User shutdown time (e.g. 10:00)

[1384] Specific operation: The user enters the shutdown time in a dedicated application and registers it on the server.

[1385] Output: Shutdown time data is saved on the server.

[1386] Step 2:

[1387] The server prepares a notification when it is time to shut down.

[1388] Input: Shutdown time stored on the server

[1389] Specific operation: The server prepares a shutdown notification based on the configured shutdown time (e.g., 10:00).

[1390] Output: Shutdown notification is ready.

[1391] Step 3:

[1392] The server sends a shutdown signal to the home appliance.

[1393] Input: Shutdown notification

[1394] Specific operation: When the set shutdown time arrives, the server sends a shutdown signal to the specified home devices (TVs and Wi-Fi routers).

[1395] Output: A shutdown signal is sent to the home appliance.

[1396] Step 4:

[1397] The terminal initiates a shutdown of the target device.

[1398] Input: Shutdown signal

[1399] Specific operation: The terminal receives the shutdown signal and starts the shutdown process of the specified device.

[1400] Output: Equipment shutdown is initiated.

[1401] Step 5:

[1402] The terminal confirms that the device has been shut down and reports this to the server.

[1403] Input: Device shutdown state

[1404] Specific operation: The device checks whether the devices have been shut down successfully and reports the result to the server, for example, notifying the server that "The TV and Wi-Fi router have been shut down successfully."

[1405] Output: The shutdown state is reported to the server.

[1406] (Application example 2)

[1407] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1408] Conventional home appliance management systems send uniform notifications and alarms without considering the user's emotions, resulting in a lack of personalization tailored to the user's psychological state. Furthermore, in-store shopping experiences are also challenged by the inability to provide support tailored to the individual user's emotional state, which can easily lead to dissatisfaction, especially when the user's emotions are unstable.

[1409] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time and adjusting the alarm sound based on the emotion analysis result, and means for checking the user's response to the alarm and sending the emotion analysis result to the server. This makes it possible to provide personalized alarm notifications and daytime activity reminders according to the user's emotional state.

[1410] The system also includes a means for providing personalized navigation and product recommendations based on the user's emotional state in a physical store, thereby enabling optimal support according to the user's emotions even in the physical store, and providing a more satisfying shopping experience.

[1411] The "means for setting a wake-up time" is a function that allows the user to set in advance the wake-up time each morning.

[1412] "Means for sending alarms" is a function that notifies you by voice or visually at a set time.

[1413] "Emotion analysis results" are the results of analyzing the user's emotional state in real time from their face and behavior, and providing the results as data.

[1414] "Means for adjusting alarm sound" is a function that selects the optimal volume and tone based on the results of emotion analysis.

[1415] The "means for checking the user's response to the alarm" is a function that detects the operation the user has performed on the alarm (such as snooze or stop) and collects that information.

[1416] The "means for setting daytime activity times" is a function that allows the user to set specific activity times (e.g., work time or exercise time).

[1417] The "means for sending a reminder" is a function for sending a reminder notification at a set activity time and notifying the user of the same.

[1418] The "means for adjusting reminder content" is a function that optimizes the content of reminder messages and notifications based on the user's emotional state.

[1419] The "means for setting bedtime" is a function that allows the user to set the time they go to bed each night.

[1420] The "means for sending a notification to urge the user to get ready for bed" is a function that sends a notification to urge the user to get ready for bed when the set bedtime approaches.

[1421] "Means for shutting down household devices" refers to a function that turns off household devices such as televisions and Wi-Fi routers at bedtime.

[1422] The "emotion analysis means" is a function for recognizing and analyzing the user's emotional state in real time and providing the results to the system.

[1423] "Means for personalized navigation and product recommendations" refers to a function that recommends optimal routes and products based on the user's emotional state within a physical store.

[1424] The present invention is a system for personalizing user experiences in home devices and brick-and-mortar stores using emotion analysis. The system consists of four main components: a server, a terminal, an emotion analysis means, and a user.

[1425] System configuration

[1426] 1. Server

[1427] The server manages schedule data set by the user, such as wake-up time, daytime activity times, and bedtime, and also collects emotional data provided by an emotion analysis tool, optimizing various notifications and recommendations based on this data.

[1428] 2. Terminal

[1429] The device can be a smartphone, smart glasses, or a head-mounted display (HMD), and it displays notifications to the user, collects the user's responses, and sends them to a server. It also provides an interface for performing actions such as shutting down home appliances.

[1430] 3. Emotion analysis method

[1431] Emotion analyzers are software that uses cameras and sensors to capture a user's face and behavior, and analyzes them to identify their emotional state, including the Microsoft Azure Emotion API and Affectiva SDK.

[1432] 4. Users

[1433] Users set their own schedules on the server and receive notifications from their devices based on those schedules. In physical stores, users can also use navigation and product recommendations based on their emotional state.

[1434] Program processing

[1435] Hardware / Software used

[1436] Smartphone (iOS or Android)

[1437] Emotion analysis software (Microsoft Azure Emotion API, Affectiva SDK)

[1438] Server (AWS Lambda + DynamoDB)

[1439] UI framework (React Native)

[1440] Data processing and calculation

[1441] The server collects and manages user settings and emotion analysis data, and determines optimized alarm sounds and reminder content based on this data.

[1442] The emotion analysis means analyzes the emotions of the user from their face and actions in real time, and sends the results to the server as data.

[1443] The device receives and displays notifications from the server, reports user responses, and, in physical stores, displays data for navigation and product recommendations.

[1444] Specific examples

[1445] 1. Manage your wake-up time

[1446] The user sets the time to wake up at 7am every morning.

[1447] At 7:00, the server determines the optimal alarm sound based on the emotion analysis results and sends a notification to the device.

[1448] The terminal sounds an alarm, and when the user presses the wake-up button, the information is sent to the server, and the emotion analysis means analyzes the emotion and provides the result to the server.

[1449] 2. Daytime activity management

[1450] A user sets their work hours as 9:00 AM to 12:00 PM.

[1451] At 9 a.m., the server uses the results of emotion analysis to send a reminder to the device, including the encouraging message, "It's time to get to work."

[1452] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[1453] 3. Bedtime notifications

[1454] The user sets the bedtime to 10 o'clock every night.

[1455] At 9:30 p.m., the server uses the results of the emotion analysis to send a bedtime notification to the device, including a relaxing message.

[1456] The device will display a notification to remind the user to get ready for bed.

[1457] 4. Shut down the device

[1458] A user sets the TV and Wi-Fi router in their home to shut down at 10:00.

[1459] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[1460] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[1461] Example prompts to input to the generative AI model

[1462] "The user's current emotional state is joy. If the emotion of joy is high, what is the next recommended product in the store?"

[1463] "If a user's emotional state is low, what message would be effective in encouraging them?"

[1464] By utilizing sentiment analysis, this system significantly personalizes users' lifestyles and in-store experiences, helping them live richer lives.

[1465] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1466] Step 1:

[1467] The user sets the wake-up time using a smartphone device. The input is the wake-up time entered by the user into the app, and the output is the wake-up time data stored on the server. The wake-up time set on the smartphone app is sent to the server and stored in a database.

[1468] Step 2:

[1469] When the set wake-up time arrives, the server requests the emotion analysis means to capture the user's face and behavior. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analysis means captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[1470] Step 3:

[1471] The server determines the optimal alarm volume and tone based on the acquired emotional data. The input is the emotional analysis data, and the output is the alarm sound setting. The server adjusts the alarm sound according to the emotional state and sends the setting to the device.

[1472] Step 4:

[1473] The device plays the adjusted alarm sound to prompt the user to wake up. The input is the alarm sound setting sent from the server, and the output is the alarm sound and wake-up notification. The device waits for the user to press the wake-up button.

[1474] Step 5:

[1475] When a user presses the wake-up button, the device reports the operation to the server. The input is the user pressing the wake-up button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with the emotion data at the time of waking up.

[1476] Step 6:

[1477] The user sets the activity time during the day (e.g., work hours). The input is the activity time entered by the user into the app, and the output is the activity time data stored on the server. The activity time set on the smartphone app is sent to the server and stored in the database.

[1478] Step 7:

[1479] When the set activity time arrives, the server requests the user's emotional data from the emotion analysis means. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analysis means captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[1480] Step 8:

[1481] The server generates an appropriate reminder message based on the acquired emotional data. The input is the emotion analysis data, and the output is the reminder message. The server optimizes the reminder content according to the emotional state and sends the message to the device.

[1482] Step 9:

[1483] The terminal displays the generated reminder message and prompts the user to start the activity. The input is the reminder message sent from the server, and the output is the reminder notification. The terminal waits for the user to press the confirmation button.

[1484] Step 10:

[1485] When the user presses the confirmation button, the device reports the operation to the server. The input is the user pressing the confirmation button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with the emotion data during the activity.

[1486] Step 11:

[1487] The user sets a bedtime. The input is the bedtime entered by the user into the app, and the output is the bedtime data stored on the server. The bedtime set on the smartphone app is sent to the server and stored in a database.

[1488] Step 12:

[1489] Thirty minutes before bedtime, the server requests the user's emotional data from the emotion analyzer. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analyzer captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[1490] Step 13:

[1491] The server generates an appropriate bedtime notification based on the acquired emotional data. The input is the emotion analysis data, and the output is the bedtime notification. The server optimizes the notification content according to the emotional state and sends the message to the device.

[1492] Step 14:

[1493] The terminal displays the generated bedtime notification and prompts the user to get ready for bed. The input is the bedtime notification sent from the server, and the output is the bedtime notification. The terminal waits for the user to press the bedtime button.

[1494] Step 15:

[1495] When a user presses the "Get ready for bed" button, the device reports the operation to the server. The input is the user pressing the "Get ready for bed" button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with emotional data at bedtime.

[1496] Step 16:

[1497] When the set bedtime arrives, the server sends a shutdown signal to home devices. The input is the set shutdown time, and the output is the shutdown signal. The server sends a signal to home devices such as TVs and Wi-Fi routers to initiate shutdown.

[1498] Step 17:

[1499] The terminal confirms receipt of the shutdown signal and checks whether the shutdown of the home device has been completed successfully. The input is the shutdown signal sent from the server, and the output is the shutdown confirmation result. Once the terminal confirms that the shutdown has been completed, it reports the result to the server.

[1500] This series of steps will enable the realization of a system that provides personalized support according to the user's emotional state, thereby enriching daily life and shopping experiences.

[1501] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1502] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1503] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1504] [Third embodiment]

[1505] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1506] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1507] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1508] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1509] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1510] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1511] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1512] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1513] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1514] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1515] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1516] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1517] The present invention relates to a home appliance management system for leading a healthy lifestyle.

[1518] System configuration

[1519] The system of the present invention consists of three main components: a server, a terminal, and a user.

[1520] The server manages the data of the user's set wake-up time, activity time, and bedtime, and is responsible for sending corresponding notifications to the device based on this data.

[1521] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server.

[1522] The user sets his or her own schedule on the server and responds to notifications from the terminal based on that setting.

[1523] Program processing

[1524] The processing of the program of the present invention will be explained in natural language below.

[1525] 1. Manage your wake-up time

[1526] The user sets the wake-up time each morning on the server.

[1527] The server sends an alarm notification to the terminal at the set time.

[1528] The device will sound an alarm and continue to notify you until the user presses the wake-up button.

[1529] When the user presses the button, the terminal sends wake-up confirmation data to the server.

[1530] 2. Daytime activity management

[1531] The user sets the daytime activity times on the server (for example, 9:00 AM to 12:00 PM).

[1532] The server sends a reminder to the device when it is time to start the activity.

[1533] The device will display a reminder and continue to notify you until you press the confirmation button.

[1534] When the user presses the confirmation button, the terminal transmits confirmation data for the start of the activity to the server.

[1535] 3. Bedtime notifications

[1536] The user sets the time they go to bed each night on the server.

[1537] The server sends a notification to the device to prepare for bed 30 minutes before going to bed.

[1538] The device will display a notification to remind the user to get ready for bed.

[1539] 4. Shut down the device

[1540] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[1541] The server sends a shutdown signal to these devices at the set time.

[1542] The terminal checks whether the device has been shut down normally and reports the result to the server.

[1543] Specific examples

[1544] 1. Manage your wake-up time

[1545] For example, a user configures the server to wake up at 7am every morning.

[1546] The server sends an alarm notification to the device at exactly 7:00, and the device sounds an alarm.

[1547] When the user presses the wake-up button, the terminal transmits the information to the server to confirm the wake-up.

[1548] 2. Daytime activity management

[1549] For example, a user may set his working hours from 9:00 AM to 12:00 PM.

[1550] At 9:00 AM, the server sends a reminder to the device saying, "It's time to get to work."

[1551] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[1552] 3. Bedtime notifications

[1553] For example, a user sets the time to go to bed at 10 o'clock every night.

[1554] At 9:30 p.m., the server sends a notification to the device saying, "It's bedtime in 30 minutes."

[1555] The device will display a notification to remind the user to get ready for bed.

[1556] 4. Shut down the device

[1557] For example, a user can set the TV and Wi-Fi router in their home to shut down at 10:00.

[1558] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[1559] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[1560] As described above, the system of the present invention manages the user's life cycle according to a set time and provides a specific means for supporting a healthy lifestyle.

[1561] The processing flow will be explained below.

[1562] Wake-up time management

[1563] Step 1:

[1564] The user logs into the server and sets the time to wake up each morning.

[1565] Step 2:

[1566] The server stores the wake-up time entered by the user in a database.

[1567] Step 3:

[1568] Five minutes before the set wake-up time, the server prepares for the alarm notification.

[1569] Step 4:

[1570] When the set wake-up time arrives, the server sends an alarm notification to the terminal.

[1571] Step 5:

[1572] The device will sound an alarm and display a wake-up notification on the screen.

[1573] Step 6:

[1574] When the user presses the wake-up button on the terminal, the terminal reports the operation to the server.

[1575] Step 7:

[1576] The server confirms that the user has pressed the wake-up button and records the wake-up confirmation information in the database.

[1577] Daytime activity management

[1578] Step 1:

[1579] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[1580] Step 2:

[1581] The server stores the activity time entered by the user in a database.

[1582] Step 3:

[1583] Five minutes before the activity start time, the server prepares the reminder notification.

[1584] Step 4:

[1585] When the activity start time arrives, the server sends a reminder notification to the device.

[1586] Step 5:

[1587] The device will display a reminder on the screen, telling the user, "It's time to start work."

[1588] Step 6:

[1589] When the user presses the confirmation button on the terminal, the terminal reports the operation to the server.

[1590] Step 7:

[1591] The server confirms that the user has pressed the confirmation button and records the confirmation information for the start of the activity in the database.

[1592] Bedtime notifications

[1593] Step 1:

[1594] The user sets the time they go to bed each night on the server.

[1595] Step 2:

[1596] The server stores the bedtime entered by the user in a database.

[1597] Step 3:

[1598] Thirty minutes before bedtime, the server prepares the bedtime notification.

[1599] Step 4:

[1600] Thirty minutes before bedtime, the server sends a notification to the device to prepare for bed.

[1601] Step 5:

[1602] The device displays the message "Bedtime in 30 minutes" on the screen, encouraging the user to get ready for bed.

[1603] Step 6:

[1604] When the user presses the prepare button, the terminal reports the operation to the server.

[1605] Step 7:

[1606] The server confirms that the user has pressed the prepare button and records the confirmation of preparation in the database.

[1607] Equipment shutdown

[1608] Step 1:

[1609] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[1610] Step 2:

[1611] The server stores the shutdown time entered by the user in a database.

[1612] Step 3:

[1613] Five minutes before the shutdown time, the server prepares for the shutdown notification.

[1614] Step 4:

[1615] When the set shutdown time arrives, the server sends a shutdown signal to the target device.

[1616] Step 5:

[1617] The terminal receives the shutdown signal and initiates shutdown of the target device.

[1618] Step 6:

[1619] The terminal checks whether the shutdown of the target device has been completed successfully.

[1620] Step 7:

[1621] The terminal reports that the shutdown is complete to the server, and the server records the information in a database.

[1622] Example 1

[1623] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1624] Maintaining a regular lifestyle is important for improving health and productivity. However, in today's busy lives, daily time management is becoming increasingly difficult. Conventional methods require users to manually manage their time, including waking up, daily activities, going to bed, and managing home devices, which can be cumbersome. This creates a particularly difficult problem for maintaining a healthy lifestyle.

[1625] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1626] In this invention, the server includes means for a user to set a wake-up time, means for sending an alarm when the set wake-up time arrives, means for the terminal to sound the alarm and confirm the user's response, means for a user to set a daytime activity time, means for sending a reminder at the set activity time, means for the terminal to display the reminder and confirm the user's response, means for a user to set a bedtime, means for sending a notification urging the user to get ready for bed 30 minutes before the set bedtime, means for a user to set a shutdown time for home appliances, and means for sending a shutdown signal to the home appliances at the set shutdown time. This makes it possible to automatically manage a user's lifestyle via the server.

[1627] "User" refers to an individual who uses this system.

[1628] "Server" refers to a computer system that records and manages user settings and transmits each piece of information to the terminal.

[1629] "Terminal" refers to a device that displays notifications and reminders from the server to the user and receives the user's responses.

[1630] "Means for setting a wake-up time" refers to the specific functions and methods by which a user inputs and sets the wake-up time each morning.

[1631] "Means for sending an alarm" refers to a specific function or method for sending an alarm notification from the server to the terminal at the set wake-up time.

[1632] The term "means for the terminal to sound an alarm and confirm the user's response" refers to a function or method for the terminal to sound an alarm and receive a user's wake-up confirmation operation.

[1633] The "means for setting daytime activity times" refers to a specific function or method for a user to set a specific time period as a daytime activity time.

[1634] "Means for sending reminders" refers to specific functions and methods for sending reminder notifications from the server to the device when the set activity time arrives.

[1635] The "means for the terminal to display the reminder and confirm the user's response" refers to a function or method for the terminal to display the reminder and receive a confirmation operation from the user.

[1636] "Means for setting bedtime" refers to the specific functions and methods by which a user can input and set their nightly bedtime.

[1637] "Means for sending a notification encouraging users to get ready for bed" refers to the specific functions and methods for sending a notification from the server to the device prompting users to get ready for bed 30 minutes before the set bedtime.

[1638] "Means for setting the shutdown time of devices in the home" refers to specific functions and methods that allow a user to input and set the shutdown time of a specific device in the home.

[1639] "Means for sending a shutdown signal to a device in the home" refers to a specific function or method for sending a shutdown signal from the server to a specific device in the home when the set shutdown time arrives.

[1640] "Means for confirming that home devices have been shut down normally" refers to specific functions and methods that allow a terminal to check the shutdown status of home devices and report the results to a server.

[1641] This invention relates to a home appliance management system for promoting a healthy lifestyle. This system consists of three main components: a server, a terminal, and a user. Below, we will explain in detail how each component operates, including the hardware and software used.

[1642] Server Roles

[1643] The server manages various schedules set by the user (wake-up time, activity time, bedtime, shutdown time of home devices) and sends appropriate notifications to the device based on those schedules. The server records the user's setting data in a database (e.g., MySQL), and generates corresponding notifications at specific times and sends them to the device. For example, the server sends an alarm notification to the device when the set wake-up time (e.g., 7:00 AM) arrives.

[1644] Device Role

[1645] The device is responsible for displaying notifications sent from the server (such as alarms, reminders, and bedtime notifications) to the user and receiving the user's responses. The device also has the function of reporting the user's responses (such as pressing a button) to the server. For example, if the device sounds an alarm and the user presses the "wake-up button," that information is sent to the server.

[1646] User Roles

[1647] Users set various schedules on the server using a smartphone app, etc. This allows the server to generate and manage notifications that fit the user's lifestyle. For example, if a user sets the server as their active time from 9:00 AM to 12:00 PM, the server will send a reminder to the device at 9:00 AM.

[1648] Specific examples

[1649] Consider a scenario where a user wakes up at 7am every morning, sets their work hours from 9am to 12pm, goes to bed at 10pm every night, and shuts down the TV and Wi-Fi router in their home at the same time.

[1650] 1. Manage your wake-up time:

[1651] The user opens the smartphone app and sets the wake-up time to 7:00 AM every morning. The server receives the wake-up time data (7:00 AM) from the user and records it in a database. When the set time of 7:00 AM arrives, the server sends an alarm notification to the device, which then sounds the alarm continuously. When the user taps the "wake-up button," the device sends "wake-up confirmation data" to the server, which records it in the database.

[1652] 2. Daytime activity management:

[1653] The user sets the working hours from 9:00 AM to 12:00 PM. The server records the received activity time data (9:00 AM - 12:00 PM) in the database. At 9:00 AM, the server sends a reminder notification to the device saying "It's time to start work." The device displays the reminder, and when the user taps the "confirm button," "activity start confirmation data" is sent to the server, which records it in the database.

[1654] 3. Bedtime notifications:

[1655] The user sets the bedtime to 10 PM every night. The server receives the bedtime data (10:00 PM) and records it in the database. At 9:30 PM, the server sends a notification to the device saying, "It's 30 minutes until bedtime." The device continues to display the notification, encouraging the user to get ready for bed.

[1656] 4. Shutdown the equipment:

[1657] A user sets the shutdown time for specific devices in the home (TV and Wi-Fi router). The server records the received shutdown time data (10:00 PM) in a database. At 10 PM, the server sends a shutdown signal to these devices. The terminal checks whether these devices have been shut down successfully and reports the results to the server. The server records this information in a database.

[1658] Prompt Sentence Examples

[1659] Here are some examples of prompts:

[1660] 1. Explain specific examples of managing your wake-up time:

[1661] Let's say a user configures the system to wake up at 7:00 every morning. The server sends a notification at exactly 7:00, and the device continues to ring the alarm. When the user presses the wake-up button, that information is sent to the server.

[1662] 2. Describe specific examples of daytime activity management:

[1663] If a user sets their active time from 9:00 AM to 12:00 PM, the server will send a reminder at 9:00 AM and the device will display the reminder. When the user presses the confirmation button, the information is sent to the server.

[1664] 3. Explain specific examples of bedtime notifications:

[1665] For users who have set their bedtime schedule to 10 p.m. every night, the server will send a notification at 9:30 p.m. and the device will display the message "Bedtime in 30 minutes."

[1666] 4. Describe an example of equipment shutdown:

[1667] If the TV and Wi-Fi router in the home are set to shut down at 10 PM, the server will send a shutdown signal to these devices at 10 PM. The device will check to see if the devices have shut down and report the results to the server.

[1668] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1669] Wake-up time management

[1670] Step 1:

[1671] The user uses a smartphone app to set the time they wake up each morning.

[1672] Input: Wake-up time (e.g. 7:00 AM)

[1673] Output: Configuration data sent to the server

[1674] Specific operation: The user enters the wake-up time on the app's settings screen and presses the Set button.

[1675] Step 2:

[1676] The server records the received wake-up time data in a database.

[1677] Input: Wake-up time data sent by the user

[1678] Output: Wake-up time data recorded in a database

[1679] Specific operation: The server executes a query to store the wake-up time in a database (e.g. MySQL).

[1680] Step 3:

[1681] When the server reaches the set wake-up time, it sends an alarm notification to the terminal.

[1682] Input: Current time and wake-up time data recorded in the database

[1683] Output: Alarm notification is sent to the device

[1684] Specific operation: The server detects that a specific time has arrived, generates an alarm notification, and sends a push notification to the device.

[1685] Step 4:

[1686] The device will sound an alarm and continue to notify you.

[1687] Input: Alarm notification sent from the server

[1688] Output: The device continues to ring

[1689] Specific behavior: The device will start playing a sound file and display a "wake-up button" on the screen.

[1690] Step 5:

[1691] The user presses the wake-up button.

[1692] Input: User presses a button

[1693] Output: Wake-up confirmation data is generated

[1694] Specific action: The user taps the "wake-up button" on the device.

[1695] Step 6:

[1696] The terminal transmits wake-up confirmation data to the server.

[1697] Input: User button press information

[1698] Output: Wake-up confirmation data is sent to the server and recorded in the database

[1699] Specific operation: The device sends an HTTP request to the server, and the server stores the wake-up confirmation data in a database.

[1700] Daytime activity management

[1701] Step 1:

[1702] The user sets the time for daytime activity using a smartphone app.

[1703] Input: Activity time (e.g. 9:00 AM - 12:00 PM)

[1704] Output: Configuration data sent to the server

[1705] Specific operation: The user enters the activity time on the app's settings screen and presses the settings button.

[1706] Step 2:

[1707] The server records the received activity time data in a database.

[1708] Input: Activity time data sent by the user

[1709] Output: Activity time data recorded in a database

[1710] Specific operation: The server executes a query to store the activity time in the database.

[1711] Step 3:

[1712] The server sends a reminder to the device at the start time of the activity.

[1713] Input: Current time and activity time data recorded in the database

[1714] Output: Reminder notification sent to device

[1715] Specific operation: The server detects when a specific time has arrived, generates a reminder notification, and sends it as a push notification to the device.

[1716] Step 4:

[1717] Your device will display reminders and keep you informed.

[1718] Input: Reminder notification sent from the server

[1719] Output: The device continues to display the reminder

[1720] Specific behavior: The device will display a reminder notification and a "Confirm" button on the screen.

[1721] Step 5:

[1722] The user presses the confirm button.

[1723] Input: User presses a button

[1724] Output: Activity start confirmation data is generated

[1725] Specific action: The user taps the "confirmation button" on the device.

[1726] Step 6:

[1727] The terminal transmits activity start confirmation data to the server.

[1728] Input: User button press information

[1729] Output: Activity start confirmation data is sent to the server and recorded in the database

[1730] Specific operation: The terminal sends an HTTP request to the server, and the server stores the activity start confirmation data in the database.

[1731] Bedtime notifications

[1732] Step 1:

[1733] The user uses a smartphone app to set their bedtime each night.

[1734] Input: Bedtime (e.g. 10:00 PM)

[1735] Output: Configuration data sent to the server

[1736] Specific behavior: The user enters bedtime on the app's settings screen and presses the Set button.

[1737] Step 2:

[1738] The server records the received bedtime data in a database.

[1739] Input: Bedtime data submitted by the user

[1740] Output: Bedtime data recorded in a database

[1741] What happens: The server runs a query to store the bedtime in the database.

[1742] Step 3:

[1743] The server sends a bedtime preparation notification to the device 30 minutes before bedtime.

[1744] Input: Current time and bedtime data recorded in the database

[1745] Output: Bedtime notification sent to device

[1746] Specific operation: The server detects when a specific time has arrived, generates a bedtime preparation notification, and sends it as a push notification to the device.

[1747] Step 4:

[1748] The device displays a notification to remind the user to get ready for bed.

[1749] Input: Bedtime notification sent from the server

[1750] Output: The device keeps displaying the bedtime notification

[1751] What it does: Your device will display a bedtime notification and display "Start getting ready for bed" on the screen.

[1752] Equipment shutdown

[1753] Step 1:

[1754] A user uses a smartphone app to set shutdown times for specific devices in the home.

[1755] Input: Shutdown time (e.g. 10:00 PM)

[1756] Output: Configuration data sent to the server

[1757] Specific behavior: The user enters the shutdown time on the app's settings screen and presses the Set button.

[1758] Step 2:

[1759] The shutdown time data received by the server is recorded in a database.

[1760] Input: Shutdown time data sent by the user

[1761] Output: Shutdown time data recorded in the database

[1762] What happens: The server runs a query to store the shutdown time in the database.

[1763] Step 3:

[1764] The server transmits a shutdown signal to a specific device in the home at a specified time.

[1765] Input: Current time and shutdown time data recorded in the database

[1766] Output: A shutdown signal is sent to the device

[1767] Specific operation: The server detects that a specific time has arrived, generates a shutdown signal, and sends it to the home appliance control system (e.g., IoT device).

[1768] Step 4:

[1769] The terminal checks whether the device has been shut down normally and reports the result to the server.

[1770] Input: Device response data to shutdown signal

[1771] Output: Shutdown confirmation data is sent to the server and recorded in the database.

[1772] Specific operation: The terminal checks the device status and reports to the server that the shutdown is complete.

[1773] (Application example 1)

[1774] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1775] Conventional lifestyle management in homes and cars is often performed separately, lacking integrated management. In particular, in autonomous vehicles, time management for users while driving or napping is difficult, and methods for realizing a comfortable and efficient lifestyle are required. Appropriate shutdown management of in-car entertainment systems and lighting is also required.

[1776] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1777] In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time, means for confirming a user's response to the alarm, means for setting a daytime activity time, means for sending a reminder at the set activity time, means for confirming a user's response to the reminder, means for setting a bedtime, means for sending a notification encouraging the user to get ready for bed at the set bedtime, means for shutting down home appliances at bedtime, means for sending specified reminders and notifications for lifestyle cycle management in the autonomous vehicle, and means for shutting down the entertainment system and lighting in the autonomous vehicle at set times, thereby enabling integrated and efficient lifestyle cycle management at home and in the autonomous vehicle.

[1778] A "wake-up time" is a specific time that a user sets to wake up.

[1779] An "alarm" is a means of sending a notification to the user at a set wake-up time to prompt them to wake up.

[1780] The "user's response to the alarm" is an action that the user takes in response to the alarm notification, such as pressing a button or using voice recognition to confirm that the user is awake.

[1781] The "daytime activity time" is a set time indicating the activities (for example, work or exercise) that the user should perform within a certain period of time.

[1782] A "reminder" is a notification means that prompts the user to start an activity at a set time.

[1783] A "user's response to a reminder" is a confirmation action that a user takes in response to a reminder notification.

[1784] A "bedtime" is a specific time that a user sets for going to bed.

[1785] The "notification to urge the user to get ready for bed" is a notification means for urging the user to get ready for bed prior to the set bedtime.

[1786] The "means for shutting down devices" is a means for automatically shutting down specific devices in a home or car at a set time.

[1787] "In-vehicle entertainment system" refers to devices and software that allow users to enjoy entertainment in an autonomous vehicle.

[1788] "Lighting" refers to a lighting device for providing brightness inside an autonomous vehicle.

[1789] This invention is a system for managing a life cycle in a home and an autonomous vehicle to lead a healthy life cycle. This system consists of three main components: a server, a terminal, and a user.

[1790] The server manages the data of the wake-up time, activity time, and bedtime set by the user, and sends corresponding notifications to the device based on this. For example, if the user sets the server to wake up at 7:00 every morning, the server will send an alarm notification to the device at exactly 7:00, and the device will ring the alarm.

[1791] The device is responsible for displaying notifications to the user, receiving the user's response (such as pressing a button), and reporting it to the server. It can be implemented as a smartphone, tablet, or a dedicated device in an autonomous vehicle. For example, the device might display a reminder saying "It's time to start work" at 9:00 AM, and when the user presses the confirmation button, the response is reported to the server.

[1792] Users can set their own schedules on the server and respond to notifications from their devices based on those settings, making it easier for users to manage their daily activity cycles.

[1793] When used in an autonomous vehicle, for example, if a user sets a wake-up time during a nap, a wake-up notification will be sent 10 minutes before the scheduled arrival time. The entertainment system and lighting will also be shut down at the set time, allowing for a comfortable in-car experience.

[1794] Hardware and software used

[1795] To implement this system, a cloud server (e.g., AWS, Google Cloud Platform) is used on the server side, and a smartphone (e.g., iPhone, Android), tablet, or in-car device (e.g., in-car infotainment system) is used on the terminal side. Such devices are connected to the Internet and have an application installed to communicate with the server.

[1796] Data calculation and processing

[1797] The server receives the life cycle data set by the user and creates a notification schedule based on this. For example, when it receives wake-up time data, it stores that data and generates a notification when the alarm time arrives. It also receives response data from the device and manages the user's behavior history. A database (e.g., MySQL, PostgreSQL) is used to process this data.

[1798] Examples of concrete examples and prompts

[1799] As a concrete example, consider the case where a user sets the device to wake up at 7:00 every morning. At 7:00, the server sends an alarm notification to the device, and the device sounds the alarm. When the user checks the alarm and presses a button, that information is sent to the server.

[1800] An example prompt might look like this:

[1801] "Generate Python code to send a nap notification so that the user can wake up 10 minutes before their scheduled arrival time."

[1802] "Generate Python code to manage work time in the car and display a reminder 10 minutes before a meeting."

[1803] This approach enables integrated and efficient life cycle management within the home and in autonomous vehicles, enabling users to maintain a healthy and comfortable lifestyle.

[1804] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1805] Step 1:

[1806] (Server) Setting user lifecycle data

[1807] Input: The user sets the wake-up time, daytime activity times, bedtime, and device shutdown time to the server.

[1808] Processing: The server receives this data and stores it in a database (e.g., MySQL, PostgreSQL).

[1809] Output: The server is ready to generate a notification schedule based on the configured data.

[1810] Step 2:

[1811] (Server) Wake-up time management

[1812] Input: The server retrieves the user's wake-up time information from the database.

[1813] Processing: When the set wake-up time approaches, the server generates an alarm notification and sends it to the device.

[1814] Output: The server sends an alarm notification to the terminal.

[1815] Step 3:

[1816] (Device) Display alarm notifications

[1817] Input: Receives alarm notifications sent from the server.

[1818] Action: The device will sound an alarm and display a wake-up notification on the screen until the user presses the wake-up button.

[1819] Output: When the user presses the button, the device sends the response data to the server.

[1820] Step 4:

[1821] (Server) Managing daytime activity times

[1822] Input: Retrieve information about the user's daily activity times from the database.

[1823] Processing: When the set activity start time approaches, the server generates a reminder notification and sends it to the device.

[1824] Output: The server sends a reminder notification to the device.

[1825] Step 5:

[1826] (Device) Displaying reminder notifications

[1827] Input: Receive reminder notifications sent from the server.

[1828] Processing: The device displays a reminder to prompt the user to start the activity. The notification continues until the user presses the confirmation button.

[1829] Output: When the user presses the button, the device sends the response data to the server.

[1830] Step 6:

[1831] (Server) Bedtime management

[1832] Input: Get the bedtime information set by the user from the database.

[1833] Processing: 30 minutes before the set bedtime, the server generates a notification to prompt the user to get ready for bed and sends it to the device.

[1834] Output: The server sends a notification to the device to prepare for bed.

[1835] Step 7:

[1836] (Device) Display bedtime notification

[1837] Input: Receive bedtime notifications sent from the server.

[1838] Processing: The device displays a notification to prompt the user to prepare for bed. The notification continues until the user presses the confirmation button.

[1839] Output: When the user presses the button, the device sends the response data to the server.

[1840] Step 8:

[1841] (Server) Device Shutdown

[1842] Input: Retrieve information about the device shutdown time set by the user from the database.

[1843] Processing: When the scheduled shutdown time arrives, the server sends a shutdown signal to the target device, and also sends a similar signal to the entertainment system and lighting in the autonomous vehicle.

[1844] Output: The server checks for shutdown and records the result.

[1845] These steps will enable integrated and efficient life cycle management within homes and autonomous vehicles.

[1846] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1847] The present invention relates to a system that provides more personalized support by combining an emotion engine that recognizes the user's emotions with a home appliance management system for living a healthy lifestyle.

[1848] System configuration

[1849] The system of the present invention consists of four main components: a server, a terminal, an emotion engine, and a user.

[1850] The server manages the data of the user's set wake-up time, activity time, and bedtime, and is responsible for sending corresponding notifications to the device based on this data.

[1851] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server.

[1852] The emotion engine is responsible for recognizing the user's emotions in real time and providing the analysis results to the server.

[1853] The user sets his or her own schedule on the server and responds to notifications from the terminal based on that setting.

[1854] Program processing

[1855] The processing of the program of the present invention will be explained in natural language below.

[1856] 1. Manage your wake-up time

[1857] The user sets the wake-up time each morning on the server.

[1858] When the set time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device.

[1859] The device will sound an alarm and display a wake-up notification on the screen.

[1860] When the user presses the wake-up button, the terminal reports the operation to the server.

[1861] The emotion engine analyzes the user's emotional state in real time when they wake up and provides the results to the server.

[1862] 2. Daytime activity management

[1863] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[1864] When the time to start the activity arrives, the server adjusts the content of the reminder message based on the analysis results of the emotion engine and sends a reminder notification to the device.

[1865] For example, if the user's emotional state is low, the device will display a reminder with an encouraging message.

[1866] When the user presses the confirmation button, the terminal reports the operation to the server.

[1867] The emotion engine analyzes the user's emotional state in real time while they are active and provides information to the server as needed.

[1868] 3. Bedtime notifications

[1869] The user sets the time they go to bed each night on the server.

[1870] Thirty minutes before bedtime, the server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device.

[1871] For example, if the user is relaxing, the notification may be displayed with soft music in the background.

[1872] The device will display a notification to remind the user to get ready for bed.

[1873] When the user presses the prepare button, the terminal reports the operation to the server.

[1874] 4. Shut down the device

[1875] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[1876] When the shutdown time arrives, the server sends a shutdown signal to these devices.

[1877] The terminal receives the shutdown signal and initiates shutdown of the target device.

[1878] The terminal checks whether the device shutdown has been completed successfully and reports the result to the server.

[1879] Specific examples

[1880] 1. Manage your wake-up time

[1881] For example, a user configures the server to wake up at 7am every morning.

[1882] The server sends an alarm notification to the device at exactly 7:00, and the device plays an appropriate alarm sound based on the results of the emotion engine.

[1883] When the user presses the wake-up button, the device sends the information to the server, and the emotion engine analyzes and reports the user's emotions.

[1884] 2. Daytime activity management

[1885] For example, a user may set his working hours from 9:00 AM to 12:00 PM.

[1886] At 9 a.m., the server uses the emotion engine's analysis to send a reminder to the device, including the encouraging message, "It's time to get to work."

[1887] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[1888] 3. Bedtime notifications

[1889] For example, a user sets the time to go to bed at 10 o'clock every night.

[1890] At 9:30 p.m., the server uses the emotion engine's analysis results to send a bedtime notification to the device, including a relaxing message.

[1891] The device will display a notification to remind the user to get ready for bed.

[1892] 4. Shut down the device

[1893] For example, a user can set the TV and Wi-Fi router in their home to shut down at 10:00.

[1894] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[1895] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[1896] As described above, the system of the present invention not only manages household appliances but also provides more personalized lifestyle management by taking into account the user's emotions.

[1897] The processing flow will be explained below.

[1898] Wake-up time management

[1899] Step 1:

[1900] The user logs into the server and sets the time to wake up each morning.

[1901] Step 2:

[1902] The server stores the wake-up time entered by the user in a database.

[1903] Step 3:

[1904] Five minutes before the set wake-up time, the server prepares for the alarm notification.

[1905] Step 4:

[1906] The emotion engine analyzes the user's current emotional state.

[1907] Step 5:

[1908] When the set wake-up time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device.

[1909] Step 6:

[1910] The device will sound an alarm and display a wake-up notification on the screen.

[1911] Step 7:

[1912] When the user presses the wake-up button on the terminal, the terminal reports the operation to the server.

[1913] Step 8:

[1914] The server confirms that the user has pressed the wake-up button and records the wake-up confirmation information in the database.

[1915] Daytime activity management

[1916] Step 1:

[1917] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[1918] Step 2:

[1919] The server stores the activity time entered by the user in a database.

[1920] Step 3:

[1921] Five minutes before the activity start time, the server prepares the reminder notification.

[1922] Step 4:

[1923] The emotion engine analyzes the user's current emotional state.

[1924] Step 5:

[1925] When the time to start the activity arrives, the server adjusts the content of the reminder message based on the analysis results of the emotion engine and sends a reminder notification to the device.

[1926] Step 6:

[1927] The device will display a reminder on the screen, telling the user, "It's time to start work."

[1928] Step 7:

[1929] When the user presses the confirmation button on the terminal, the terminal reports the operation to the server.

[1930] Step 8:

[1931] The server confirms that the user has pressed the confirmation button and records the confirmation information for the start of the activity in the database.

[1932] Bedtime notifications

[1933] Step 1:

[1934] The user sets the time they go to bed each night on the server.

[1935] Step 2:

[1936] The server stores the bedtime entered by the user in a database.

[1937] Step 3:

[1938] Thirty minutes before bedtime, the server prepares the bedtime notification.

[1939] Step 4:

[1940] The emotion engine analyzes the user's current emotional state.

[1941] Step 5:

[1942] The server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device.

[1943] Step 6:

[1944] The device displays the message "Bedtime in 30 minutes" on the screen, encouraging the user to get ready for bed.

[1945] Step 7:

[1946] When the user presses the prepare button, the terminal reports the operation to the server.

[1947] Step 8:

[1948] The server confirms that the user has pressed the prepare button and records the confirmation of preparation in the database.

[1949] Equipment shutdown

[1950] Step 1:

[1951] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[1952] Step 2:

[1953] The server stores the shutdown time entered by the user in a database.

[1954] Step 3:

[1955] Five minutes before the shutdown time, the server prepares for the shutdown notification.

[1956] Step 4:

[1957] When the shutdown time arrives, the server sends a shutdown signal to the target device.

[1958] Step 5:

[1959] The terminal receives the shutdown signal and initiates shutdown of the target device.

[1960] Step 6:

[1961] The terminal checks whether the shutdown of the target device has been completed successfully.

[1962] Step 7:

[1963] The terminal reports that the shutdown is complete to the server, and the server records the information in a database.

[1964] The above is a specific flow of processing steps in the embodiment of the present invention. By having the server, terminal, emotion engine, and user fulfill their respective roles, it becomes possible to maintain a healthy lifestyle.

[1965] Example 2

[1966] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1967] Conventional home device management systems have the ability to send notifications and reminders based on the user's schedule, but they have the problem of not being able to provide a personalized experience because they do not take the user's emotional state into consideration.In addition, they cannot appropriately adjust the content of notifications or alarm sounds depending on the user's emotional state, resulting in situations where users do not receive sufficient support or reminders.

[1968] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1969] In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time, means for checking the user's response to the alarm, means for setting a daytime activity time, means for sending a reminder at the set activity time, means for checking the user's response to the reminder, means for setting a bedtime, means for sending a notification urging the user to get ready for bed at the set bedtime, means for shutting down home appliances at bedtime, means for analyzing the user's emotional state in real time, and means for adjusting the notification content and alarm volume based on the user's emotional state. This makes it possible to provide personalized notifications and alarms according to the user's emotional state, thereby enabling the user's lifestyle to be managed more healthily and efficiently.

[1970] The "means for setting the wake-up time" is a function that allows the user to specify the wake-up time each morning and register it in the server.

[1971] The "means for sending an alarm at a set wake-up time" is a function that sends an alarm notification from the server to the terminal at a set time.

[1972] The "means for checking the user's response to the alarm" is a function that detects whether the user has responded to the alarm (for example, by pressing the wake-up button) and reports that information to the server.

[1973] The "means for setting daytime activity times" is a function that allows the user to specify specific activity times during the day and register them on the server.

[1974] The "means for sending a reminder at a set activity time" is a function for sending a reminder notification from the server to the terminal when the set activity time arrives.

[1975] The "means for confirming the user's response to the reminder" is a function that detects whether the user has responded to the reminder notification by pressing a confirmation button, and reports that information to the server.

[1976] The "means for setting bedtime" is a function that allows the user to specify the time they go to bed each night and register it in the server.

[1977] The "means for sending a notification prompting the user to get ready for bed at a set bedtime" is a function for sending a notification prompting the user to get ready for bed from the server to the terminal before the set bedtime.

[1978] The "means for shutting down devices in the home at bedtime" is a function that sends a shutdown signal from the server to specific devices in the home when the set bedtime arrives.

[1979] "Means for analyzing the user's emotional state in real time" is a function that analyzes the user's facial expressions, voice, etc. using emotion recognition technology to grasp the user's emotional state in real time.

[1980] "Means for adjusting notification content and alarm volume based on the user's emotional state" is a function for optimizing the content and volume of notifications and alarms based on the analysis of the user's emotional state.

[1981] The present invention relates to a home appliance management system that supports users in maintaining a healthy lifestyle. The system recognizes the user's emotional state in real time and adjusts notification content and alarm volume based on that information to provide more personalized support.

[1982] System configuration

[1983] The system of the present invention consists of four main components: a server, a terminal, an emotion engine, and a user.

[1984] The server manages the data of the user's set wake-up time, daytime activity time, and bedtime, and sends corresponding notifications to the device based on this data. Specifically, a cloud server such as Amazon EC2 or Google Cloud can be used.

[1985] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server. The terminal can be an iOS or Android device.

[1986] The emotion engine recognizes the user's emotions in real time and provides the analysis results to the server. Specifically, it uses emotion recognition APIs from IBM Watson and Microsoft Azure.

[1987] The user sets his or her own schedule in the server and responds to notifications from the terminal based on that schedule.

[1988] Wake-up time management

[1989] The user sets the wake-up time each morning on the server through a dedicated application. When the set time arrives, the server determines the optimal alarm volume and tone based on the analysis results of the emotion engine and sends an alarm notification to the device. The device then plays the set music and displays "It's time to wake up" on the screen. When the user presses the wake-up button, the device sends that information to the server, and the emotion engine analyzes and reports the user's emotions.

[1990] Examples:

[1991] For example, a user sets the server to wake up at 7:00 every morning. The server sends an alarm notification to the device at exactly 7:00, and the device sounds an appropriate alarm sound based on the results of the emotion engine. When the user presses the wake-up button, the device sends that information to the server, and the emotion engine analyzes and reports the user's emotions. An example of a prompt sentence is, "Write a program that sets the user to wake up at 7:00, and adjusts the alarm volume based on the analysis results of the emotion engine."

[1992] Daytime activity management

[1993] The user sets the time of daytime activities on the server through a dedicated app. For example, they can set "work hours from 9:00 AM to 12:00 PM." When it's time to start the activity, the server adjusts the reminder message based on the analysis results of the emotion engine and sends it to the device. The device displays the reminder, and when the user presses the confirmation button, it reports the operation to the server.

[1994] Examples:

[1995] For example, a user sets their working hours from 9:00 AM to 12:00 PM. At 9:00 AM, the server reflects the analysis results of the emotion engine and sends a reminder to the device containing an encouraging message saying, "It's time to start work." The device displays the reminder, and when the user presses the confirmation button, the information is sent to the server. An example of a prompt sentence is, "Write a program that sets the user's daytime activity hours as 9:00 AM to 12:00 PM and adjusts the reminder content based on the results of the emotion engine."

[1996] Bedtime notifications

[1997] The user sets their nightly bedtime on the server via a dedicated app. 30 minutes before their set bedtime, the server adjusts the notification content based on the analysis results of the emotion engine and sends a bedtime preparation notification to the device. The device displays the notification, and when the user presses the bedtime preparation button, the information is sent to the server.

[1998] Examples:

[1999] For example, a user sets the bedtime to 10 o'clock every night. At 9:30 p.m., the server reflects the analysis results of the emotion engine and sends a bedtime notification with a relaxing message to the device. The device displays the notification, and when the user presses the ready button, the information is sent to the server. An example of a prompt sentence is, "Write a program that sets the user's bedtime to 10 o'clock and plays calm music based on the results of the emotion engine."

[2000] Shut down household devices

[2001] The user sets the shutdown time for a specific device in the home on the server through a dedicated application. When the shutdown time arrives, the server sends a shutdown signal to the target device, and the device receives the signal and starts shutting down the device. The device confirms that the device has been shut down successfully and reports the status to the server.

[2002] Examples:

[2003] For example, a user sets the TV and Wi-Fi router in their home to shut down at 10 o'clock. At 10 pm, the server sends a shutdown signal to the TV and Wi-Fi router. The device confirms that these devices have shut down successfully and reports the results to the server. An example of a prompt sentence is, "Write a program that shuts down the TV and Wi-Fi router in your home every night at 10 o'clock. Please also add a function to report the results to the server."

[2004] As described above, the system of the present invention not only manages household appliances but also provides more personalized lifestyle support by taking into account the emotional state of the user.

[2005] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2006] Program processing steps

[2007] Wake-up time management

[2008] Step 1:

[2009] The user sets the wake-up time on the server.

[2010] Input: User's wake-up time (e.g. 7:00)

[2011] Specific operation: The user enters the wake-up time in a dedicated application and registers it on the server.

[2012] Output: The wake-up time data is saved on the server.

[2013] Step 2:

[2014] The server prepares the notification based on the configured wake-up time.

[2015] Input: Wake-up time stored on the server

[2016] Specific operation: The server prepares an alarm notification based on the set wake-up time (e.g., 7:00).

[2017] Output: Alarm notification is ready.

[2018] Step 3:

[2019] An emotion engine analyzes the user's emotional state.

[2020] Input: Real-time user data (face recognition, voice data, etc.)

[2021] Specific operation: The emotion engine analyzes the user's face and voice through the camera and microphone to analyze their emotional state.

[2022] Output: Emotional state data (e.g., relaxed state)

[2023] Step 4:

[2024] The server determines the optimal alarm volume and tone and sends a notification to the device.

[2025] Input: Emotional state data, saved wake-up time

[2026] Specific operation: Based on the results of the emotion engine, the server selects the optimal alarm volume and tone and sends an alarm notification to the device.

[2027] Output: An alarm notification is sent to the device.

[2028] Step 5:

[2029] Your device will sound an alarm and display a wake-up notification.

[2030] Input: Alarm notification

[2031] Specific actions: The device receives a notification, plays the set music, and displays a wake-up time notification on the screen.

[2032] Output: An alarm will sound and a wake-up notification will appear on the screen.

[2033] Step 6:

[2034] The user presses the wake-up button.

[2035] Input: User button presses

[2036] Specific operation: The user presses the alarm stop button on the terminal.

[2037] Output: The alarm stops and operation data is generated on the terminal.

[2038] Step 7:

[2039] The terminal reports the user's operations to the server.

[2040] Input: Operation data (alarm stop button pressed)

[2041] Specific operation: The terminal transmits the user's button operation information to the server.

[2042] Output: Operation information is reported to the server.

[2043] Step 8:

[2044] The emotion engine analyzes the user's emotional state when they wake up in real time and provides the information to the server.

[2045] Input: Real-time user data (face recognition, voice data, etc.)

[2046] Specific operation: The emotion engine re-analyzes the user's emotions upon waking up and sends the results to the server.

[2047] Output: The emotional state data is provided to the server.

[2048] Daytime activity management

[2049] Step 1:

[2050] The user sets the daytime activity times on the server.

[2051] Input: User activity time (e.g. 9:00 to 12:00)

[2052] Specific operation: The user inputs the activity time in a dedicated application and registers it on the server.

[2053] Output: Activity time data is saved on the server.

[2054] Step 2:

[2055] The server prepares notifications based on configured active times.

[2056] Input: Activity time stored on the server

[2057] Specific behavior: The server prepares a reminder notification based on the set activity time (e.g. 9:00).

[2058] Output: Reminder notification is ready.

[2059] Step 3:

[2060] The emotion engine analyzes the user's emotional state in real time during activity.

[2061] Input: Real-time user data (face recognition, voice data, etc.)

[2062] Specific operation: The emotion engine analyzes the user's facial expressions and voice to analyze the user's emotional state in real time during an activity.

[2063] Output: Emotional state data during the activity

[2064] Step 4:

[2065] The server determines the optimal message content and sends a notification to the device.

[2066] Input: Emotional state data during activity, saved activity time

[2067] Specific operation: Based on the results of the emotion engine, the server selects a message, such as "Good luck," and sends a reminder notification to the device.

[2068] Output: A reminder notification is sent to your device.

[2069] Step 5:

[2070] Your device will display the reminder.

[2071] Input: Reminder

[2072] Specific behavior: The device receives a notification and displays something like, "It's 9 o'clock, time to get to work."

[2073] Output: The reminder will be displayed on your device.

[2074] Step 6:

[2075] The user presses the confirm button.

[2076] Input: User confirmation button operation

[2077] Specific operation: The user presses the confirmation button in response to the reminder notification.

[2078] Output: Operation data is generated when the confirmation button is pressed.

[2079] Step 7:

[2080] The terminal reports the user's operations to the server.

[2081] Input: Operation data (Confirm button pressed)

[2082] Specific operation: The terminal transmits the user's confirmation button operation information to the server.

[2083] Output: Operation information is reported to the server.

[2084] Bedtime notifications

[2085] Step 1:

[2086] The user sets the bedtime on the server.

[2087] Input: User's bedtime (e.g. 10:00)

[2088] Specific operation: The user enters the bedtime in a dedicated application and registers it on the server.

[2089] Output: Bedtime data is saved on the server.

[2090] Step 2:

[2091] The server prepares a notification before bedtime.

[2092] Input: Bedtime stored on the server

[2093] Specific operation: The server prepares a bedtime notification based on the set time 30 minutes before bedtime (e.g. 9:30).

[2094] Output: Bedtime notification is ready.

[2095] Step 3:

[2096] An emotion engine analyzes the user's emotional state.

[2097] Input: Real-time user data (face recognition, voice data, etc.)

[2098] Specific operation: The emotion engine analyzes the user's face and voice to analyze their emotional state before going to sleep.

[2099] Output: Emotional state data before going to bed

[2100] Step 4:

[2101] The server determines the optimal notification content and sends it to the device.

[2102] Input: emotional state data before bedtime, saved bedtime

[2103] Specific operation: Based on the results of the emotion engine, the server selects calm music if the user is in a relaxed state and sends a notification to the device to prepare for bed.

[2104] Output: A bedtime notification is sent to the device.

[2105] Step 5:

[2106] Your device will display a bedtime notification.

[2107] Input: Bedtime notification

[2108] Specific behavior: The device receives a notification and displays "Start getting ready for bed."

[2109] Output: A bedtime notification will appear on your device.

[2110] Step 6:

[2111] The user presses the ready for bed button.

[2112] Input: User confirmation button operation

[2113] Specific Action: The user presses the Ready for Bed button.

[2114] Output: Operation data is generated when the ready button is pressed.

[2115] Step 7:

[2116] The terminal reports the user's operations to the server.

[2117] Input: Operation data (Ready button pressed)

[2118] Specific operation: The terminal sends the user's preparation completion button operation information to the server.

[2119] Output: Operation information is reported to the server.

[2120] Shut down household devices

[2121] Step 1:

[2122] The user sets the shutdown time of the home appliances in the server.

[2123] Input: User shutdown time (e.g. 10:00)

[2124] Specific operation: The user enters the shutdown time in a dedicated application and registers it on the server.

[2125] Output: Shutdown time data is saved on the server.

[2126] Step 2:

[2127] The server prepares a notification when it is time to shut down.

[2128] Input: Shutdown time stored on the server

[2129] Specific operation: The server prepares a shutdown notification based on the configured shutdown time (e.g., 10:00).

[2130] Output: Shutdown notification is ready.

[2131] Step 3:

[2132] The server sends a shutdown signal to the home appliance.

[2133] Input: Shutdown notification

[2134] Specific operation: When the set shutdown time arrives, the server sends a shutdown signal to the specified home devices (TVs and Wi-Fi routers).

[2135] Output: A shutdown signal is sent to the home appliance.

[2136] Step 4:

[2137] The terminal initiates a shutdown of the target device.

[2138] Input: Shutdown signal

[2139] Specific operation: The terminal receives the shutdown signal and starts the shutdown process of the specified device.

[2140] Output: Equipment shutdown is initiated.

[2141] Step 5:

[2142] The terminal confirms that the device has been shut down and reports this to the server.

[2143] Input: Device shutdown state

[2144] Specific operation: The device checks whether the devices have been shut down successfully and reports the result to the server, for example, notifying the server that "The TV and Wi-Fi router have been shut down successfully."

[2145] Output: The shutdown state is reported to the server.

[2146] (Application example 2)

[2147] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2148] Conventional home appliance management systems send uniform notifications and alarms without considering the user's emotions, resulting in a lack of personalization tailored to the user's psychological state. Furthermore, in-store shopping experiences are also challenged by the inability to provide support tailored to the individual user's emotional state, which can easily lead to dissatisfaction, especially when the user's emotions are unstable.

[2149] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time and adjusting the alarm sound based on the emotion analysis result, and means for checking the user's response to the alarm and sending the emotion analysis result to the server. This makes it possible to provide personalized alarm notifications and daytime activity reminders according to the user's emotional state.

[2150] The system also includes a means for providing personalized navigation and product recommendations based on the user's emotional state in a physical store, thereby enabling optimal support according to the user's emotions even in the physical store, and providing a more satisfying shopping experience.

[2151] The "means for setting a wake-up time" is a function that allows the user to set in advance the wake-up time each morning.

[2152] "Means for sending alarms" is a function that notifies you by voice or visually at a set time.

[2153] "Emotion analysis results" are the results of analyzing the user's emotional state in real time from their face and behavior, and providing the results as data.

[2154] "Means for adjusting alarm sound" is a function that selects the optimal volume and tone based on the results of emotion analysis.

[2155] The "means for checking the user's response to the alarm" is a function that detects the operation the user has performed on the alarm (such as snooze or stop) and collects that information.

[2156] The "means for setting daytime activity times" is a function that allows the user to set specific activity times (e.g., work time or exercise time).

[2157] The "means for sending a reminder" is a function for sending a reminder notification at a set activity time and notifying the user of the same.

[2158] The "means for adjusting reminder content" is a function that optimizes the content of reminder messages and notifications based on the user's emotional state.

[2159] The "means for setting bedtime" is a function that allows the user to set the time they go to bed each night.

[2160] The "means for sending a notification to urge the user to get ready for bed" is a function that sends a notification to urge the user to get ready for bed when the set bedtime approaches.

[2161] "Means for shutting down household devices" refers to a function that turns off household devices such as televisions and Wi-Fi routers at bedtime.

[2162] The "emotion analysis means" is a function for recognizing and analyzing the user's emotional state in real time and providing the results to the system.

[2163] "Means for personalized navigation and product recommendations" refers to a function that recommends optimal routes and products based on the user's emotional state within a physical store.

[2164] The present invention is a system for personalizing user experiences in home devices and brick-and-mortar stores using emotion analysis. The system consists of four main components: a server, a terminal, an emotion analysis means, and a user.

[2165] System configuration

[2166] 1. Server

[2167] The server manages schedule data set by the user, such as wake-up time, daytime activity times, and bedtime, and also collects emotional data provided by an emotion analysis tool, optimizing various notifications and recommendations based on this data.

[2168] 2. Terminal

[2169] The device can be a smartphone, smart glasses, or a head-mounted display (HMD), and it displays notifications to the user, collects the user's responses, and sends them to a server. It also provides an interface for performing actions such as shutting down home appliances.

[2170] 3. Emotion analysis method

[2171] Emotion analyzers are software that uses cameras and sensors to capture a user's face and behavior, and analyzes them to identify their emotional state, including the Microsoft Azure Emotion API and Affectiva SDK.

[2172] 4. Users

[2173] Users set their own schedules on the server and receive notifications from their devices based on those schedules. In physical stores, users can also use navigation and product recommendations based on their emotional state.

[2174] Program processing

[2175] Hardware / Software used

[2176] Smartphone (iOS or Android)

[2177] Emotion analysis software (Microsoft Azure Emotion API, Affectiva SDK)

[2178] Server (AWS Lambda + DynamoDB)

[2179] UI framework (React Native)

[2180] Data processing and calculation

[2181] The server collects and manages user settings and emotion analysis data, and determines optimized alarm sounds and reminder content based on this data.

[2182] The emotion analysis means analyzes the emotions of the user from their face and actions in real time, and sends the results to the server as data.

[2183] The device receives and displays notifications from the server, reports user responses, and, in physical stores, displays data for navigation and product recommendations.

[2184] Specific examples

[2185] 1. Manage your wake-up time

[2186] The user sets the time to wake up at 7am every morning.

[2187] At 7:00, the server determines the optimal alarm sound based on the emotion analysis results and sends a notification to the device.

[2188] The terminal sounds an alarm, and when the user presses the wake-up button, the information is sent to the server, and the emotion analysis means analyzes the emotion and provides the result to the server.

[2189] 2. Daytime activity management

[2190] A user sets their work hours as 9:00 AM to 12:00 PM.

[2191] At 9 a.m., the server uses the results of emotion analysis to send a reminder to the device, including the encouraging message, "It's time to get to work."

[2192] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[2193] 3. Bedtime notifications

[2194] The user sets the bedtime to 10 o'clock every night.

[2195] At 9:30 p.m., the server uses the results of the emotion analysis to send a bedtime notification to the device, including a relaxing message.

[2196] The device will display a notification to remind the user to get ready for bed.

[2197] 4. Shut down the device

[2198] A user sets the TV and Wi-Fi router in their home to shut down at 10:00.

[2199] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[2200] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[2201] Example prompts to input to the generative AI model

[2202] "The user's current emotional state is joy. If the emotion of joy is high, what is the next recommended product in the store?"

[2203] "If a user's emotional state is low, what message would be effective in encouraging them?"

[2204] By utilizing sentiment analysis, this system significantly personalizes users' lifestyles and in-store experiences, helping them live richer lives.

[2205] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2206] Step 1:

[2207] The user sets the wake-up time using a smartphone device. The input is the wake-up time entered by the user into the app, and the output is the wake-up time data stored on the server. The wake-up time set on the smartphone app is sent to the server and stored in a database.

[2208] Step 2:

[2209] When the set wake-up time arrives, the server requests the emotion analysis means to capture the user's face and behavior. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analysis means captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[2210] Step 3:

[2211] The server determines the optimal alarm volume and tone based on the acquired emotional data. The input is the emotional analysis data, and the output is the alarm sound setting. The server adjusts the alarm sound according to the emotional state and sends the setting to the device.

[2212] Step 4:

[2213] The device plays the adjusted alarm sound to prompt the user to wake up. The input is the alarm sound setting sent from the server, and the output is the alarm sound and wake-up notification. The device waits for the user to press the wake-up button.

[2214] Step 5:

[2215] When a user presses the wake-up button, the device reports the operation to the server. The input is the user pressing the wake-up button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with the emotion data at the time of waking up.

[2216] Step 6:

[2217] The user sets the activity time during the day (e.g., work hours). The input is the activity time entered by the user into the app, and the output is the activity time data stored on the server. The activity time set on the smartphone app is sent to the server and stored in the database.

[2218] Step 7:

[2219] When the set activity time arrives, the server requests the user's emotional data from the emotion analysis means. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analysis means captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[2220] Step 8:

[2221] The server generates an appropriate reminder message based on the acquired emotional data. The input is the emotion analysis data, and the output is the reminder message. The server optimizes the reminder content according to the emotional state and sends the message to the device.

[2222] Step 9:

[2223] The terminal displays the generated reminder message and prompts the user to start the activity. The input is the reminder message sent from the server, and the output is the reminder notification. The terminal waits for the user to press the confirmation button.

[2224] Step 10:

[2225] When the user presses the confirmation button, the device reports the operation to the server. The input is the user pressing the confirmation button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with the emotion data during the activity.

[2226] Step 11:

[2227] The user sets a bedtime. The input is the bedtime entered by the user into the app, and the output is the bedtime data stored on the server. The bedtime set on the smartphone app is sent to the server and stored in a database.

[2228] Step 12:

[2229] Thirty minutes before bedtime, the server requests the user's emotional data from the emotion analyzer. The input is the set time and a request to capture emotional data, and the output is data indicating the user's emotional state. The emotion analyzer captures the user's face and analyzes the emotional data in real time using the Microsoft Azure Emotion API.

[2230] Step 13:

[2231] The server generates an appropriate bedtime notification based on the acquired emotional data. The input is the emotion analysis data, and the output is the bedtime notification. The server optimizes the notification content according to the emotional state and sends the message to the device.

[2232] Step 14:

[2233] The terminal displays the generated bedtime notification and prompts the user to get ready for bed. The input is the bedtime notification sent from the server, and the output is the bedtime notification. The terminal waits for the user to press the bedtime button.

[2234] Step 15:

[2235] When a user presses the "Get ready for bed" button, the device reports the operation to the server. The input is the user pressing the "Get ready for bed" button, and the output is the user's response data sent to the server. The server receives the user's response data and records it along with emotional data at bedtime.

[2236] Step 16:

[2237] When the set bedtime arrives, the server sends a shutdown signal to home devices. The input is the set shutdown time, and the output is the shutdown signal. The server sends a signal to home devices such as TVs and Wi-Fi routers to initiate shutdown.

[2238] Step 17:

[2239] The terminal confirms receipt of the shutdown signal and checks whether the shutdown of the home device has been completed successfully. The input is the shutdown signal sent from the server, and the output is the shutdown confirmation result. Once the terminal confirms that the shutdown has been completed, it reports the result to the server.

[2240] This series of steps will enable the realization of a system that provides personalized support according to the user's emotional state, thereby enriching daily life and shopping experiences.

[2241] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2242] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2243] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2244] [Fourth embodiment]

[2245] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2246] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2247] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2248] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2249] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2250] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2251] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2252] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2253] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2254] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2255] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2256] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2257] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2258] The present invention relates to a home appliance management system for leading a healthy lifestyle.

[2259] System configuration

[2260] The system of the present invention consists of three main components: a server, a terminal, and a user.

[2261] The server manages the data of the user's set wake-up time, activity time, and bedtime, and is responsible for sending corresponding notifications to the device based on this data.

[2262] The terminal is responsible for displaying notifications to the user, receiving the user's responses (such as pressing a button) and reporting them to the server.

[2263] The user sets his or her own schedule on the server and responds to notifications from the terminal based on that setting.

[2264] Program processing

[2265] The processing of the program of the present invention will be explained in natural language below.

[2266] 1. Manage your wake-up time

[2267] The user sets the wake-up time each morning on the server.

[2268] The server sends an alarm notification to the terminal at the set time.

[2269] The device will sound an alarm and continue to notify you until the user presses the wake-up button.

[2270] When the user presses the button, the terminal sends wake-up confirmation data to the server.

[2271] 2. Daytime activity management

[2272] The user sets the daytime activity times on the server (for example, 9:00 AM to 12:00 PM).

[2273] The server sends a reminder to the device when it is time to start the activity.

[2274] The device will display a reminder and continue to notify you until you press the confirmation button.

[2275] When the user presses the confirmation button, the terminal transmits confirmation data for the start of the activity to the server.

[2276] 3. Bedtime notifications

[2277] The user sets the time they go to bed each night on the server.

[2278] The server sends a notification to the device to prepare for bed 30 minutes before going to bed.

[2279] The device will display a notification to remind the user to get ready for bed.

[2280] 4. Shut down the device

[2281] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[2282] The server sends a shutdown signal to these devices at the set time.

[2283] The terminal checks whether the device has been shut down normally and reports the result to the server.

[2284] Specific examples

[2285] 1. Manage your wake-up time

[2286] For example, a user configures the server to wake up at 7am every morning.

[2287] The server sends an alarm notification to the device at exactly 7:00, and the device sounds an alarm.

[2288] When the user presses the wake-up button, the terminal transmits the information to the server to confirm the wake-up.

[2289] 2. Daytime activity management

[2290] For example, a user may set his working hours from 9:00 AM to 12:00 PM.

[2291] At 9:00 AM, the server sends a reminder to the device saying, "It's time to get to work."

[2292] The terminal displays the reminder, and when the user presses the confirmation button, confirmation information is sent to the server.

[2293] 3. Bedtime notifications

[2294] For example, a user sets the time to go to bed at 10 o'clock every night.

[2295] At 9:30 p.m., the server sends a notification to the device saying, "It's bedtime in 30 minutes."

[2296] The device will display a notification to remind the user to get ready for bed.

[2297] 4. Shut down the device

[2298] For example, a user can set the TV and Wi-Fi router in their home to shut down at 10:00.

[2299] At 10 p.m., the server sends a shutdown signal to the TV and Wi-Fi router.

[2300] The terminal checks whether these devices have been shut down normally and reports the result to the server.

[2301] As described above, the system of the present invention manages the user's life cycle according to a set time and provides a specific means for supporting a healthy lifestyle.

[2302] The processing flow will be explained below.

[2303] Wake-up time management

[2304] Step 1:

[2305] The user logs into the server and sets the time to wake up each morning.

[2306] Step 2:

[2307] The server stores the wake-up time entered by the user in a database.

[2308] Step 3:

[2309] Five minutes before the set wake-up time, the server prepares for the alarm notification.

[2310] Step 4:

[2311] When the set wake-up time arrives, the server sends an alarm notification to the terminal.

[2312] Step 5:

[2313] The device will sound an alarm and display a wake-up notification on the screen.

[2314] Step 6:

[2315] When the user presses the wake-up button on the terminal, the terminal reports the operation to the server.

[2316] Step 7:

[2317] The server confirms that the user has pressed the wake-up button and records the wake-up confirmation information in the database.

[2318] Daytime activity management

[2319] Step 1:

[2320] The user sets the daytime activity hours (for example, 9:00 AM to 12:00 PM) on the server.

[2321] Step 2:

[2322] The server stores the activity time entered by the user in a database.

[2323] Step 3:

[2324] Five minutes before the activity start time, the server prepares the reminder notification.

[2325] Step 4:

[2326] When the activity start time arrives, the server sends a reminder notification to the device.

[2327] Step 5:

[2328] The device will display a reminder on the screen, telling the user, "It's time to start work."

[2329] Step 6:

[2330] When the user presses the confirmation button on the terminal, the terminal reports the operation to the server.

[2331] Step 7:

[2332] The server confirms that the user has pressed the confirmation button and records the confirmation information for the start of the activity in the database.

[2333] Bedtime notifications

[2334] Step 1:

[2335] The user sets the time they go to bed each night on the server.

[2336] Step 2:

[2337] The server stores the bedtime entered by the user in a database.

[2338] Step 3:

[2339] Thirty minutes before bedtime, the server prepares the bedtime notification.

[2340] Step 4:

[2341] Thirty minutes before bedtime, the server sends a notification to the device to prepare for bed.

[2342] Step 5:

[2343] The device displays the message "Bedtime in 30 minutes" on the screen, encouraging the user to get ready for bed.

[2344] Step 6:

[2345] When the user presses the prepare button, the terminal reports the operation to the server.

[2346] Step 7:

[2347] The server confirms that the user has pressed the prepare button and records the confirmation of preparation in the database.

[2348] Equipment shutdown

[2349] Step 1:

[2350] Users set shutdown times for specific devices in their homes (such as televisions and Wi-Fi routers) on the server.

[2351] Step 2:

[2352] The server stores the shutdown time entered by the user in a database.

[2353] Step 3:

[2354] Five minutes before the shutdown time, the server prepares for the shutdown notification.

[2355] Step 4:

[2356] When the set shutdown time arrives, the server sends a shutdown signal to the target device.

[2357] Step 5:

[2358] The terminal receives the shutdown signal and initiates shutdown of the target device.

[2359] Step 6:

[2360] The terminal checks whether the shutdown of the target device has been completed successfully.

[2361] Step 7:

[2362] The terminal reports that the shutdown is complete to the server, and the server records the information in a database.

[2363] Example 1

[2364] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2365] Maintaining a regular lifestyle is important for improving health and productivity. However, in today's busy lives, daily time management is becoming increasingly difficult. Conventional methods require users to manually manage their time, including waking up, daily activities, going to bed, and managing home devices, which can be cumbersome. This creates a particularly difficult problem for maintaining a healthy lifestyle.

[2366] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2367] In this invention, the server includes means for a user to set a wake-up time, means for sending an alarm when the set wake-up time arrives, means for the terminal to sound the alarm and confirm the user's response, means for a user to set a daytime activity time, means for sending a reminder at the set activity time, means for the terminal to display the reminder and confirm the user's response, means for a user to set a bedtime, means for sending a notification urging the user to get ready for bed 30 minutes before the set bedtime, means for a user to set a shutdown time for home appliances, and means for sending a shutdown signal to the home appliances at the set shutdown time. This makes it possible to automatically manage a user's lifestyle via the server.

[2368] "User" refers to an individual who uses this system.

[2369] "Server" refers to a computer system that records and manages user settings and transmits each piece of information to the terminal.

[2370] "Terminal" refers to a device that displays notifications and reminders from the server to the user and receives the user's responses.

[2371] "Means for setting a wake-up time" refers to the specific functions and methods by which a user inputs and sets the wake-up time each morning.

[2372] "Means for sending an alarm" refers to a specific function or method for sending an alarm notification from the server to the terminal at the set wake-up time.

[2373] The term "means for the terminal to sound an alarm and confirm the user's response" refers to a function or method for the terminal to sound an alarm and receive a user's wake-up confirmation operation.

[2374] The "means for setting daytime activity times" refers to a specific function or method for a user to set a specific time period as a daytime activity time.

[2375] "Means for sending reminders" refers to specific functions and methods for sending reminder notifications from the server to the device when the set activity time arrives.

[2376] The "means for the terminal to display the reminder and confirm the user's response" refers to a function or method for the terminal to display the reminder and receive a confirmation operation from the user.

[2377] "Means for setting bedtime" refers to the specific functions and methods by which a user can input and set their nightly bedtime.

[2378] "Means for sending a notification encouraging users to get ready for bed" refers to the specific functions and methods for sending a notification from the server to the device prompting users to get ready for bed 30 minutes before the set bedtime.

[2379] "Means for setting the shutdown time of devices in the home" refers to specific functions and methods that allow a user to input and set the shutdown time of a specific device in the home.

[2380] "Means for sending a shutdown signal to a device in the home" refers to a specific function or method for sending a shutdown signal from the server to a specific device in the home when the set shutdown time arrives.

[2381] "Means for confirming that home devices have been shut down normally" refers to specific functions and methods that allow a terminal to check the shutdown status of home devices and report the results to a server.

[2382] This invention relates to a home appliance management system for promoting a healthy lifestyle. This system consists of three main components: a server, a terminal, and a user. Below, we will explain in detail how each component operates, including the hardware and software used.

[2383] Server Roles

[2384] The server manages various schedules set by the user (wake-up time, activity time, bedtime, shutdown time of home devices) and sends appropriate notifications to the device based on those schedules. The server records the user's setting data in a database (e.g., MySQL), and generates corresponding notifications at specific times and sends them to the device. For example, the server sends an alarm notification to the device when the set wake-up time (e.g., 7:00 AM) arrives.

[2385] Device Role

[2386] The device is responsible for displaying notifications sent from the server (such as alarms, reminders, and bedtime notifications) to the user and receiving the user's responses. The device also has the function of reporting the user's responses (such as pressing a button) to the server. For example, if the device sounds an alarm and the user presses the "wake-up button," that information is sent to the server.

[2387] User Roles

[2388] Users set various schedules on the server using a smartphone app, etc. This allows the server to generate and manage notifications that fit the user's lifestyle. For example, if a user sets the server as their active time from 9:00 AM to 12:00 PM, the server will send a reminder to the device at 9:00 AM.

[2389] Specific examples

[2390] Consider a scenario where a user wakes up at 7am every morning, sets their work hours from 9am to 12pm, goes to bed at 10pm every night, and shuts down the TV and Wi-Fi router in their home at the same time.

[2391] 1. Manage your wake-up time:

[2392] The user opens the smartphone app and sets the wake-up time to 7:00 AM every morning. The server receives the wake-up time data (7:00 AM) from the user and records it in a database. When the set time of 7:00 AM arrives, the server sends an alarm notification to the device, which then sounds the alarm continuously. When the user taps the "wake-up button," the device sends "wake-up confirmation data" to the server, which records it in the database.

[2393] 2. Daytime activity management:

[2394] The user sets the working hours from 9:00 AM to 12:00 PM. The server records the received activity time data (9:00 AM - 12:00 PM) in the database. At 9:00 AM, the server sends a reminder notification to the device saying "It's time to start work." The device displays the reminder, and when the user taps the "confirm button," "activity start confirmation data" is sent to the server, which records it in the database.

[2395] 3. Bedtime notifications:

[2396] The user sets the bedtime to 10 PM every night. The server receives the bedtime data (10:00 PM) and records it in the database. At 9:30 PM, the server sends a notification to the device saying, "It's 30 minutes until bedtime." The device continues to display the notification, encouraging the user to get ready for bed.

[2397] 4. Shutdown the equipment:

[2398] A user sets the shutdown time for specific devices in the home (TV and Wi-Fi router). The server records the received shutdown time data (10:00 PM) in a database. At 10 PM, the server sends a shutdown signal to these devices. The terminal checks whether these devices have been shut down successfully and reports the results to the server. The server records this information in a database.

[2399] Prompt Sentence Examples

[2400] Here are some examples of prompts:

[2401] 1. Explain specific examples of managing your wake-up time:

[2402] Let's say a user configures the system to wake up at 7:00 every morning. The server sends a notification at exactly 7:00, and the device continues to ring the alarm. When the user presses the wake-up button, that information is sent to the server.

[2403] 2. Describe specific examples of daytime activity management:

[2404] If a user sets their active time from 9:00 AM to 12:00 PM, the server will send a reminder at 9:00 AM and the device will display the reminder. When the user presses the confirmation button, the information is sent to the server.

[2405] 3. Explain specific examples of bedtime notifications:

[2406] For users who have set their bedtime schedule to 10 p.m. every night, the server will send a notification at 9:30 p.m. and the device will display the message "Bedtime in 30 minutes."

[2407] 4. Describe an example of equipment shutdown:

[2408] If the TV and Wi-Fi router in the home are set to shut down at 10 PM, the server will send a shutdown signal to these devices at 10 PM. The device will check to see if the devices have shut down and report the results to the server.

[2409] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2410] Wake-up time management

[2411] Step 1:

[2412] The user uses a smartphone app to set the time they wake up each morning.

[2413] Input: Wake-up time (e.g. 7:00 AM)

[2414] Output: Configuration data sent to the server

[2415] Specific operation: The user enters the wake-up time on the app's settings screen and presses the Set button.

[2416] Step 2:

[2417] The server records the received wake-up time data in a database.

[2418] Input: Wake-up time data sent by the user

[2419] Output: Wake-up time data recorded in a database

[2420] Specific operation: The server executes a query to store the wake-up time in a database (e.g. MySQL).

[2421] Step 3:

[2422] When the server reaches the set wake-up time, it sends an alarm notification to the terminal.

[2423] Input: Current time and wake-up time data recorded in the database

[2424] Output: Alarm notification is sent to the device

[2425] Specific operation: The server detects that a specific time has arrived, generates an alarm notification, and sends a push notification to the device.

[2426] Step 4:

[2427] The device will sound an alarm and continue to notify you.

[2428] Input: Alarm notification sent from the server

[2429] Output: The device continues to ring

[2430] Specific behavior: The device will start playing a sound file and display a "wake-up button" on the screen.

[2431] Step 5:

[2432] The user presses the wake-up button.

[2433] Input: User presses a button

[2434] Output: Wake-up confirmation data is generated

[2435] Specific action: The user taps the "wake-up button" on the device.

[2436] Step 6:

[2437] The terminal transmits wake-up confirmation data to the server.

[2438] Input: User button press information

[2439] Output: Wake-up confirmation data is sent to the server and recorded in the database

[2440] Specific operation: The device sends an HTTP request to the server, and the server stores the wake-up confirmation data in a database.

[2441] Daytime activity management

[2442] Step 1:

[2443] The user sets the time for daytime activity using a smartphone app.

[2444] Input: Activity time (e.g. 9:00 AM - 12:00 PM)

[2445] Output: Configuration data sent to the server

[2446] Specific operation: The user enters the activity time on the app's settings screen and presses the settings button.

[2447] Step 2:

[2448] The server records the received activity time data in a database.

[2449] Input: Activity time data sent by the user

[2450] Output: Activity time data recorded in a database

[2451] Specific operation: The server executes a query to store the activity time in the database.

[2452] Step 3:

[2453] The server sends a reminder to the device at the start time of the activity.

[2454] Input: Current time and activity time data recorded in the database

[2455] Output: Reminder notification sent to device

[2456] Specific operation: The server detects when a specific time has arrived, generates a reminder notification, and sends it as a push notification to the device.

[2457] Step 4:

[2458] Your device will display reminders and keep you informed.

[2459] Input: Reminder notification sent from the server

[2460] Output: The device continues to display the reminder

[2461] Specific behavior: The device will display a reminder notification and a "Confirm" button on the screen.

[2462] Step 5:

[2463] The user presses the confirm button.

[2464] Input: User presses a button

[2465] Output: Activity start confirmation data is generated

[2466] Specific action: The user taps the "confirmation button" on the device.

[2467] Step 6:

[2468] The terminal transmits activity start confirmation data to the server.

[2469] Input: User button press information

[2470] Output: Activity start confirmation data is sent to the server and recorded in the database

[2471] Specific operation: The terminal sends an HTTP request to the server, and the server stores the activity start confirmation data in the database.

[2472] Bedtime notifications

[2473] Step 1:

[2474] The user uses a smartphone app to set their bedtime each night.

[2475] Input: Bedtime (e.g. 10:00 PM)

[2476] Output: Configuration data sent to the server

[2477] Specific behavior: The user enters bedtime on the app's settings screen and presses the Set button.

[2478] Step 2:

[2479] The server records the received bedtime data in a database.

[2480] Input: Bedtime data submitted by the user

[2481] Output: Bedtime data recorded in a database

[2482] What happens: The server runs a query to store the bedtime in the database.

[2483] Step 3:

[2484] The server sends a bedtime preparation notification to the device 30 minutes before bedtime.

[2485] Input: Current time and bedtime data recorded in the database

[2486] Output: Bedtime notification sent to device

[2487] Specific operation: The server detects when a specific time has arrived, generates a bedtime preparation notification, and sends it as a push notification to the device.

[2488] Step 4:

[2489] The device displays a notification to remind the user to get ready for bed.

[2490] Input: Bedtime notification sent from the server

[2491] Output: The device keeps displaying the bedtime notification

[2492] What it does: Your device will display a bedtime notification and display "Start getting ready for bed" on the screen.

[2493] Equipment shutdown

[2494] Step 1:

[2495] A user uses a smartphone app to set shutdown times for specific devices in the home.

[2496] Input: Shutdown time (e.g. 10:00 PM)

[2497] Output: Configuration data sent to the server

[2498] Specific behavior: The user enters the shutdown time on the app's settings screen and presses the Set button.

[2499] Step 2:

[2500] The shutdown time data received by the server is recorded in a database.

[2501] Input: Shutdown time data sent by the user

[2502] Output: Shutdown time data recorded in the database

[2503] What happens: The server runs a query to store the shutdown time in the database.

[2504] Step 3:

[2505] The server transmits a shutdown signal to a specific device in the home at a specified time.

[2506] Input: Current time and shutdown time data recorded in the database

[2507] Output: A shutdown signal is sent to the device

[2508] Specific operation: The server detects that a specific time has arrived, generates a shutdown signal, and sends it to the home appliance control system (e.g., IoT device).

[2509] Step 4:

[2510] The terminal checks whether the device has been shut down normally and reports the result to the server.

[2511] Input: Device response data to shutdown signal

[2512] Output: Shutdown confirmation data is sent to the server and recorded in the database.

[2513] Specific operation: The terminal checks the device status and reports to the server that the shutdown is complete.

[2514] (Application example 1)

[2515] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2516] Conventional lifestyle management in homes and cars is often performed separately, lacking integrated management. In particular, in autonomous vehicles, time management for users while driving or napping is difficult, and methods for realizing a comfortable and efficient lifestyle are required. Appropriate shutdown management of in-car entertainment systems and lighting is also required.

[2517] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2518] In this invention, the server includes means for setting a wake-up time, means for sending an alarm at the set wake-up time, means for confirming a user's response to the alarm, means for setting a daytime activity time, means for sending a reminder at the set activity time, means for confirming a user's response to the reminder, means for setting a bedtime, means for sending a notification encouraging the user to get ready for bed at the set bedtime, means for shutting down home appliances at bedtime, means for sending specified reminders and notifications for lifestyle cycle management in the autonomous vehicle, and means for shutting down the entertainment system and lighting in the autonomous vehicle at set times, thereby enabling integrated and efficient lifestyle cycle management at home and in the autonomous vehicle.

[2519] A "wake-up time" is a specific time that a user sets to wake up.

[2520] An "alarm" is a means of sending a notification to the user at a set wake-up time to prompt them to wake up.

[2521] The "user's response to the alarm" is an action that the user takes in response to the alarm notification, such as pressing a button or using voice recognition to confirm that the user is awake.

[2522] The "daytime activity time" is a set time indicating the activities (for example, work or exercise) that the user should perform within a certain period of time.

[2523] A "reminder" is a notification means that prompts the user to start an activity at a set time.

[2524] A "user's response to a reminder" is a confirmation action that a user takes in response to a reminder notification.

[2525] A "bedtime" is a specific time that a user sets for going to bed.

[2526] The "notification to urge the user to get ready for bed" is a notification means for urging the user to get ready for bed prior to the set bedtime.

[2527] The "means for shutting down devices" is a means for automatically shutting down specific devices in a home or car at a set time.

[2528] "In-vehicle entertainment system" refers to devices and software that allow users to enjoy entertainment in an autonomous vehicle.

[2529] "Lighting" refers to a lighting device for providing brightness inside an autonomous vehicle.

[2530] This invention is a system for managing a life cycle in a home and an autonomous vehicle to lead a healthy life cycle. This system consists of three main components: a server, a terminal, and a user.

[2531] The server manages the data of the wake-up time, activity time, and bedtime set by the user, and sends corresponding notifications to the device based on this. For example, if the user sets the server to wake up at 7:00 every morning, the server will send an alarm notification to the device at exactly 7:00, and the device will ring the alarm.

[2532] The device is responsible for displaying notifications to the user, receiving the user's response (such as pressing a button), and reporting it to the server. It can be implemented as a smartphone, tablet, or a dedicated device in an autonomous vehicle. For example, the device might display a reminder saying "It's time to start work" at 9:00 AM, and when the user presses the confirmation button, the response is reported to the server.

[2533] Users can set their own schedules on the server and respond to notifications from their devices based on those settings, making it easier for users to manage their daily activity cycles.

[2534] When used in an autonomous vehicle, for example, if a user sets a wake-up time during a nap, a wake-up notification will be sent 10 minutes before the scheduled arrival time. The entertainment system and lighting will also be shut down at the set time, allowing for a comfortable in-car experience.

[2535] Hardware and software used

[2536] To implement this system, a cloud server (e.g., AWS, Google Cloud Platform) is used on the server side, and a smartphone (e.g., iPhone, Android), tablet, or in-car device (e.g., in-car infotainment system) is used on the terminal side. Such devices are connected to the Internet and have an application installed to communicate with the server.

[2537] Data calculation and processing

[2538] The server receives the life cycle data set by the user and creates a notific...

Claims

1. a means for setting a wake-up time; means for sending an alarm at a set wake-up time; means for ascertaining a user's response to the alarm; a means for setting the time for daytime activity; a means for sending reminders at set activity times; a means for confirming a user's response to the reminder; a means for setting a bedtime; A means for sending a notification at a set bedtime prompting the user to get ready for bed; A means to shut down devices in the home at bedtime; A system including:

2. The system of claim 1 further comprising means for verifying that home appliances are properly shut down at said bedtime.

3. 2. The system of claim 1, wherein each of the alarm, reminder, and notification means further comprises means for reporting a user's response to a server.

Images