System

Abstract

A system is provided.SOLUTION: A system comprising: a means for receiving an input of a schedule and a meal content by voice or text from a user terminal; a means for converting the voice into text; a means for transmitting the text data to a server; a means for analyzing the text data in the server and storing schedule information in a database; a means for setting a reminder based on the schedule; a means for analyzing a meal record and generating advice about nutritional balance; a means for acquiring seasonal event information and seasonal food information; and a means for receiving a comment from a parent and notifying at an appropriate timing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, busy work and family schedules make it difficult for parents to communicate with their children and manage their lives efficiently. In particular, when parents are too busy at work to communicate directly with their children, it can be difficult to provide them with the necessary advice and information in a timely manner. Information about individual dietary balance and seasonal events is also often lacking. This can lead to a decline in quality of life and a potential estrangement between parents and children. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means: A system accepts voice or text input from a user terminal, converts it into text data, and sends it to a server. The server analyzes the received text data, stores it in a database as schedule information, and sets reminders. It also analyzes food records and generates advice on nutritional balance. It also provides information on seasonal events and in-season foods, and receives comments from parents and notifies them at appropriate times. In this way, a system is provided that supports efficient lifestyle management and communication between parents and children, improving quality of life.

[0006] "User terminal" refers to a device that allows a user to input voice or text.

[0007] "Speech recognition library" refers to a software component for converting voice data into text data.

[0008] "Text data" refers to character string data converted by a voice recognition library or input directly by the user.

[0009] A "server" refers to a computer system that receives data sent from multiple user terminals, analyzes it, and performs appropriate processing.

[0010] "Database" refers to a system for storing schedule information and meal records analyzed by a server.

[0011] "Reminder" refers to a function for sending notifications to the user at specified times.

[0012] "Meal record" refers to text data entered by the user containing details of meals.

[0013] "Nutritional balance" refers to a criterion for assessing how balanced a user's diet is with respect to specific nutrients.

[0014] "Advice" refers to recommendations or suggestions provided to the user by the server based on the results of its analysis.

[0015] "Seasonal event information" refers to information about events and activities related to a particular season.

[0016] "Seasonal food information" refers to information about foods that are recommended to be eaten in that season.

[0017] "Parent comments" refer to messages or advice that parents want to send to their children via the user terminal.

[0018] "Appropriate timing" refers to the most effective time to notify based on the child's schedule and circumstances. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention relates to a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[0041] System Overview

[0042] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user.It also supports communication between parents and children by receiving comments from parents and notifying their children.

[0043] Program processing flow

[0044] User terminal

[0045] A user terminal is a device that allows a user to input schedules and meal details by voice or text. When a user inputs by voice, the user terminal uses a voice recognition library to convert the voice into text and sends this text data to the server. For example, if a user inputs "Meeting tomorrow at 10 AM" by voice, the terminal converts this into text data "Meeting tomorrow at 10 AM" and sends it to the server.

[0046] server

[0047] The server is responsible for receiving and analyzing text data sent from the device. For example, the server uses Natural Language Processing (NLP) technology to analyze the received text data and save the schedule information in a database. It also sets reminders and sends reminder notifications to the user's device at appropriate times. If a user inputs "Today's breakfast is bread and banana," the server analyzes the meal contents and generates advice about nutritional balance. For example, it generates advice such as "You should add a few more vegetables" and sends this to the user's device.

[0048] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[0049] Parent comment notification

[0050] When a parent enters a comment by voice or text from their device, the data is sent to the server. The server saves the received comment and checks the child's schedule. For example, if a parent sends a comment such as "Please do your homework properly," the server will send a notification to the child's device saying "Please do your homework properly" at an appropriate time based on the child's schedule. An appropriate time could be, for example, a time when the child has free time or before bedtime.

[0051] Specific examples

[0052] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[0053] 1. The user device converts the speech into text and sends it to the server.

[0054] 2. The server receives the text data and uses NLP technology to analyze the schedule information for "tomorrow at 10:00 AM" and saves it in a database.

[0055] 3. The server sets a reminder and sends a reminder notification to the user terminal, for example, "tomorrow at 9:30 AM."

[0056] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[0057] 1. The parent device converts the voice into text and sends it to the server.

[0058] 2. The server receives the comment and checks the child's schedule.

[0059] 3. At the appropriate time, the server notifies the child's device, "Please do your homework properly."

[0060] The specific system configuration and programs required to implement the present invention are those that properly perform the above-mentioned processes, thereby supporting efficient communication between the user and their family and lifestyle management.

[0061] The processing flow will be explained below.

[0062] Specific processing flow of the program

[0063] Schedule management process

[0064] Step 1:

[0065] The user inputs the schedule details by voice. For example, the user may say, "Meeting tomorrow at 10:00 AM."

[0066] Step 2:

[0067] The device uses a speech recognition library to convert the voice data into text data. Specifically, the voice "Meeting tomorrow at 10 AM" is converted into text "Meeting tomorrow at 10 AM."

[0068] Step 3:

[0069] The terminal sends the converted text data to the server via an HTTP request.

[0070] Step 4:

[0071] The server receives the text data and analyzes it using Natural Language Processing (NLP) technology, extracting the date and time information "tomorrow at 10 AM" from the text.

[0072] Step 5:

[0073] The server saves the extracted schedule information in a database. For example, it saves the meeting information for "December 20, 2023 at 10:00 AM."

[0074] Step 6:

[0075] The server sets a reminder based on the saved schedule, for example, "December 20, 2023 at 9:30 AM."

[0076] Step 7:

[0077] When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There will be a meeting at 10:00."

[0078] Food record and advice processing

[0079] Step 1:

[0080] The user inputs the details of their meal by voice, for example, saying, "Today's breakfast is bread and bananas."

[0081] Step 2:

[0082] The device converts the voice data into text data. The voice "Today's breakfast is bread and bananas" is converted into text "Today's breakfast is bread and bananas."

[0083] Step 3:

[0084] The terminal transmits the converted text data to the server.

[0085] Step 4:

[0086] The server receives the text data and uses the Nutrient Analysis API to analyze the dietary content, evaluating the carbohydrate and vitamin content as analysis results.

[0087] Step 5:

[0088] The server generates nutritional advice based on the analysis results, such as "You might want to add a few more vegetables."

[0089] Step 6:

[0090] The server transmits the generated advice to the user terminal and notifies it.

[0091] Parent comment notification processing

[0092] Step 1:

[0093] Parents can comment by voice or text, for example, saying, "Please do your homework."

[0094] Step 2:

[0095] The device converts the voice to text. The voice "Do your homework properly" is converted to text "Do your homework properly."

[0096] Step 3:

[0097] The terminal transmits the converted text data to the server.

[0098] Step 4:

[0099] The server receives and stores the comment data.

[0100] Step 5:

[0101] The server checks the child's schedule and determines the appropriate notification time. For example, if a child has free time at 4 p.m., it sets the notification to be sent at "4 p.m."

[0102] Step 6:

[0103] At the set time, the server sends a message to the child's device saying, "Please do your homework properly."

[0104] Providing information on seasonal events and foods

[0105] Step 1:

[0106] The server checks the current date and retrieves information about seasonal events and foods. For example, if it is October, it retrieves information about "moon viewing."

[0107] Step 2:

[0108] Based on the information acquired by the server, a message to be provided to the user is generated. For example, a message such as "Today is the season for viewing the moon. Let's enjoy some dumplings" is generated.

[0109] Step 3:

[0110] The server sends the generated message to the user terminal for notification.

[0111] This is the specific program flow of the AI ​​Mom System. This system will improve the efficiency of users' lifestyle management and support communication between parents and children.

[0112] Example 1

[0113] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] Currently, there are many schedule management systems and lifestyle support systems, but they are difficult to flexibly respond to the diverse needs of users. Furthermore, there are a lack of means for smooth communication between parents and children. Furthermore, there are few systems that provide integrated functions such as nutritional balance advice and reminders. For these reasons, there is a demand for a system that can comprehensively support users' daily lives.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0116] In this invention, the server includes means for accepting input of schedules and meal details by voice or text from a user terminal, means for converting the voice into text, means for transmitting the text data to the server, means for analyzing the text data in the server and storing the schedule information in a database, means for setting reminders based on the schedule, means for analyzing the meal records and generating advice on nutritional balance, means for acquiring information on seasonal events and in-season foods, means for receiving comments from parents and notifying them at appropriate times, means for saving the comments sent by the parents and notifying them at appropriate times based on the comments, means for the user to request generation of reminders or advice by voice or text, and means for analyzing the request using natural language processing technology and for the user to generate reminders or advice. This provides comprehensive support for the user's daily life and facilitates communication between parents and children.

[0117] 1. "User terminal" refers to a device that allows a user to input schedules and meal details by voice or text.

[0118] 2. "Means for converting speech to text" refers to a function that converts a user's speech into text data using a speech recognition library or the like.

[0119] 3. "Means for transmitting text data to a server" means a function that uses a communication protocol to transmit text data to a server via a network.

[0120] 4. "Server" means a computing device that analyzes received data, stores necessary information in a database, and generates and sends reminders and advice to the user's device.

[0121] 5. "Means for analyzing text data" means a function that uses Natural Language Processing technology to analyze input text data and extract necessary information.

[0122] 6. "Means for saving to a database" refers to the function of saving the analyzed text data to persistent storage.

[0123] 7. "Means for setting reminders" refers to a function for setting notification timing based on a schedule.

[0124] 8. "Means for generating advice on nutritional balance" refers to a function that analyzes the input dietary information and provides appropriate nutritional advice based on the results.

[0125] 9. "Means for obtaining seasonal event information and in-season food information" is a function for obtaining information on related events and foods based on the current date.

[0126] 10. "Means for receiving comments from parents and notifying them at an appropriate time" refers to a function that receives voice or text comments from parents and notifies them at an appropriate time based on the child's schedule.

[0127] 11. "Natural Language Processing Technology" is a general term for natural language processing technology used to analyze text data.

[0128] 12. "Means for analyzing requests and generating reminders and advice for users" refers to a function that analyzes requests from users and generates reminders and advice based on the results.

[0129] MODE FOR CARRYING OUT THE INVENTION

[0130] The present invention relates to a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[0131] User device functions

[0132] A user terminal is a device that allows users to input schedules and meal details by voice or text. Specifically, this applies to smartphones and tablets. The terminal has the following main functions that operate based on user input:

[0133] Speech Recognition Library: The Google® Cloud Speech-to-Text API is used to convert the user's speech into text. This converted text data is sent to the server described below.

[0134] Communication means: After speech recognition, the system has a communication protocol for transmitting the converted text data to the server. For example, the system transmits the data using an HTTP POST request.

[0135] Server Features

[0136] The server receives and analyzes text data sent from the user's device, processes and stores the necessary information, and also generates reminders and advice and notifies the user's device.

[0137] Data reception: Use a web framework such as Flask or Django to receive HTTP POST requests from the user's device.

[0138] Text analysis: Using Natural Language Processing (NLP) techniques, such as spaCy, the received text data is analyzed to extract date and time information and event information.

[0139] Database: Analyzed schedule information is saved in a database such as MySQL (registered trademark) or MongoDB. Saved data includes date and time, events, user IDs, etc.

[0140] Set reminders: Use a cron job to send reminder notifications at a set time, for example, "Tomorrow at 9:30 AM."

[0141] Advice generation: Analyzes the dietary information entered by the user and generates appropriate nutritional advice based on the results. For example, if the user enters "Today's breakfast is bread and banana," the system will provide advice such as "You should add a few more vegetables."

[0142] Obtaining seasonal event information: Obtaining seasonal event and seasonal food information based on the current date and notifying the user.

[0143] Parent comment notifications: Receive and save comments sent by parents and notify them at appropriate times based on the child's schedule. For example, if a parent comments "Please do your homework properly," a notification "Please do your homework properly" will be sent to the child during their free time.

[0144] Specific examples

[0145] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[0146] The user's device converts the speech into text and sends it to the server using the Google Cloud Speech-to-Text API.

[0147] The server receives the text data and uses NLP techniques to parse the schedule information for "tomorrow at 10 AM" and uses spaCy to store it in the database.

[0148] The server sets a reminder and uses a cron job to send a reminder notification to the user's device at "tomorrow at 9:30 AM."

[0149] The server generates dietary advice and sends it to the user, such as "You should add a few more vegetables."

[0150] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[0151] The parent device converts the speech into text and sends it to the server, again using the Google Cloud Speech-to-Text API.

[0152] The server receives the comments and uses spaCy to check the child's schedule.

[0153] The server sends a notification to the child's device at an appropriate time, such as during the child's free time, saying, "Please do your homework properly."

[0154] Prompt Sentence Examples

[0155] Here are some examples of prompts that users might enter using a generative AI model:

[0156] "Set a meeting reminder for tomorrow at 10 AM"

[0157] "Today's breakfast is bread and bananas. I'd like some advice on nutritional balance."

[0158] "Remind my child to do their homework properly"

[0159] In this way, the system of the present invention can comprehensively support the daily lives of users and their families, facilitate smooth communication between parents and children, and enable efficient schedule management.

[0160] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0161] Program processing flow

[0162] Step 1:

[0163] The user enters data by voice or text.

[0164] Specific actions: A user uses a device such as a smartphone or tablet to say, "Meeting tomorrow at 10:00 AM," or type in text such as, "Today's breakfast is bread and bananas."

[0165] Input: Audio or text data

[0166] Output: Audio or text data on the user's device

[0167] Step 2:

[0168] The user terminal converts the speech into text.

[0169] Specific operation: The user device uses the Google Cloud Speech-to-Text API to convert the voice data into text data, and the converted text is displayed on the device screen.

[0170] Input: Audio data

[0171] Output: Text data (e.g. "Meeting tomorrow at 10 AM")

[0172] Step 3:

[0173] The user terminal transmits text data to the server.

[0174] Specific operation: The device sends the converted text data to the server using an HTTP POST request. The user's device screen displays the status of the transmission.

[0175] Input: Text data (e.g. "Meeting tomorrow at 10 AM")

[0176] Output: Text data sent to the server

[0177] Step 4:

[0178] The server receives the text data.

[0179] Specific operation: The server uses Flask or Django to receive HTTP POST requests from user terminals and retrieve text data. The received data is recorded in the server log.

[0180] Input: HTTP POST request

[0181] Output: Received text data

[0182] Step 5:

[0183] The server parses the text data.

[0184] Specific operation: The server uses NLP technology such as spaCy to analyze the text data and extract the date and time information for "tomorrow at 10 AM" and the event information for "meeting." The analysis results are recorded in the server log.

[0185] Input: Text data (e.g. "Meeting tomorrow at 10 AM")

[0186] Output: Parsed information (e.g., date and time "tomorrow 10:00 AM", event "meeting")

[0187] Step 6:

[0188] The server stores the analysis results in a database.

[0189] Specific operation: The server stores the analyzed information in MySQL or MongoDB. The stored data includes user ID, date and time, event, etc. The server generates a log to confirm the data stored in the database.

[0190] Input: Parsed information

[0191] Output: Information stored in the database

[0192] Step 7:

[0193] The server generates reminders and advice.

[0194] Specific operation: The server sets up a cron job to send a reminder notification at "9:30 AM tomorrow." It also generates nutritional advice based on "Today's breakfast is bread and bananas," and generates a message saying, "You should add a few more vegetables."

[0195] Input: Saved schedule information and meal information

[0196] Output: Reminder notifications and advice messages

[0197] Step 8:

[0198] The server sends reminders and advice to the user terminal.

[0199] Specific operation: The server uses push notification to send a notification to the user's device at 9:30 AM tomorrow saying, "There will be a meeting at 10 AM." It also sends an advice message.

[0200] Input: Reminder notification setting information and advice message

[0201] Output: Notifications and messages sent to the user's device

[0202] Step 9:

[0203] Parents enter comments.

[0204] Specific operation: The parent speaks into their smartphone, "Please do your homework properly." This voice data is sent to the device.

[0205] Input: Audio data

[0206] Output: Audio data on the parent device

[0207] Step 10:

[0208] The server saves the parent's comments and notifies the child.

[0209] Specific operation: The server receives comments from the parent's device, analyzes them, and stores them in a database. Furthermore, based on the child's schedule, it sends a message to the child's device during their free time saying, "Please do your homework properly."

[0210] Input: Parsed parent comments, child schedule information

[0211] Output: Notification sent to child device

[0212] (Application example 1)

[0213] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] It is often difficult to efficiently communicate between workers and managers in factories and manage work schedules. In busy work environments, workers can forget their schedules or miss necessary safety instructions, resulting in reduced efficiency and safety. Furthermore, there are concerns that workers' workloads may increase if they are not properly provided with information related to seasonal changes or specific tasks.

[0215] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0216] In this invention, the server includes means for accepting voice or text input of schedules and work details from a user terminal, means for converting the voice into text, means for transmitting the text data to the server, means for analyzing the text data in the server and storing the schedule information in a database, means for setting reminders based on the schedule, means for analyzing work records and generating work-related advice, means for acquiring seasonal event information and work-related information, and means for receiving comments from a manager and notifying them at an appropriate time, thereby enabling factory workers to efficiently manage their schedules and receive necessary instructions and advice in a timely manner.

[0217] A "user terminal" is a device that accepts voice or text input.

[0218] A "speech recognition library" is a software library for converting voice data into text.

[0219] "Text data" refers to character string data obtained by voice input or text input.

[0220] A "server" is a central processing device that receives text data and performs analysis, storage, and notification.

[0221] "Natural Language Processing technology" refers to computer processing technology for understanding and analyzing human language.

[0222] "Database" refers to a data storage system for storing analyzed schedule information and work records.

[0223] "Reminder" is a function that notifies the user based on a set schedule.

[0224] "Work records" refers to information about specific tasks and time management within a factory.

[0225] "Advice" means advice or recommendations provided based on the analyzed work record.

[0226] A "manager" is a person who is responsible for giving comments and instructions to workers.

[0227] "Seasonal event information" refers to information about specific events for each season.

[0228] "Work-related information" refers to various types of information provided to improve work efficiency and safety.

[0229] The present invention is an in-factory work management and communication system including a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[0230] System Overview

[0231] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user.It also supports communication between managers and workers by receiving comments from managers and notifying them to workers.

[0232] Program processing flow

[0233] User terminal

[0234] A user terminal is a device that allows a user to input schedules and work details by voice or text. When a user inputs by voice, the user terminal uses a speech recognition library (e.g., Google Cloud Speech-to-Text API) to convert the voice into text and sends this text data to the server. For example, if a user inputs "Maintenance inspection of Line 1 at 10:00 AM" by voice, the terminal converts this into text data "Maintenance inspection of Line 1 at 10:00 AM" and sends it to the server.

[0235] server

[0236] The server is responsible for receiving and analyzing the text data sent from the terminal. The server uses Natural Language Processing (NLP) technology (e.g., NLTK, Spacy) to analyze the received text data and saves the schedule information in a database (e.g., PostgreSQL). It also sets reminders and sends reminder notifications to the user terminal at appropriate times. For example, it sends a reminder 30 minutes before the maintenance inspection saying, "Maintenance inspection of Line 1 is coming up soon."

[0237] Furthermore, the server analyzes work records, generates work-related advice (e.g., "Please wear protective equipment before maintenance inspections"), and sends this to the user's terminal. The server also obtains information related to seasonal changes and specific tasks and provides it to the user. For example, in the summer, the server notifies users with information such as "Be careful of heatstroke."

[0238] Administrator comment notification

[0239] Managers input comments by voice or text from their own devices, and the data is sent to the server. The server saves the received comments and checks the worker's schedule. For example, if a manager sends a comment such as "Please wear protective equipment before maintenance inspection," the server will send a notification to the worker's device at the appropriate time based on the worker's schedule, stating "Please wear protective equipment before maintenance inspection."

[0240] Specific examples

[0241] When a worker voice-inputs "Maintenance inspection of Line 1 at 10:00 AM," the following process takes place:

[0242] 1. The user device converts the speech into text and sends it to the server.

[0243] 2. The server receives the text data and uses NLP technology to analyze the schedule information for the "10:00 AM maintenance inspection" and save it in the database.

[0244] 3. The server sets a reminder and sends a reminder notification to the user terminal 30 minutes before the maintenance inspection.

[0245] 4. When the administrator terminal inputs "Please wear protective equipment before maintenance inspection" by voice, the instruction is notified to the user terminal at the appropriate time.

[0246] Prompt Sentence Examples

[0247] "We have a maintenance inspection scheduled for Line 1 at 10:00 AM. Please send me a reminder 30 minutes before. I would also like a reminder to put on protective gear before the inspection."

[0248] As described above, by using this system, factory workers can efficiently manage their schedules and receive necessary instructions and advice in a timely manner.

[0249] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0250] Program processing flow

[0251] Step 1:

[0252] The user inputs schedules and work details using voice or text. The user's device receives the input voice data and converts it into text data using a speech recognition library (e.g., Google Cloud Speech-to-Text API). The text data obtained through this conversion becomes the input for the next step.

[0253] Step 2:

[0254] The user terminal sends the converted text data to the server. The server receives the text data from the user terminal and prepares to analyze it. The input here is text data.

[0255] Step 3:

[0256] The server analyzes the received text data using Natural Language Processing (NLP) techniques (e.g., NLTK, Spacy). From this analysis, schedule information and task details are extracted. The results of this analysis are stored in a database for further processing. The output of the analysis is database entries for schedule information and task details.

[0257] Step 4:

[0258] The server sets a schedule-based reminder based on the analysis results. A specific time, such as 30 minutes before a maintenance inspection, is specified for the reminder setting. This reminder setting information is used to send reminder notifications. The reminder setting information is output.

[0259] Step 5:

[0260] The set reminder is sent to the user's device. The contents of the reminder are personalized for each user and notified at the appropriate time. The input here is the set reminder information, and the output is the reminder notification displayed on the user's device.

[0261] Step 6:

[0262] The server analyzes the work records and generates work-related advice, such as specific advice like "wear protective equipment before maintenance inspection." This advice information becomes the input for the next processing step.

[0263] Step 7:

[0264] The generated advice information is sent to the user terminal and notified to the user at the appropriate time. This allows the worker to receive the necessary advice. The input here is the generated advice information, and the output is the advice notification sent to the user terminal.

[0265] Step 8:

[0266] The administrator inputs a comment by voice or text from their own terminal. The administrator terminal sends this comment data to the server. The sent comment data becomes the input for the next step.

[0267] Step 9:

[0268] The server saves comments received from the administrator and checks the worker's schedule. The comments are notified to the worker's terminal at the appropriate time. This notification allows the worker to receive important instructions and comments. The input here is the comment data from the administrator, and the output is a notification to the worker's terminal.

[0269] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0270] The present invention is a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them, and by combining an emotion engine, it is also able to respond to the emotions of the user. Specific embodiments are described below.

[0271] System Overview

[0272] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user. It also uses an emotion engine to recognize the user's emotions and responds based on that information. It also supports communication between parents and children by receiving comments from parents and notifying the child.

[0273] Program processing flow

[0274] User terminal

[0275] The user terminal is a device that allows users to input schedules and meal details by voice or text. When a user inputs by voice, the user terminal uses a voice recognition library to convert the voice into text and sends this text data to the server. It also has an emotion engine that recognizes the user's emotions from the input voice and text data. For example, if a user inputs "Meeting tomorrow at 10 AM," the terminal converts this into text data "Meeting tomorrow at 10 AM" and sends it to the server, while also analyzing the user's emotions (e.g., stress, anxiety).

[0276] server

[0277] The server is responsible for receiving and analyzing the text data and emotion data sent from the device. For example, the server uses Natural Language Processing (NLP) technology to analyze the received text data and store the schedule information in a database. Furthermore, the server adjusts the content of reminders and advice based on the results of the emotion engine. For example, if the user's emotion is recognized as "stress," the server adjusts the wording of the reminder to be softer.

[0278] For example, if a user inputs "Today's breakfast is bread and bananas," the server analyzes the meal contents and generates advice on nutritional balance. For example, it generates advice such as "You should add a few more vegetables," and sends this advice to the user's terminal.

[0279] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[0280] Parent comment notification

[0281] When a parent enters a comment via voice or text from their device, the data is sent to the server. The server saves the received comment and checks the child's schedule. For example, if a parent sends a comment saying "Please do your homework properly," the server will send a notification saying "Please do your homework properly" to the child's device at an appropriate time based on the child's schedule. Here too, the content of the notification is adjusted taking into account the child's emotional information (e.g., if they are feeling depressed).

[0282] Specific examples

[0283] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[0284] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects "stress" from the user's voice.

[0285] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[0286] 3. The server receives the text data and analyzes it using NLP technology. It extracts the date and time information for "tomorrow at 10:00 AM" and adjusts the reminder based on the emotional data "stress."

[0287] 4. The server saves the schedule information in a database and sets a reminder. For example, it sets a reminder for "tomorrow at 9:30 AM" and adjusts the message content to reduce "stress."

[0288] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[0289] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[0290] 1. The parent device converts the voice to text and analyzes the emotion using an emotion engine. For example, it detects the emotion of "encouragement."

[0291] 2. The device sends the converted text data and emotion data to the server.

[0292] 3. The server receives and stores the comment data. Based on the emotional data "encouragement," it sets up notifications to be sent at the optimal time for the child's situation.

[0293] 4. At the set time, the server sends a message to the child's device saying, "Please do your homework properly." For example, at 4:00 PM, the server sends a message saying, "Please do your homework properly. Good luck!"

[0294] The specific system configuration and programs required to implement the present invention are those that properly perform the above-mentioned processes, thereby improving the efficiency of users' lifestyle management and supporting communication between parents and children in a way that takes emotion into consideration.

[0295] The processing flow will be explained below.

[0296] Specific processing flow of the program

[0297] Schedule management and emotion recognition processing

[0298] Step 1:

[0299] The user inputs the schedule details by voice. For example, the user may say, "Meeting tomorrow at 10:00 AM."

[0300] Step 2:

[0301] The device uses a speech recognition library to convert the voice data into text data. Specifically, the voice "Meeting tomorrow at 10 AM" is converted into text "Meeting tomorrow at 10 AM."

[0302] Step 3:

[0303] The device uses an emotion engine to analyze the user's emotion from the input voice data. For example, the emotion "stress" is analyzed.

[0304] Step 4:

[0305] The device sends the converted text data and analyzed emotion data to the server via an HTTP request.

[0306] Step 5:

[0307] The server receives the text data and emotion data and analyzes the text data using Natural Language Processing (NLP) technology, extracting the date and time information for "tomorrow at 10:00 AM."

[0308] Step 6:

[0309] The server saves the extracted schedule information in a database. For example, it saves the meeting information for "December 20, 2023 at 10:00 AM."

[0310] Step 7:

[0311] The server adjusts and sets the content of the reminder based on the emotion data. For example, if the user's emotion is recognized as "stressed," the server adjusts the reminder to a more soothing message. For example, it sets the reminder to "Tomorrow at 9:30 AM" and adjusts the notification content to "You have a meeting at 10 AM. Relax and enjoy."

[0312] Step 8:

[0313] When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[0314] Food record and advice processing

[0315] Step 1:

[0316] The user inputs the details of their meal by voice, for example, saying, "Today's breakfast is bread and bananas."

[0317] Step 2:

[0318] The device converts the voice data into text data. The voice "Today's breakfast is bread and bananas" is converted into text "Today's breakfast is bread and bananas."

[0319] Step 3:

[0320] The terminal transmits the converted text data to the server.

[0321] Step 4:

[0322] The server receives the text data and uses the Nutrient Analysis API to analyze the dietary content, evaluating the carbohydrate and vitamin content as analysis results.

[0323] Step 5:

[0324] The server generates nutritional advice based on the analysis results, such as "You might want to add a few more vegetables."

[0325] Step 6:

[0326] The server transmits the generated advice to the user terminal and notifies it.

[0327] Parent comment notification processing

[0328] Step 1:

[0329] Parents can comment by voice or text, for example, saying, "Please do your homework."

[0330] Step 2:

[0331] The device converts the voice to text. The voice "Do your homework properly" is converted to text "Do your homework properly."

[0332] Step 3:

[0333] The terminal transmits the converted text data and the emotion data generated by the emotion engine to the server.

[0334] Step 4:

[0335] The server receives the comment data and the emotion data and stores them. The emotion data is analyzed as, for example, "encouragement."

[0336] Step 5:

[0337] The server checks the child's schedule and calculates the appropriate time to notify them of the comment. For example, if the child has free time at 4 p.m., it will set the notification to be sent at that time.

[0338] Step 6:

[0339] The server sends the message "Do your homework properly" to the child's device at the set time. For example, at 4:00 PM, it sends the message "Do your homework properly. Good luck!"

[0340] Providing information on seasonal events and foods

[0341] Step 1:

[0342] The server checks the current date and retrieves information about seasonal events and foods. For example, if it is October, it retrieves information about "moon viewing."

[0343] Step 2:

[0344] Based on the information acquired by the server, a message to be provided to the user is generated. For example, a message such as "Today is the season for viewing the moon. Let's enjoy some dumplings" is generated.

[0345] Step 3:

[0346] The server sends the generated message to the user terminal for notification.

[0347] This is the specific program flow of the AI ​​Mom System. This system will improve the efficiency of users' lifestyle management and support communication between parents and children in a way that takes emotions into consideration.

[0348] Example 2

[0349] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0350] Conventional schedule management systems and lifestyle support systems simply process information mechanically without considering the user's feelings, which means they are ineffective at reducing the user's psychological burden. Specifically, it is difficult to provide appropriate reminders and support to users who are stressed or anxious. Furthermore, they lack functionality to support communication between parents and children, making it difficult to maintain a smooth parent-child relationship.

[0351] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0352] In this invention, the server includes means for analyzing text data and emotion data sent from the user terminal and saving schedule information in a database, means for setting reminders based on the schedule and adjusting them based on the emotion data, and means for analyzing the meal record, generating advice on nutritional balance, and adjusting them based on the emotion data. This makes it possible to provide reminders and support that take the user's emotions into consideration. Furthermore, by providing a function for receiving comments from parents and notifying them at an appropriate time while taking into account the child's emotion data, it is possible to facilitate communication between parents and support the maintenance of a smooth parent-child relationship.

[0353] A "user terminal" is a device that allows a user to input schedules and meal details by voice or text.

[0354] A "voice recognition library" is a software component for converting voice data into text data.

[0355] "Emotion data" is information about a user's emotional state that has been analyzed using emotion recognition technology.

[0356] "Server" means a central device that receives, analyzes, and processes data sent from user terminals.

[0357] "Text data" is data in text format that has been converted by voice recognition.

[0358] "Natural Language Processing technology" is a technology for understanding and generating natural language.

[0359] A "database" is a system for organizing and storing information.

[0360] "Reminder" is a function for notifying the user of a specific date, time or event.

[0361] "Diet record" is data about dietary details entered by the user.

[0362] "Nutritional balance" refers to the proper distribution of nutrients in food.

[0363] "Event information" is information about specific activities or events related to seasons, dates, and times.

[0364] "Parent comment" is a message entered by a parent to their child.

[0365] This invention is a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them, and by combining it with an emotion engine, it can also respond to the user's emotions. This system is configured as follows.

[0366] System Overview

[0367] A user terminal is a device that allows a user to input schedules and meal details by voice or text. Examples of such devices include smartphones and tablet devices. When a user inputs voice, the user terminal uses a speech recognition library (e.g., Google Cloud Speech-to-Text API) to convert the voice into text and sends this text data to a server. In addition, an emotion recognition API (e.g., Microsoft® Azure® Cognitive Services) is used to recognize the user's emotions from the input voice data or text data.

[0368] The server is responsible for receiving and analyzing the text data and emotion data sent from the user terminal. The server analyzes the received text data using Natural Language Processing technology (e.g., Google Cloud Natural Language API) and stores the schedule information in a database (e.g., MySQL).

[0369] Furthermore, the content of reminders and advice is adjusted based on the results of the emotion engine. For example, if the user's emotion is recognized as "stress," the wording of the reminder is softened or adjusted accordingly. Also, if the user inputs "Today's breakfast is bread and banana," the server analyzes the meal contents and generates advice on nutritional balance. For example, advice such as "You should add a few more vegetables" is generated and sent to the user's device.

[0370] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[0371] The system also receives comments from parents and checks their children's schedules. For example, if a parent sends a comment saying, "Please do your homework properly," the server will send a notification saying, "Please do your homework properly" to the child's device at an appropriate time based on the child's schedule. At this time, the content of the notification will be adjusted taking into account the child's emotional information (e.g., if they are feeling depressed).

[0372] Specific operation example

[0373] Schedule management example

[0374] If a user speaks "Meeting tomorrow at 10 AM," the following occurs:

[0375] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects "stress" from the user's voice.

[0376] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[0377] 3. The server receives the text data and analyzes it using NLP technology. It extracts the date and time information for "tomorrow at 10:00 AM" and adjusts the reminder based on the emotional data "stress."

[0378] 4. The server saves the schedule information in a database and sets a reminder. For example, it sets a reminder for "tomorrow at 9:30 AM" and adjusts the message content to reduce "stress."

[0379] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[0380] Example of parent comment notification

[0381] When a parent sends a comment saying "Please do your homework properly," the following occurs.

[0382] 1. The parent device converts the voice to text and analyzes the emotion using an emotion engine. For example, it detects the emotion of "encouragement."

[0383] 2. The device sends the converted text data and emotion data to the server.

[0384] 3. The server receives and stores the comment data. Based on the emotional data "encouragement," it sets up notifications to be sent at the optimal time for the child's situation.

[0385] 4. At the set time, the server sends a message to the child's device saying, "Please do your homework properly." For example, at 4:00 PM, the server sends a message saying, "Please do your homework properly. Good luck!"

[0386] Prompt Sentence Examples

[0387] Here are some examples of prompts to input to a generative AI model:

[0388] If a user says "Meet tomorrow at 10 AM," how does the system handle this?

[0389]

[0390] If a parent sends a comment like "Do your homework properly," how does the system handle that?

[0391] conclusion

[0392] This system will help users manage their daily lives more efficiently and provide emotionally sensitive reminders and advice, reducing the user's psychological burden. It will also facilitate communication between parents and children, helping to maintain parent-child relationships.

[0393] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0394] Specific processing steps for schedule management

[0395] Step 1: User enters schedule by voice

[0396] The user inputs by voice, "Meeting tomorrow at 10:00 AM." This input data is sent to the user's terminal as voice data.

[0397] Step 2: Speech to text

[0398] The device uses the Google Cloud Speech-to-Text API to convert the voice data entered by the user into text data.

[0399] Input: Voice data "Meeting tomorrow at 10 AM"

[0400] Data processing: Convert speech to text using a speech recognition library

[0401] Output: Text data "Meeting tomorrow at 10 AM"

[0402] Step 3: Sentiment analysis

[0403] The device uses the emotion recognition API of Microsoft Azure Cognitive Services to analyze the user's emotions from the converted text data.

[0404] Input: Text data "Meeting tomorrow at 10 AM"

[0405] Data processing: Emotion analysis using emotion recognition API

[0406] Output: Emotion data "Stress"

[0407] Step 4: Sending data

[0408] The device encodes the converted text data and emotion data into JSON format and sends it to the server via HTTPS.

[0409] Input: Text data "Meeting tomorrow at 10 AM", emotion data "Stress"

[0410] Data processing: Encode text data and emotion data into JSON format

[0411] Output: JSON data

[0412] Step 5: Data reception and analysis

[0413] The server receives the JSON data and uses the Google Cloud Natural Language API to parse the text data and extract the date and time information for "tomorrow at 10:00 AM."

[0414] Input: JSON data

[0415] Data processing: Analyze text data using natural language processing technology and extract date and time information

[0416] Output: Date and time information "Tomorrow 10:00 AM"

[0417] Step 6: Save the schedule

[0418] The server stores the extracted date and time information and text data in a MySQL database.

[0419] Input: Date and time information "Tomorrow 10:00 AM", text data "Meeting tomorrow at 10:00 AM"

[0420] Data processing: Execute SQL queries and save to database

[0421] Output: Schedule information stored in the database

[0422] Step 7: Set reminders

[0423] The server adjusts the content of the reminder based on the emotional data "stress" and sets reminder information based on the schedule, for example, setting a reminder 30 minutes before a meeting.

[0424] Input: Emotion data "Stress", Date and time information "Tomorrow 10:00 AM"

[0425] Data processing: Generate emotion-sensitive reminder messages and set reminder information

[0426] Output: Reminder information (date, time and message)

[0427] Step 8: Send reminders

[0428] When the server reaches the set time (e.g. 9:30 AM), it sends a reminder notification to the user's device.

[0429] Input: Reminder information

[0430] Data processing: Send reminders via HTTP POST request

[0431] Output: Reminder message "There's a meeting at 10 o'clock. Relax and enjoy."

[0432] Specific steps for parent comment notification

[0433] Step 1: Parents enter comments by voice

[0434] The parent can input "Please do your homework properly" by voice. This input data is sent to the parent device as voice data.

[0435] Step 2: Speech to text

[0436] The parent device uses the Google Cloud Speech-to-Text API to convert the voice data input by the parent into text data.

[0437] Input: Voice data "Do your homework properly"

[0438] Data processing: Convert speech to text using a speech recognition library

[0439] Output: Text data "Please do your homework properly."

[0440] Step 3: Sentiment analysis

[0441] The parent device uses the emotion recognition API of Microsoft Azure Cognitive Services to analyze emotions from the converted text data.

[0442] Input: Text data "Please do your homework properly"

[0443] Data processing: Emotion analysis using emotion recognition API

[0444] Output: Emotion data "Encouragement"

[0445] Step 4: Sending data

[0446] The parent device encodes the converted text data and emotion data into JSON format and sends it to the server via HTTPS.

[0447] Input: Text data "Do your homework properly", emotion data "Encouragement"

[0448] Data processing: Encode text data and emotion data into JSON format

[0449] Output: JSON data

[0450] Step 5: Data reception and analysis

[0451] The server interprets the received JSON data and extracts text and emotion data.

[0452] Input: JSON data

[0453] Data processing: Analyze JSON data and extract text and sentiment data

[0454] Output: Text data, emotion data

[0455] Step 6: Save comments

[0456] The server stores the extracted text data and emotion data in a database.

[0457] Input: Text data, emotion data

[0458] Data manipulation: Save to database using SQL queries

[0459] Output: Comment information stored in the database

[0460] Step 7: Notification Settings

[0461] The server checks the child's schedule and configures it to send notifications at appropriate times based on emotional data.

[0462] Input: Child's schedule information, emotional data "encouragement"

[0463] Data processing: Set notification timing and content

[0464] Output: Notification settings information

[0465] Step 8: Send notification

[0466] When the server reaches the set time (e.g. 3:50 pm), it will notify the child's device of the parent's comment, "Make sure you do your homework properly."

[0467] Input: Notification settings information

[0468] Data processing: Send parent comment via HTTP POST request

[0469] Output: Notification message "Do your homework properly. Good luck!"

[0470] conclusion

[0471] In this way, the user's lifestyle management becomes more efficient, and it becomes possible to provide emotionally sensitive reminders and advice. It also facilitates communication between parents and children, helping to maintain parent-child relationships.

[0472] (Application example 2)

[0473] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0474] Currently, there is a demand for improved efficiency in factory operations and smoother communication with workers. In particular, it is important to be able to quickly and accurately interpret voice instructions from workers and respond accordingly. There is also a need to understand workers' emotions and provide appropriate advice and reminders to reduce their stress. However, current systems are unable to meet these requirements.

[0475] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting voice or text input of schedules and meal details from a user terminal; means for converting the voice into text; means for transmitting the text data to the server; means for analyzing the text data in the server and storing the schedule information in a database; means for setting reminders based on the schedule; means for analyzing meal records and generating advice on nutritional balance; means for acquiring information on seasonal events and in-season foods; means for receiving comments from parents and notifying them at appropriate times; means for accepting voice instructions from workers in a factory environment, converting them into text in real time, and analyzing schedule and emotion data; and means for adjusting reminders and advice to workers based on the emotion data and notifying them at appropriate times. This enables improved work efficiency in the factory and smoother communication with workers.

[0476] A "user terminal" is a device that accepts input of schedules and meal details by voice or text.

[0477] "Means for converting speech to text" refers to technology for converting speech input by a user into text data.

[0478] The "means for transmitting text data to a server" is a function for transmitting the converted text data to a server via a network.

[0479] A "server" is a computer system for analyzing and storing data sent from a user terminal.

[0480] The "means for saving schedule information in a database" is a function for storing analyzed schedule information in a database.

[0481] The "means for setting a reminder" is a function for notifying the user based on a set schedule.

[0482] "Means for analyzing dietary records" refers to technology for analyzing dietary details entered by a user.

[0483] The "means for generating advice regarding nutritional balance" is a function for providing advice regarding nutrition based on the analyzed dietary record.

[0484] The "means for acquiring seasonal event information and seasonal food information" is a function for collecting event information and seasonal food information corresponding to each season and providing it to the user.

[0485] The "means for receiving comments from parents and notifying them at an appropriate time" is a function for receiving comments entered by parents and notifying them in accordance with the child's schedule.

[0486] "Factory environment" refers to the location where manufacturing and product assembly operations occur.

[0487] The "means for receiving voice instructions from workers" is a function that enables workers working in the factory to input voice instructions.

[0488] "Means for converting to text in real time" refers to technology for converting voice instructions into text data in real time.

[0489] "Emotion data" is data that indicates the emotional state of a worker analyzed from his / her voice.

[0490] The "means for adjusting reminders and advice to workers based on emotional data" is a function for providing appropriate reminders and advice to workers based on the analyzed emotional data.

[0491] This invention is a system for improving work efficiency in factories and facilitating communication with workers. The system includes a user terminal, a server, and a communication means for exchanging data between them. By combining it with an emotion engine, the system can also respond to the emotions of workers.

[0492] System Overview

[0493] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user. In particular, it uses an emotion engine to recognize the worker's emotions and responds based on that information. It also has a function to receive comments from parents and notify the child.

[0494] User terminal

[0495] The user terminal is a device that allows workers to input instructions and schedules by voice or text. When using voice input, the user terminal uses a voice recognition library (e.g., the SpeechRecognition library) to convert the voice into text and sends this text data to the server. It also has an emotion engine (e.g., an emotion recognition model using Transformers) that recognizes the worker's emotions from the input voice and text data. For example, if a worker inputs "Maintenance work will be performed at 3:00 PM today," the terminal converts this into text data that reads "Maintenance work will be performed at 3:00 PM today" and sends it to the server. At the same time, the terminal also analyzes the worker's emotions (e.g., sense of relief).

[0496] server

[0497] The server is responsible for receiving and analyzing the text and emotion data sent from the device. Specifically, it uses Natural Language Processing (NLP) technology to analyze the received text data and store the schedule information in a database. It also adjusts the content of reminders and advice based on the results of the emotion engine. For example, if the worker's emotion is recognized as "relief," the wording of the reminder will be adjusted accordingly. The server then sends the processed notification to the user's device at a specific time.

[0498] Specific examples

[0499] For example, if a worker says, "Maintenance work will be performed at 3:00 p.m. today. Please don't forget," the following process will occur:

[0500] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects a "sense of security."

[0501] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[0502] 3. The server receives the text data and analyzes it using NLP technology. It extracts the time information "3 PM" and adjusts the reminder based on the emotional data "sense of relief."

[0503] 4. The server saves the schedule information to a database and sets a reminder, for example, for 2:45 PM, with a tailored message tailored to the user's emotional state.

[0504] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 2:45 PM, it sends a notification saying, "Maintenance work will be performed at 3:00. Please proceed without worry."

[0505] Prompt Sentence Examples

[0506] An example of a voice instruction that a worker inputs into a terminal is a prompt such as, "Maintenance work will be performed at 3:00 PM today. Please don't forget." An example of a notification is a statement such as, "Maintenance work will be performed at 3:00 PM. Please proceed without worry."

[0507] In this way, work efficiency within the factory is improved and communication with workers is facilitated.

[0508] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0509] Step 1:

[0510] The user inputs by voice, "Maintenance work will be performed at 3:00 PM today. Please don't forget." This input is received by the terminal as voice data.

[0511] Step 2:

[0512] The device uses a speech recognition library (for example, the SpeechRecognition library) to convert the received voice data into text data. The input is voice data, and the output is text data: "Maintenance work will be carried out at 3:00 p.m. today. Please remember."

[0513] Step 3:

[0514] The device analyzes the converted text data using an emotion engine (e.g., an emotion recognition model using Transformers). The input is text data, and the device outputs emotion data (e.g., a sense of relief) as the result of emotion analysis.

[0515] Step 4:

[0516] The device sends the converted text data and emotion data to the server via an HTTP request. The input is text data and emotion data, and the device sends them to the server as output.

[0517] Step 5:

[0518] The text data received by the server is analyzed using Natural Language Processing (NLP) technology. The input is text data, and the analysis results extract date and time information (e.g., "3:00 PM") and process it into schedule information. The output is schedule information.

[0519] Step 6:

[0520] The server saves the schedule information in a database. The input is the extracted schedule information, and the output is the information saved in the database.

[0521] Step 7:

[0522] The server adjusts the reminder and advice content based on the emotion data and schedule information. The input is the emotion data and schedule information, and the output is the adjusted reminder content.

[0523] Step 8:

[0524] When the server reaches the reminder setting time (for example, 2:45 PM), it sends a reminder notification to the user terminal. The input is the adjusted reminder content, and the output is the notification sent to the user terminal.

[0525] Step 9:

[0526] The user device receives the notification and notifies the user by voice or text. The input is a reminder notification, and the output is a notification to the user saying, "There will be maintenance work at 3:00 PM. Please proceed with confidence."

[0527] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0528] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0529] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0530] [Second embodiment]

[0531] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0532] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0533] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0534] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0535] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0536] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0537] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0538] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0539] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0540] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0541] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0542] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0543] The present invention relates to a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[0544] System Overview

[0545] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user.It also supports communication between parents and children by receiving comments from parents and notifying their children.

[0546] Program processing flow

[0547] User terminal

[0548] A user terminal is a device that allows a user to input schedules and meal details by voice or text. When a user inputs by voice, the user terminal uses a voice recognition library to convert the voice into text and sends this text data to the server. For example, if a user inputs "Meeting tomorrow at 10 AM" by voice, the terminal converts this into text data "Meeting tomorrow at 10 AM" and sends it to the server.

[0549] server

[0550] The server is responsible for receiving and analyzing text data sent from the device. For example, the server uses Natural Language Processing (NLP) technology to analyze the received text data and save the schedule information in a database. It also sets reminders and sends reminder notifications to the user's device at appropriate times. If a user inputs "Today's breakfast is bread and banana," the server analyzes the meal contents and generates advice about nutritional balance. For example, it generates advice such as "You should add a few more vegetables" and sends this to the user's device.

[0551] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[0552] Parent comment notification

[0553] When a parent enters a comment by voice or text from their device, the data is sent to the server. The server saves the received comment and checks the child's schedule. For example, if a parent sends a comment such as "Please do your homework properly," the server will send a notification to the child's device saying "Please do your homework properly" at an appropriate time based on the child's schedule. An appropriate time could be, for example, a time when the child has free time or before bedtime.

[0554] Specific examples

[0555] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[0556] 1. The user device converts the speech into text and sends it to the server.

[0557] 2. The server receives the text data and uses NLP technology to analyze the schedule information for "tomorrow at 10:00 AM" and saves it in a database.

[0558] 3. The server sets a reminder and sends a reminder notification to the user terminal, for example, "tomorrow at 9:30 AM."

[0559] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[0560] 1. The parent device converts the voice into text and sends it to the server.

[0561] 2. The server receives the comment and checks the child's schedule.

[0562] 3. At the appropriate time, the server notifies the child's device, "Please do your homework properly."

[0563] The specific system configuration and programs required to implement the present invention are those that properly perform the above-mentioned processes, thereby supporting efficient communication between the user and their family and lifestyle management.

[0564] The processing flow will be explained below.

[0565] Specific processing flow of the program

[0566] Schedule management process

[0567] Step 1:

[0568] The user inputs the schedule details by voice. For example, the user may say, "Meeting tomorrow at 10:00 AM."

[0569] Step 2:

[0570] The device uses a speech recognition library to convert the voice data into text data. Specifically, the voice "Meeting tomorrow at 10 AM" is converted into text "Meeting tomorrow at 10 AM."

[0571] Step 3:

[0572] The terminal sends the converted text data to the server via an HTTP request.

[0573] Step 4:

[0574] The server receives the text data and analyzes it using Natural Language Processing (NLP) technology, extracting the date and time information "tomorrow at 10 AM" from the text.

[0575] Step 5:

[0576] The server saves the extracted schedule information in a database. For example, it saves the meeting information for "December 20, 2023 at 10:00 AM."

[0577] Step 6:

[0578] The server sets a reminder based on the saved schedule, for example, "December 20, 2023 at 9:30 AM."

[0579] Step 7:

[0580] When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There will be a meeting at 10:00."

[0581] Food record and advice processing

[0582] Step 1:

[0583] The user inputs the details of their meal by voice, for example, saying, "Today's breakfast is bread and bananas."

[0584] Step 2:

[0585] The device converts the voice data into text data. The voice "Today's breakfast is bread and bananas" is converted into text "Today's breakfast is bread and bananas."

[0586] Step 3:

[0587] The terminal transmits the converted text data to the server.

[0588] Step 4:

[0589] The server receives the text data and uses the Nutrient Analysis API to analyze the dietary content, evaluating the carbohydrate and vitamin content as analysis results.

[0590] Step 5:

[0591] The server generates nutritional advice based on the analysis results, such as "You might want to add a few more vegetables."

[0592] Step 6:

[0593] The server transmits the generated advice to the user terminal and notifies it.

[0594] Parent comment notification processing

[0595] Step 1:

[0596] Parents can comment by voice or text, for example, saying, "Please do your homework."

[0597] Step 2:

[0598] The device converts the voice to text. The voice "Do your homework properly" is converted to text "Do your homework properly."

[0599] Step 3:

[0600] The terminal transmits the converted text data to the server.

[0601] Step 4:

[0602] The server receives and stores the comment data.

[0603] Step 5:

[0604] The server checks the child's schedule and determines the appropriate notification time. For example, if a child has free time at 4 p.m., it sets the notification to be sent at "4 p.m."

[0605] Step 6:

[0606] At the set time, the server sends a message to the child's device saying, "Please do your homework properly."

[0607] Providing information on seasonal events and foods

[0608] Step 1:

[0609] The server checks the current date and retrieves information about seasonal events and foods. For example, if it is October, it retrieves information about "moon viewing."

[0610] Step 2:

[0611] Based on the information acquired by the server, a message to be provided to the user is generated. For example, a message such as "Today is the season for viewing the moon. Let's enjoy some dumplings" is generated.

[0612] Step 3:

[0613] The server sends the generated message to the user terminal for notification.

[0614] This is the specific program flow of the AI ​​Mom System. This system will improve the efficiency of users' lifestyle management and support communication between parents and children.

[0615] Example 1

[0616] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0617] Currently, there are many schedule management systems and lifestyle support systems, but they are difficult to flexibly respond to the diverse needs of users. Furthermore, there are a lack of means for smooth communication between parents and children. Furthermore, there are few systems that provide integrated functions such as nutritional balance advice and reminders. For these reasons, there is a demand for a system that can comprehensively support users' daily lives.

[0618] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0619] In this invention, the server includes means for accepting input of schedules and meal details by voice or text from a user terminal, means for converting the voice into text, means for transmitting the text data to the server, means for analyzing the text data in the server and storing the schedule information in a database, means for setting reminders based on the schedule, means for analyzing the meal records and generating advice on nutritional balance, means for acquiring information on seasonal events and in-season foods, means for receiving comments from parents and notifying them at appropriate times, means for saving the comments sent by the parents and notifying them at appropriate times based on the comments, means for the user to request generation of reminders or advice by voice or text, and means for analyzing the request using natural language processing technology and for the user to generate reminders or advice. This provides comprehensive support for the user's daily life and facilitates communication between parents and children.

[0620] 1. "User terminal" refers to a device that allows a user to input schedules and meal details by voice or text.

[0621] 2. "Means for converting speech to text" refers to a function that converts a user's speech into text data using a speech recognition library or the like.

[0622] 3. "Means for transmitting text data to a server" means a function that uses a communication protocol to transmit text data to a server via a network.

[0623] 4. "Server" means a computing device that analyzes received data, stores necessary information in a database, and generates and sends reminders and advice to the user's device.

[0624] 5. "Means for analyzing text data" means a function that uses Natural Language Processing technology to analyze input text data and extract necessary information.

[0625] 6. "Means for saving to a database" refers to the function of saving the analyzed text data to persistent storage.

[0626] 7. "Means for setting reminders" refers to a function for setting notification timing based on a schedule.

[0627] 8. "Means for generating advice on nutritional balance" refers to a function that analyzes the input dietary information and provides appropriate nutritional advice based on the results.

[0628] 9. "Means for obtaining seasonal event information and in-season food information" is a function for obtaining information on related events and foods based on the current date.

[0629] 10. "Means for receiving comments from parents and notifying them at an appropriate time" refers to a function that receives voice or text comments from parents and notifies them at an appropriate time based on the child's schedule.

[0630] 11. "Natural Language Processing Technology" is a general term for natural language processing technology used to analyze text data.

[0631] 12. "Means for analyzing requests and generating reminders and advice for users" refers to a function that analyzes requests from users and generates reminders and advice based on the results.

[0632] MODE FOR CARRYING OUT THE INVENTION

[0633] The present invention relates to a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[0634] User device functions

[0635] A user terminal is a device that allows users to input schedules and meal details by voice or text. Specifically, this applies to smartphones and tablets. The terminal has the following main functions that operate based on user input:

[0636] Speech Recognition Library: The Google Cloud Speech-to-Text API is used to convert the user's speech into text. This converted text data is sent to the server described below.

[0637] Communication means: After speech recognition, the system has a communication protocol for transmitting the converted text data to the server. For example, the system transmits the data using an HTTP POST request.

[0638] Server Features

[0639] The server receives and analyzes text data sent from the user's device, processes and stores the necessary information, and also generates reminders and advice and notifies the user's device.

[0640] Data reception: Use a web framework such as Flask or Django to receive HTTP POST requests from the user's device.

[0641] Text analysis: Using Natural Language Processing (NLP) techniques, such as spaCy, the received text data is analyzed to extract date and time information and event information.

[0642] Database: Analyzed schedule information is saved in a database such as MySQL or MongoDB. Saved data includes date and time, events, user IDs, etc.

[0643] Set reminders: Use a cron job to send reminder notifications at a set time, for example, "Tomorrow at 9:30 AM."

[0644] Advice generation: Analyzes the dietary information entered by the user and generates appropriate nutritional advice based on the results. For example, if the user enters "Today's breakfast is bread and banana," the system will provide advice such as "You should add a few more vegetables."

[0645] Obtaining seasonal event information: Obtaining seasonal event and seasonal food information based on the current date and notifying the user.

[0646] Parent comment notifications: Receive and save comments sent by parents and notify them at appropriate times based on the child's schedule. For example, if a parent comments "Please do your homework properly," a notification "Please do your homework properly" will be sent to the child during their free time.

[0647] Specific examples

[0648] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[0649] The user's device converts the speech into text and sends it to the server using the Google Cloud Speech-to-Text API.

[0650] The server receives the text data and uses NLP techniques to parse the schedule information for "tomorrow at 10 AM" and uses spaCy to store it in the database.

[0651] The server sets a reminder and uses a cron job to send a reminder notification to the user's device at "tomorrow at 9:30 AM."

[0652] The server generates dietary advice and sends it to the user, such as "You should add a few more vegetables."

[0653] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[0654] The parent device converts the speech into text and sends it to the server, again using the Google Cloud Speech-to-Text API.

[0655] The server receives the comments and uses spaCy to check the child's schedule.

[0656] The server sends a notification to the child's device at an appropriate time, such as during the child's free time, saying, "Please do your homework properly."

[0657] Prompt Sentence Examples

[0658] Here are some examples of prompts that users might enter using a generative AI model:

[0659] "Set a meeting reminder for tomorrow at 10 AM"

[0660] "Today's breakfast is bread and bananas. I'd like some advice on nutritional balance."

[0661] "Remind my child to do their homework properly"

[0662] In this way, the system of the present invention can comprehensively support the daily lives of users and their families, facilitate smooth communication between parents and children, and enable efficient schedule management.

[0663] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0664] Program processing flow

[0665] Step 1:

[0666] The user enters data by voice or text.

[0667] Specific actions: A user uses a device such as a smartphone or tablet to say, "Meeting tomorrow at 10:00 AM," or type in text such as, "Today's breakfast is bread and bananas."

[0668] Input: Audio or text data

[0669] Output: Audio or text data on the user's device

[0670] Step 2:

[0671] The user terminal converts the speech into text.

[0672] Specific operation: The user device uses the Google Cloud Speech-to-Text API to convert the voice data into text data, and the converted text is displayed on the device screen.

[0673] Input: Audio data

[0674] Output: Text data (e.g. "Meeting tomorrow at 10 AM")

[0675] Step 3:

[0676] The user terminal transmits text data to the server.

[0677] Specific operation: The device sends the converted text data to the server using an HTTP POST request. The user's device screen displays the status of the transmission.

[0678] Input: Text data (e.g. "Meeting tomorrow at 10 AM")

[0679] Output: Text data sent to the server

[0680] Step 4:

[0681] The server receives the text data.

[0682] Specific operation: The server uses Flask or Django to receive HTTP POST requests from user terminals and retrieve text data. The received data is recorded in the server log.

[0683] Input: HTTP POST request

[0684] Output: Received text data

[0685] Step 5:

[0686] The server parses the text data.

[0687] Specific operation: The server uses NLP technology such as spaCy to analyze the text data and extract the date and time information for "tomorrow at 10 AM" and the event information for "meeting." The analysis results are recorded in the server log.

[0688] Input: Text data (e.g. "Meeting tomorrow at 10 AM")

[0689] Output: Parsed information (e.g., date and time "tomorrow 10:00 AM", event "meeting")

[0690] Step 6:

[0691] The server stores the analysis results in a database.

[0692] Specific operation: The server stores the analyzed information in MySQL or MongoDB. The stored data includes user ID, date and time, event, etc. The server generates a log to confirm the data stored in the database.

[0693] Input: Parsed information

[0694] Output: Information stored in the database

[0695] Step 7:

[0696] The server generates reminders and advice.

[0697] Specific operation: The server sets up a cron job to send a reminder notification at "9:30 AM tomorrow." It also generates nutritional advice based on "Today's breakfast is bread and bananas," and generates a message saying, "You should add a few more vegetables."

[0698] Input: Saved schedule information and meal information

[0699] Output: Reminder notifications and advice messages

[0700] Step 8:

[0701] The server sends reminders and advice to the user terminal.

[0702] Specific operation: The server uses push notification to send a notification to the user's device at 9:30 AM tomorrow saying, "There will be a meeting at 10 AM." It also sends an advice message.

[0703] Input: Reminder notification setting information and advice message

[0704] Output: Notifications and messages sent to the user's device

[0705] Step 9:

[0706] Parents enter comments.

[0707] Specific operation: The parent speaks into their smartphone, "Please do your homework properly." This voice data is sent to the device.

[0708] Input: Audio data

[0709] Output: Audio data on the parent device

[0710] Step 10:

[0711] The server saves the parent's comments and notifies the child.

[0712] Specific operation: The server receives comments from the parent's device, analyzes them, and stores them in a database. Furthermore, based on the child's schedule, it sends a message to the child's device during their free time saying, "Please do your homework properly."

[0713] Input: Parsed parent comments, child schedule information

[0714] Output: Notification sent to child device

[0715] (Application example 1)

[0716] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0717] It is often difficult to efficiently communicate between workers and managers in factories and manage work schedules. In busy work environments, workers can forget their schedules or miss necessary safety instructions, resulting in reduced efficiency and safety. Furthermore, there are concerns that workers' workloads may increase if they are not properly provided with information related to seasonal changes or specific tasks.

[0718] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0719] In this invention, the server includes means for accepting voice or text input of schedules and work details from a user terminal, means for converting the voice into text, means for transmitting the text data to the server, means for analyzing the text data in the server and storing the schedule information in a database, means for setting reminders based on the schedule, means for analyzing work records and generating work-related advice, means for acquiring seasonal event information and work-related information, and means for receiving comments from a manager and notifying them at an appropriate time, thereby enabling factory workers to efficiently manage their schedules and receive necessary instructions and advice in a timely manner.

[0720] A "user terminal" is a device that accepts voice or text input.

[0721] A "speech recognition library" is a software library for converting voice data into text.

[0722] "Text data" refers to character string data obtained by voice input or text input.

[0723] A "server" is a central processing device that receives text data and performs analysis, storage, and notification.

[0724] "Natural Language Processing technology" refers to computer processing technology for understanding and analyzing human language.

[0725] "Database" refers to a data storage system for storing analyzed schedule information and work records.

[0726] "Reminder" is a function that notifies the user based on a set schedule.

[0727] "Work records" refers to information about specific tasks and time management within a factory.

[0728] "Advice" means advice or recommendations provided based on the analyzed work record.

[0729] A "manager" is a person who is responsible for giving comments and instructions to workers.

[0730] "Seasonal event information" refers to information about specific events for each season.

[0731] "Work-related information" refers to various types of information provided to improve work efficiency and safety.

[0732] The present invention is an in-factory work management and communication system including a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[0733] System Overview

[0734] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user.It also supports communication between managers and workers by receiving comments from managers and notifying them to workers.

[0735] Program processing flow

[0736] User terminal

[0737] A user terminal is a device that allows a user to input schedules and work details by voice or text. When a user inputs by voice, the user terminal uses a speech recognition library (e.g., Google Cloud Speech-to-Text API) to convert the voice into text and sends this text data to the server. For example, if a user inputs "Maintenance inspection of Line 1 at 10:00 AM" by voice, the terminal converts this into text data "Maintenance inspection of Line 1 at 10:00 AM" and sends it to the server.

[0738] server

[0739] The server is responsible for receiving and analyzing the text data sent from the terminal. The server uses Natural Language Processing (NLP) technology (e.g., NLTK, Spacy) to analyze the received text data and saves the schedule information in a database (e.g., PostgreSQL). It also sets reminders and sends reminder notifications to the user terminal at appropriate times. For example, it sends a reminder 30 minutes before the maintenance inspection saying, "Maintenance inspection of Line 1 is coming up soon."

[0740] Furthermore, the server analyzes work records, generates work-related advice (e.g., "Please wear protective equipment before maintenance inspections"), and sends this to the user's terminal. The server also obtains information related to seasonal changes and specific tasks and provides it to the user. For example, in the summer, the server notifies users with information such as "Be careful of heatstroke."

[0741] Administrator comment notification

[0742] Managers input comments by voice or text from their own devices, and the data is sent to the server. The server saves the received comments and checks the worker's schedule. For example, if a manager sends a comment such as "Please wear protective equipment before maintenance inspection," the server will send a notification to the worker's device at the appropriate time based on the worker's schedule, stating "Please wear protective equipment before maintenance inspection."

[0743] Specific examples

[0744] When a worker voice-inputs "Maintenance inspection of Line 1 at 10:00 AM," the following process takes place:

[0745] 1. The user device converts the speech into text and sends it to the server.

[0746] 2. The server receives the text data and uses NLP technology to analyze the schedule information for the "10:00 AM maintenance inspection" and save it in the database.

[0747] 3. The server sets a reminder and sends a reminder notification to the user terminal 30 minutes before the maintenance inspection.

[0748] 4. When the administrator terminal inputs "Please wear protective equipment before maintenance inspection" by voice, the instruction is notified to the user terminal at the appropriate time.

[0749] Prompt Sentence Examples

[0750] "We have a maintenance inspection scheduled for Line 1 at 10:00 AM. Please send me a reminder 30 minutes before. I would also like a reminder to put on protective gear before the inspection."

[0751] As described above, by using this system, factory workers can efficiently manage their schedules and receive necessary instructions and advice in a timely manner.

[0752] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0753] Program processing flow

[0754] Step 1:

[0755] The user inputs schedules and work details using voice or text. The user's device receives the input voice data and converts it into text data using a speech recognition library (e.g., Google Cloud Speech-to-Text API). The text data obtained through this conversion becomes the input for the next step.

[0756] Step 2:

[0757] The user terminal sends the converted text data to the server. The server receives the text data from the user terminal and prepares to analyze it. The input here is text data.

[0758] Step 3:

[0759] The server analyzes the received text data using Natural Language Processing (NLP) techniques (e.g., NLTK, Spacy). From this analysis, schedule information and task details are extracted. The results of this analysis are stored in a database for further processing. The output of the analysis is database entries for schedule information and task details.

[0760] Step 4:

[0761] The server sets a schedule-based reminder based on the analysis results. A specific time, such as 30 minutes before a maintenance inspection, is specified for the reminder setting. This reminder setting information is used to send reminder notifications. The reminder setting information is output.

[0762] Step 5:

[0763] The set reminder is sent to the user's device. The contents of the reminder are personalized for each user and notified at the appropriate time. The input here is the set reminder information, and the output is the reminder notification displayed on the user's device.

[0764] Step 6:

[0765] The server analyzes the work records and generates work-related advice, such as specific advice like "wear protective equipment before maintenance inspection." This advice information becomes the input for the next processing step.

[0766] Step 7:

[0767] The generated advice information is sent to the user terminal and notified to the user at the appropriate time. This allows the worker to receive the necessary advice. The input here is the generated advice information, and the output is the advice notification sent to the user terminal.

[0768] Step 8:

[0769] The administrator inputs a comment by voice or text from their own terminal. The administrator terminal sends this comment data to the server. The sent comment data becomes the input for the next step.

[0770] Step 9:

[0771] The server saves comments received from the administrator and checks the worker's schedule. The comments are notified to the worker's terminal at the appropriate time. This notification allows the worker to receive important instructions and comments. The input here is the comment data from the administrator, and the output is a notification to the worker's terminal.

[0772] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0773] The present invention is a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them, and by combining an emotion engine, it is also able to respond to the emotions of the user. Specific embodiments are described below.

[0774] System Overview

[0775] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user. It also uses an emotion engine to recognize the user's emotions and responds based on that information. It also supports communication between parents and children by receiving comments from parents and notifying the child.

[0776] Program processing flow

[0777] User terminal

[0778] The user terminal is a device that allows users to input schedules and meal details by voice or text. When a user inputs by voice, the user terminal uses a voice recognition library to convert the voice into text and sends this text data to the server. It also has an emotion engine that recognizes the user's emotions from the input voice and text data. For example, if a user inputs "Meeting tomorrow at 10 AM," the terminal converts this into text data "Meeting tomorrow at 10 AM" and sends it to the server, while also analyzing the user's emotions (e.g., stress, anxiety).

[0779] server

[0780] The server is responsible for receiving and analyzing the text data and emotion data sent from the device. For example, the server uses Natural Language Processing (NLP) technology to analyze the received text data and store the schedule information in a database. Furthermore, the server adjusts the content of reminders and advice based on the results of the emotion engine. For example, if the user's emotion is recognized as "stress," the server adjusts the wording of the reminder to be softer.

[0781] For example, if a user inputs "Today's breakfast is bread and bananas," the server analyzes the meal contents and generates advice on nutritional balance. For example, it generates advice such as "You should add a few more vegetables," and sends this advice to the user's terminal.

[0782] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[0783] Parent comment notification

[0784] When a parent enters a comment via voice or text from their device, the data is sent to the server. The server saves the received comment and checks the child's schedule. For example, if a parent sends a comment saying "Please do your homework properly," the server will send a notification saying "Please do your homework properly" to the child's device at an appropriate time based on the child's schedule. Here too, the content of the notification is adjusted taking into account the child's emotional information (e.g., if they are feeling depressed).

[0785] Specific examples

[0786] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[0787] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects "stress" from the user's voice.

[0788] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[0789] 3. The server receives the text data and analyzes it using NLP technology. It extracts the date and time information for "tomorrow at 10:00 AM" and adjusts the reminder based on the emotional data "stress."

[0790] 4. The server saves the schedule information in a database and sets a reminder. For example, it sets a reminder for "tomorrow at 9:30 AM" and adjusts the message content to reduce "stress."

[0791] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[0792] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[0793] 1. The parent device converts the voice to text and analyzes the emotion using an emotion engine. For example, it detects the emotion of "encouragement."

[0794] 2. The device sends the converted text data and emotion data to the server.

[0795] 3. The server receives and stores the comment data. Based on the emotional data "encouragement," it sets up notifications to be sent at the optimal time for the child's situation.

[0796] 4. At the set time, the server sends a message to the child's device saying, "Please do your homework properly." For example, at 4:00 PM, the server sends a message saying, "Please do your homework properly. Good luck!"

[0797] The specific system configuration and programs required to implement the present invention are those that properly perform the above-mentioned processes, thereby improving the efficiency of users' lifestyle management and supporting communication between parents and children in a way that takes emotion into consideration.

[0798] The processing flow will be explained below.

[0799] Specific processing flow of the program

[0800] Schedule management and emotion recognition processing

[0801] Step 1:

[0802] The user inputs the schedule details by voice. For example, the user may say, "Meeting tomorrow at 10:00 AM."

[0803] Step 2:

[0804] The device uses a speech recognition library to convert the voice data into text data. Specifically, the voice "Meeting tomorrow at 10 AM" is converted into text "Meeting tomorrow at 10 AM."

[0805] Step 3:

[0806] The device uses an emotion engine to analyze the user's emotion from the input voice data. For example, the emotion "stress" is analyzed.

[0807] Step 4:

[0808] The device sends the converted text data and analyzed emotion data to the server via an HTTP request.

[0809] Step 5:

[0810] The server receives the text data and emotion data and analyzes the text data using Natural Language Processing (NLP) technology, extracting the date and time information for "tomorrow at 10:00 AM."

[0811] Step 6:

[0812] The server saves the extracted schedule information in a database. For example, it saves the meeting information for "December 20, 2023 at 10:00 AM."

[0813] Step 7:

[0814] The server adjusts and sets the content of the reminder based on the emotion data. For example, if the user's emotion is recognized as "stressed," the server adjusts the reminder to a more soothing message. For example, it sets the reminder to "Tomorrow at 9:30 AM" and adjusts the notification content to "You have a meeting at 10 AM. Relax and enjoy."

[0815] Step 8:

[0816] When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[0817] Food record and advice processing

[0818] Step 1:

[0819] The user inputs the details of their meal by voice, for example, saying, "Today's breakfast is bread and bananas."

[0820] Step 2:

[0821] The device converts the voice data into text data. The voice "Today's breakfast is bread and bananas" is converted into text "Today's breakfast is bread and bananas."

[0822] Step 3:

[0823] The terminal transmits the converted text data to the server.

[0824] Step 4:

[0825] The server receives the text data and uses the Nutrient Analysis API to analyze the dietary content, evaluating the carbohydrate and vitamin content as analysis results.

[0826] Step 5:

[0827] The server generates nutritional advice based on the analysis results, such as "You might want to add a few more vegetables."

[0828] Step 6:

[0829] The server transmits the generated advice to the user terminal and notifies it.

[0830] Parent comment notification processing

[0831] Step 1:

[0832] Parents can comment by voice or text, for example, saying, "Please do your homework."

[0833] Step 2:

[0834] The device converts the voice to text. The voice "Do your homework properly" is converted to text "Do your homework properly."

[0835] Step 3:

[0836] The terminal transmits the converted text data and the emotion data generated by the emotion engine to the server.

[0837] Step 4:

[0838] The server receives the comment data and the emotion data and stores them. The emotion data is analyzed as, for example, "encouragement."

[0839] Step 5:

[0840] The server checks the child's schedule and calculates the appropriate time to notify them of the comment. For example, if the child has free time at 4 p.m., it will set the notification to be sent at that time.

[0841] Step 6:

[0842] The server sends the message "Do your homework properly" to the child's device at the set time. For example, at 4:00 PM, it sends the message "Do your homework properly. Good luck!"

[0843] Providing information on seasonal events and foods

[0844] Step 1:

[0845] The server checks the current date and retrieves information about seasonal events and foods. For example, if it is October, it retrieves information about "moon viewing."

[0846] Step 2:

[0847] Based on the information acquired by the server, a message to be provided to the user is generated. For example, a message such as "Today is the season for viewing the moon. Let's enjoy some dumplings" is generated.

[0848] Step 3:

[0849] The server sends the generated message to the user terminal for notification.

[0850] This is the specific program flow of the AI ​​Mom System. This system will improve the efficiency of users' lifestyle management and support communication between parents and children in a way that takes emotions into consideration.

[0851] Example 2

[0852] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0853] Conventional schedule management systems and lifestyle support systems simply process information mechanically without considering the user's feelings, which means they are ineffective at reducing the user's psychological burden. Specifically, it is difficult to provide appropriate reminders and support to users who are stressed or anxious. Furthermore, they lack functionality to support communication between parents and children, making it difficult to maintain a smooth parent-child relationship.

[0854] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0855] In this invention, the server includes means for analyzing text data and emotion data sent from the user terminal and saving schedule information in a database, means for setting reminders based on the schedule and adjusting them based on the emotion data, and means for analyzing the meal record, generating advice on nutritional balance, and adjusting them based on the emotion data. This makes it possible to provide reminders and support that take the user's emotions into consideration. Furthermore, by providing a function for receiving comments from parents and notifying them at an appropriate time while taking into account the child's emotion data, it is possible to facilitate communication between parents and support the maintenance of a smooth parent-child relationship.

[0856] A "user terminal" is a device that allows a user to input schedules and meal details by voice or text.

[0857] A "voice recognition library" is a software component for converting voice data into text data.

[0858] "Emotion data" is information about a user's emotional state that has been analyzed using emotion recognition technology.

[0859] "Server" means a central device that receives, analyzes, and processes data sent from user terminals.

[0860] "Text data" is data in text format that has been converted by voice recognition.

[0861] "Natural Language Processing technology" is a technology for understanding and generating natural language.

[0862] A "database" is a system for organizing and storing information.

[0863] "Reminder" is a function for notifying the user of a specific date, time or event.

[0864] "Diet record" is data about dietary details entered by the user.

[0865] "Nutritional balance" refers to the proper distribution of nutrients in food.

[0866] "Event information" is information about specific activities or events related to seasons, dates, and times.

[0867] "Parent comment" is a message entered by a parent to their child.

[0868] This invention is a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them, and by combining it with an emotion engine, it can also respond to the user's emotions. This system is configured as follows.

[0869] System Overview

[0870] A user terminal is a device that allows a user to input schedules and meal details by voice or text. Examples of such devices include smartphones and tablet devices. When a user inputs voice information, the user terminal uses a speech recognition library (e.g., Google Cloud Speech-to-Text API) to convert the speech into text and sends this text data to a server. In addition, an emotion recognition API (e.g., Microsoft Azure Cognitive Services) is used to recognize the user's emotions from the input voice or text data.

[0871] The server is responsible for receiving and analyzing the text data and emotion data sent from the user terminal. The server analyzes the received text data using Natural Language Processing technology (e.g., Google Cloud Natural Language API) and stores the schedule information in a database (e.g., MySQL).

[0872] Furthermore, the content of reminders and advice is adjusted based on the results of the emotion engine. For example, if the user's emotion is recognized as "stress," the wording of the reminder is softened or adjusted accordingly. Also, if the user inputs "Today's breakfast is bread and banana," the server analyzes the meal contents and generates advice on nutritional balance. For example, advice such as "You should add a few more vegetables" is generated and sent to the user's device.

[0873] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[0874] The system also receives comments from parents and checks their children's schedules. For example, if a parent sends a comment saying, "Please do your homework properly," the server will send a notification saying, "Please do your homework properly" to the child's device at an appropriate time based on the child's schedule. At this time, the content of the notification will be adjusted taking into account the child's emotional information (e.g., if they are feeling depressed).

[0875] Specific operation example

[0876] Schedule management example

[0877] If a user speaks "Meeting tomorrow at 10 AM," the following occurs:

[0878] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects "stress" from the user's voice.

[0879] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[0880] 3. The server receives the text data and analyzes it using NLP technology. It extracts the date and time information for "tomorrow at 10:00 AM" and adjusts the reminder based on the emotional data "stress."

[0881] 4. The server saves the schedule information in a database and sets a reminder. For example, it sets a reminder for "tomorrow at 9:30 AM" and adjusts the message content to reduce "stress."

[0882] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[0883] Example of parent comment notification

[0884] When a parent sends a comment saying "Please do your homework properly," the following occurs.

[0885] 1. The parent device converts the voice to text and analyzes the emotion using an emotion engine. For example, it detects the emotion of "encouragement."

[0886] 2. The device sends the converted text data and emotion data to the server.

[0887] 3. The server receives and stores the comment data. Based on the emotional data "encouragement," it sets up notifications to be sent at the optimal time for the child's situation.

[0888] 4. At the set time, the server sends a message to the child's device saying, "Please do your homework properly." For example, at 4:00 PM, the server sends a message saying, "Please do your homework properly. Good luck!"

[0889] Prompt Sentence Examples

[0890] Here are some examples of prompts to input to a generative AI model:

[0891] If a user says "Meet tomorrow at 10 AM," how does the system handle this?

[0892]

[0893] If a parent sends a comment like "Do your homework properly," how does the system handle that?

[0894] conclusion

[0895] This system will help users manage their daily lives more efficiently and provide emotionally sensitive reminders and advice, reducing the user's psychological burden. It will also facilitate communication between parents and children, helping to maintain parent-child relationships.

[0896] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0897] Specific processing steps for schedule management

[0898] Step 1: User enters schedule by voice

[0899] The user inputs by voice, "Meeting tomorrow at 10:00 AM." This input data is sent to the user's terminal as voice data.

[0900] Step 2: Speech to text

[0901] The device uses the Google Cloud Speech-to-Text API to convert the voice data entered by the user into text data.

[0902] Input: Voice data "Meeting tomorrow at 10 AM"

[0903] Data processing: Convert speech to text using a speech recognition library

[0904] Output: Text data "Meeting tomorrow at 10 AM"

[0905] Step 3: Sentiment analysis

[0906] The device uses the emotion recognition API of Microsoft Azure Cognitive Services to analyze the user's emotions from the converted text data.

[0907] Input: Text data "Meeting tomorrow at 10 AM"

[0908] Data processing: Emotion analysis using emotion recognition API

[0909] Output: Emotion data "Stress"

[0910] Step 4: Sending data

[0911] The device encodes the converted text data and emotion data into JSON format and sends it to the server via HTTPS.

[0912] Input: Text data "Meeting tomorrow at 10 AM", emotion data "Stress"

[0913] Data processing: Encode text data and emotion data into JSON format

[0914] Output: JSON data

[0915] Step 5: Data reception and analysis

[0916] The server receives the JSON data and uses the Google Cloud Natural Language API to parse the text data and extract the date and time information for "tomorrow at 10:00 AM."

[0917] Input: JSON data

[0918] Data processing: Analyze text data using natural language processing technology and extract date and time information

[0919] Output: Date and time information "Tomorrow 10:00 AM"

[0920] Step 6: Save the schedule

[0921] The server stores the extracted date and time information and text data in a MySQL database.

[0922] Input: Date and time information "Tomorrow 10:00 AM", text data "Meeting tomorrow at 10:00 AM"

[0923] Data processing: Execute SQL queries and save to database

[0924] Output: Schedule information stored in the database

[0925] Step 7: Set reminders

[0926] The server adjusts the content of the reminder based on the emotional data "stress" and sets reminder information based on the schedule, for example, setting a reminder 30 minutes before a meeting.

[0927] Input: Emotion data "Stress", Date and time information "Tomorrow 10:00 AM"

[0928] Data processing: Generate emotion-sensitive reminder messages and set reminder information

[0929] Output: Reminder information (date, time and message)

[0930] Step 8: Send reminders

[0931] When the server reaches the set time (e.g. 9:30 AM), it sends a reminder notification to the user's device.

[0932] Input: Reminder information

[0933] Data processing: Send reminders via HTTP POST request

[0934] Output: Reminder message "There's a meeting at 10 o'clock. Relax and enjoy."

[0935] Specific steps for parent comment notification

[0936] Step 1: Parents enter comments by voice

[0937] The parent can input "Please do your homework properly" by voice. This input data is sent to the parent device as voice data.

[0938] Step 2: Speech to text

[0939] The parent device uses the Google Cloud Speech-to-Text API to convert the voice data input by the parent into text data.

[0940] Input: Voice data "Do your homework properly"

[0941] Data processing: Convert speech to text using a speech recognition library

[0942] Output: Text data "Please do your homework properly."

[0943] Step 3: Sentiment analysis

[0944] The parent device uses the emotion recognition API of Microsoft Azure Cognitive Services to analyze emotions from the converted text data.

[0945] Input: Text data "Please do your homework properly"

[0946] Data processing: Emotion analysis using emotion recognition API

[0947] Output: Emotion data "Encouragement"

[0948] Step 4: Sending data

[0949] The parent device encodes the converted text data and emotion data into JSON format and sends it to the server via HTTPS.

[0950] Input: Text data "Do your homework properly", emotion data "Encouragement"

[0951] Data processing: Encode text data and emotion data into JSON format

[0952] Output: JSON data

[0953] Step 5: Data reception and analysis

[0954] The server interprets the received JSON data and extracts text and emotion data.

[0955] Input: JSON data

[0956] Data processing: Analyze JSON data and extract text and sentiment data

[0957] Output: Text data, emotion data

[0958] Step 6: Save comments

[0959] The server stores the extracted text data and emotion data in a database.

[0960] Input: Text data, emotion data

[0961] Data manipulation: Save to database using SQL queries

[0962] Output: Comment information stored in the database

[0963] Step 7: Notification Settings

[0964] The server checks the child's schedule and configures it to send notifications at appropriate times based on emotional data.

[0965] Input: Child's schedule information, emotional data "encouragement"

[0966] Data processing: Set notification timing and content

[0967] Output: Notification settings information

[0968] Step 8: Send notification

[0969] When the server reaches the set time (e.g. 3:50 pm), it will notify the child's device of the parent's comment, "Make sure you do your homework properly."

[0970] Input: Notification settings information

[0971] Data processing: Send parent comment via HTTP POST request

[0972] Output: Notification message "Do your homework properly. Good luck!"

[0973] conclusion

[0974] In this way, the user's lifestyle management becomes more efficient, and it becomes possible to provide emotionally sensitive reminders and advice. It also facilitates communication between parents and children, helping to maintain parent-child relationships.

[0975] (Application example 2)

[0976] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0977] Currently, there is a demand for improved efficiency in factory operations and smoother communication with workers. In particular, it is important to be able to quickly and accurately interpret voice instructions from workers and respond accordingly. There is also a need to understand workers' emotions and provide appropriate advice and reminders to reduce their stress. However, current systems are unable to meet these requirements.

[0978] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting voice or text input of schedules and meal details from a user terminal; means for converting the voice into text; means for transmitting the text data to the server; means for analyzing the text data in the server and storing the schedule information in a database; means for setting reminders based on the schedule; means for analyzing meal records and generating advice on nutritional balance; means for acquiring information on seasonal events and in-season foods; means for receiving comments from parents and notifying them at appropriate times; means for accepting voice instructions from workers in a factory environment, converting them into text in real time, and analyzing schedule and emotion data; and means for adjusting reminders and advice to workers based on the emotion data and notifying them at appropriate times. This enables improved work efficiency in the factory and smoother communication with workers.

[0979] A "user terminal" is a device that accepts input of schedules and meal details by voice or text.

[0980] "Means for converting speech to text" refers to technology for converting speech input by a user into text data.

[0981] The "means for transmitting text data to a server" is a function for transmitting the converted text data to a server via a network.

[0982] A "server" is a computer system for analyzing and storing data sent from a user terminal.

[0983] The "means for saving schedule information in a database" is a function for storing analyzed schedule information in a database.

[0984] The "means for setting a reminder" is a function for notifying the user based on a set schedule.

[0985] "Means for analyzing dietary records" refers to technology for analyzing dietary details entered by a user.

[0986] The "means for generating advice regarding nutritional balance" is a function for providing advice regarding nutrition based on the analyzed dietary record.

[0987] The "means for acquiring seasonal event information and seasonal food information" is a function for collecting event information and seasonal food information corresponding to each season and providing it to the user.

[0988] The "means for receiving comments from parents and notifying them at an appropriate time" is a function for receiving comments entered by parents and notifying them in accordance with the child's schedule.

[0989] "Factory environment" refers to the location where manufacturing and product assembly operations occur.

[0990] The "means for receiving voice instructions from workers" is a function that enables workers working in the factory to input voice instructions.

[0991] "Means for converting to text in real time" refers to technology for converting voice instructions into text data in real time.

[0992] "Emotion data" is data that indicates the emotional state of a worker analyzed from his / her voice.

[0993] The "means for adjusting reminders and advice to workers based on emotional data" is a function for providing appropriate reminders and advice to workers based on the analyzed emotional data.

[0994] This invention is a system for improving work efficiency in factories and facilitating communication with workers. The system includes a user terminal, a server, and a communication means for exchanging data between them. By combining it with an emotion engine, the system can also respond to the emotions of workers.

[0995] System Overview

[0996] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user. In particular, it uses an emotion engine to recognize the worker's emotions and responds based on that information. It also has a function to receive comments from parents and notify the child.

[0997] User terminal

[0998] The user terminal is a device that allows workers to input instructions and schedules by voice or text. When using voice input, the user terminal uses a voice recognition library (e.g., the SpeechRecognition library) to convert the voice into text and sends this text data to the server. It also has an emotion engine (e.g., an emotion recognition model using Transformers) that recognizes the worker's emotions from the input voice and text data. For example, if a worker inputs "Maintenance work will be performed at 3:00 PM today," the terminal converts this into text data that reads "Maintenance work will be performed at 3:00 PM today" and sends it to the server. At the same time, the terminal also analyzes the worker's emotions (e.g., sense of relief).

[0999] server

[1000] The server is responsible for receiving and analyzing the text and emotion data sent from the device. Specifically, it uses Natural Language Processing (NLP) technology to analyze the received text data and store the schedule information in a database. It also adjusts the content of reminders and advice based on the results of the emotion engine. For example, if the worker's emotion is recognized as "relief," the wording of the reminder will be adjusted accordingly. The server then sends the processed notification to the user's device at a specific time.

[1001] Specific examples

[1002] For example, if a worker says, "Maintenance work will be performed at 3:00 p.m. today. Please don't forget," the following process will occur:

[1003] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects a "sense of security."

[1004] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[1005] 3. The server receives the text data and analyzes it using NLP technology. It extracts the time information "3 PM" and adjusts the reminder based on the emotional data "sense of relief."

[1006] 4. The server saves the schedule information to a database and sets a reminder, for example, for 2:45 PM, with a tailored message tailored to the user's emotional state.

[1007] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 2:45 PM, it sends a notification saying, "Maintenance work will be performed at 3:00. Please proceed without worry."

[1008] Prompt Sentence Examples

[1009] An example of a voice instruction that a worker inputs into a terminal is a prompt such as, "Maintenance work will be performed at 3:00 PM today. Please don't forget." An example of a notification is a statement such as, "Maintenance work will be performed at 3:00 PM. Please proceed without worry."

[1010] In this way, work efficiency within the factory is improved and communication with workers is facilitated.

[1011] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1012] Step 1:

[1013] The user inputs by voice, "Maintenance work will be performed at 3:00 PM today. Please don't forget." This input is received by the terminal as voice data.

[1014] Step 2:

[1015] The device uses a speech recognition library (for example, the SpeechRecognition library) to convert the received voice data into text data. The input is voice data, and the output is text data: "Maintenance work will be carried out at 3:00 p.m. today. Please remember."

[1016] Step 3:

[1017] The device analyzes the converted text data using an emotion engine (e.g., an emotion recognition model using Transformers). The input is text data, and the device outputs emotion data (e.g., a sense of relief) as the result of emotion analysis.

[1018] Step 4:

[1019] The device sends the converted text data and emotion data to the server via an HTTP request. The input is text data and emotion data, and the device sends them to the server as output.

[1020] Step 5:

[1021] The text data received by the server is analyzed using Natural Language Processing (NLP) technology. The input is text data, and the analysis results extract date and time information (e.g., "3:00 PM") and process it into schedule information. The output is schedule information.

[1022] Step 6:

[1023] The server saves the schedule information in a database. The input is the extracted schedule information, and the output is the information saved in the database.

[1024] Step 7:

[1025] The server adjusts the reminder and advice content based on the emotion data and schedule information. The input is the emotion data and schedule information, and the output is the adjusted reminder content.

[1026] Step 8:

[1027] When the server reaches the reminder setting time (for example, 2:45 PM), it sends a reminder notification to the user terminal. The input is the adjusted reminder content, and the output is the notification sent to the user terminal.

[1028] Step 9:

[1029] The user device receives the notification and notifies the user by voice or text. The input is a reminder notification, and the output is a notification to the user saying, "There will be maintenance work at 3:00 PM. Please proceed with confidence."

[1030] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1031] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1032] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1033] [Third embodiment]

[1034] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1035] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1037] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1038] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1039] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1041] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1042] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1044] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1045] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1046] The present invention relates to a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[1047] System Overview

[1048] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user.It also supports communication between parents and children by receiving comments from parents and notifying their children.

[1049] Program processing flow

[1050] User terminal

[1051] A user terminal is a device that allows a user to input schedules and meal details by voice or text. When a user inputs by voice, the user terminal uses a voice recognition library to convert the voice into text and sends this text data to the server. For example, if a user inputs "Meeting tomorrow at 10 AM" by voice, the terminal converts this into text data "Meeting tomorrow at 10 AM" and sends it to the server.

[1052] server

[1053] The server is responsible for receiving and analyzing text data sent from the device. For example, the server uses Natural Language Processing (NLP) technology to analyze the received text data and save the schedule information in a database. It also sets reminders and sends reminder notifications to the user's device at appropriate times. If a user inputs "Today's breakfast is bread and banana," the server analyzes the meal contents and generates advice about nutritional balance. For example, it generates advice such as "You should add a few more vegetables" and sends this to the user's device.

[1054] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[1055] Parent comment notification

[1056] When a parent enters a comment by voice or text from their device, the data is sent to the server. The server saves the received comment and checks the child's schedule. For example, if a parent sends a comment such as "Please do your homework properly," the server will send a notification to the child's device saying "Please do your homework properly" at an appropriate time based on the child's schedule. An appropriate time could be, for example, a time when the child has free time or before bedtime.

[1057] Specific examples

[1058] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[1059] 1. The user device converts the speech into text and sends it to the server.

[1060] 2. The server receives the text data and uses NLP technology to analyze the schedule information for "tomorrow at 10:00 AM" and saves it in a database.

[1061] 3. The server sets a reminder and sends a reminder notification to the user terminal, for example, "tomorrow at 9:30 AM."

[1062] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[1063] 1. The parent device converts the voice into text and sends it to the server.

[1064] 2. The server receives the comment and checks the child's schedule.

[1065] 3. At the appropriate time, the server notifies the child's device, "Please do your homework properly."

[1066] The specific system configuration and programs required to implement the present invention are those that properly perform the above-mentioned processes, thereby supporting efficient communication between the user and their family and lifestyle management.

[1067] The processing flow will be explained below.

[1068] Specific processing flow of the program

[1069] Schedule management process

[1070] Step 1:

[1071] The user inputs the schedule details by voice. For example, the user may say, "Meeting tomorrow at 10:00 AM."

[1072] Step 2:

[1073] The device uses a speech recognition library to convert the voice data into text data. Specifically, the voice "Meeting tomorrow at 10 AM" is converted into text "Meeting tomorrow at 10 AM."

[1074] Step 3:

[1075] The terminal sends the converted text data to the server via an HTTP request.

[1076] Step 4:

[1077] The server receives the text data and analyzes it using Natural Language Processing (NLP) technology, extracting the date and time information "tomorrow at 10 AM" from the text.

[1078] Step 5:

[1079] The server saves the extracted schedule information in a database. For example, it saves the meeting information for "December 20, 2023 at 10:00 AM."

[1080] Step 6:

[1081] The server sets a reminder based on the saved schedule, for example, "December 20, 2023 at 9:30 AM."

[1082] Step 7:

[1083] When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There will be a meeting at 10:00."

[1084] Food record and advice processing

[1085] Step 1:

[1086] The user inputs the details of their meal by voice, for example, saying, "Today's breakfast is bread and bananas."

[1087] Step 2:

[1088] The device converts the voice data into text data. The voice "Today's breakfast is bread and bananas" is converted into text "Today's breakfast is bread and bananas."

[1089] Step 3:

[1090] The terminal transmits the converted text data to the server.

[1091] Step 4:

[1092] The server receives the text data and uses the Nutrient Analysis API to analyze the dietary content, evaluating the carbohydrate and vitamin content as analysis results.

[1093] Step 5:

[1094] The server generates nutritional advice based on the analysis results, such as "You might want to add a few more vegetables."

[1095] Step 6:

[1096] The server transmits the generated advice to the user terminal and notifies it.

[1097] Parent comment notification processing

[1098] Step 1:

[1099] Parents can comment by voice or text, for example, saying, "Please do your homework."

[1100] Step 2:

[1101] The device converts the voice to text. The voice "Do your homework properly" is converted to text "Do your homework properly."

[1102] Step 3:

[1103] The terminal transmits the converted text data to the server.

[1104] Step 4:

[1105] The server receives and stores the comment data.

[1106] Step 5:

[1107] The server checks the child's schedule and determines the appropriate notification time. For example, if a child has free time at 4 p.m., it sets the notification to be sent at "4 p.m."

[1108] Step 6:

[1109] At the set time, the server sends a message to the child's device saying, "Please do your homework properly."

[1110] Providing information on seasonal events and foods

[1111] Step 1:

[1112] The server checks the current date and retrieves information about seasonal events and foods. For example, if it is October, it retrieves information about "moon viewing."

[1113] Step 2:

[1114] Based on the information acquired by the server, a message to be provided to the user is generated. For example, a message such as "Today is the season for viewing the moon. Let's enjoy some dumplings" is generated.

[1115] Step 3:

[1116] The server sends the generated message to the user terminal for notification.

[1117] This is the specific program flow of the AI ​​Mom System. This system will improve the efficiency of users' lifestyle management and support communication between parents and children.

[1118] Example 1

[1119] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1120] Currently, there are many schedule management systems and lifestyle support systems, but they are difficult to flexibly respond to the diverse needs of users. Furthermore, there are a lack of means for smooth communication between parents and children. Furthermore, there are few systems that provide integrated functions such as nutritional balance advice and reminders. For these reasons, there is a demand for a system that can comprehensively support users' daily lives.

[1121] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1122] In this invention, the server includes means for accepting input of schedules and meal details by voice or text from a user terminal, means for converting the voice into text, means for transmitting the text data to the server, means for analyzing the text data in the server and storing the schedule information in a database, means for setting reminders based on the schedule, means for analyzing the meal records and generating advice on nutritional balance, means for acquiring information on seasonal events and in-season foods, means for receiving comments from parents and notifying them at appropriate times, means for saving the comments sent by the parents and notifying them at appropriate times based on the comments, means for the user to request generation of reminders or advice by voice or text, and means for analyzing the request using natural language processing technology and for the user to generate reminders or advice. This provides comprehensive support for the user's daily life and facilitates communication between parents and children.

[1123] 1. "User terminal" refers to a device that allows a user to input schedules and meal details by voice or text.

[1124] 2. "Means for converting speech to text" refers to a function that converts a user's speech into text data using a speech recognition library or the like.

[1125] 3. "Means for transmitting text data to a server" means a function that uses a communication protocol to transmit text data to a server via a network.

[1126] 4. "Server" means a computing device that analyzes received data, stores necessary information in a database, and generates and sends reminders and advice to the user's device.

[1127] 5. "Means for analyzing text data" means a function that uses Natural Language Processing technology to analyze input text data and extract necessary information.

[1128] 6. "Means for saving to a database" refers to the function of saving the analyzed text data to persistent storage.

[1129] 7. "Means for setting reminders" refers to a function for setting notification timing based on a schedule.

[1130] 8. "Means for generating advice on nutritional balance" refers to a function that analyzes the input dietary information and provides appropriate nutritional advice based on the results.

[1131] 9. "Means for obtaining seasonal event information and in-season food information" is a function for obtaining information on related events and foods based on the current date.

[1132] 10. "Means for receiving comments from parents and notifying them at an appropriate time" refers to a function that receives voice or text comments from parents and notifies them at an appropriate time based on the child's schedule.

[1133] 11. "Natural Language Processing Technology" is a general term for natural language processing technology used to analyze text data.

[1134] 12. "Means for analyzing requests and generating reminders and advice for users" refers to a function that analyzes requests from users and generates reminders and advice based on the results.

[1135] MODE FOR CARRYING OUT THE INVENTION

[1136] The present invention relates to a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[1137] User device functions

[1138] A user terminal is a device that allows users to input schedules and meal details by voice or text. Specifically, this applies to smartphones and tablets. The terminal has the following main functions that operate based on user input:

[1139] Speech Recognition Library: The Google Cloud Speech-to-Text API is used to convert the user's speech into text. This converted text data is sent to the server described below.

[1140] Communication means: After speech recognition, the system has a communication protocol for transmitting the converted text data to the server. For example, the system transmits the data using an HTTP POST request.

[1141] Server Features

[1142] The server receives and analyzes text data sent from the user's device, processes and stores the necessary information, and also generates reminders and advice and notifies the user's device.

[1143] Data reception: Use a web framework such as Flask or Django to receive HTTP POST requests from the user's device.

[1144] Text analysis: Using Natural Language Processing (NLP) techniques, such as spaCy, the received text data is analyzed to extract date and time information and event information.

[1145] Database: Analyzed schedule information is saved in a database such as MySQL or MongoDB. Saved data includes date and time, events, user IDs, etc.

[1146] Set reminders: Use a cron job to send reminder notifications at a set time, for example, "Tomorrow at 9:30 AM."

[1147] Advice generation: Analyzes the dietary information entered by the user and generates appropriate nutritional advice based on the results. For example, if the user enters "Today's breakfast is bread and banana," the system will provide advice such as "You should add a few more vegetables."

[1148] Obtaining seasonal event information: Obtaining seasonal event and seasonal food information based on the current date and notifying the user.

[1149] Parent comment notifications: Receive and save comments sent by parents and notify them at appropriate times based on the child's schedule. For example, if a parent comments "Please do your homework properly," a notification "Please do your homework properly" will be sent to the child during their free time.

[1150] Specific examples

[1151] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[1152] The user's device converts the speech into text and sends it to the server using the Google Cloud Speech-to-Text API.

[1153] The server receives the text data and uses NLP techniques to parse the schedule information for "tomorrow at 10 AM" and uses spaCy to store it in the database.

[1154] The server sets a reminder and uses a cron job to send a reminder notification to the user's device at "tomorrow at 9:30 AM."

[1155] The server generates dietary advice and sends it to the user, such as "You should add a few more vegetables."

[1156] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[1157] The parent device converts the speech into text and sends it to the server, again using the Google Cloud Speech-to-Text API.

[1158] The server receives the comments and uses spaCy to check the child's schedule.

[1159] The server sends a notification to the child's device at an appropriate time, such as during the child's free time, saying, "Please do your homework properly."

[1160] Prompt Sentence Examples

[1161] Here are some examples of prompts that users might enter using a generative AI model:

[1162] "Set a meeting reminder for tomorrow at 10 AM"

[1163] "Today's breakfast is bread and bananas. I'd like some advice on nutritional balance."

[1164] "Remind my child to do their homework properly"

[1165] In this way, the system of the present invention can comprehensively support the daily lives of users and their families, facilitate smooth communication between parents and children, and enable efficient schedule management.

[1166] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1167] Program processing flow

[1168] Step 1:

[1169] The user enters data by voice or text.

[1170] Specific actions: A user uses a device such as a smartphone or tablet to say, "Meeting tomorrow at 10:00 AM," or type in text such as, "Today's breakfast is bread and bananas."

[1171] Input: Audio or text data

[1172] Output: Audio or text data on the user's device

[1173] Step 2:

[1174] The user terminal converts the speech into text.

[1175] Specific operation: The user device uses the Google Cloud Speech-to-Text API to convert the voice data into text data, and the converted text is displayed on the device screen.

[1176] Input: Audio data

[1177] Output: Text data (e.g. "Meeting tomorrow at 10 AM")

[1178] Step 3:

[1179] The user terminal transmits text data to the server.

[1180] Specific operation: The device sends the converted text data to the server using an HTTP POST request. The user's device screen displays the status of the transmission.

[1181] Input: Text data (e.g. "Meeting tomorrow at 10 AM")

[1182] Output: Text data sent to the server

[1183] Step 4:

[1184] The server receives the text data.

[1185] Specific operation: The server uses Flask or Django to receive HTTP POST requests from user terminals and retrieve text data. The received data is recorded in the server log.

[1186] Input: HTTP POST request

[1187] Output: Received text data

[1188] Step 5:

[1189] The server parses the text data.

[1190] Specific operation: The server uses NLP technology such as spaCy to analyze the text data and extract the date and time information for "tomorrow at 10 AM" and the event information for "meeting." The analysis results are recorded in the server log.

[1191] Input: Text data (e.g. "Meeting tomorrow at 10 AM")

[1192] Output: Parsed information (e.g., date and time "tomorrow 10:00 AM", event "meeting")

[1193] Step 6:

[1194] The server stores the analysis results in a database.

[1195] Specific operation: The server stores the analyzed information in MySQL or MongoDB. The stored data includes user ID, date and time, event, etc. The server generates a log to confirm the data stored in the database.

[1196] Input: Parsed information

[1197] Output: Information stored in the database

[1198] Step 7:

[1199] The server generates reminders and advice.

[1200] Specific operation: The server sets up a cron job to send a reminder notification at "9:30 AM tomorrow." It also generates nutritional advice based on "Today's breakfast is bread and bananas," and generates a message saying, "You should add a few more vegetables."

[1201] Input: Saved schedule information and meal information

[1202] Output: Reminder notifications and advice messages

[1203] Step 8:

[1204] The server sends reminders and advice to the user terminal.

[1205] Specific operation: The server uses push notification to send a notification to the user's device at 9:30 AM tomorrow saying, "There will be a meeting at 10 AM." It also sends an advice message.

[1206] Input: Reminder notification setting information and advice message

[1207] Output: Notifications and messages sent to the user's device

[1208] Step 9:

[1209] Parents enter comments.

[1210] Specific operation: The parent speaks into their smartphone, "Please do your homework properly." This voice data is sent to the device.

[1211] Input: Audio data

[1212] Output: Audio data on the parent device

[1213] Step 10:

[1214] The server saves the parent's comments and notifies the child.

[1215] Specific operation: The server receives comments from the parent's device, analyzes them, and stores them in a database. Furthermore, based on the child's schedule, it sends a message to the child's device during their free time saying, "Please do your homework properly."

[1216] Input: Parsed parent comments, child schedule information

[1217] Output: Notification sent to child device

[1218] (Application example 1)

[1219] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1220] It is often difficult to efficiently communicate between workers and managers in factories and manage work schedules. In busy work environments, workers can forget their schedules or miss necessary safety instructions, resulting in reduced efficiency and safety. Furthermore, there are concerns that workers' workloads may increase if they are not properly provided with information related to seasonal changes or specific tasks.

[1221] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1222] In this invention, the server includes means for accepting voice or text input of schedules and work details from a user terminal, means for converting the voice into text, means for transmitting the text data to the server, means for analyzing the text data in the server and storing the schedule information in a database, means for setting reminders based on the schedule, means for analyzing work records and generating work-related advice, means for acquiring seasonal event information and work-related information, and means for receiving comments from a manager and notifying them at an appropriate time, thereby enabling factory workers to efficiently manage their schedules and receive necessary instructions and advice in a timely manner.

[1223] A "user terminal" is a device that accepts voice or text input.

[1224] A "speech recognition library" is a software library for converting voice data into text.

[1225] "Text data" refers to character string data obtained by voice input or text input.

[1226] A "server" is a central processing device that receives text data and performs analysis, storage, and notification.

[1227] "Natural Language Processing technology" refers to computer processing technology for understanding and analyzing human language.

[1228] "Database" refers to a data storage system for storing analyzed schedule information and work records.

[1229] "Reminder" is a function that notifies the user based on a set schedule.

[1230] "Work records" refers to information about specific tasks and time management within a factory.

[1231] "Advice" means advice or recommendations provided based on the analyzed work record.

[1232] A "manager" is a person who is responsible for giving comments and instructions to workers.

[1233] "Seasonal event information" refers to information about specific events for each season.

[1234] "Work-related information" refers to various types of information provided to improve work efficiency and safety.

[1235] The present invention is an in-factory work management and communication system including a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[1236] System Overview

[1237] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user.It also supports communication between managers and workers by receiving comments from managers and notifying them to workers.

[1238] Program processing flow

[1239] User terminal

[1240] A user terminal is a device that allows a user to input schedules and work details by voice or text. When a user inputs by voice, the user terminal uses a speech recognition library (e.g., Google Cloud Speech-to-Text API) to convert the voice into text and sends this text data to the server. For example, if a user inputs "Maintenance inspection of Line 1 at 10:00 AM" by voice, the terminal converts this into text data "Maintenance inspection of Line 1 at 10:00 AM" and sends it to the server.

[1241] server

[1242] The server is responsible for receiving and analyzing the text data sent from the terminal. The server uses Natural Language Processing (NLP) technology (e.g., NLTK, Spacy) to analyze the received text data and saves the schedule information in a database (e.g., PostgreSQL). It also sets reminders and sends reminder notifications to the user terminal at appropriate times. For example, it sends a reminder 30 minutes before the maintenance inspection saying, "Maintenance inspection of Line 1 is coming up soon."

[1243] Furthermore, the server analyzes work records, generates work-related advice (e.g., "Please wear protective equipment before maintenance inspections"), and sends this to the user's terminal. The server also obtains information related to seasonal changes and specific tasks and provides it to the user. For example, in the summer, the server notifies users with information such as "Be careful of heatstroke."

[1244] Administrator comment notification

[1245] Managers input comments by voice or text from their own devices, and the data is sent to the server. The server saves the received comments and checks the worker's schedule. For example, if a manager sends a comment such as "Please wear protective equipment before maintenance inspection," the server will send a notification to the worker's device at the appropriate time based on the worker's schedule, stating "Please wear protective equipment before maintenance inspection."

[1246] Specific examples

[1247] When a worker voice-inputs "Maintenance inspection of Line 1 at 10:00 AM," the following process takes place:

[1248] 1. The user device converts the speech into text and sends it to the server.

[1249] 2. The server receives the text data and uses NLP technology to analyze the schedule information for the "10:00 AM maintenance inspection" and save it in the database.

[1250] 3. The server sets a reminder and sends a reminder notification to the user terminal 30 minutes before the maintenance inspection.

[1251] 4. When the administrator terminal inputs "Please wear protective equipment before maintenance inspection" by voice, the instruction is notified to the user terminal at the appropriate time.

[1252] Prompt Sentence Examples

[1253] "We have a maintenance inspection scheduled for Line 1 at 10:00 AM. Please send me a reminder 30 minutes before. I would also like a reminder to put on protective gear before the inspection."

[1254] As described above, by using this system, factory workers can efficiently manage their schedules and receive necessary instructions and advice in a timely manner.

[1255] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1256] Program processing flow

[1257] Step 1:

[1258] The user inputs schedules and work details using voice or text. The user's device receives the input voice data and converts it into text data using a speech recognition library (e.g., Google Cloud Speech-to-Text API). The text data obtained through this conversion becomes the input for the next step.

[1259] Step 2:

[1260] The user terminal sends the converted text data to the server. The server receives the text data from the user terminal and prepares to analyze it. The input here is text data.

[1261] Step 3:

[1262] The server analyzes the received text data using Natural Language Processing (NLP) techniques (e.g., NLTK, Spacy). From this analysis, schedule information and task details are extracted. The results of this analysis are stored in a database for further processing. The output of the analysis is database entries for schedule information and task details.

[1263] Step 4:

[1264] The server sets a schedule-based reminder based on the analysis results. A specific time, such as 30 minutes before a maintenance inspection, is specified for the reminder setting. This reminder setting information is used to send reminder notifications. The reminder setting information is output.

[1265] Step 5:

[1266] The set reminder is sent to the user's device. The contents of the reminder are personalized for each user and notified at the appropriate time. The input here is the set reminder information, and the output is the reminder notification displayed on the user's device.

[1267] Step 6:

[1268] The server analyzes the work records and generates work-related advice, such as specific advice like "wear protective equipment before maintenance inspection." This advice information becomes the input for the next processing step.

[1269] Step 7:

[1270] The generated advice information is sent to the user terminal and notified to the user at the appropriate time. This allows the worker to receive the necessary advice. The input here is the generated advice information, and the output is the advice notification sent to the user terminal.

[1271] Step 8:

[1272] The administrator inputs a comment by voice or text from their own terminal. The administrator terminal sends this comment data to the server. The sent comment data becomes the input for the next step.

[1273] Step 9:

[1274] The server saves comments received from the administrator and checks the worker's schedule. The comments are notified to the worker's terminal at the appropriate time. This notification allows the worker to receive important instructions and comments. The input here is the comment data from the administrator, and the output is a notification to the worker's terminal.

[1275] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1276] The present invention is a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them, and by combining an emotion engine, it is also able to respond to the emotions of the user. Specific embodiments are described below.

[1277] System Overview

[1278] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user. It also uses an emotion engine to recognize the user's emotions and responds based on that information. It also supports communication between parents and children by receiving comments from parents and notifying the child.

[1279] Program processing flow

[1280] User terminal

[1281] The user terminal is a device that allows users to input schedules and meal details by voice or text. When a user inputs by voice, the user terminal uses a voice recognition library to convert the voice into text and sends this text data to the server. It also has an emotion engine that recognizes the user's emotions from the input voice and text data. For example, if a user inputs "Meeting tomorrow at 10 AM," the terminal converts this into text data "Meeting tomorrow at 10 AM" and sends it to the server, while also analyzing the user's emotions (e.g., stress, anxiety).

[1282] server

[1283] The server is responsible for receiving and analyzing the text data and emotion data sent from the device. For example, the server uses Natural Language Processing (NLP) technology to analyze the received text data and store the schedule information in a database. Furthermore, the server adjusts the content of reminders and advice based on the results of the emotion engine. For example, if the user's emotion is recognized as "stress," the server adjusts the wording of the reminder to be softer.

[1284] For example, if a user inputs "Today's breakfast is bread and bananas," the server analyzes the meal contents and generates advice on nutritional balance. For example, it generates advice such as "You should add a few more vegetables," and sends this advice to the user's terminal.

[1285] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[1286] Parent comment notification

[1287] When a parent enters a comment via voice or text from their device, the data is sent to the server. The server saves the received comment and checks the child's schedule. For example, if a parent sends a comment saying "Please do your homework properly," the server will send a notification saying "Please do your homework properly" to the child's device at an appropriate time based on the child's schedule. Here too, the content of the notification is adjusted taking into account the child's emotional information (e.g., if they are feeling depressed).

[1288] Specific examples

[1289] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[1290] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects "stress" from the user's voice.

[1291] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[1292] 3. The server receives the text data and analyzes it using NLP technology. It extracts the date and time information for "tomorrow at 10:00 AM" and adjusts the reminder based on the emotional data "stress."

[1293] 4. The server saves the schedule information in a database and sets a reminder. For example, it sets a reminder for "tomorrow at 9:30 AM" and adjusts the message content to reduce "stress."

[1294] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[1295] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[1296] 1. The parent device converts the voice to text and analyzes the emotion using an emotion engine. For example, it detects the emotion of "encouragement."

[1297] 2. The device sends the converted text data and emotion data to the server.

[1298] 3. The server receives and stores the comment data. Based on the emotional data "encouragement," it sets up notifications to be sent at the optimal time for the child's situation.

[1299] 4. At the set time, the server sends a message to the child's device saying, "Please do your homework properly." For example, at 4:00 PM, the server sends a message saying, "Please do your homework properly. Good luck!"

[1300] The specific system configuration and programs required to implement the present invention are those that properly perform the above-mentioned processes, thereby improving the efficiency of users' lifestyle management and supporting communication between parents and children in a way that takes emotion into consideration.

[1301] The processing flow will be explained below.

[1302] Specific processing flow of the program

[1303] Schedule management and emotion recognition processing

[1304] Step 1:

[1305] The user inputs the schedule details by voice. For example, the user may say, "Meeting tomorrow at 10:00 AM."

[1306] Step 2:

[1307] The device uses a speech recognition library to convert the voice data into text data. Specifically, the voice "Meeting tomorrow at 10 AM" is converted into text "Meeting tomorrow at 10 AM."

[1308] Step 3:

[1309] The device uses an emotion engine to analyze the user's emotion from the input voice data. For example, the emotion "stress" is analyzed.

[1310] Step 4:

[1311] The device sends the converted text data and analyzed emotion data to the server via an HTTP request.

[1312] Step 5:

[1313] The server receives the text data and emotion data and analyzes the text data using Natural Language Processing (NLP) technology, extracting the date and time information for "tomorrow at 10:00 AM."

[1314] Step 6:

[1315] The server saves the extracted schedule information in a database. For example, it saves the meeting information for "December 20, 2023 at 10:00 AM."

[1316] Step 7:

[1317] The server adjusts and sets the content of the reminder based on the emotion data. For example, if the user's emotion is recognized as "stressed," the server adjusts the reminder to a more soothing message. For example, it sets the reminder to "Tomorrow at 9:30 AM" and adjusts the notification content to "You have a meeting at 10 AM. Relax and enjoy."

[1318] Step 8:

[1319] When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[1320] Food record and advice processing

[1321] Step 1:

[1322] The user inputs the details of their meal by voice, for example, saying, "Today's breakfast is bread and bananas."

[1323] Step 2:

[1324] The device converts the voice data into text data. The voice "Today's breakfast is bread and bananas" is converted into text "Today's breakfast is bread and bananas."

[1325] Step 3:

[1326] The terminal transmits the converted text data to the server.

[1327] Step 4:

[1328] The server receives the text data and uses the Nutrient Analysis API to analyze the dietary content, evaluating the carbohydrate and vitamin content as analysis results.

[1329] Step 5:

[1330] The server generates nutritional advice based on the analysis results, such as "You might want to add a few more vegetables."

[1331] Step 6:

[1332] The server transmits the generated advice to the user terminal and notifies it.

[1333] Parent comment notification processing

[1334] Step 1:

[1335] Parents can comment by voice or text, for example, saying, "Please do your homework."

[1336] Step 2:

[1337] The device converts the voice to text. The voice "Do your homework properly" is converted to text "Do your homework properly."

[1338] Step 3:

[1339] The terminal transmits the converted text data and the emotion data generated by the emotion engine to the server.

[1340] Step 4:

[1341] The server receives the comment data and the emotion data and stores them. The emotion data is analyzed as, for example, "encouragement."

[1342] Step 5:

[1343] The server checks the child's schedule and calculates the appropriate time to notify them of the comment. For example, if the child has free time at 4 p.m., it will set the notification to be sent at that time.

[1344] Step 6:

[1345] The server sends the message "Do your homework properly" to the child's device at the set time. For example, at 4:00 PM, it sends the message "Do your homework properly. Good luck!"

[1346] Providing information on seasonal events and foods

[1347] Step 1:

[1348] The server checks the current date and retrieves information about seasonal events and foods. For example, if it is October, it retrieves information about "moon viewing."

[1349] Step 2:

[1350] Based on the information acquired by the server, a message to be provided to the user is generated. For example, a message such as "Today is the season for viewing the moon. Let's enjoy some dumplings" is generated.

[1351] Step 3:

[1352] The server sends the generated message to the user terminal for notification.

[1353] This is the specific program flow of the AI ​​Mom System. This system will improve the efficiency of users' lifestyle management and support communication between parents and children in a way that takes emotions into consideration.

[1354] Example 2

[1355] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1356] Conventional schedule management systems and lifestyle support systems simply process information mechanically without considering the user's feelings, which means they are ineffective at reducing the user's psychological burden. Specifically, it is difficult to provide appropriate reminders and support to users who are stressed or anxious. Furthermore, they lack functionality to support communication between parents and children, making it difficult to maintain a smooth parent-child relationship.

[1357] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1358] In this invention, the server includes means for analyzing text data and emotion data sent from the user terminal and saving schedule information in a database, means for setting reminders based on the schedule and adjusting them based on the emotion data, and means for analyzing the meal record, generating advice on nutritional balance, and adjusting them based on the emotion data. This makes it possible to provide reminders and support that take the user's emotions into consideration. Furthermore, by providing a function for receiving comments from parents and notifying them at an appropriate time while taking into account the child's emotion data, it is possible to facilitate communication between parents and support the maintenance of a smooth parent-child relationship.

[1359] A "user terminal" is a device that allows a user to input schedules and meal details by voice or text.

[1360] A "voice recognition library" is a software component for converting voice data into text data.

[1361] "Emotion data" is information about a user's emotional state that has been analyzed using emotion recognition technology.

[1362] "Server" means a central device that receives, analyzes, and processes data sent from user terminals.

[1363] "Text data" is data in text format that has been converted by voice recognition.

[1364] "Natural Language Processing technology" is a technology for understanding and generating natural language.

[1365] A "database" is a system for organizing and storing information.

[1366] "Reminder" is a function for notifying the user of a specific date, time or event.

[1367] "Diet record" is data about dietary details entered by the user.

[1368] "Nutritional balance" refers to the proper distribution of nutrients in food.

[1369] "Event information" is information about specific activities or events related to seasons, dates, and times.

[1370] "Parent comment" is a message entered by a parent to their child.

[1371] This invention is a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them, and by combining it with an emotion engine, it can also respond to the user's emotions. This system is configured as follows.

[1372] System Overview

[1373] A user terminal is a device that allows a user to input schedules and meal details by voice or text. Examples of such devices include smartphones and tablet devices. When a user inputs voice information, the user terminal uses a speech recognition library (e.g., Google Cloud Speech-to-Text API) to convert the speech into text and sends this text data to a server. In addition, an emotion recognition API (e.g., Microsoft Azure Cognitive Services) is used to recognize the user's emotions from the input voice or text data.

[1374] The server is responsible for receiving and analyzing the text data and emotion data sent from the user terminal. The server analyzes the received text data using Natural Language Processing technology (e.g., Google Cloud Natural Language API) and stores the schedule information in a database (e.g., MySQL).

[1375] Furthermore, the content of reminders and advice is adjusted based on the results of the emotion engine. For example, if the user's emotion is recognized as "stress," the wording of the reminder is softened or adjusted accordingly. Also, if the user inputs "Today's breakfast is bread and banana," the server analyzes the meal contents and generates advice on nutritional balance. For example, advice such as "You should add a few more vegetables" is generated and sent to the user's device.

[1376] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[1377] The system also receives comments from parents and checks their children's schedules. For example, if a parent sends a comment saying, "Please do your homework properly," the server will send a notification saying, "Please do your homework properly" to the child's device at an appropriate time based on the child's schedule. At this time, the content of the notification will be adjusted taking into account the child's emotional information (e.g., if they are feeling depressed).

[1378] Specific operation example

[1379] Schedule management example

[1380] If a user speaks "Meeting tomorrow at 10 AM," the following occurs:

[1381] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects "stress" from the user's voice.

[1382] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[1383] 3. The server receives the text data and analyzes it using NLP technology. It extracts the date and time information for "tomorrow at 10:00 AM" and adjusts the reminder based on the emotional data "stress."

[1384] 4. The server saves the schedule information in a database and sets a reminder. For example, it sets a reminder for "tomorrow at 9:30 AM" and adjusts the message content to reduce "stress."

[1385] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[1386] Example of parent comment notification

[1387] When a parent sends a comment saying "Please do your homework properly," the following occurs.

[1388] 1. The parent device converts the voice to text and analyzes the emotion using an emotion engine. For example, it detects the emotion of "encouragement."

[1389] 2. The device sends the converted text data and emotion data to the server.

[1390] 3. The server receives and stores the comment data. Based on the emotional data "encouragement," it sets up notifications to be sent at the optimal time for the child's situation.

[1391] 4. At the set time, the server sends a message to the child's device saying, "Please do your homework properly." For example, at 4:00 PM, the server sends a message saying, "Please do your homework properly. Good luck!"

[1392] Prompt Sentence Examples

[1393] Here are some examples of prompts to input to a generative AI model:

[1394] If a user says "Meet tomorrow at 10 AM," how does the system handle this?

[1395]

[1396] If a parent sends a comment like "Do your homework properly," how does the system handle that?

[1397] conclusion

[1398] This system will help users manage their daily lives more efficiently and provide emotionally sensitive reminders and advice, reducing the user's psychological burden. It will also facilitate communication between parents and children, helping to maintain parent-child relationships.

[1399] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1400] Specific processing steps for schedule management

[1401] Step 1: User enters schedule by voice

[1402] The user inputs by voice, "Meeting tomorrow at 10:00 AM." This input data is sent to the user's terminal as voice data.

[1403] Step 2: Speech to text

[1404] The device uses the Google Cloud Speech-to-Text API to convert the voice data entered by the user into text data.

[1405] Input: Voice data "Meeting tomorrow at 10 AM"

[1406] Data processing: Convert speech to text using a speech recognition library

[1407] Output: Text data "Meeting tomorrow at 10 AM"

[1408] Step 3: Sentiment analysis

[1409] The device uses the emotion recognition API of Microsoft Azure Cognitive Services to analyze the user's emotions from the converted text data.

[1410] Input: Text data "Meeting tomorrow at 10 AM"

[1411] Data processing: Emotion analysis using emotion recognition API

[1412] Output: Emotion data "Stress"

[1413] Step 4: Sending data

[1414] The device encodes the converted text data and emotion data into JSON format and sends it to the server via HTTPS.

[1415] Input: Text data "Meeting tomorrow at 10 AM", emotion data "Stress"

[1416] Data processing: Encode text data and emotion data into JSON format

[1417] Output: JSON data

[1418] Step 5: Data reception and analysis

[1419] The server receives the JSON data and uses the Google Cloud Natural Language API to parse the text data and extract the date and time information for "tomorrow at 10:00 AM."

[1420] Input: JSON data

[1421] Data processing: Analyze text data using natural language processing technology and extract date and time information

[1422] Output: Date and time information "Tomorrow 10:00 AM"

[1423] Step 6: Save the schedule

[1424] The server stores the extracted date and time information and text data in a MySQL database.

[1425] Input: Date and time information "Tomorrow 10:00 AM", text data "Meeting tomorrow at 10:00 AM"

[1426] Data processing: Execute SQL queries and save to database

[1427] Output: Schedule information stored in the database

[1428] Step 7: Set reminders

[1429] The server adjusts the content of the reminder based on the emotional data "stress" and sets reminder information based on the schedule, for example, setting a reminder 30 minutes before a meeting.

[1430] Input: Emotion data "Stress", Date and time information "Tomorrow 10:00 AM"

[1431] Data processing: Generate emotion-sensitive reminder messages and set reminder information

[1432] Output: Reminder information (date, time and message)

[1433] Step 8: Send reminders

[1434] When the server reaches the set time (e.g. 9:30 AM), it sends a reminder notification to the user's device.

[1435] Input: Reminder information

[1436] Data processing: Send reminders via HTTP POST request

[1437] Output: Reminder message "There's a meeting at 10 o'clock. Relax and enjoy."

[1438] Specific steps for parent comment notification

[1439] Step 1: Parents enter comments by voice

[1440] The parent can input "Please do your homework properly" by voice. This input data is sent to the parent device as voice data.

[1441] Step 2: Speech to text

[1442] The parent device uses the Google Cloud Speech-to-Text API to convert the voice data input by the parent into text data.

[1443] Input: Voice data "Do your homework properly"

[1444] Data processing: Convert speech to text using a speech recognition library

[1445] Output: Text data "Please do your homework properly."

[1446] Step 3: Sentiment analysis

[1447] The parent device uses the emotion recognition API of Microsoft Azure Cognitive Services to analyze emotions from the converted text data.

[1448] Input: Text data "Please do your homework properly"

[1449] Data processing: Emotion analysis using emotion recognition API

[1450] Output: Emotion data "Encouragement"

[1451] Step 4: Sending data

[1452] The parent device encodes the converted text data and emotion data into JSON format and sends it to the server via HTTPS.

[1453] Input: Text data "Do your homework properly", emotion data "Encouragement"

[1454] Data processing: Encode text data and emotion data into JSON format

[1455] Output: JSON data

[1456] Step 5: Data reception and analysis

[1457] The server interprets the received JSON data and extracts text and emotion data.

[1458] Input: JSON data

[1459] Data processing: Analyze JSON data and extract text and sentiment data

[1460] Output: Text data, emotion data

[1461] Step 6: Save comments

[1462] The server stores the extracted text data and emotion data in a database.

[1463] Input: Text data, emotion data

[1464] Data manipulation: Save to database using SQL queries

[1465] Output: Comment information stored in the database

[1466] Step 7: Notification Settings

[1467] The server checks the child's schedule and configures it to send notifications at appropriate times based on emotional data.

[1468] Input: Child's schedule information, emotional data "encouragement"

[1469] Data processing: Set notification timing and content

[1470] Output: Notification settings information

[1471] Step 8: Send notification

[1472] When the server reaches the set time (e.g. 3:50 pm), it will notify the child's device of the parent's comment, "Make sure you do your homework properly."

[1473] Input: Notification settings information

[1474] Data processing: Send parent comment via HTTP POST request

[1475] Output: Notification message "Do your homework properly. Good luck!"

[1476] conclusion

[1477] In this way, the user's lifestyle management becomes more efficient, and it becomes possible to provide emotionally sensitive reminders and advice. It also facilitates communication between parents and children, helping to maintain parent-child relationships.

[1478] (Application example 2)

[1479] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1480] Currently, there is a demand for improved efficiency in factory operations and smoother communication with workers. In particular, it is important to be able to quickly and accurately interpret voice instructions from workers and respond accordingly. There is also a need to understand workers' emotions and provide appropriate advice and reminders to reduce their stress. However, current systems are unable to meet these requirements.

[1481] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting voice or text input of schedules and meal details from a user terminal; means for converting the voice into text; means for transmitting the text data to the server; means for analyzing the text data in the server and storing the schedule information in a database; means for setting reminders based on the schedule; means for analyzing meal records and generating advice on nutritional balance; means for acquiring information on seasonal events and in-season foods; means for receiving comments from parents and notifying them at appropriate times; means for accepting voice instructions from workers in a factory environment, converting them into text in real time, and analyzing schedule and emotion data; and means for adjusting reminders and advice to workers based on the emotion data and notifying them at appropriate times. This enables improved work efficiency in the factory and smoother communication with workers.

[1482] A "user terminal" is a device that accepts input of schedules and meal details by voice or text.

[1483] "Means for converting speech to text" refers to technology for converting speech input by a user into text data.

[1484] The "means for transmitting text data to a server" is a function for transmitting the converted text data to a server via a network.

[1485] A "server" is a computer system for analyzing and storing data sent from a user terminal.

[1486] The "means for saving schedule information in a database" is a function for storing analyzed schedule information in a database.

[1487] The "means for setting a reminder" is a function for notifying the user based on a set schedule.

[1488] "Means for analyzing dietary records" refers to technology for analyzing dietary details entered by a user.

[1489] The "means for generating advice regarding nutritional balance" is a function for providing advice regarding nutrition based on the analyzed dietary record.

[1490] The "means for acquiring seasonal event information and seasonal food information" is a function for collecting event information and seasonal food information corresponding to each season and providing it to the user.

[1491] The "means for receiving comments from parents and notifying them at an appropriate time" is a function for receiving comments entered by parents and notifying them in accordance with the child's schedule.

[1492] "Factory environment" refers to the location where manufacturing and product assembly operations occur.

[1493] The "means for receiving voice instructions from workers" is a function that enables workers working in the factory to input voice instructions.

[1494] "Means for converting to text in real time" refers to technology for converting voice instructions into text data in real time.

[1495] "Emotion data" is data that indicates the emotional state of a worker analyzed from his / her voice.

[1496] The "means for adjusting reminders and advice to workers based on emotional data" is a function for providing appropriate reminders and advice to workers based on the analyzed emotional data.

[1497] This invention is a system for improving work efficiency in factories and facilitating communication with workers. The system includes a user terminal, a server, and a communication means for exchanging data between them. By combining it with an emotion engine, the system can also respond to the emotions of workers.

[1498] System Overview

[1499] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user. In particular, it uses an emotion engine to recognize the worker's emotions and responds based on that information. It also has a function to receive comments from parents and notify the child.

[1500] User terminal

[1501] The user terminal is a device that allows workers to input instructions and schedules by voice or text. When using voice input, the user terminal uses a voice recognition library (e.g., the SpeechRecognition library) to convert the voice into text and sends this text data to the server. It also has an emotion engine (e.g., an emotion recognition model using Transformers) that recognizes the worker's emotions from the input voice and text data. For example, if a worker inputs "Maintenance work will be performed at 3:00 PM today," the terminal converts this into text data that reads "Maintenance work will be performed at 3:00 PM today" and sends it to the server. At the same time, the terminal also analyzes the worker's emotions (e.g., sense of relief).

[1502] server

[1503] The server is responsible for receiving and analyzing the text and emotion data sent from the device. Specifically, it uses Natural Language Processing (NLP) technology to analyze the received text data and store the schedule information in a database. It also adjusts the content of reminders and advice based on the results of the emotion engine. For example, if the worker's emotion is recognized as "relief," the wording of the reminder will be adjusted accordingly. The server then sends the processed notification to the user's device at a specific time.

[1504] Specific examples

[1505] For example, if a worker says, "Maintenance work will be performed at 3:00 p.m. today. Please don't forget," the following process will occur:

[1506] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects a "sense of security."

[1507] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[1508] 3. The server receives the text data and analyzes it using NLP technology. It extracts the time information "3 PM" and adjusts the reminder based on the emotional data "sense of relief."

[1509] 4. The server saves the schedule information to a database and sets a reminder, for example, for 2:45 PM, with a tailored message tailored to the user's emotional state.

[1510] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 2:45 PM, it sends a notification saying, "Maintenance work will be performed at 3:00. Please proceed without worry."

[1511] Prompt Sentence Examples

[1512] An example of a voice instruction that a worker inputs into a terminal is a prompt such as, "Maintenance work will be performed at 3:00 PM today. Please don't forget." An example of a notification is a statement such as, "Maintenance work will be performed at 3:00 PM. Please proceed without worry."

[1513] In this way, work efficiency within the factory is improved and communication with workers is facilitated.

[1514] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1515] Step 1:

[1516] The user inputs by voice, "Maintenance work will be performed at 3:00 PM today. Please don't forget." This input is received by the terminal as voice data.

[1517] Step 2:

[1518] The device uses a speech recognition library (for example, the SpeechRecognition library) to convert the received voice data into text data. The input is voice data, and the output is text data: "Maintenance work will be carried out at 3:00 p.m. today. Please remember."

[1519] Step 3:

[1520] The device analyzes the converted text data using an emotion engine (e.g., an emotion recognition model using Transformers). The input is text data, and the device outputs emotion data (e.g., a sense of relief) as the result of emotion analysis.

[1521] Step 4:

[1522] The device sends the converted text data and emotion data to the server via an HTTP request. The input is text data and emotion data, and the device sends them to the server as output.

[1523] Step 5:

[1524] The text data received by the server is analyzed using Natural Language Processing (NLP) technology. The input is text data, and the analysis results extract date and time information (e.g., "3:00 PM") and process it into schedule information. The output is schedule information.

[1525] Step 6:

[1526] The server saves the schedule information in a database. The input is the extracted schedule information, and the output is the information saved in the database.

[1527] Step 7:

[1528] The server adjusts the reminder and advice content based on the emotion data and schedule information. The input is the emotion data and schedule information, and the output is the adjusted reminder content.

[1529] Step 8:

[1530] When the server reaches the reminder setting time (for example, 2:45 PM), it sends a reminder notification to the user terminal. The input is the adjusted reminder content, and the output is the notification sent to the user terminal.

[1531] Step 9:

[1532] The user device receives the notification and notifies the user by voice or text. The input is a reminder notification, and the output is a notification to the user saying, "There will be maintenance work at 3:00 PM. Please proceed with confidence."

[1533] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1534] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1535] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1536] [Fourth embodiment]

[1537] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1538] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1539] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1540] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1541] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1542] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1543] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1544] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1545] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1546] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1547] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1548] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1549] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1550] The present invention relates to a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[1551] System Overview

[1552] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user.It also supports communication between parents and children by receiving comments from parents and notifying their children.

[1553] Program processing flow

[1554] User terminal

[1555] A user terminal is a device that allows a user to input schedules and meal details by voice or text. When a user inputs by voice, the user terminal uses a voice recognition library to convert the voice into text and sends this text data to the server. For example, if a user inputs "Meeting tomorrow at 10 AM" by voice, the terminal converts this into text data "Meeting tomorrow at 10 AM" and sends it to the server.

[1556] server

[1557] The server is responsible for receiving and analyzing text data sent from the device. For example, the server uses Natural Language Processing (NLP) technology to analyze the received text data and save the schedule information in a database. It also sets reminders and sends reminder notifications to the user's device at appropriate times. If a user inputs "Today's breakfast is bread and banana," the server analyzes the meal contents and generates advice about nutritional balance. For example, it generates advice such as "You should add a few more vegetables" and sends this to the user's device.

[1558] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[1559] Parent comment notification

[1560] When a parent enters a comment by voice or text from their device, the data is sent to the server. The server saves the received comment and checks the child's schedule. For example, if a parent sends a comment such as "Please do your homework properly," the server will send a notification to the child's device saying "Please do your homework properly" at an appropriate time based on the child's schedule. An appropriate time could be, for example, a time when the child has free time or before bedtime.

[1561] Specific examples

[1562] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[1563] 1. The user device converts the speech into text and sends it to the server.

[1564] 2. The server receives the text data and uses NLP technology to analyze the schedule information for "tomorrow at 10:00 AM" and saves it in a database.

[1565] 3. The server sets a reminder and sends a reminder notification to the user terminal, for example, "tomorrow at 9:30 AM."

[1566] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[1567] 1. The parent device converts the voice into text and sends it to the server.

[1568] 2. The server receives the comment and checks the child's schedule.

[1569] 3. At the appropriate time, the server notifies the child's device, "Please do your homework properly."

[1570] The specific system configuration and programs required to implement the present invention are those that properly perform the above-mentioned processes, thereby supporting efficient communication between the user and their family and lifestyle management.

[1571] The processing flow will be explained below.

[1572] Specific processing flow of the program

[1573] Schedule management process

[1574] Step 1:

[1575] The user inputs the schedule details by voice. For example, the user may say, "Meeting tomorrow at 10:00 AM."

[1576] Step 2:

[1577] The device uses a speech recognition library to convert the voice data into text data. Specifically, the voice "Meeting tomorrow at 10 AM" is converted into text "Meeting tomorrow at 10 AM."

[1578] Step 3:

[1579] The terminal sends the converted text data to the server via an HTTP request.

[1580] Step 4:

[1581] The server receives the text data and analyzes it using Natural Language Processing (NLP) technology, extracting the date and time information "tomorrow at 10 AM" from the text.

[1582] Step 5:

[1583] The server saves the extracted schedule information in a database. For example, it saves the meeting information for "December 20, 2023 at 10:00 AM."

[1584] Step 6:

[1585] The server sets a reminder based on the saved schedule, for example, "December 20, 2023 at 9:30 AM."

[1586] Step 7:

[1587] When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There will be a meeting at 10:00."

[1588] Food record and advice processing

[1589] Step 1:

[1590] The user inputs the details of their meal by voice, for example, saying, "Today's breakfast is bread and bananas."

[1591] Step 2:

[1592] The device converts the voice data into text data. The voice "Today's breakfast is bread and bananas" is converted into text "Today's breakfast is bread and bananas."

[1593] Step 3:

[1594] The terminal transmits the converted text data to the server.

[1595] Step 4:

[1596] The server receives the text data and uses the Nutrient Analysis API to analyze the dietary content, evaluating the carbohydrate and vitamin content as analysis results.

[1597] Step 5:

[1598] The server generates nutritional advice based on the analysis results, such as "You might want to add a few more vegetables."

[1599] Step 6:

[1600] The server transmits the generated advice to the user terminal and notifies it.

[1601] Parent comment notification processing

[1602] Step 1:

[1603] Parents can comment by voice or text, for example, saying, "Please do your homework."

[1604] Step 2:

[1605] The device converts the voice to text. The voice "Do your homework properly" is converted to text "Do your homework properly."

[1606] Step 3:

[1607] The terminal transmits the converted text data to the server.

[1608] Step 4:

[1609] The server receives and stores the comment data.

[1610] Step 5:

[1611] The server checks the child's schedule and determines the appropriate notification time. For example, if a child has free time at 4 p.m., it sets the notification to be sent at "4 p.m."

[1612] Step 6:

[1613] At the set time, the server sends a message to the child's device saying, "Please do your homework properly."

[1614] Providing information on seasonal events and foods

[1615] Step 1:

[1616] The server checks the current date and retrieves information about seasonal events and foods. For example, if it is October, it retrieves information about "moon viewing."

[1617] Step 2:

[1618] Based on the information acquired by the server, a message to be provided to the user is generated. For example, a message such as "Today is the season for viewing the moon. Let's enjoy some dumplings" is generated.

[1619] Step 3:

[1620] The server sends the generated message to the user terminal for notification.

[1621] This is the specific program flow of the AI ​​Mom System. This system will improve the efficiency of users' lifestyle management and support communication between parents and children.

[1622] Example 1

[1623] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1624] Currently, there are many schedule management systems and lifestyle support systems, but they are difficult to flexibly respond to the diverse needs of users. Furthermore, there are a lack of means for smooth communication between parents and children. Furthermore, there are few systems that provide integrated functions such as nutritional balance advice and reminders. For these reasons, there is a demand for a system that can comprehensively support users' daily lives.

[1625] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1626] In this invention, the server includes means for accepting input of schedules and meal details by voice or text from a user terminal, means for converting the voice into text, means for transmitting the text data to the server, means for analyzing the text data in the server and storing the schedule information in a database, means for setting reminders based on the schedule, means for analyzing the meal records and generating advice on nutritional balance, means for acquiring information on seasonal events and in-season foods, means for receiving comments from parents and notifying them at appropriate times, means for saving the comments sent by the parents and notifying them at appropriate times based on the comments, means for the user to request generation of reminders or advice by voice or text, and means for analyzing the request using natural language processing technology and for the user to generate reminders or advice. This provides comprehensive support for the user's daily life and facilitates communication between parents and children.

[1627] 1. "User terminal" refers to a device that allows a user to input schedules and meal details by voice or text.

[1628] 2. "Means for converting speech to text" refers to a function that converts a user's speech into text data using a speech recognition library or the like.

[1629] 3. "Means for transmitting text data to a server" means a function that uses a communication protocol to transmit text data to a server via a network.

[1630] 4. "Server" means a computing device that analyzes received data, stores necessary information in a database, and generates and sends reminders and advice to the user's device.

[1631] 5. "Means for analyzing text data" means a function that uses Natural Language Processing technology to analyze input text data and extract necessary information.

[1632] 6. "Means for saving to a database" refers to the function of saving the analyzed text data to persistent storage.

[1633] 7. "Means for setting reminders" refers to a function for setting notification timing based on a schedule.

[1634] 8. "Means for generating advice on nutritional balance" refers to a function that analyzes the input dietary information and provides appropriate nutritional advice based on the results.

[1635] 9. "Means for obtaining seasonal event information and in-season food information" is a function for obtaining information on related events and foods based on the current date.

[1636] 10. "Means for receiving comments from parents and notifying them at an appropriate time" refers to a function that receives voice or text comments from parents and notifies them at an appropriate time based on the child's schedule.

[1637] 11. "Natural Language Processing Technology" is a general term for natural language processing technology used to analyze text data.

[1638] 12. "Means for analyzing requests and generating reminders and advice for users" refers to a function that analyzes requests from users and generates reminders and advice based on the results.

[1639] MODE FOR CARRYING OUT THE INVENTION

[1640] The present invention relates to a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[1641] User device functions

[1642] A user terminal is a device that allows users to input schedules and meal details by voice or text. Specifically, this applies to smartphones and tablets. The terminal has the following main functions that operate based on user input:

[1643] Speech Recognition Library: The Google Cloud Speech-to-Text API is used to convert the user's speech into text. This converted text data is sent to the server described below.

[1644] Communication means: After speech recognition, the system has a communication protocol for transmitting the converted text data to the server. For example, the system transmits the data using an HTTP POST request.

[1645] Server Features

[1646] The server receives and analyzes text data sent from the user's device, processes and stores the necessary information, and also generates reminders and advice and notifies the user's device.

[1647] Data reception: Use a web framework such as Flask or Django to receive HTTP POST requests from the user's device.

[1648] Text analysis: Using Natural Language Processing (NLP) techniques, such as spaCy, the received text data is analyzed to extract date and time information and event information.

[1649] Database: Analyzed schedule information is saved in a database such as MySQL or MongoDB. Saved data includes date and time, events, user IDs, etc.

[1650] Set reminders: Use a cron job to send reminder notifications at a set time, for example, "Tomorrow at 9:30 AM."

[1651] Advice generation: Analyzes the dietary information entered by the user and generates appropriate nutritional advice based on the results. For example, if the user enters "Today's breakfast is bread and banana," the system will provide advice such as "You should add a few more vegetables."

[1652] Obtaining seasonal event information: Obtaining seasonal event and seasonal food information based on the current date and notifying the user.

[1653] Parent comment notifications: Receive and save comments sent by parents and notify them at appropriate times based on the child's schedule. For example, if a parent comments "Please do your homework properly," a notification "Please do your homework properly" will be sent to the child during their free time.

[1654] Specific examples

[1655] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[1656] The user's device converts the speech into text and sends it to the server using the Google Cloud Speech-to-Text API.

[1657] The server receives the text data and uses NLP techniques to parse the schedule information for "tomorrow at 10 AM" and uses spaCy to store it in the database.

[1658] The server sets a reminder and uses a cron job to send a reminder notification to the user's device at "tomorrow at 9:30 AM."

[1659] The server generates dietary advice and sends it to the user, such as "You should add a few more vegetables."

[1660] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[1661] The parent device converts the speech into text and sends it to the server, again using the Google Cloud Speech-to-Text API.

[1662] The server receives the comments and uses spaCy to check the child's schedule.

[1663] The server sends a notification to the child's device at an appropriate time, such as during the child's free time, saying, "Please do your homework properly."

[1664] Prompt Sentence Examples

[1665] Here are some examples of prompts that users might enter using a generative AI model:

[1666] "Set a meeting reminder for tomorrow at 10 AM"

[1667] "Today's breakfast is bread and bananas. I'd like some advice on nutritional balance."

[1668] "Remind my child to do their homework properly"

[1669] In this way, the system of the present invention can comprehensively support the daily lives of users and their families, facilitate smooth communication between parents and children, and enable efficient schedule management.

[1670] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1671] Program processing flow

[1672] Step 1:

[1673] The user enters data by voice or text.

[1674] Specific actions: A user uses a device such as a smartphone or tablet to say, "Meeting tomorrow at 10:00 AM," or type in text such as, "Today's breakfast is bread and bananas."

[1675] Input: Audio or text data

[1676] Output: Audio or text data on the user's device

[1677] Step 2:

[1678] The user terminal converts the speech into text.

[1679] Specific operation: The user device uses the Google Cloud Speech-to-Text API to convert the voice data into text data, and the converted text is displayed on the device screen.

[1680] Input: Audio data

[1681] Output: Text data (e.g. "Meeting tomorrow at 10 AM")

[1682] Step 3:

[1683] The user terminal transmits text data to the server.

[1684] Specific operation: The device sends the converted text data to the server using an HTTP POST request. The user's device screen displays the status of the transmission.

[1685] Input: Text data (e.g. "Meeting tomorrow at 10 AM")

[1686] Output: Text data sent to the server

[1687] Step 4:

[1688] The server receives the text data.

[1689] Specific operation: The server uses Flask or Django to receive HTTP POST requests from user terminals and retrieve text data. The received data is recorded in the server log.

[1690] Input: HTTP POST request

[1691] Output: Received text data

[1692] Step 5:

[1693] The server parses the text data.

[1694] Specific operation: The server uses NLP technology such as spaCy to analyze the text data and extract the date and time information for "tomorrow at 10 AM" and the event information for "meeting." The analysis results are recorded in the server log.

[1695] Input: Text data (e.g. "Meeting tomorrow at 10 AM")

[1696] Output: Parsed information (e.g., date and time "tomorrow 10:00 AM", event "meeting")

[1697] Step 6:

[1698] The server stores the analysis results in a database.

[1699] Specific operation: The server stores the analyzed information in MySQL or MongoDB. The stored data includes user ID, date and time, event, etc. The server generates a log to confirm the data stored in the database.

[1700] Input: Parsed information

[1701] Output: Information stored in the database

[1702] Step 7:

[1703] The server generates reminders and advice.

[1704] Specific operation: The server sets up a cron job to send a reminder notification at "9:30 AM tomorrow." It also generates nutritional advice based on "Today's breakfast is bread and bananas," and generates a message saying, "You should add a few more vegetables."

[1705] Input: Saved schedule information and meal information

[1706] Output: Reminder notifications and advice messages

[1707] Step 8:

[1708] The server sends reminders and advice to the user terminal.

[1709] Specific operation: The server uses push notification to send a notification to the user's device at 9:30 AM tomorrow saying, "There will be a meeting at 10 AM." It also sends an advice message.

[1710] Input: Reminder notification setting information and advice message

[1711] Output: Notifications and messages sent to the user's device

[1712] Step 9:

[1713] Parents enter comments.

[1714] Specific operation: The parent speaks into their smartphone, "Please do your homework properly." This voice data is sent to the device.

[1715] Input: Audio data

[1716] Output: Audio data on the parent device

[1717] Step 10:

[1718] The server saves the parent's comments and notifies the child.

[1719] Specific operation: The server receives comments from the parent's device, analyzes them, and stores them in a database. Furthermore, based on the child's schedule, it sends a message to the child's device during their free time saying, "Please do your homework properly."

[1720] Input: Parsed parent comments, child schedule information

[1721] Output: Notification sent to child device

[1722] (Application example 1)

[1723] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1724] It is often difficult to efficiently communicate between workers and managers in factories and manage work schedules. In busy work environments, workers can forget their schedules or miss necessary safety instructions, resulting in reduced efficiency and safety. Furthermore, there are concerns that workers' workloads may increase if they are not properly provided with information related to seasonal changes or specific tasks.

[1725] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1726] In this invention, the server includes means for accepting voice or text input of schedules and work details from a user terminal, means for converting the voice into text, means for transmitting the text data to the server, means for analyzing the text data in the server and storing the schedule information in a database, means for setting reminders based on the schedule, means for analyzing work records and generating work-related advice, means for acquiring seasonal event information and work-related information, and means for receiving comments from a manager and notifying them at an appropriate time, thereby enabling factory workers to efficiently manage their schedules and receive necessary instructions and advice in a timely manner.

[1727] A "user terminal" is a device that accepts voice or text input.

[1728] A "speech recognition library" is a software library for converting voice data into text.

[1729] "Text data" refers to character string data obtained by voice input or text input.

[1730] A "server" is a central processing device that receives text data and performs analysis, storage, and notification.

[1731] "Natural Language Processing technology" refers to computer processing technology for understanding and analyzing human language.

[1732] "Database" refers to a data storage system for storing analyzed schedule information and work records.

[1733] "Reminder" is a function that notifies the user based on a set schedule.

[1734] "Work records" refers to information about specific tasks and time management within a factory.

[1735] "Advice" means advice or recommendations provided based on the analyzed work record.

[1736] A "manager" is a person who is responsible for giving comments and instructions to workers.

[1737] "Seasonal event information" refers to information about specific events for each season.

[1738] "Work-related information" refers to various types of information provided to improve work efficiency and safety.

[1739] The present invention is an in-factory work management and communication system including a user terminal, a server, and a communication means for exchanging data between them. Specific embodiments will be described below.

[1740] System Overview

[1741] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user.It also supports communication between managers and workers by receiving comments from managers and notifying them to workers.

[1742] Program processing flow

[1743] User terminal

[1744] A user terminal is a device that allows a user to input schedules and work details by voice or text. When a user inputs by voice, the user terminal uses a speech recognition library (e.g., Google Cloud Speech-to-Text API) to convert the voice into text and sends this text data to the server. For example, if a user inputs "Maintenance inspection of Line 1 at 10:00 AM" by voice, the terminal converts this into text data "Maintenance inspection of Line 1 at 10:00 AM" and sends it to the server.

[1745] server

[1746] The server is responsible for receiving and analyzing the text data sent from the terminal. The server uses Natural Language Processing (NLP) technology (e.g., NLTK, Spacy) to analyze the received text data and saves the schedule information in a database (e.g., PostgreSQL). It also sets reminders and sends reminder notifications to the user terminal at appropriate times. For example, it sends a reminder 30 minutes before the maintenance inspection saying, "Maintenance inspection of Line 1 is coming up soon."

[1747] Furthermore, the server analyzes work records, generates work-related advice (e.g., "Please wear protective equipment before maintenance inspections"), and sends this to the user's terminal. The server also obtains information related to seasonal changes and specific tasks and provides it to the user. For example, in the summer, the server notifies users with information such as "Be careful of heatstroke."

[1748] Administrator comment notification

[1749] Managers input comments by voice or text from their own devices, and the data is sent to the server. The server saves the received comments and checks the worker's schedule. For example, if a manager sends a comment such as "Please wear protective equipment before maintenance inspection," the server will send a notification to the worker's device at the appropriate time based on the worker's schedule, stating "Please wear protective equipment before maintenance inspection."

[1750] Specific examples

[1751] When a worker voice-inputs "Maintenance inspection of Line 1 at 10:00 AM," the following process takes place:

[1752] 1. The user device converts the speech into text and sends it to the server.

[1753] 2. The server receives the text data and uses NLP technology to analyze the schedule information for the "10:00 AM maintenance inspection" and save it in the database.

[1754] 3. The server sets a reminder and sends a reminder notification to the user terminal 30 minutes before the maintenance inspection.

[1755] 4. When the administrator terminal inputs "Please wear protective equipment before maintenance inspection" by voice, the instruction is notified to the user terminal at the appropriate time.

[1756] Prompt Sentence Examples

[1757] "We have a maintenance inspection scheduled for Line 1 at 10:00 AM. Please send me a reminder 30 minutes before. I would also like a reminder to put on protective gear before the inspection."

[1758] As described above, by using this system, factory workers can efficiently manage their schedules and receive necessary instructions and advice in a timely manner.

[1759] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1760] Program processing flow

[1761] Step 1:

[1762] The user inputs schedules and work details using voice or text. The user's device receives the input voice data and converts it into text data using a speech recognition library (e.g., Google Cloud Speech-to-Text API). The text data obtained through this conversion becomes the input for the next step.

[1763] Step 2:

[1764] The user terminal sends the converted text data to the server. The server receives the text data from the user terminal and prepares to analyze it. The input here is text data.

[1765] Step 3:

[1766] The server analyzes the received text data using Natural Language Processing (NLP) techniques (e.g., NLTK, Spacy). From this analysis, schedule information and task details are extracted. The results of this analysis are stored in a database for further processing. The output of the analysis is database entries for schedule information and task details.

[1767] Step 4:

[1768] The server sets a schedule-based reminder based on the analysis results. A specific time, such as 30 minutes before a maintenance inspection, is specified for the reminder setting. This reminder setting information is used to send reminder notifications. The reminder setting information is output.

[1769] Step 5:

[1770] The set reminder is sent to the user's device. The contents of the reminder are personalized for each user and notified at the appropriate time. The input here is the set reminder information, and the output is the reminder notification displayed on the user's device.

[1771] Step 6:

[1772] The server analyzes the work records and generates work-related advice, such as specific advice like "wear protective equipment before maintenance inspection." This advice information becomes the input for the next processing step.

[1773] Step 7:

[1774] The generated advice information is sent to the user terminal and notified to the user at the appropriate time. This allows the worker to receive the necessary advice. The input here is the generated advice information, and the output is the advice notification sent to the user terminal.

[1775] Step 8:

[1776] The administrator inputs a comment by voice or text from their own terminal. The administrator terminal sends this comment data to the server. The sent comment data becomes the input for the next step.

[1777] Step 9:

[1778] The server saves comments received from the administrator and checks the worker's schedule. The comments are notified to the worker's terminal at the appropriate time. This notification allows the worker to receive important instructions and comments. The input here is the comment data from the administrator, and the output is a notification to the worker's terminal.

[1779] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1780] The present invention is a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them, and by combining an emotion engine, it is also able to respond to the emotions of the user. Specific embodiments are described below.

[1781] System Overview

[1782] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user. It also uses an emotion engine to recognize the user's emotions and responds based on that information. It also supports communication between parents and children by receiving comments from parents and notifying the child.

[1783] Program processing flow

[1784] User terminal

[1785] The user terminal is a device that allows users to input schedules and meal details by voice or text. When a user inputs by voice, the user terminal uses a voice recognition library to convert the voice into text and sends this text data to the server. It also has an emotion engine that recognizes the user's emotions from the input voice and text data. For example, if a user inputs "Meeting tomorrow at 10 AM," the terminal converts this into text data "Meeting tomorrow at 10 AM" and sends it to the server, while also analyzing the user's emotions (e.g., stress, anxiety).

[1786] server

[1787] The server is responsible for receiving and analyzing the text data and emotion data sent from the device. For example, the server uses Natural Language Processing (NLP) technology to analyze the received text data and store the schedule information in a database. Furthermore, the server adjusts the content of reminders and advice based on the results of the emotion engine. For example, if the user's emotion is recognized as "stress," the server adjusts the wording of the reminder to be softer.

[1788] For example, if a user inputs "Today's breakfast is bread and bananas," the server analyzes the meal contents and generates advice on nutritional balance. For example, it generates advice such as "You should add a few more vegetables," and sends this advice to the user's terminal.

[1789] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[1790] Parent comment notification

[1791] When a parent enters a comment via voice or text from their device, the data is sent to the server. The server saves the received comment and checks the child's schedule. For example, if a parent sends a comment saying "Please do your homework properly," the server will send a notification saying "Please do your homework properly" to the child's device at an appropriate time based on the child's schedule. Here too, the content of the notification is adjusted taking into account the child's emotional information (e.g., if they are feeling depressed).

[1792] Specific examples

[1793] For example, if a user speaks "Meeting tomorrow at 10 AM," the following process occurs:

[1794] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects "stress" from the user's voice.

[1795] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[1796] 3. The server receives the text data and analyzes it using NLP technology. It extracts the date and time information for "tomorrow at 10:00 AM" and adjusts the reminder based on the emotional data "stress."

[1797] 4. The server saves the schedule information in a database and sets a reminder. For example, it sets a reminder for "tomorrow at 9:30 AM" and adjusts the message content to reduce "stress."

[1798] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[1799] Also, if a parent sends a comment saying "Please do your homework properly," the processing is as follows:

[1800] 1. The parent device converts the voice to text and analyzes the emotion using an emotion engine. For example, it detects the emotion of "encouragement."

[1801] 2. The device sends the converted text data and emotion data to the server.

[1802] 3. The server receives and stores the comment data. Based on the emotional data "encouragement," it sets up notifications to be sent at the optimal time for the child's situation.

[1803] 4. At the set time, the server sends a message to the child's device saying, "Please do your homework properly." For example, at 4:00 PM, the server sends a message saying, "Please do your homework properly. Good luck!"

[1804] The specific system configuration and programs required to implement the present invention are those that properly perform the above-mentioned processes, thereby improving the efficiency of users' lifestyle management and supporting communication between parents and children in a way that takes emotion into consideration.

[1805] The processing flow will be explained below.

[1806] Specific processing flow of the program

[1807] Schedule management and emotion recognition processing

[1808] Step 1:

[1809] The user inputs the schedule details by voice. For example, the user may say, "Meeting tomorrow at 10:00 AM."

[1810] Step 2:

[1811] The device uses a speech recognition library to convert the voice data into text data. Specifically, the voice "Meeting tomorrow at 10 AM" is converted into text "Meeting tomorrow at 10 AM."

[1812] Step 3:

[1813] The device uses an emotion engine to analyze the user's emotion from the input voice data. For example, the emotion "stress" is analyzed.

[1814] Step 4:

[1815] The device sends the converted text data and analyzed emotion data to the server via an HTTP request.

[1816] Step 5:

[1817] The server receives the text data and emotion data and analyzes the text data using Natural Language Processing (NLP) technology, extracting the date and time information for "tomorrow at 10:00 AM."

[1818] Step 6:

[1819] The server saves the extracted schedule information in a database. For example, it saves the meeting information for "December 20, 2023 at 10:00 AM."

[1820] Step 7:

[1821] The server adjusts and sets the content of the reminder based on the emotion data. For example, if the user's emotion is recognized as "stressed," the server adjusts the reminder to a more soothing message. For example, it sets the reminder to "Tomorrow at 9:30 AM" and adjusts the notification content to "You have a meeting at 10 AM. Relax and enjoy."

[1822] Step 8:

[1823] When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[1824] Food record and advice processing

[1825] Step 1:

[1826] The user inputs the details of their meal by voice, for example, saying, "Today's breakfast is bread and bananas."

[1827] Step 2:

[1828] The device converts the voice data into text data. The voice "Today's breakfast is bread and bananas" is converted into text "Today's breakfast is bread and bananas."

[1829] Step 3:

[1830] The terminal transmits the converted text data to the server.

[1831] Step 4:

[1832] The server receives the text data and uses the Nutrient Analysis API to analyze the dietary content, evaluating the carbohydrate and vitamin content as analysis results.

[1833] Step 5:

[1834] The server generates nutritional advice based on the analysis results, such as "You might want to add a few more vegetables."

[1835] Step 6:

[1836] The server transmits the generated advice to the user terminal and notifies it.

[1837] Parent comment notification processing

[1838] Step 1:

[1839] Parents can comment by voice or text, for example, saying, "Please do your homework."

[1840] Step 2:

[1841] The device converts the voice to text. The voice "Do your homework properly" is converted to text "Do your homework properly."

[1842] Step 3:

[1843] The terminal transmits the converted text data and the emotion data generated by the emotion engine to the server.

[1844] Step 4:

[1845] The server receives the comment data and the emotion data and stores them. The emotion data is analyzed as, for example, "encouragement."

[1846] Step 5:

[1847] The server checks the child's schedule and calculates the appropriate time to notify them of the comment. For example, if the child has free time at 4 p.m., it will set the notification to be sent at that time.

[1848] Step 6:

[1849] The server sends the message "Do your homework properly" to the child's device at the set time. For example, at 4:00 PM, it sends the message "Do your homework properly. Good luck!"

[1850] Providing information on seasonal events and foods

[1851] Step 1:

[1852] The server checks the current date and retrieves information about seasonal events and foods. For example, if it is October, it retrieves information about "moon viewing."

[1853] Step 2:

[1854] Based on the information acquired by the server, a message to be provided to the user is generated. For example, a message such as "Today is the season for viewing the moon. Let's enjoy some dumplings" is generated.

[1855] Step 3:

[1856] The server sends the generated message to the user terminal for notification.

[1857] This is the specific program flow of the AI ​​Mom System. This system will improve the efficiency of users' lifestyle management and support communication between parents and children in a way that takes emotions into consideration.

[1858] Example 2

[1859] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1860] Conventional schedule management systems and lifestyle support systems simply process information mechanically without considering the user's feelings, which means they are ineffective at reducing the user's psychological burden. Specifically, it is difficult to provide appropriate reminders and support to users who are stressed or anxious. Furthermore, they lack functionality to support communication between parents and children, making it difficult to maintain a smooth parent-child relationship.

[1861] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1862] In this invention, the server includes means for analyzing text data and emotion data sent from the user terminal and saving schedule information in a database, means for setting reminders based on the schedule and adjusting them based on the emotion data, and means for analyzing the meal record, generating advice on nutritional balance, and adjusting them based on the emotion data. This makes it possible to provide reminders and support that take the user's emotions into consideration. Furthermore, by providing a function for receiving comments from parents and notifying them at an appropriate time while taking into account the child's emotion data, it is possible to facilitate communication between parents and support the maintenance of a smooth parent-child relationship.

[1863] A "user terminal" is a device that allows a user to input schedules and meal details by voice or text.

[1864] A "voice recognition library" is a software component for converting voice data into text data.

[1865] "Emotion data" is information about a user's emotional state that has been analyzed using emotion recognition technology.

[1866] "Server" means a central device that receives, analyzes, and processes data sent from user terminals.

[1867] "Text data" is data in text format that has been converted by voice recognition.

[1868] "Natural Language Processing technology" is a technology for understanding and generating natural language.

[1869] A "database" is a system for organizing and storing information.

[1870] "Reminder" is a function for notifying the user of a specific date, time or event.

[1871] "Diet record" is data about dietary details entered by the user.

[1872] "Nutritional balance" refers to the proper distribution of nutrients in food.

[1873] "Event information" is information about specific activities or events related to seasons, dates, and times.

[1874] "Parent comment" is a message entered by a parent to their child.

[1875] This invention is a schedule management and life support system that includes a user terminal, a server, and a communication means for exchanging data between them, and by combining it with an emotion engine, it can also respond to the user's emotions. This system is configured as follows.

[1876] System Overview

[1877] A user terminal is a device that allows a user to input schedules and meal details by voice or text. Examples of such devices include smartphones and tablet devices. When a user inputs voice information, the user terminal uses a speech recognition library (e.g., Google Cloud Speech-to-Text API) to convert the speech into text and sends this text data to a server. In addition, an emotion recognition API (e.g., Microsoft Azure Cognitive Services) is used to recognize the user's emotions from the input voice or text data.

[1878] The server is responsible for receiving and analyzing the text data and emotion data sent from the user terminal. The server analyzes the received text data using Natural Language Processing technology (e.g., Google Cloud Natural Language API) and stores the schedule information in a database (e.g., MySQL).

[1879] Furthermore, the content of reminders and advice is adjusted based on the results of the emotion engine. For example, if the user's emotion is recognized as "stress," the wording of the reminder is softened or adjusted accordingly. Also, if the user inputs "Today's breakfast is bread and banana," the server analyzes the meal contents and generates advice on nutritional balance. For example, advice such as "You should add a few more vegetables" is generated and sent to the user's device.

[1880] Furthermore, the server retrieves information about seasonal events and foods according to the current date and provides it to the user. For example, if it is October, the server will notify the user with information such as "Today is the season for viewing the moon. Let's enjoy some dumplings."

[1881] The system also receives comments from parents and checks their children's schedules. For example, if a parent sends a comment saying, "Please do your homework properly," the server will send a notification saying, "Please do your homework properly" to the child's device at an appropriate time based on the child's schedule. At this time, the content of the notification will be adjusted taking into account the child's emotional information (e.g., if they are feeling depressed).

[1882] Specific operation example

[1883] Schedule management example

[1884] If a user speaks "Meeting tomorrow at 10 AM," the following occurs:

[1885] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects "stress" from the user's voice.

[1886] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[1887] 3. The server receives the text data and analyzes it using NLP technology. It extracts the date and time information for "tomorrow at 10:00 AM" and adjusts the reminder based on the emotional data "stress."

[1888] 4. The server saves the schedule information in a database and sets a reminder. For example, it sets a reminder for "tomorrow at 9:30 AM" and adjusts the message content to reduce "stress."

[1889] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 9:30, it sends a notification saying, "There's a meeting at 10:00. Relax and enjoy."

[1890] Example of parent comment notification

[1891] When a parent sends a comment saying "Please do your homework properly," the following occurs.

[1892] 1. The parent device converts the voice to text and analyzes the emotion using an emotion engine. For example, it detects the emotion of "encouragement."

[1893] 2. The device sends the converted text data and emotion data to the server.

[1894] 3. The server receives and stores the comment data. Based on the emotional data "encouragement," it sets up notifications to be sent at the optimal time for the child's situation.

[1895] 4. At the set time, the server sends a message to the child's device saying, "Please do your homework properly." For example, at 4:00 PM, the server sends a message saying, "Please do your homework properly. Good luck!"

[1896] Prompt Sentence Examples

[1897] Here are some examples of prompts to input to a generative AI model:

[1898] If a user says "Meet tomorrow at 10 AM," how does the system handle this?

[1899]

[1900] If a parent sends a comment like "Do your homework properly," how does the system handle that?

[1901] conclusion

[1902] This system will help users manage their daily lives more efficiently and provide emotionally sensitive reminders and advice, reducing the user's psychological burden. It will also facilitate communication between parents and children, helping to maintain parent-child relationships.

[1903] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1904] Specific processing steps for schedule management

[1905] Step 1: User enters schedule by voice

[1906] The user inputs by voice, "Meeting tomorrow at 10:00 AM." This input data is sent to the user's terminal as voice data.

[1907] Step 2: Speech to text

[1908] The device uses the Google Cloud Speech-to-Text API to convert the voice data entered by the user into text data.

[1909] Input: Voice data "Meeting tomorrow at 10 AM"

[1910] Data processing: Convert speech to text using a speech recognition library

[1911] Output: Text data "Meeting tomorrow at 10 AM"

[1912] Step 3: Sentiment analysis

[1913] The device uses the emotion recognition API of Microsoft Azure Cognitive Services to analyze the user's emotions from the converted text data.

[1914] Input: Text data "Meeting tomorrow at 10 AM"

[1915] Data processing: Emotion analysis using emotion recognition API

[1916] Output: Emotion data "Stress"

[1917] Step 4: Sending data

[1918] The device encodes the converted text data and emotion data into JSON format and sends it to the server via HTTPS.

[1919] Input: Text data "Meeting tomorrow at 10 AM", emotion data "Stress"

[1920] Data processing: Encode text data and emotion data into JSON format

[1921] Output: JSON data

[1922] Step 5: Data reception and analysis

[1923] The server receives the JSON data and uses the Google Cloud Natural Language API to parse the text data and extract the date and time information for "tomorrow at 10:00 AM."

[1924] Input: JSON data

[1925] Data processing: Analyze text data using natural language processing technology and extract date and time information

[1926] Output: Date and time information "Tomorrow 10:00 AM"

[1927] Step 6: Save the schedule

[1928] The server stores the extracted date and time information and text data in a MySQL database.

[1929] Input: Date and time information "Tomorrow 10:00 AM", text data "Meeting tomorrow at 10:00 AM"

[1930] Data processing: Execute SQL queries and save to database

[1931] Output: Schedule information stored in the database

[1932] Step 7: Set reminders

[1933] The server adjusts the content of the reminder based on the emotional data "stress" and sets reminder information based on the schedule, for example, setting a reminder 30 minutes before a meeting.

[1934] Input: Emotion data "Stress", Date and time information "Tomorrow 10:00 AM"

[1935] Data processing: Generate emotion-sensitive reminder messages and set reminder information

[1936] Output: Reminder information (date, time and message)

[1937] Step 8: Send reminders

[1938] When the server reaches the set time (e.g. 9:30 AM), it sends a reminder notification to the user's device.

[1939] Input: Reminder information

[1940] Data processing: Send reminders via HTTP POST request

[1941] Output: Reminder message "There's a meeting at 10 o'clock. Relax and enjoy."

[1942] Specific steps for parent comment notification

[1943] Step 1: Parents enter comments by voice

[1944] The parent can input "Please do your homework properly" by voice. This input data is sent to the parent device as voice data.

[1945] Step 2: Speech to text

[1946] The parent device uses the Google Cloud Speech-to-Text API to convert the voice data input by the parent into text data.

[1947] Input: Voice data "Do your homework properly"

[1948] Data processing: Convert speech to text using a speech recognition library

[1949] Output: Text data "Please do your homework properly."

[1950] Step 3: Sentiment analysis

[1951] The parent device uses the emotion recognition API of Microsoft Azure Cognitive Services to analyze emotions from the converted text data.

[1952] Input: Text data "Please do your homework properly"

[1953] Data processing: Emotion analysis using emotion recognition API

[1954] Output: Emotion data "Encouragement"

[1955] Step 4: Sending data

[1956] The parent device encodes the converted text data and emotion data into JSON format and sends it to the server via HTTPS.

[1957] Input: Text data "Do your homework properly", emotion data "Encouragement"

[1958] Data processing: Encode text data and emotion data into JSON format

[1959] Output: JSON data

[1960] Step 5: Data reception and analysis

[1961] The server interprets the received JSON data and extracts text and emotion data.

[1962] Input: JSON data

[1963] Data processing: Analyze JSON data and extract text and sentiment data

[1964] Output: Text data, emotion data

[1965] Step 6: Save comments

[1966] The server stores the extracted text data and emotion data in a database.

[1967] Input: Text data, emotion data

[1968] Data manipulation: Save to database using SQL queries

[1969] Output: Comment information stored in the database

[1970] Step 7: Notification Settings

[1971] The server checks the child's schedule and configures it to send notifications at appropriate times based on emotional data.

[1972] Input: Child's schedule information, emotional data "encouragement"

[1973] Data processing: Set notification timing and content

[1974] Output: Notification settings information

[1975] Step 8: Send notification

[1976] When the server reaches the set time (e.g. 3:50 pm), it will notify the child's device of the parent's comment, "Make sure you do your homework properly."

[1977] Input: Notification settings information

[1978] Data processing: Send parent comment via HTTP POST request

[1979] Output: Notification message "Do your homework properly. Good luck!"

[1980] conclusion

[1981] In this way, the user's lifestyle management becomes more efficient, and it becomes possible to provide emotionally sensitive reminders and advice. It also facilitates communication between parents and children, helping to maintain parent-child relationships.

[1982] (Application example 2)

[1983] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1984] Currently, there is a demand for improved efficiency in factory operations and smoother communication with workers. In particular, it is important to be able to quickly and accurately interpret voice instructions from workers and respond accordingly. There is also a need to understand workers' emotions and provide appropriate advice and reminders to reduce their stress. However, current systems are unable to meet these requirements.

[1985] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting voice or text input of schedules and meal details from a user terminal; means for converting the voice into text; means for transmitting the text data to the server; means for analyzing the text data in the server and storing the schedule information in a database; means for setting reminders based on the schedule; means for analyzing meal records and generating advice on nutritional balance; means for acquiring information on seasonal events and in-season foods; means for receiving comments from parents and notifying them at appropriate times; means for accepting voice instructions from workers in a factory environment, converting them into text in real time, and analyzing schedule and emotion data; and means for adjusting reminders and advice to workers based on the emotion data and notifying them at appropriate times. This enables improved work efficiency in the factory and smoother communication with workers.

[1986] A "user terminal" is a device that accepts input of schedules and meal details by voice or text.

[1987] "Means for converting speech to text" refers to technology for converting speech input by a user into text data.

[1988] The "means for transmitting text data to a server" is a function for transmitting the converted text data to a server via a network.

[1989] A "server" is a computer system for analyzing and storing data sent from a user terminal.

[1990] The "means for saving schedule information in a database" is a function for storing analyzed schedule information in a database.

[1991] The "means for setting a reminder" is a function for notifying the user based on a set schedule.

[1992] "Means for analyzing dietary records" refers to technology for analyzing dietary details entered by a user.

[1993] The "means for generating advice regarding nutritional balance" is a function for providing advice regarding nutrition based on the analyzed dietary record.

[1994] The "means for acquiring seasonal event information and seasonal food information" is a function for collecting event information and seasonal food information corresponding to each season and providing it to the user.

[1995] The "means for receiving comments from parents and notifying them at an appropriate time" is a function for receiving comments entered by parents and notifying them in accordance with the child's schedule.

[1996] "Factory environment" refers to the location where manufacturing and product assembly operations occur.

[1997] The "means for receiving voice instructions from workers" is a function that enables workers working in the factory to input voice instructions.

[1998] "Means for converting to text in real time" refers to technology for converting voice instructions into text data in real time.

[1999] "Emotion data" is data that indicates the emotional state of a worker analyzed from his / her voice.

[2000] The "means for adjusting reminders and advice to workers based on emotional data" is a function for providing appropriate reminders and advice to workers based on the analyzed emotional data.

[2001] This invention is a system for improving work efficiency in factories and facilitating communication with workers. The system includes a user terminal, a server, and a communication means for exchanging data between them. By combining it with an emotion engine, the system can also respond to the emotions of workers.

[2002] System Overview

[2003] This system accepts voice or text input from the user's device, analyzes it on the server, and provides appropriate reminders and advice to the user. In particular, it uses an emotion engine to recognize the worker's emotions and responds based on that information. It also has a function to receive comments from parents and notify the child.

[2004] User terminal

[2005] The user terminal is a device that allows workers to input instructions and schedules by voice or text. When using voice input, the user terminal uses a voice recognition library (e.g., the SpeechRecognition library) to convert the voice into text and sends this text data to the server. It also has an emotion engine (e.g., an emotion recognition model using Transformers) that recognizes the worker's emotions from the input voice and text data. For example, if a worker inputs "Maintenance work will be performed at 3:00 PM today," the terminal converts this into text data that reads "Maintenance work will be performed at 3:00 PM today" and sends it to the server. At the same time, the terminal also analyzes the worker's emotions (e.g., sense of relief).

[2006] server

[2007] The server is responsible for receiving and analyzing the text and emotion data sent from the device. Specifically, it uses Natural Language Processing (NLP) technology to analyze the received text data and store the schedule information in a database. It also adjusts the content of reminders and advice based on the results of the emotion engine. For example, if the worker's emotion is recognized as "relief," the wording of the reminder will be adjusted accordingly. The server then sends the processed notification to the user's device at a specific time.

[2008] Specific examples

[2009] For example, if a worker says, "Maintenance work will be performed at 3:00 p.m. today. Please don't forget," the following process will occur:

[2010] 1. The user device converts the voice into text and analyzes the emotion using an emotion engine. For example, it detects a "sense of security."

[2011] 2. The device sends the converted text data and emotion data to the server via an HTTP request.

[2012] 3. The server receives the text data and analyzes it using NLP technology. It extracts the time information "3 PM" and adjusts the reminder based on the emotional data "sense of relief."

[2013] 4. The server saves the schedule information to a database and sets a reminder, for example, for 2:45 PM, with a tailored message tailored to the user's emotional state.

[2014] 5. When the server reaches the set reminder time, it sends a reminder notification to the user's device. For example, at 2:45 PM, it sends a notification saying, "Maintenance work will be performed at 3:00. Please proceed without worry."

[2015] Prompt Sentence Examples

[2016] An example of a voice instruction that a worker inputs into a terminal is a prompt such as, "Maintenance work will be performed at 3:00 PM today. Please don't forget." An example of a notification is a statement such as, "Maintenance work will be performed at 3:00 PM. Please proceed without worry."

[2017] In this way, work efficiency within the factory is improved and communication with workers is facilitated.

[2018] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2019] Step 1:

[2020] The user inputs by voice, "Maintenance work will be performed at 3:00 PM today. Please don't forget." This input is received by the terminal as voice data.

[2021] Step 2:

[2022] The device uses a speech recognition library (for example, the SpeechRecognition library) to convert the received voice data into text data. The input is voice data, and the output is text data: "Maintenance work will be carried out at 3:00 p.m. today. Please remember."

[2023] Step 3:

[2024] The device analyzes the converted text data using an emotion engine (e.g., an emotion recognition model using Transformers). The input is text data, and the device outputs emotion data (e.g., a sense of relief) as the result of emotion analysis.

[2025] Step 4:

[2026] The device sends the converted text data and emotion data to the server via an HTTP request. The input is text data and emotion data, and the device sends them to the server as output.

[2027] Step 5:

[2028] The text data received by the server is analyzed using Natural Language Processing (NLP) technology. The input is text data, and the analysis results extract date and time information (e.g., "3:00 PM") and process it into schedule information. The output is schedule information.

[2029] Step 6:

[2030] The server saves the schedule information in a database. The input is the extracted schedule information, and the output is the information saved in the database.

[2031] Step 7:

[2032] The server adjusts the reminder and advice content based on the emotion data and schedule information. The input is the emotion data and schedule information, and the output is the adjusted reminder content.

[2033] Step 8:

[2034] When the server reaches the reminder setting time (for example, 2:45 PM), it sends a reminder notification to the user terminal. The input is the adjusted reminder content, and the output is the notification sent to the user terminal.

[2035] Step 9:

[2036] The user device receives the notification and notifies the user by voice or text. The input is a reminder notification, and the output is a notification to the user saying, "There will be maintenance work at 3:00 PM. Please proceed with confidence."

[2037] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2038] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2039] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2040] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2041] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2042] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2043] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2044] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2045] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2046] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pr...

Claims

1. means for receiving input of schedule and meal details by voice or text from a user terminal; means for converting said speech to text; means for transmitting the text data to a server; A server analyzes the text data and stores schedule information in a database; means for setting schedule-based reminders; A means for analyzing the food record and generating nutritional advice; A means for obtaining seasonal event information and seasonal food information; A system that includes a means of receiving comments from parents and notifying them in a timely manner.

2. 2. The system according to claim 1, wherein the input from the user terminal is voice and is converted into text using a voice recognition library.

3. 2. The system according to claim 1, wherein the server analyzes text data using natural language processing technology and registers schedules and meal records.

Images