System

Abstract

A system is provided.SOLUTION: A system comprising: means for periodically acquiring a new message from an information transmission medium; means for analyzing content of the acquired message and extracting a designated keyword, a name, an address, a mention, and deadline date and time information; means for summarizing and arranging the extracted information as tasks; means for integrating the arranged tasks into one ToDo list and displaying the ToDo list with a priority; means for giving an alert mark or coloring to a task whose deadline is approaching; and means for notifying a terminal of a user of the ToDo list.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, where messages are increasingly received via multiple information transmission media (email, chat tools, etc.), the present invention aims to reduce the risk of overlooking important tasks and prevent missed or delayed responses. It also aims to provide a system that automatically extracts, integrates, centrally manages, and prioritizes tasks to facilitate efficient and prompt business responses. [Means for solving the problem]

[0005] The present invention provides a system that periodically retrieves new messages from multiple information transmission media, automatically extracts task information from these messages, and centrally manages them using the following means. First, it provides a means for periodically retrieving new messages from information transmission media, and then it provides a means for analyzing the content of the retrieved messages and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information. It also provides a means for summarizing the extracted information and organizing it into tasks, and a means for integrating the organized tasks into a single to-do list and displaying them in prioritized order. In particular, the above-mentioned problems are solved by configuring a system that includes a means for marking tasks with upcoming deadlines with an alert mark or color and a means for notifying the user of this to-do list on their device.

[0006] An "information transmission medium" is a communication means for supporting the sending and receiving of messages between users, such as an email server, a chat tool server, or a web chat server.

[0007] A "new message" is a message that is newly received through an information transmission medium.

[0008] A "means for acquiring" is a program or algorithm for automatically collecting new messages periodically from an information carrier.

[0009] An "analyzing means" is a program or algorithm that uses natural language processing techniques to read and understand the content of the captured messages and identify specific keywords or phrases.

[0010] The "extraction means" is a program or algorithm for extracting important information such as your name, address, mentions, deadlines, etc. from the analyzed message content.

[0011] A "summarization tool" is a program or algorithm that concisely summarizes the extracted information and organizes it in a way that makes sense for the task.

[0012] A "to-do list consolidator" is a program or algorithm that consolidates multiple tasks into a single list.

[0013] A "prioritization means" is a program or algorithm that ranks the listed tasks based on their urgency or importance.

[0014] An "alert marking or coloring means" is a program or algorithm that marks or colors tasks to visually emphasize tasks that are approaching deadlines or are important.

[0015] The "means for notifying the user's terminal" is a program or algorithm for delivering the generated ToDo list and alert information to the user's device and displaying them. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The system for implementing this invention efficiently manages tasks through the mutual cooperation of three elements: a server, a terminal, and a user. The specific roles of each element and the processing contents of the program are explained below.

[0038] Server Roles

[0039] The server plays a central role in this system and implements the following functions:

[0040] 1. Data Acquisition

[0041] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[0042] 2. Message Analysis

[0043] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract your name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as time and date information, which is then converted into a standard format.

[0044] 3. Task summary and organization

[0045] The server then summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, a sentence in an email saying "Yamada-san, please submit your report by next Monday" will generate a task called "Submit report" (deadline: next Monday).

[0046] 4. ToDo list generation

[0047] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[0048] 5. Alert Settings

[0049] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0050] 6. Notification function

[0051] The server notifies the user of the created to-do list via email or a dedicated notification API.

[0052] Device Role

[0053] The terminal is a device that is directly operated by the user and implements the following functions:

[0054] 1. Data display

[0055] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[0056] 2. Alert display

[0057] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0058] 3. Completing and updating tasks

[0059] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[0060] User Roles

[0061] The user is the ultimate beneficiary of this system and performs the following actions:

[0062] 1. Check the list

[0063] Users can check their to-do list on their device and understand the details and deadlines of each task.

[0064] 2. Task Processing

[0065] The user can then work on the displayed tasks in order of priority. After completing a task, they mark it as completed on their device and update the list.

[0066] Specific examples

[0067] An example of the system's operation is shown below.

[0068] Example 1: Creating a task from an email

[0069] Email content received by the server: "Yamada-san, please submit your report by next Monday. Thank you."

[0070] Information extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[0071] Server-generated task: "Report submission" (Deadline: next Monday)

[0072] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[0073] Example 2: Creating a task from Google Chat

[0074] Chat contents retrieved by the server: "@Yamada-san, please check the materials before tomorrow's meeting."

[0075] Information extracted by the server: mention (@Yamada), request (document confirmation), deadline (tomorrow's meeting)

[0076] Server-generated task: "Check materials" (Deadline: tomorrow's meeting)

[0077] To-do list notified to the device: "Check materials" (deadline: tomorrow's meeting), with attention alerts

[0078] This system allows users to efficiently manage tasks and prevent oversight.

[0079] The processing flow will be explained below.

[0080] Step 1:

[0081] The server periodically (for example, every 5 minutes) launches a task and accesses an information transmission medium (email server, chat tool server, web chat server).

[0082] Specifically, it logs in to the mail server using the IMAP or POP3 protocol, and retrieves new messages from the chat tool using a dedicated API.

[0083] Step 2:

[0084] The server analyzes new messages received from each information carrier.

[0085] The analysis uses natural language processing (NLP) technology to extract information such as the message body, the date and time of sending, and the sender, with particular attention paid to your name, address, mentions, and designated keywords (such as "request," "confirmation," and "question").

[0086] Step 3:

[0087] The server generates a summarized task content based on the information extracted from the parsed message.

[0088] For example, from an email that says "Yamada, please submit your report by next Monday," you can create a task called "Report submission" (deadline: next Monday). The date and time information is also converted to the standard format (YYYY-MM-DD).

[0089] Step 4:

[0090] The server stores the generated tasks in a database and integrates the to-do list.

[0091] Tasks are saved in a database, taking care to avoid duplication. Saved tasks are organized by user and managed centrally.

[0092] Step 5:

[0093] The server organizes the tasks on the to-do list based on priority.

[0094] The order of the list is determined by the urgency of the task and its deadline. Tasks with upcoming deadlines are given high priority.

[0095] Step 6:

[0096] The server monitors the deadlines of tasks and displays alert marks and colors on tasks whose deadlines are approaching.

[0097] For example, tasks with a deadline within two days will be marked with a red alert mark.

[0098] Step 7:

[0099] The server notifies the user's device of the generated ToDo list.

[0100] Delivery to devices is via email, dedicated app notification APIs, and web app update functions.

[0101] Step 8:

[0102] The terminal displays the ToDo list received from the server to the user.

[0103] The list is displayed in a table or calendar format, with tasks with alerts specifically highlighted.

[0104] Step 9:

[0105] Users can check their to-do list on their device and process tasks based on their content and priority.

[0106] When the task is complete, the user marks the task as complete on the terminal.

[0107] Step 10:

[0108] The terminal synchronizes the information of the user's task completion with the server.

[0109] The server removes the completed tasks from the database and updates the current to-do list.

[0110] The above is the specific processing flow of this system. This system efficiently manages each task in a centralized manner, preventing users from overlooking something and enabling them to respond quickly.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] In conventional task management systems, the process of effectively retrieving new messages from information transmission media and organizing them into tasks is cumbersome and not fully automated. Furthermore, the system lacks the ability to prioritize tasks based on their importance and deadlines, increasing the risk of users overlooking tasks. Furthermore, the system lacks alert functions for task deadlines and update processes after task completion, making it difficult for users to efficiently manage tasks.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes: means for periodically acquiring new data from an information transmission medium; means for analyzing the acquired data and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information; means for summarizing the extracted information and organizing it into tasks; means for integrating the organized tasks into a single to-do list and automatically prioritizing and displaying them; means for marking or coloring tasks with an alert mark if the deadline is approaching; means for monitoring task deadlines and notifying users of tasks with an approaching deadline; means for notifying the user of the to-do list on the user's device; and means for synchronizing the information with the server and updating the to-do list when the user marks a task as completed. This improves task management automation, allowing users to efficiently manage tasks without missing any tasks.

[0116] An "information transmission medium" is a communication means for exchanging information, such as an email server, a chat tool server, or a web chat server.

[0117] "Periodic acquisition" means that data is automatically acquired at preset time intervals.

[0118] "Analyzing" means understanding the content of the acquired data and classifying and extracting information according to certain rules.

[0119] "Specified keywords" are words that indicate specific conditions or content, and are set so that the system can extract them preferentially.

[0120] "Extract" means to extract specific information from within a message.

[0121] "Summarizing" means to briefly summarize extracted information.

[0122] A "task" is a specific task or action that a user must perform.

[0123] "Organizing" means arranging summarized information systematically and making it easy to manage.

[0124] A "ToDo list" is a list that displays tasks that need to be completed in a list format.

[0125] "Integrating" means combining multiple tasks into one list.

[0126] "Priority" is a criterion for determining the display order and execution order based on the importance and urgency of tasks.

[0127] A "deadline" is the deadline for completing a task.

[0128] An "alert mark" is a sign or icon that calls attention to specific information.

[0129] "Coloring" means adding color to information to draw visual attention.

[0130] "Notifying" means informing the user of information.

[0131] "Synchronizing" means keeping data consistent across different devices and systems.

[0132] "To update" means to bring existing information or data up to date.

[0133] The system for implementing this invention efficiently manages tasks through collaboration between the server, the terminal, and the user. The specific roles of each element and the processing contents of the program are explained below.

[0134] Server Roles

[0135] The server plays a central role in the system and implements the following functions:

[0136] 1. Get new messages

[0137] The server periodically retrieves new messages from the information transmission medium (e.g., email server, chat tool server, web chat server) using APIs and protocols (IMAP, POP3) specific to each medium. For example, the server retrieves new emails from the email server using the IMAP protocol.

[0138] 2. Message Content Analysis

[0139] The server analyzes the received message using natural language processing technology (for example, Google Cloud Natural Language API or spaCy) and extracts the specified keywords, the user's name, address, mentions, and deadline information. For example, from an email that says "Yamada-san, please submit your report by next Monday," the server extracts "Name: Yamada," "Request: Report submission," and "Deadline: next Monday."

[0140] 3. Summarize and organize your tasks

[0141] The server summarizes the message based on the extracted information and organizes it into meaningful tasks, such as generating a task called "Report Submission" (Deadline: Next Monday).

[0142] 4. Creating and updating a to-do list

[0143] The server consolidates the organized tasks into a single to-do list and stores it in a database, where task priorities are automatically set and adjusted based on deadlines and importance.

[0144] 5. Setting up alerts

[0145] The server monitors task deadlines and assigns alert marks and colors to tasks that are approaching their deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0146] 6. Notification function

[0147] The server notifies the user of the created to-do list via email or a dedicated notification API. For example, the server uses the Gmail API to send the new to-do list to the user's Gmail account via email.

[0148] Device Role

[0149] A terminal is a device that is directly operated by a user and has the following functions:

[0150] 1. Data display

[0151] The device receives the to-do list from the server and displays it to the user. For example, the list can be presented visually using a web app or mobile app.

[0152] 2. Alert display

[0153] The device displays tasks with alert marks and color coding to indicate which tasks the user should prioritize. Tasks with red marks are displayed at the top of the list.

[0154] 3. Completing and updating tasks

[0155] When a user completes a task, the device synchronizes the information with the server and updates the to-do list. The completed task is removed from the list. For example, when a user marks the "Submit Report" task as completed, the device sends the information to the server and the list is updated.

[0156] User Roles

[0157] The user is the ultimate beneficiary of this system and performs the following operations:

[0158] 1. Check the list

[0159] The user checks the to-do list on the device and understands the content and deadline of each task. For example, the user checks the task "Check materials" (deadline: tomorrow's meeting).

[0160] 2. Task Processing

[0161] The user will then work on the displayed tasks in order of priority. After completing a task, they will mark it as completed on their device and update the list. For example, the user will work on the "Document Review" task and mark it as completed.

[0162] Example operation

[0163] 1. Creating a task from email

[0164] The server retrieves the email from the email server using the IMAP protocol.

[0165] Email content: "Yamada-san, please submit your report by next Monday."

[0166] Information analyzed and extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[0167] Server-generated task: "Report submission" (Deadline: next Monday)

[0168] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[0169] When a user marks a task as complete, the device sends the information to the server and the task is removed from the list.

[0170] 2. Task creation from chat tools

[0171] The server uses the API to retrieve messages from the chat tool server.

[0172] Message: "@Yamada-san, please check the materials before tomorrow's meeting."

[0173] Information analyzed and extracted by the server: mention (@Yamada), request (document confirmation), deadline (tomorrow's meeting)

[0174] Server-generated task: "Check materials" (Deadline: tomorrow's meeting)

[0175] To-do list notified to the device: "Check materials" (deadline: tomorrow's meeting), with alert mark

[0176] When a user marks a task as complete, the device sends the information to the server and the task is removed from the list.

[0177] This allows users to efficiently manage tasks and prevent oversight.

[0178] Generative AI model used and example prompts

[0179] Generative AI model: OpenAI GPT-3

[0180] Example prompt

[0181] Please create a task from the email below.

[0182] Email content: Yamada-san, please submit your report by next Monday.

[0183] Expected output: Task "Report submission" (Deadline: next Monday)

[0184] Using these prompts, the AI ​​model extracts the specified information and generates a task.

[0185] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0186] Step 1:

[0187] Get new messages

[0188] Input: Information transmission medium (email server, chat tool server, web chat server)

[0189] The server periodically retrieves new messages using the API or protocol of each medium (IMAP, POP3).

[0190] Specific operation: The server retrieves new mail from the email server using the IMAP protocol.

[0191] Output: Retrieved message data

[0192] Step 2:

[0193] Message content analysis

[0194] Input: Retrieved message data

[0195] The server uses natural language processing technology (e.g., Google Cloud Natural Language API or spaCy) to analyze the content of the message and extract specified keywords, your name, address, mentions, and deadline date and time information.

[0196] Specific operation: The server extracts "Name: Yamada", "Request: Report submission", and "Deadline: Next Monday" from an email that says "Yamada-san, please submit your report by next Monday."

[0197] Output: Extracted information (name, request details, deadline, etc.)

[0198] Step 3:

[0199] Summarizing and organizing tasks

[0200] Input: Extracted information

[0201] The server uses the extracted information to summarize the messages and organize them into meaningful tasks.

[0202] Specific operation: The server generates the extracted task "Report submission" (deadline: next Monday).

[0203] Output: Organized tasks

[0204] Step 4:

[0205] Creating and updating to-do lists

[0206] Input: Organized tasks

[0207] The server consolidates the organized tasks into a single to-do list and stores it in a database, with task priorities automatically set.

[0208] Specific operation: The server integrates a new task "Report submission" (deadline: next Monday) into the ToDo list.

[0209] Output: ToDo list

[0210] Step 5:

[0211] Configuring Alerts

[0212] Input: Tasks in your ToDo list

[0213] The server monitors task deadlines and displays alert marks and colors for tasks whose deadlines are approaching.

[0214] Specific behavior: The server marks tasks with deadlines within two days with a red alert mark.

[0215] Output: ToDo list with alerts

[0216] Step 6:

[0217] Notification function

[0218] Input: To-do list with alerts

[0219] The server notifies the user of the generated to-do list via email or a dedicated notification API.

[0220] What happens: The server uses the Gmail API to email the new to-do list to the user's Gmail account.

[0221] Output: ToDo list notified to the user's device

[0222] Step 7:

[0223] Data Display

[0224] Input: ToDo list notified to the user's device

[0225] The terminal displays the received ToDo list to the user.

[0226] Specific operation: The device uses a web app or mobile app to display the task "Submit report" (deadline: next Monday).

[0227] Output: Displayed ToDo list

[0228] Step 8:

[0229] Alert display

[0230] Input: To-do list on device

[0231] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0232] What it does: The device will display tasks marked in red at the top of the list.

[0233] Output: Highlighted tasks

[0234] Step 9:

[0235] Completing and updating tasks

[0236] Input: Tasks marked as completed by user action

[0237] When a user completes a task, the device syncs the information with the server, updates the to-do list, and removes the completed task from the list.

[0238] Specific behavior: The user marks the "report submission" task as completed, the device sends the information to the server, and the list is updated.

[0239] Output: Updated ToDo list

[0240] This allows users to efficiently manage tasks and prevent oversight.

[0241] (Application example 1)

[0242] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0243] Conventional task management systems relied on email and chat messages, often requiring manual input from humans. Furthermore, they lacked a mechanism for efficiently managing error messages and operational status information from industrial robots. This resulted in the time-consuming task of responding to errors and managing tasks, resulting in reduced productivity.

[0244] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0245] In this invention, the server includes means for periodically acquiring new messages from an information transmission medium, means for analyzing the contents of the acquired messages and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them as tasks, means for integrating the organized tasks into a single ToDo list and displaying them in order of priority, means for marking or coloring tasks with an alert mark or color, means for analyzing messages from industrial robots and adding abnormality messages and operating statuses as tasks, and means for notifying the user of the ToDo list on their terminal. This makes it possible to improve the efficiency of task management and prevent oversights.

[0246] An "information transmission medium" is a communication means such as an email server, a chat tool server, or a web chat server, and is a medium for exchanging messages.

[0247] A "new message" is a message that is received for the first time over the communication medium and has not yet been processed.

[0248] A "specified keyword" is a specific word or phrase that is set in advance by the user and is used for automatic task extraction.

[0249] "My Name" refers to the user's own name or identification information, and is used to extract related tasks from the message content.

[0250] An "address" is an email address or a user's identifier in a chat tool.

[0251] "Mention" is a mechanism for notifying users who are specifically mentioned in chat tools or messages, and is implemented using the @ symbol, etc.

[0252] "Deadline date and time information" refers to a specific date and time set as the deadline for completing a task, and is used to make the user aware of the urgency of the task.

[0253] A "task" is an activity or work performed to achieve a specific purpose or goal.

[0254] A "ToDo list" is an organized collection of tasks that allows a user to view all the tasks that need to be done.

[0255] "Alert marks and colors" are symbols and colors that visually emphasize the urgency and importance of a task, and serve to draw the user's attention.

[0256] An "industrial robot" is an automated machine used in industrial fields such as manufacturing, and is used to efficiently perform specific tasks or processes.

[0257] An "abnormality message" is a message that an industrial robot sends when it detects some kind of error or abnormality.

[0258] "Operation status" refers to information that indicates the current operating state and work progress of an industrial robot.

[0259] The system for implementing this invention involves three elements, a server, a terminal, and a user, working together to efficiently manage tasks, including the operational status and error messages of industrial robots. The specific roles of each element and the processing contents of the program are explained below.

[0260] Server Roles

[0261] The server plays a central role in this system and implements the following functions:

[0262] 1. Data Acquisition

[0263] The server periodically retrieves new messages from information transmission media (email server, chat tool server, web chat server). This uses the API of each media. For example, IMAP or POP3 protocols are used to retrieve emails from the mail server, and a dedicated API is used to retrieve messages from the chat tool. In addition, data from industrial robots is also retrieved using a dedicated communication protocol.

[0264] 2. Message Analysis

[0265] The server analyzes the contents of the received messages using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"). Information about the time and date is also extracted and converted into a standard format. Furthermore, the server analyzes abnormal messages and operating status from industrial robots and recognizes them as tasks.

[0266] 3. Task summary and organization

[0267] The server summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, a sentence in an email saying, "Person in charge, please check the machine's operation by next Monday," generates a task called "Check the machine's operation" (deadline: next Monday). Similarly, a message from a robot saying, "Error code 1234: An abnormal operation has occurred," generates a task called "Error response" (immediate response).

[0268] 4. ToDo list generation

[0269] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[0270] 5. Alert Settings

[0271] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0272] 6. Notification function

[0273] The server notifies the user of the created to-do list via email or a dedicated notification API.

[0274] Device Role

[0275] The terminal is a device that is directly operated by the user and implements the following functions:

[0276] 1. Data display

[0277] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[0278] 2. Alert display

[0279] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0280] 3. Completing and updating tasks

[0281] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[0282] User Roles

[0283] The user is the ultimate beneficiary of this system and performs the following actions:

[0284] 1. Check the list

[0285] Users can check their to-do list on their device and understand the details and deadlines of each task.

[0286] 2. Task Processing

[0287] The user can then work on the displayed tasks in order of priority. After completing a task, they mark it as completed on their device and update the list.

[0288] Specific examples

[0289] An example of the system's operation is shown below.

[0290] Example 1: Creating a task from an email

[0291] Email content received by the server: "To the person in charge, please check that the machine is working properly by next Monday. Thank you."

[0292] Information extracted by the server: Name (person in charge), request details (machine operation check), deadline (next Monday)

[0293] Server-generated task: "Check machine operation" (Deadline: next Monday)

[0294] To-do list displayed on the device: "Check machine operation" (Deadline: next Monday)

[0295] Example 2: Task generation from an industrial robot

[0296] Message received from the server: "Error code 1234: Abnormal operation"

[0297] Information extracted by the server: Abnormality details (operational abnormality), priority (high)

[0298] Server-generated tasks: "Error handling" (immediate response)

[0299] To-do list notified on the device: "Error handling" (immediate response)

[0300] Prompt Sentence Examples

[0301] text

[0302] The body of the email should follow the format below. Please extract the task details and deadline. For example:

[0303] Email body: "Person in charge, please check the operation by next Monday."

[0304] Task: "Check operation"

[0305] Due: "Next Monday"

[0306] This system allows users to efficiently manage tasks and respond to abnormalities in industrial robots in real time.

[0307] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0308] Step 1:

[0309] The server periodically retrieves new messages from the information carrier. This includes retrieving emails using the IMAP and POP3 protocols, and retrieving messages from chat tools using a dedicated API. The input is the new messages, and the output is the retrieved raw data. This data is used in the subsequent analysis steps.

[0310] Step 2:

[0311] The server analyzes the acquired raw data using natural language processing (NLP) technology. This analysis extracts the user's name, address, mentions, specified keywords (such as "request," "confirmation," or "question"), and deadline date and time information. It also analyzes abnormality messages and operating status from the industrial robot to extract information about the abnormality and operating status. The input is raw data, and the output is extracted metadata.

[0312] Step 3:

[0313] The server summarizes each message based on the extracted metadata and organizes it into meaningful tasks. For example, a sentence in an email saying, "Person in charge, please check the machine's operation by next Monday," generates a task called "Check the machine's operation" (deadline: next Monday). Similarly, a message from a robot saying, "Error code 1234: An abnormal operation has occurred," generates a task called "Error response" (immediate response). The input is metadata, and the output is the generated task.

[0314] Step 4:

[0315] The server consolidates the organized tasks into a single to-do list. This list is stored in a database and displayed based on task priority. Priorities are automatically set based on the importance and urgency of the tasks. The input is the generated tasks, and the output is the consolidated to-do list.

[0316] Step 5:

[0317] The server monitors the deadlines of tasks in the to-do list and assigns alert marks and colors to tasks with upcoming deadlines. For example, a red alert mark is assigned to a task with a deadline within two days. The input is a to-do list, and the output is a visually emphasized to-do list.

[0318] Step 6:

[0319] The server notifies the user of the generated ToDo list via email or a dedicated notification API. The input is a visually enhanced ToDo list, and the output is the notified ToDo list.

[0320] Step 7:

[0321] The terminal displays the ToDo list received from the server to the user. The user manages tasks based on this list. The input is the notified ToDo list, and the output is the list displayed on the terminal.

[0322] Step 8:

[0323] The terminal highlights tasks with alert marks and colors to indicate which tasks the user should prioritize. The input is a highlighted to-do list, and the output is the priority tasks recognized by the user.

[0324] Step 9:

[0325] When a user completes a task, the device synchronizes the information with the server and updates the to-do list. At the same time, the completed task is removed from the list. The input is the task that was notified as completed, and the output is the updated to-do list.

[0326] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0327] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions. The specific roles of each element and the processing content of the program are explained below.

[0328] Server Roles

[0329] The server plays a central role in this system and implements the following functions:

[0330] 1. Data Acquisition

[0331] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[0332] 2. Message Analysis

[0333] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as information about the time and date, and converts it into a standard format.

[0334] 3. Task summary and organization

[0335] The server then summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, an email message saying "Yamada-san, please submit your report by next Monday" generates a task called "Submit report" (deadline: next Monday).

[0336] 4. ToDo list generation

[0337] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[0338] 5. Alert Settings

[0339] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0340] 6. Emotion engine integration

[0341] The server receives input from the emotion engine to recognize the user's current emotional state, which evaluates emotions using techniques such as voice recognition, facial expression analysis, and text analysis.

[0342] 7. Changing task priorities based on emotions

[0343] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[0344] 8. Change notification methods based on emotions

[0345] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[0346] Device Role

[0347] The terminal is a device that is directly operated by the user and implements the following functions:

[0348] 1. Data display

[0349] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[0350] 2. Alert display

[0351] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0352] 3. Collecting Emotional Data

[0353] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[0354] 4. Completing and updating tasks

[0355] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[0356] The role of the emotional engine

[0357] The emotion engine analyzes the user's emotional state and implements the following functions:

[0358] 1. Recognizing emotional states

[0359] The emotion engine uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[0360] 2. Data transmission

[0361] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[0362] User Roles

[0363] The user is the ultimate beneficiary of this system and performs the following actions:

[0364] 1. Check the list

[0365] Users can check their to-do list on their device and understand the details and deadlines of each task.

[0366] 2. Task Processing

[0367] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[0368] 3. Providing Emotion Data

[0369] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[0370] Specific examples

[0371] An example of the system's operation is shown below.

[0372] Example 1: Creating a task from an email

[0373] Email content received by the server: "Yamada-san, please submit your report by next Monday. Thank you."

[0374] Information extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[0375] Server-generated task: "Report submission" (Deadline: next Monday)

[0376] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[0377] Example 2: Changing task priorities with an emotion engine

[0378] The emotion engine analyzes the user's voice and facial expressions to recognize stress levels

[0379] The server dynamically changes the priority of tasks based on emotional information, and displays tasks that are easy to handle at the top.

[0380] Updated ToDo list notified on device: Priorities have been changed, with "Check documents" (deadline: tomorrow's meeting, light task) at the top

[0381] This system not only allows users to efficiently manage tasks and prevent oversights, but also allows them to respond flexibly according to their emotional state.

[0382] The processing flow will be explained below.

[0383] Step 1:

[0384] The server periodically (for example, every 5 minutes) launches a task and accesses an information transmission medium (email server, chat tool server, web chat server).

[0385] Specifically, it logs in to the mail server using the IMAP or POP3 protocol, and retrieves new messages from the chat tool using a dedicated API.

[0386] Step 2:

[0387] The server analyzes new messages received from each information carrier.

[0388] The analysis uses natural language processing (NLP) technology to extract information such as the message body, the date and time of sending, and the sender, with particular attention paid to your name, address, mentions, and designated keywords (such as "request," "confirmation," and "question").

[0389] Step 3:

[0390] The server generates a summarized task content based on the information extracted from the parsed message.

[0391] For example, from an email that says, "Yamada, please submit your report by next Monday," you can extract the name "Yamada," the request to "submit the report," and the deadline "next Monday," and create a task called "Submit report" (deadline: next Monday).

[0392] Step 4:

[0393] The server stores the generated tasks in a database and integrates the to-do list.

[0394] Tasks are saved in a database, taking care to avoid duplication. Saved tasks are organized by user and managed centrally.

[0395] Step 5:

[0396] The server organizes the tasks on the to-do list based on priority.

[0397] The order of the list is determined by the urgency of the task and its deadline. Tasks with upcoming deadlines are given high priority.

[0398] Step 6:

[0399] The server monitors the deadlines of tasks and displays alert marks and colors on tasks whose deadlines are approaching.

[0400] For example, tasks with a deadline within two days will be marked with a red alert mark.

[0401] Step 7:

[0402] The emotion engine collects data such as the user's voice, facial expressions, and input text to analyze the user's emotional state.

[0403] Using voice analysis and facial expression analysis technology, the system assesses the user's stress level, fatigue state, concentration, etc.

[0404] Step 8:

[0405] Based on the emotion data from the emotion engine, the server dynamically changes the priority of tasks.

[0406] For example, if the emotion engine detects that the user is in a high stress state, the server will prioritize and display relatively light tasks.

[0407] Step 9:

[0408] Based on the emotion data from the emotion engine, the server adjusts how the task is notified.

[0409] For example, if the server detects that the user is concentrating, it will make adjustments such as reducing notification sounds and reducing the number of pop-up notifications.

[0410] Step 10:

[0411] The server notifies the user's device of the generated ToDo list.

[0412] Delivery to devices is via email, dedicated app notification APIs, and web app update functions.

[0413] Step 11:

[0414] The terminal displays the ToDo list received from the server to the user.

[0415] The list is displayed in a table or calendar format, with tasks with alerts highlighted, and the list view is based on an emotion engine.

[0416] Step 12:

[0417] Users can check their to-do list on their device and process tasks based on their content and priority.

[0418] When the task is complete, the user marks the task as complete on the terminal.

[0419] Step 13:

[0420] The terminal synchronizes the information of the user's task completion with the server.

[0421] The server removes the completed tasks from the database and updates the current to-do list.

[0422] The above is the specific processing flow of this system. By combining it with an emotion engine, flexible task management that responds to the user's emotional state is realized, enabling efficient work execution.

[0423] Example 2

[0424] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0425] In modern society, with the massive volume of messages received daily from various information transmission media, it is difficult for users to efficiently manage their tasks. Simply listing tasks without considering the user's emotions and state increases stress and burden, and reduces efficiency. Furthermore, there is a lack of methods for properly communicating task deadlines and importance. This leads to problems such as users overlooking tasks or failing to respond appropriately to important tasks.

[0426] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for periodically acquiring new messages from an information transmission medium, means for analyzing the content of the acquired messages and extracting specified keywords, the user's name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them as tasks, means for integrating the organized tasks into a single to-do list and displaying them with priority, means for marking or coloring tasks with an alert mark when the deadline is approaching, means for collecting user emotion data from the terminal and transmitting it to the emotion engine, means for receiving input from the emotion engine and dynamically changing task priorities in consideration of the user's emotional state, means for adjusting the notification method based on the input from the emotion engine, and means for notifying the user of the to-do list on the terminal. This allows the user to efficiently manage tasks and flexibly respond to their emotional state.

[0427] "Information transmission medium" refers to systems and services for transmitting information and sending and receiving messages and data, such as email servers, chat tool servers, and web chat servers.

[0428] A "new message" refers to a message that is received for the first time through an information transmission medium and has not yet been processed by a user.

[0429] "Analysis" refers to the process of analyzing the content of captured messages and extracting specific information or patterns.

[0430] "Keywords" are words or phrases with a specific meaning or purpose that are used to extract important information from the content of a message.

[0431] An "emotion engine" is a system that recognizes the user's emotional state and collects and analyzes data from voice, facial expressions, input text, etc.

[0432] A "task" refers to a specific action or task that a user needs to perform, and is organized and managed as a to-do list.

[0433] A "ToDo list" is a list that lists a user's tasks and indicates information such as their priority and deadlines.

[0434] An "alert mark" refers to a mark or color that is displayed to warn or draw attention to a specific situation or task that is approaching a deadline.

[0435] "Priority" refers to a ranking that determines which tasks should be processed first based on the importance and urgency of the tasks.

[0436] "Emotional state" refers to the user's current psychological state or feelings, and includes specific sensations and emotions such as stress, fatigue, and concentration.

[0437] "Notification method" refers to the means or format for notifying the user of a task or alert, and includes audio, visual, banner display, etc.

[0438] "Device" refers to a device that is directly operated by a user, including, for example, a computer, a smartphone, or a tablet.

[0439] "Integration" refers to the process of combining multiple elements or pieces of information into one system or list.

[0440] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions. The specific roles of each element and the processing content of the program are explained below.

[0441] Server Roles

[0442] 1. Data Acquisition

[0443] The server periodically retrieves new messages from the email server, chat tool server, and web chat server, using the API of each medium. For example, IMAP or POP3 protocols are used to retrieve emails from the mail server, and a dedicated API is used to retrieve messages from the chat tool.

[0444] 2. Message Analysis

[0445] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as information about the time and date, and converts it into a standard format.

[0446] 3. Task summary and organization

[0447] The server summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, an email message saying "Customer, please submit the report by next Monday" generates a task called "Submit report" (deadline: next Monday).

[0448] 4. ToDo list generation

[0449] The server consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[0450] 5. Alert Settings

[0451] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0452] 6. Emotion engine integration

[0453] The server receives input from the emotion engine to recognize the user's current emotional state, which evaluates emotions using techniques such as voice recognition, facial expression analysis, and text analysis.

[0454] 7. Changing task priorities based on emotions

[0455] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[0456] 8. Change notification methods based on emotions

[0457] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[0458] Device Role

[0459] The terminal is a device that is directly operated by the user and implements the following functions:

[0460] 1. Data display

[0461] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[0462] 2. Alert display

[0463] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0464] 3. Collecting Emotional Data

[0465] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[0466] 4. Completing and updating tasks

[0467] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[0468] The role of the emotional engine

[0469] The emotion engine analyzes the user's emotional state and implements the following functions:

[0470] 1. Recognizing emotional states

[0471] The emotion engine uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[0472] 2. Data transmission

[0473] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[0474] User Roles

[0475] The user is the ultimate beneficiary of this system and performs the following actions:

[0476] 1. Check the list

[0477] Users can check their to-do list on their device and understand the details and deadlines of each task.

[0478] 2. Task Processing

[0479] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[0480] 3. Providing Emotion Data

[0481] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[0482] Specific examples

[0483] An example of the system's operation is shown below.

[0484] Example 1: Creating a task from an email

[0485] The server receives the following email: "Dear customer, please submit your report by next Monday."

[0486] Information extracted by the server: Name (customer), request (report submission), deadline (next Monday)

[0487] Server-generated task: "Report submission" (Deadline: next Monday)

[0488] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[0489] Example 2: Changing task priorities with an emotion engine

[0490] The emotion engine analyzes the user's voice and facial expressions to recognize stress levels

[0491] The server dynamically changes the priority of tasks based on emotional information, and displays tasks that are easy to handle at the top.

[0492] Updated ToDo list notified on device: Priorities have been changed, with "Check documents" (deadline: tomorrow's meeting, light task) at the top

[0493] This system not only allows users to efficiently manage tasks and prevent oversights, but also allows them to respond flexibly according to their emotional state.

[0494] Prompt Sentence Examples

[0495] Here are some examples of prompts for generative AI models:

[0496] The user is in a stressful state and is suddenly given another task: "Customer, please prepare materials for the meeting by the end of this week." In this case, how can the system change the task priority and adjust the notification method? Please explain the specific steps.

[0497] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0498] Processing Steps

[0499] Step 1:

[0500] Data Acquisition

[0501] The server periodically retrieves new messages from the email server, chat tool server, and web chat server.

[0502] Input: New messages on email servers, chat tool servers, and web chat servers

[0503] Data processing: Retrieve messages using APIs, such as Gmail's IMAP protocol or Slack's API.

[0504] Output: A list of retrieved message data

[0505] Specific behavior:

[0506] Retrieves unread emails from the inbox using Gmail's IMAP protocol.

[0507] Use the Slack API to get the latest messages in a specific channel.

[0508] Step 2:

[0509] Message Parsing

[0510] The server analyzes the content of the received message using natural language processing (NLP) technology.

[0511] Input: List of retrieved message data

[0512] Data processing: Use an NLP library (e.g., Spacy or NLTK) to parse messages and extract names, addresses, mentions, keywords, and date information.

[0513] Output: A list of parsed information (names, keywords, date information, etc.)

[0514] Specific behavior:

[0515] Convert the phrase "next Monday" to standard format (YYYY-MM-DD).

[0516] Detects keywords such as "request," "confirmation," and "question" to determine the type of task.

[0517] Step 3:

[0518] Task summary and organization

[0519] The server summarizes the messages based on the extracted information and organizes them into meaningful tasks.

[0520] Input: A list of parsed information

[0521] Data processing: Extract important information from messages (e.g., "report submission" and "deadline: next Monday") and generate tasks.

[0522] Output: A list of generated tasks

[0523] Specific behavior:

[0524] Extract "Name," "Report submission," and "Deadline" from the message and save them as task data.

[0525] Step 4:

[0526] ToDo list generation

[0527] The server combines the generated tasks into a single ToDo list and saves it in a database.

[0528] Input: A list of generated tasks

[0529] Data processing: Sort tasks based on priority and combine them into one to-do list.

[0530] Output: Consolidated ToDo list

[0531] Specific behavior:

[0532] Calculate the importance and urgency score for each task and place them at the top of the list.

[0533] Save the integrated to-do list in a database.

[0534] Step 5:

[0535] Alert Settings

[0536] The server monitors task deadlines and sets alerts for upcoming deadlines.

[0537] Input: Integrated ToDo List

[0538] Data processing: Compare the current time with the task deadline and mark tasks with an alert if the deadline is approaching.

[0539] Output: ToDo list with alerts

[0540] Specific behavior:

[0541] Detects tasks with deadlines within two days and marks them with a red alert.

[0542] Reflect tasks with alert marks in your ToDo list.

[0543] Step 6:

[0544] Collecting Emotional Data

[0545] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[0546] Input: User's voice, facial expressions, input text

[0547] Data processing: Convert analog data into digital data and send it to the emotion engine.

[0548] Output: Emotion data sent to the emotion engine

[0549] Specific behavior:

[0550] The user's facial expressions are captured by a camera and analyzed using a facial expression analysis algorithm (such as OpenCV).

[0551] The voice recorded by the microphone is input into the emotion engine.

[0552] Step 7:

[0553] Changing task priorities based on emotions

[0554] The emotion engine recognizes the user's emotional state (e.g., stress or fatigue) and transmits it to the server, which then dynamically changes task priorities based on this information.

[0555] Input: emotion data from emotion engine, integrated to-do list

[0556] Data processing: Analyze sentiment data and reprioritize tasks.

[0557] Output: ToDo list with changed priorities

[0558] Specific behavior:

[0559] If the emotion engine determines that the user's stress level is "high," it temporarily lowers the priority of the current high-load task.

[0560] Move lighter tasks to the top of the list and notify the user.

[0561] Step 8:

[0562] Changing notification methods based on emotions

[0563] Based on the information from the emotion engine, the server adjusts the notification method.

[0564] Input: Emotion data from the emotion engine

[0565] Data processing: Emotional data is used to change the notification method (volume, display format, etc.).

[0566] Output: Adjusted notification method

[0567] Specific behavior:

[0568] If the emotion engine determines that the user is in a "concentrated state," it will mute the notification volume.

[0569] Switch to banner display for email and message notifications.

[0570] Step 9:

[0571] Task completion and updates

[0572] After the user completes a task, they mark it as complete on their device, which then syncs it with the server, and the completed task is removed from their to-do list.

[0573] Input: Task information that the user marked as complete

[0574] Data processing: Delete completed tasks from the database and update the to-do list.

[0575] Output: Updated ToDo list

[0576] Specific behavior:

[0577] When the user marks a task as completed, the device sends that information to the server.

[0578] The server removes the completed task from the database and updates the to-do list.

[0579] (Application example 2)

[0580] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0581] Conventional task management systems set task priorities without considering the user's emotional state, which can lead to inefficient task processing. Furthermore, notification methods are uniform, potentially disrupting user concentration. In crowded environments, such as brick-and-mortar stores, employees are prone to stress and fatigue, making efficient task management difficult. Under these circumstances, a flexible task management method that responds to employees' emotional state is needed.

[0582] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0583] In this invention, the server includes means for periodically retrieving new messages from an information transmission medium, means for analyzing the content of the retrieved messages and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them into tasks, means for integrating the organized tasks into a single ToDo list and displaying them with priority, means for marking or coloring tasks with an alert mark or color that are approaching their deadline, means for notifying the user of the ToDo list on their device, means for analyzing the user's emotional state using an emotion engine and dynamically changing task priorities based on the emotional state, and means for adjusting the notification method according to the user's emotional state. This enables efficient task management based on the user's emotional state, reducing employee stress and maintaining their concentration.

[0584] An "information transmission medium" is a medium that provides communication means such as an email server, a chat tool server, or a web chat server.

[0585] A "new message" is a new message obtained from a predetermined information transmission medium.

[0586] "Analysis" is the process of analyzing the content of the acquired message using natural language processing technology and extracting specified elements.

[0587] A "specified keyword" is a keyword for identifying a specific action or information from the message content.

[0588] A "ToDo list" is a list that organizes extracted tasks and allows users to manage them.

[0589] "Alert marks and colors" are visual display means for visually indicating the urgency and importance of a task.

[0590] A "terminal" is a device that is directly operated by a user, and includes a smartphone, smart glasses, etc.

[0591] "Emotional state" refers to the user's emotional level or type, such as stress, fatigue, concentration, or joy.

[0592] An "emotion engine" is a system that assesses a user's emotional state using technologies such as voice recognition, facial expression analysis, and text analysis.

[0593] "Dynamic change" refers to adjusting task priorities in real time based on the user's emotional state.

[0594] "Notification method" refers to the means by which a user is notified of a task or alert, and includes adjusting the notification sound and visual display.

[0595] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions.

[0596] Server Roles

[0597] The server plays a central role in this system and implements the following functions:

[0598] 1. Data Acquisition:

[0599] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[0600] 2. Message analysis:

[0601] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract specified keywords, the user's name, address, mentions, and deadline information, thereby understanding the meaning of the message and extracting important information.

[0602] 3. Task summary and organization:

[0603] Based on the extracted information, each message is summarized and organized into a meaningful task. For example, a task such as "Report submission" (deadline: next Monday) is generated.

[0604] 4. ToDo List Generation:

[0605] Organize your tasks into a single to-do list, stored in a database and displayed based on your priorities, which are automatically set based on the importance and urgency of the tasks.

[0606] 5. Alert Settings:

[0607] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0608] 6. Emotion engine integration:

[0609] It receives input from the emotion engine to recognize the user's current emotional state, which uses techniques such as voice recognition, facial expression analysis, and text analysis to assess emotions.

[0610] 7. Changing task priorities based on emotions:

[0611] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[0612] 8. Change notification methods based on emotions:

[0613] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[0614] Device Role

[0615] The terminal is a device that is directly operated by the user and implements the following functions:

[0616] 1. Data display:

[0617] The terminal displays the ToDo list received from the server to the user.

[0618] 2. Alert display:

[0619] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0620] 3. Collecting Emotional Data:

[0621] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[0622] 4. Completing and updating tasks:

[0623] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[0624] The role of the emotional engine

[0625] The emotion engine analyzes the user's emotional state and implements the following functions:

[0626] 1. Recognition of emotional states:

[0627] It uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[0628] 2. Data transmission:

[0629] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[0630] User Roles

[0631] The user is the ultimate beneficiary of this system and performs the following actions:

[0632] 1. Check the list:

[0633] Users can check their to-do list on their device and understand the details and deadlines of each task.

[0634] 2. Task processing:

[0635] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[0636] 3. Providing Emotion Data:

[0637] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[0638] Examples and prompts

[0639] For example, if an employee says they're "very stressed" in a busy store, the emotion engine will recognize this increased stress and move lower priority tasks higher on the list.

[0640] An example of a prompt to pass to a generative AI model:

[0641] Emotion engine emotion assessment: "Evaluate the user's emotion from this message: 'Very stressed'"

[0642] Task Management: "Here's your current task list. Reprioritize it based on the sentiment score."

[0643] This enables efficient task management based on emotional state, reducing stress and helping employees maintain focus.

[0644] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0645] Step 1:

[0646] The server periodically retrieves new messages from information transmission media. In this process, messages are collected from email servers, chat tool servers, and web chat servers using APIs. The input is message data retrieved from the APIs of each media, and the output is an unanalyzed message list stored in the server.

[0647] Step 2:

[0648] The server analyzes the contents of the received messages and extracts the specified keywords, the user's name, address, mentions, and deadline information. This process uses natural language processing (NLP). The input is the unanalyzed message data, and the output is a dataset containing the extracted important information (keywords, name, address, mentions, deadline).

[0649] Step 3:

[0650] The server summarizes the extracted information and organizes it into meaningful tasks. This process uses a summarization algorithm to convert messages into concise tasks. The input is a dataset containing important information, and the output is an organized list of tasks to display to the user.

[0651] Step 4:

[0652] The server then consolidates the organized tasks into a single to-do list and displays them prioritized. This process uses an algorithm that automatically prioritizes tasks based on their importance and urgency. The input is an organized task list, and the output is a prioritized to-do list.

[0653] Step 5:

[0654] The server assigns alert marks and colors to tasks with upcoming deadlines. This process uses logic that assigns a red alert mark to tasks with due dates within two days. The input is a to-do list, and the output is an updated to-do list with alert marks and colors.

[0655] Step 6:

[0656] The terminal displays the ToDo list received from the server to the user. This process visually displays the list using a UI (user interface). The input is a prioritized ToDo list, and the output is a task list displayed to the user.

[0657] Step 7:

[0658] The device collects emotion data from the user's voice, facial expressions, input text, etc., and sends it to the emotion engine. This process uses various sensors and analysis technologies to obtain emotion data. The input is the user's voice, facial expressions, and text data, and the output is emotion data sent to the emotion engine.

[0659] Step 8:

[0660] The emotion engine analyzes the user's emotional state and sends data to the server based on the emotional state. This process uses speech recognition, facial expression analysis, and text analysis to evaluate emotions. The input is emotion data, and the output is analyzed emotional state data.

[0661] Step 9:

[0662] The server receives input from the emotion engine and dynamically reprioritizes tasks based on the emotional state. This process uses an algorithm that reprioritizes tasks based on the emotional state score. The input is the parsed emotional state data, and the output is an updated to-do list that has been reprioritized based on the emotion.

[0663] Step 10:

[0664] The server adjusts the notification method according to the user's emotional state. This process involves making adjustments such as reducing the notification sound so as not to disturb the user's concentration. The input is the emotional state data, and the output is the adjusted notification method.

[0665] The following are examples of prompts that can be passed to a generative AI model:

[0666] Emotion engine emotion assessment: "Evaluate the user's emotion from this message: 'Very stressed'"

[0667] Task Management: "Here's your current task list. Reprioritize it based on the sentiment score."

[0668] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0669] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0670] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0671] [Second embodiment]

[0672] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0673] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0674] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0675] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0676] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0677] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0678] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0679] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0680] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0681] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0682] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0683] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0684] The system for implementing this invention efficiently manages tasks through the mutual cooperation of three elements: a server, a terminal, and a user. The specific roles of each element and the processing contents of the program are explained below.

[0685] Server Roles

[0686] The server plays a central role in this system and implements the following functions:

[0687] 1. Data Acquisition

[0688] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[0689] 2. Message Analysis

[0690] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract your name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as time and date information, which is then converted into a standard format.

[0691] 3. Task summary and organization

[0692] The server then summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, a sentence in an email saying "Yamada-san, please submit your report by next Monday" will generate a task called "Submit report" (deadline: next Monday).

[0693] 4. ToDo list generation

[0694] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[0695] 5. Alert Settings

[0696] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0697] 6. Notification function

[0698] The server notifies the user of the created to-do list via email or a dedicated notification API.

[0699] Device Role

[0700] The terminal is a device that is directly operated by the user and implements the following functions:

[0701] 1. Data display

[0702] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[0703] 2. Alert display

[0704] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0705] 3. Completing and updating tasks

[0706] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[0707] User Roles

[0708] The user is the ultimate beneficiary of this system and performs the following actions:

[0709] 1. Check the list

[0710] Users can check their to-do list on their device and understand the details and deadlines of each task.

[0711] 2. Task Processing

[0712] The user can then work on the displayed tasks in order of priority. After completing a task, they mark it as completed on their device and update the list.

[0713] Specific examples

[0714] An example of the system's operation is shown below.

[0715] Example 1: Creating a task from an email

[0716] Email content received by the server: "Yamada-san, please submit your report by next Monday. Thank you."

[0717] Information extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[0718] Server-generated task: "Report submission" (Deadline: next Monday)

[0719] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[0720] Example 2: Creating a task from Google Chat

[0721] Chat contents retrieved by the server: "@Yamada-san, please check the materials before tomorrow's meeting."

[0722] Information extracted by the server: mention (@Yamada), request (document confirmation), deadline (tomorrow's meeting)

[0723] Server-generated task: "Check materials" (Deadline: tomorrow's meeting)

[0724] To-do list notified to the device: "Check materials" (deadline: tomorrow's meeting), with attention alerts

[0725] This system allows users to efficiently manage tasks and prevent oversight.

[0726] The processing flow will be explained below.

[0727] Step 1:

[0728] The server periodically (for example, every 5 minutes) launches a task and accesses an information transmission medium (email server, chat tool server, web chat server).

[0729] Specifically, it logs in to the mail server using the IMAP or POP3 protocol, and retrieves new messages from the chat tool using a dedicated API.

[0730] Step 2:

[0731] The server analyzes new messages received from each information carrier.

[0732] The analysis uses natural language processing (NLP) technology to extract information such as the message body, the date and time of sending, and the sender, with particular attention paid to your name, address, mentions, and designated keywords (such as "request," "confirmation," and "question").

[0733] Step 3:

[0734] The server generates a summarized task content based on the information extracted from the parsed message.

[0735] For example, from an email that says "Yamada, please submit your report by next Monday," you can create a task called "Report submission" (deadline: next Monday). The date and time information is also converted to the standard format (YYYY-MM-DD).

[0736] Step 4:

[0737] The server stores the generated tasks in a database and integrates the to-do list.

[0738] Tasks are saved in a database, taking care to avoid duplication. Saved tasks are organized by user and managed centrally.

[0739] Step 5:

[0740] The server organizes the tasks on the to-do list based on priority.

[0741] The order of the list is determined by the urgency of the task and its deadline. Tasks with upcoming deadlines are given high priority.

[0742] Step 6:

[0743] The server monitors the deadlines of tasks and displays alert marks and colors on tasks whose deadlines are approaching.

[0744] For example, tasks with a deadline within two days will be marked with a red alert mark.

[0745] Step 7:

[0746] The server notifies the user's device of the generated ToDo list.

[0747] Delivery to devices is via email, dedicated app notification APIs, and web app update functions.

[0748] Step 8:

[0749] The terminal displays the ToDo list received from the server to the user.

[0750] The list is displayed in a table or calendar format, with tasks with alerts specifically highlighted.

[0751] Step 9:

[0752] Users can check their to-do list on their device and process tasks based on their content and priority.

[0753] When the task is complete, the user marks the task as complete on the terminal.

[0754] Step 10:

[0755] The terminal synchronizes the information of the user's task completion with the server.

[0756] The server removes the completed tasks from the database and updates the current to-do list.

[0757] The above is the specific processing flow of this system. This system efficiently manages each task in a centralized manner, preventing users from overlooking something and enabling them to respond quickly.

[0758] Example 1

[0759] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0760] In conventional task management systems, the process of effectively retrieving new messages from information transmission media and organizing them into tasks is cumbersome and not fully automated. Furthermore, the system lacks the ability to prioritize tasks based on their importance and deadlines, increasing the risk of users overlooking tasks. Furthermore, the system lacks alert functions for task deadlines and update processes after task completion, making it difficult for users to efficiently manage tasks.

[0761] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0762] In this invention, the server includes: means for periodically acquiring new data from an information transmission medium; means for analyzing the acquired data and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information; means for summarizing the extracted information and organizing it into tasks; means for integrating the organized tasks into a single to-do list and automatically prioritizing and displaying them; means for marking or coloring tasks with an alert mark if the deadline is approaching; means for monitoring task deadlines and notifying users of tasks with an approaching deadline; means for notifying the user of the to-do list on the user's device; and means for synchronizing the information with the server and updating the to-do list when the user marks a task as completed. This improves task management automation, allowing users to efficiently manage tasks without missing any tasks.

[0763] An "information transmission medium" is a communication means for exchanging information, such as an email server, a chat tool server, or a web chat server.

[0764] "Periodic acquisition" means that data is automatically acquired at preset time intervals.

[0765] "Analyzing" means understanding the content of the acquired data and classifying and extracting information according to certain rules.

[0766] "Specified keywords" are words that indicate specific conditions or content, and are set so that the system can extract them preferentially.

[0767] "Extract" means to extract specific information from within a message.

[0768] "Summarizing" means to briefly summarize extracted information.

[0769] A "task" is a specific task or action that a user must perform.

[0770] "Organizing" means arranging summarized information systematically and making it easy to manage.

[0771] A "ToDo list" is a list that displays tasks that need to be completed in a list format.

[0772] "Integrating" means combining multiple tasks into one list.

[0773] "Priority" is a criterion for determining the display order and execution order based on the importance and urgency of tasks.

[0774] A "deadline" is the deadline for completing a task.

[0775] An "alert mark" is a sign or icon that calls attention to specific information.

[0776] "Coloring" means adding color to information to draw visual attention.

[0777] "Notifying" means informing the user of information.

[0778] "Synchronizing" means keeping data consistent across different devices and systems.

[0779] "To update" means to bring existing information or data up to date.

[0780] The system for implementing this invention efficiently manages tasks through collaboration between the server, the terminal, and the user. The specific roles of each element and the processing contents of the program are explained below.

[0781] Server Roles

[0782] The server plays a central role in the system and implements the following functions:

[0783] 1. Get new messages

[0784] The server periodically retrieves new messages from the information transmission medium (e.g., email server, chat tool server, web chat server) using APIs and protocols (IMAP, POP3) specific to each medium. For example, the server retrieves new emails from the email server using the IMAP protocol.

[0785] 2. Message Content Analysis

[0786] The server analyzes the received message using natural language processing technology (for example, Google Cloud Natural Language API or spaCy) and extracts the specified keywords, the user's name, address, mentions, and deadline information. For example, from an email that says "Yamada-san, please submit your report by next Monday," the server extracts "Name: Yamada," "Request: Report submission," and "Deadline: next Monday."

[0787] 3. Summarize and organize your tasks

[0788] The server summarizes the message based on the extracted information and organizes it into meaningful tasks, such as generating a task called "Report Submission" (Deadline: Next Monday).

[0789] 4. Creating and updating a to-do list

[0790] The server consolidates the organized tasks into a single to-do list and stores it in a database, where task priorities are automatically set and adjusted based on deadlines and importance.

[0791] 5. Setting up alerts

[0792] The server monitors task deadlines and assigns alert marks and colors to tasks that are approaching their deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0793] 6. Notification function

[0794] The server notifies the user of the created to-do list via email or a dedicated notification API. For example, the server uses the Gmail API to send the new to-do list to the user's Gmail account via email.

[0795] Device Role

[0796] A terminal is a device that is directly operated by a user and has the following functions:

[0797] 1. Data display

[0798] The device receives the to-do list from the server and displays it to the user. For example, the list can be presented visually using a web app or mobile app.

[0799] 2. Alert display

[0800] The device displays tasks with alert marks and color coding to indicate which tasks the user should prioritize. Tasks with red marks are displayed at the top of the list.

[0801] 3. Completing and updating tasks

[0802] When a user completes a task, the device synchronizes the information with the server and updates the to-do list. The completed task is removed from the list. For example, when a user marks the "Submit Report" task as completed, the device sends the information to the server and the list is updated.

[0803] User Roles

[0804] The user is the ultimate beneficiary of this system and performs the following operations:

[0805] 1. Check the list

[0806] The user checks the to-do list on the device and understands the content and deadline of each task. For example, the user checks the task "Check materials" (deadline: tomorrow's meeting).

[0807] 2. Task Processing

[0808] The user will then work on the displayed tasks in order of priority. After completing a task, they will mark it as completed on their device and update the list. For example, the user will work on the "Document Review" task and mark it as completed.

[0809] Example operation

[0810] 1. Creating a task from email

[0811] The server retrieves the email from the email server using the IMAP protocol.

[0812] Email content: "Yamada-san, please submit your report by next Monday."

[0813] Information analyzed and extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[0814] Server-generated task: "Report submission" (Deadline: next Monday)

[0815] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[0816] When a user marks a task as complete, the device sends the information to the server and the task is removed from the list.

[0817] 2. Task creation from chat tools

[0818] The server uses the API to retrieve messages from the chat tool server.

[0819] Message: "@Yamada-san, please check the materials before tomorrow's meeting."

[0820] Information analyzed and extracted by the server: mention (@Yamada), request (document confirmation), deadline (tomorrow's meeting)

[0821] Server-generated task: "Check materials" (Deadline: tomorrow's meeting)

[0822] To-do list notified to the device: "Check materials" (deadline: tomorrow's meeting), with alert mark

[0823] When a user marks a task as complete, the device sends the information to the server and the task is removed from the list.

[0824] This allows users to efficiently manage tasks and prevent oversight.

[0825] Generative AI model used and example prompts

[0826] Generative AI model: OpenAI GPT-3

[0827] Example prompt

[0828] Please create a task from the email below.

[0829] Email content: Yamada-san, please submit your report by next Monday.

[0830] Expected output: Task "Report submission" (Deadline: next Monday)

[0831] Using these prompts, the AI ​​model extracts the specified information and generates a task.

[0832] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0833] Step 1:

[0834] Get new messages

[0835] Input: Information transmission medium (email server, chat tool server, web chat server)

[0836] The server periodically retrieves new messages using the API or protocol of each medium (IMAP, POP3).

[0837] Specific operation: The server retrieves new mail from the email server using the IMAP protocol.

[0838] Output: Retrieved message data

[0839] Step 2:

[0840] Message content analysis

[0841] Input: Retrieved message data

[0842] The server uses natural language processing technology (e.g., Google Cloud Natural Language API or spaCy) to analyze the content of the message and extract specified keywords, your name, address, mentions, and deadline date and time information.

[0843] Specific operation: The server extracts "Name: Yamada", "Request: Report submission", and "Deadline: Next Monday" from an email that says "Yamada-san, please submit your report by next Monday."

[0844] Output: Extracted information (name, request details, deadline, etc.)

[0845] Step 3:

[0846] Summarizing and organizing tasks

[0847] Input: Extracted information

[0848] The server uses the extracted information to summarize the messages and organize them into meaningful tasks.

[0849] Specific operation: The server generates the extracted task "Report submission" (deadline: next Monday).

[0850] Output: Organized tasks

[0851] Step 4:

[0852] Creating and updating to-do lists

[0853] Input: Organized tasks

[0854] The server consolidates the organized tasks into a single to-do list and stores it in a database, with task priorities automatically set.

[0855] Specific operation: The server integrates a new task "Report submission" (deadline: next Monday) into the ToDo list.

[0856] Output: ToDo list

[0857] Step 5:

[0858] Configuring Alerts

[0859] Input: Tasks in your ToDo list

[0860] The server monitors task deadlines and displays alert marks and colors for tasks whose deadlines are approaching.

[0861] Specific behavior: The server marks tasks with deadlines within two days with a red alert mark.

[0862] Output: ToDo list with alerts

[0863] Step 6:

[0864] Notification function

[0865] Input: To-do list with alerts

[0866] The server notifies the user of the generated to-do list via email or a dedicated notification API.

[0867] What happens: The server uses the Gmail API to email the new to-do list to the user's Gmail account.

[0868] Output: ToDo list notified to the user's device

[0869] Step 7:

[0870] Data Display

[0871] Input: ToDo list notified to the user's device

[0872] The terminal displays the received ToDo list to the user.

[0873] Specific operation: The device uses a web app or mobile app to display the task "Submit report" (deadline: next Monday).

[0874] Output: Displayed ToDo list

[0875] Step 8:

[0876] Alert display

[0877] Input: To-do list on device

[0878] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0879] What it does: The device will display tasks marked in red at the top of the list.

[0880] Output: Highlighted tasks

[0881] Step 9:

[0882] Completing and updating tasks

[0883] Input: Tasks marked as completed by user action

[0884] When a user completes a task, the device syncs the information with the server, updates the to-do list, and removes the completed task from the list.

[0885] Specific behavior: The user marks the "report submission" task as completed, the device sends the information to the server, and the list is updated.

[0886] Output: Updated ToDo list

[0887] This allows users to efficiently manage tasks and prevent oversight.

[0888] (Application example 1)

[0889] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0890] Conventional task management systems relied on email and chat messages, often requiring manual input from humans. Furthermore, they lacked a mechanism for efficiently managing error messages and operational status information from industrial robots. This resulted in the time-consuming task of responding to errors and managing tasks, resulting in reduced productivity.

[0891] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0892] In this invention, the server includes means for periodically acquiring new messages from an information transmission medium, means for analyzing the contents of the acquired messages and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them as tasks, means for integrating the organized tasks into a single ToDo list and displaying them in order of priority, means for marking or coloring tasks with an alert mark or color, means for analyzing messages from industrial robots and adding abnormality messages and operating statuses as tasks, and means for notifying the user of the ToDo list on their terminal. This makes it possible to improve the efficiency of task management and prevent oversights.

[0893] An "information transmission medium" is a communication means such as an email server, a chat tool server, or a web chat server, and is a medium for exchanging messages.

[0894] A "new message" is a message that is received for the first time over the communication medium and has not yet been processed.

[0895] A "specified keyword" is a specific word or phrase that is set in advance by the user and is used for automatic task extraction.

[0896] "My Name" refers to the user's own name or identification information, and is used to extract related tasks from the message content.

[0897] An "address" is an email address or a user's identifier in a chat tool.

[0898] "Mention" is a mechanism for notifying users who are specifically mentioned in chat tools or messages, and is implemented using the @ symbol, etc.

[0899] "Deadline date and time information" refers to a specific date and time set as the deadline for completing a task, and is used to make the user aware of the urgency of the task.

[0900] A "task" is an activity or work performed to achieve a specific purpose or goal.

[0901] A "ToDo list" is an organized collection of tasks that allows a user to view all the tasks that need to be done.

[0902] "Alert marks and colors" are symbols and colors that visually emphasize the urgency and importance of a task, and serve to draw the user's attention.

[0903] An "industrial robot" is an automated machine used in industrial fields such as manufacturing, and is used to efficiently perform specific tasks or processes.

[0904] An "abnormality message" is a message that an industrial robot sends when it detects some kind of error or abnormality.

[0905] "Operation status" refers to information that indicates the current operating state and work progress of an industrial robot.

[0906] The system for implementing this invention involves three elements, a server, a terminal, and a user, working together to efficiently manage tasks, including the operational status and error messages of industrial robots. The specific roles of each element and the processing contents of the program are explained below.

[0907] Server Roles

[0908] The server plays a central role in this system and implements the following functions:

[0909] 1. Data Acquisition

[0910] The server periodically retrieves new messages from information transmission media (email server, chat tool server, web chat server). This uses the API of each media. For example, IMAP or POP3 protocols are used to retrieve emails from the mail server, and a dedicated API is used to retrieve messages from the chat tool. In addition, data from industrial robots is also retrieved using a dedicated communication protocol.

[0911] 2. Message Analysis

[0912] The server analyzes the contents of the received messages using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"). Information about the time and date is also extracted and converted into a standard format. Furthermore, the server analyzes abnormal messages and operating status from industrial robots and recognizes them as tasks.

[0913] 3. Task summary and organization

[0914] The server summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, a sentence in an email saying, "Person in charge, please check the machine's operation by next Monday," generates a task called "Check the machine's operation" (deadline: next Monday). Similarly, a message from a robot saying, "Error code 1234: An abnormal operation has occurred," generates a task called "Error response" (immediate response).

[0915] 4. ToDo list generation

[0916] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[0917] 5. Alert Settings

[0918] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0919] 6. Notification function

[0920] The server notifies the user of the created to-do list via email or a dedicated notification API.

[0921] Device Role

[0922] The terminal is a device that is directly operated by the user and implements the following functions:

[0923] 1. Data display

[0924] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[0925] 2. Alert display

[0926] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0927] 3. Completing and updating tasks

[0928] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[0929] User Roles

[0930] The user is the ultimate beneficiary of this system and performs the following actions:

[0931] 1. Check the list

[0932] Users can check their to-do list on their device and understand the details and deadlines of each task.

[0933] 2. Task Processing

[0934] The user can then work on the displayed tasks in order of priority. After completing a task, they mark it as completed on their device and update the list.

[0935] Specific examples

[0936] An example of the system's operation is shown below.

[0937] Example 1: Creating a task from an email

[0938] Email content received by the server: "To the person in charge, please check that the machine is working properly by next Monday. Thank you."

[0939] Information extracted by the server: Name (person in charge), request details (machine operation check), deadline (next Monday)

[0940] Server-generated task: "Check machine operation" (Deadline: next Monday)

[0941] To-do list displayed on the device: "Check machine operation" (Deadline: next Monday)

[0942] Example 2: Task generation from an industrial robot

[0943] Message received from the server: "Error code 1234: Abnormal operation"

[0944] Information extracted by the server: Abnormality details (operational abnormality), priority (high)

[0945] Server-generated tasks: "Error handling" (immediate response)

[0946] To-do list notified on the device: "Error handling" (immediate response)

[0947] Prompt Sentence Examples

[0948] text

[0949] The body of the email should follow the format below. Please extract the task details and deadline. For example:

[0950] Email body: "Person in charge, please check the operation by next Monday."

[0951] Task: "Check operation"

[0952] Due: "Next Monday"

[0953] This system allows users to efficiently manage tasks and respond to abnormalities in industrial robots in real time.

[0954] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0955] Step 1:

[0956] The server periodically retrieves new messages from the information carrier. This includes retrieving emails using the IMAP and POP3 protocols, and retrieving messages from chat tools using a dedicated API. The input is the new messages, and the output is the retrieved raw data. This data is used in the subsequent analysis steps.

[0957] Step 2:

[0958] The server analyzes the acquired raw data using natural language processing (NLP) technology. This analysis extracts the user's name, address, mentions, specified keywords (such as "request," "confirmation," or "question"), and deadline date and time information. It also analyzes abnormality messages and operating status from the industrial robot to extract information about the abnormality and operating status. The input is raw data, and the output is extracted metadata.

[0959] Step 3:

[0960] The server summarizes each message based on the extracted metadata and organizes it into meaningful tasks. For example, a sentence in an email saying, "Person in charge, please check the machine's operation by next Monday," generates a task called "Check the machine's operation" (deadline: next Monday). Similarly, a message from a robot saying, "Error code 1234: An abnormal operation has occurred," generates a task called "Error response" (immediate response). The input is metadata, and the output is the generated task.

[0961] Step 4:

[0962] The server consolidates the organized tasks into a single to-do list. This list is stored in a database and displayed based on task priority. Priorities are automatically set based on the importance and urgency of the tasks. The input is the generated tasks, and the output is the consolidated to-do list.

[0963] Step 5:

[0964] The server monitors the deadlines of tasks in the to-do list and assigns alert marks and colors to tasks with upcoming deadlines. For example, a red alert mark is assigned to a task with a deadline within two days. The input is a to-do list, and the output is a visually emphasized to-do list.

[0965] Step 6:

[0966] The server notifies the user of the generated ToDo list via email or a dedicated notification API. The input is a visually enhanced ToDo list, and the output is the notified ToDo list.

[0967] Step 7:

[0968] The terminal displays the ToDo list received from the server to the user. The user manages tasks based on this list. The input is the notified ToDo list, and the output is the list displayed on the terminal.

[0969] Step 8:

[0970] The terminal highlights tasks with alert marks and colors to indicate which tasks the user should prioritize. The input is a highlighted to-do list, and the output is the priority tasks recognized by the user.

[0971] Step 9:

[0972] When a user completes a task, the device synchronizes the information with the server and updates the to-do list. At the same time, the completed task is removed from the list. The input is the task that was notified as completed, and the output is the updated to-do list.

[0973] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0974] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions. The specific roles of each element and the processing content of the program are explained below.

[0975] Server Roles

[0976] The server plays a central role in this system and implements the following functions:

[0977] 1. Data Acquisition

[0978] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[0979] 2. Message Analysis

[0980] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as information about the time and date, and converts it into a standard format.

[0981] 3. Task summary and organization

[0982] The server then summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, an email message saying "Yamada-san, please submit your report by next Monday" generates a task called "Submit report" (deadline: next Monday).

[0983] 4. ToDo list generation

[0984] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[0985] 5. Alert Settings

[0986] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[0987] 6. Emotion engine integration

[0988] The server receives input from the emotion engine to recognize the user's current emotional state, which evaluates emotions using techniques such as voice recognition, facial expression analysis, and text analysis.

[0989] 7. Changing task priorities based on emotions

[0990] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[0991] 8. Change notification methods based on emotions

[0992] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[0993] Device Role

[0994] The terminal is a device that is directly operated by the user and implements the following functions:

[0995] 1. Data display

[0996] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[0997] 2. Alert display

[0998] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[0999] 3. Collecting Emotional Data

[1000] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[1001] 4. Completing and updating tasks

[1002] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[1003] The role of the emotional engine

[1004] The emotion engine analyzes the user's emotional state and implements the following functions:

[1005] 1. Recognizing emotional states

[1006] The emotion engine uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[1007] 2. Data transmission

[1008] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[1009] User Roles

[1010] The user is the ultimate beneficiary of this system and performs the following actions:

[1011] 1. Check the list

[1012] Users can check their to-do list on their device and understand the details and deadlines of each task.

[1013] 2. Task Processing

[1014] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[1015] 3. Providing Emotion Data

[1016] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[1017] Specific examples

[1018] An example of the system's operation is shown below.

[1019] Example 1: Creating a task from an email

[1020] Email content received by the server: "Yamada-san, please submit your report by next Monday. Thank you."

[1021] Information extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[1022] Server-generated task: "Report submission" (Deadline: next Monday)

[1023] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[1024] Example 2: Changing task priorities with an emotion engine

[1025] The emotion engine analyzes the user's voice and facial expressions to recognize stress levels

[1026] The server dynamically changes the priority of tasks based on emotional information, and displays tasks that are easy to handle at the top.

[1027] Updated ToDo list notified on device: Priorities have been changed, with "Check documents" (deadline: tomorrow's meeting, light task) at the top

[1028] This system not only allows users to efficiently manage tasks and prevent oversights, but also allows them to respond flexibly according to their emotional state.

[1029] The processing flow will be explained below.

[1030] Step 1:

[1031] The server periodically (for example, every 5 minutes) launches a task and accesses an information transmission medium (email server, chat tool server, web chat server).

[1032] Specifically, it logs in to the mail server using the IMAP or POP3 protocol, and retrieves new messages from the chat tool using a dedicated API.

[1033] Step 2:

[1034] The server analyzes new messages received from each information carrier.

[1035] The analysis uses natural language processing (NLP) technology to extract information such as the message body, the date and time of sending, and the sender, with particular attention paid to your name, address, mentions, and designated keywords (such as "request," "confirmation," and "question").

[1036] Step 3:

[1037] The server generates a summarized task content based on the information extracted from the parsed message.

[1038] For example, from an email that says, "Yamada, please submit your report by next Monday," you can extract the name "Yamada," the request to "submit the report," and the deadline "next Monday," and create a task called "Submit report" (deadline: next Monday).

[1039] Step 4:

[1040] The server stores the generated tasks in a database and integrates the to-do list.

[1041] Tasks are saved in a database, taking care to avoid duplication. Saved tasks are organized by user and managed centrally.

[1042] Step 5:

[1043] The server organizes the tasks on the to-do list based on priority.

[1044] The order of the list is determined by the urgency of the task and its deadline. Tasks with upcoming deadlines are given high priority.

[1045] Step 6:

[1046] The server monitors the deadlines of tasks and displays alert marks and colors on tasks whose deadlines are approaching.

[1047] For example, tasks with a deadline within two days will be marked with a red alert mark.

[1048] Step 7:

[1049] The emotion engine collects data such as the user's voice, facial expressions, and input text to analyze the user's emotional state.

[1050] Using voice analysis and facial expression analysis technology, the system assesses the user's stress level, fatigue state, concentration, etc.

[1051] Step 8:

[1052] Based on the emotion data from the emotion engine, the server dynamically changes the priority of tasks.

[1053] For example, if the emotion engine detects that the user is in a high stress state, the server will prioritize and display relatively light tasks.

[1054] Step 9:

[1055] Based on the emotion data from the emotion engine, the server adjusts how the task is notified.

[1056] For example, if the server detects that the user is concentrating, it will make adjustments such as reducing notification sounds and reducing the number of pop-up notifications.

[1057] Step 10:

[1058] The server notifies the user's device of the generated ToDo list.

[1059] Delivery to devices is via email, dedicated app notification APIs, and web app update functions.

[1060] Step 11:

[1061] The terminal displays the ToDo list received from the server to the user.

[1062] The list is displayed in a table or calendar format, with tasks with alerts highlighted, and the list view is based on an emotion engine.

[1063] Step 12:

[1064] Users can check their to-do list on their device and process tasks based on their content and priority.

[1065] When the task is complete, the user marks the task as complete on the terminal.

[1066] Step 13:

[1067] The terminal synchronizes the information of the user's task completion with the server.

[1068] The server removes the completed tasks from the database and updates the current to-do list.

[1069] The above is the specific processing flow of this system. By combining it with an emotion engine, flexible task management that responds to the user's emotional state is realized, enabling efficient work execution.

[1070] Example 2

[1071] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1072] In modern society, with the massive volume of messages received daily from various information transmission media, it is difficult for users to efficiently manage their tasks. Simply listing tasks without considering the user's emotions and state increases stress and burden, and reduces efficiency. Furthermore, there is a lack of methods for properly communicating task deadlines and importance. This leads to problems such as users overlooking tasks or failing to respond appropriately to important tasks.

[1073] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for periodically acquiring new messages from an information transmission medium, means for analyzing the content of the acquired messages and extracting specified keywords, the user's name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them as tasks, means for integrating the organized tasks into a single to-do list and displaying them with priority, means for marking or coloring tasks with an alert mark when the deadline is approaching, means for collecting user emotion data from the terminal and transmitting it to the emotion engine, means for receiving input from the emotion engine and dynamically changing task priorities in consideration of the user's emotional state, means for adjusting the notification method based on the input from the emotion engine, and means for notifying the user of the to-do list on the terminal. This allows the user to efficiently manage tasks and flexibly respond to their emotional state.

[1074] "Information transmission medium" refers to systems and services for transmitting information and sending and receiving messages and data, such as email servers, chat tool servers, and web chat servers.

[1075] A "new message" refers to a message that is received for the first time through an information transmission medium and has not yet been processed by a user.

[1076] "Analysis" refers to the process of analyzing the content of captured messages and extracting specific information or patterns.

[1077] "Keywords" are words or phrases with a specific meaning or purpose that are used to extract important information from the content of a message.

[1078] An "emotion engine" is a system that recognizes the user's emotional state and collects and analyzes data from voice, facial expressions, input text, etc.

[1079] A "task" refers to a specific action or task that a user needs to perform, and is organized and managed as a to-do list.

[1080] A "ToDo list" is a list that lists a user's tasks and indicates information such as their priority and deadlines.

[1081] An "alert mark" refers to a mark or color that is displayed to warn or draw attention to a specific situation or task that is approaching a deadline.

[1082] "Priority" refers to a ranking that determines which tasks should be processed first based on the importance and urgency of the tasks.

[1083] "Emotional state" refers to the user's current psychological state or feelings, and includes specific sensations and emotions such as stress, fatigue, and concentration.

[1084] "Notification method" refers to the means or format for notifying the user of a task or alert, and includes audio, visual, banner display, etc.

[1085] "Device" refers to a device that is directly operated by a user, including, for example, a computer, a smartphone, or a tablet.

[1086] "Integration" refers to the process of combining multiple elements or pieces of information into one system or list.

[1087] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions. The specific roles of each element and the processing content of the program are explained below.

[1088] Server Roles

[1089] 1. Data Acquisition

[1090] The server periodically retrieves new messages from the email server, chat tool server, and web chat server, using the API of each medium. For example, IMAP or POP3 protocols are used to retrieve emails from the mail server, and a dedicated API is used to retrieve messages from the chat tool.

[1091] 2. Message Analysis

[1092] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as information about the time and date, and converts it into a standard format.

[1093] 3. Task summary and organization

[1094] The server summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, an email message saying "Customer, please submit the report by next Monday" generates a task called "Submit report" (deadline: next Monday).

[1095] 4. ToDo list generation

[1096] The server consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[1097] 5. Alert Settings

[1098] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1099] 6. Emotion engine integration

[1100] The server receives input from the emotion engine to recognize the user's current emotional state, which evaluates emotions using techniques such as voice recognition, facial expression analysis, and text analysis.

[1101] 7. Changing task priorities based on emotions

[1102] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[1103] 8. Change notification methods based on emotions

[1104] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[1105] Device Role

[1106] The terminal is a device that is directly operated by the user and implements the following functions:

[1107] 1. Data display

[1108] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[1109] 2. Alert display

[1110] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[1111] 3. Collecting Emotional Data

[1112] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[1113] 4. Completing and updating tasks

[1114] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[1115] The role of the emotional engine

[1116] The emotion engine analyzes the user's emotional state and implements the following functions:

[1117] 1. Recognizing emotional states

[1118] The emotion engine uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[1119] 2. Data transmission

[1120] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[1121] User Roles

[1122] The user is the ultimate beneficiary of this system and performs the following actions:

[1123] 1. Check the list

[1124] Users can check their to-do list on their device and understand the details and deadlines of each task.

[1125] 2. Task Processing

[1126] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[1127] 3. Providing Emotion Data

[1128] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[1129] Specific examples

[1130] An example of the system's operation is shown below.

[1131] Example 1: Creating a task from an email

[1132] The server receives the following email: "Dear customer, please submit your report by next Monday."

[1133] Information extracted by the server: Name (customer), request (report submission), deadline (next Monday)

[1134] Server-generated task: "Report submission" (Deadline: next Monday)

[1135] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[1136] Example 2: Changing task priorities with an emotion engine

[1137] The emotion engine analyzes the user's voice and facial expressions to recognize stress levels

[1138] The server dynamically changes the priority of tasks based on emotional information, and displays tasks that are easy to handle at the top.

[1139] Updated ToDo list notified on device: Priorities have been changed, with "Check documents" (deadline: tomorrow's meeting, light task) at the top

[1140] This system not only allows users to efficiently manage tasks and prevent oversights, but also allows them to respond flexibly according to their emotional state.

[1141] Prompt Sentence Examples

[1142] Here are some examples of prompts for generative AI models:

[1143] The user is in a stressful state and is suddenly given another task: "Customer, please prepare materials for the meeting by the end of this week." In this case, how can the system change the task priority and adjust the notification method? Please explain the specific steps.

[1144] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1145] Processing Steps

[1146] Step 1:

[1147] Data Acquisition

[1148] The server periodically retrieves new messages from the email server, chat tool server, and web chat server.

[1149] Input: New messages on email servers, chat tool servers, and web chat servers

[1150] Data processing: Retrieve messages using APIs, such as Gmail's IMAP protocol or Slack's API.

[1151] Output: A list of retrieved message data

[1152] Specific behavior:

[1153] Retrieves unread emails from the inbox using Gmail's IMAP protocol.

[1154] Use the Slack API to get the latest messages in a specific channel.

[1155] Step 2:

[1156] Message Parsing

[1157] The server analyzes the content of the received message using natural language processing (NLP) technology.

[1158] Input: List of retrieved message data

[1159] Data processing: Use an NLP library (e.g., Spacy or NLTK) to parse messages and extract names, addresses, mentions, keywords, and date information.

[1160] Output: A list of parsed information (names, keywords, date information, etc.)

[1161] Specific behavior:

[1162] Convert the phrase "next Monday" to standard format (YYYY-MM-DD).

[1163] Detects keywords such as "request," "confirmation," and "question" to determine the type of task.

[1164] Step 3:

[1165] Task summary and organization

[1166] The server summarizes the messages based on the extracted information and organizes them into meaningful tasks.

[1167] Input: A list of parsed information

[1168] Data processing: Extract important information from messages (e.g., "report submission" and "deadline: next Monday") and generate tasks.

[1169] Output: A list of generated tasks

[1170] Specific behavior:

[1171] Extract "Name," "Report submission," and "Deadline" from the message and save them as task data.

[1172] Step 4:

[1173] ToDo list generation

[1174] The server combines the generated tasks into a single ToDo list and saves it in a database.

[1175] Input: A list of generated tasks

[1176] Data processing: Sort tasks based on priority and combine them into one to-do list.

[1177] Output: Consolidated ToDo list

[1178] Specific behavior:

[1179] Calculate the importance and urgency score for each task and place them at the top of the list.

[1180] Save the integrated to-do list in a database.

[1181] Step 5:

[1182] Alert Settings

[1183] The server monitors task deadlines and sets alerts for upcoming deadlines.

[1184] Input: Integrated ToDo List

[1185] Data processing: Compare the current time with the task deadline and mark tasks with an alert if the deadline is approaching.

[1186] Output: ToDo list with alerts

[1187] Specific behavior:

[1188] Detects tasks with deadlines within two days and marks them with a red alert.

[1189] Reflect tasks with alert marks in your ToDo list.

[1190] Step 6:

[1191] Collecting Emotional Data

[1192] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[1193] Input: User's voice, facial expressions, input text

[1194] Data processing: Convert analog data into digital data and send it to the emotion engine.

[1195] Output: Emotion data sent to the emotion engine

[1196] Specific behavior:

[1197] The user's facial expressions are captured by a camera and analyzed using a facial expression analysis algorithm (such as OpenCV).

[1198] The voice recorded by the microphone is input into the emotion engine.

[1199] Step 7:

[1200] Changing task priorities based on emotions

[1201] The emotion engine recognizes the user's emotional state (e.g., stress or fatigue) and transmits it to the server, which then dynamically changes task priorities based on this information.

[1202] Input: emotion data from emotion engine, integrated to-do list

[1203] Data processing: Analyze sentiment data and reprioritize tasks.

[1204] Output: ToDo list with changed priorities

[1205] Specific behavior:

[1206] If the emotion engine determines that the user's stress level is "high," it temporarily lowers the priority of the current high-load task.

[1207] Move lighter tasks to the top of the list and notify the user.

[1208] Step 8:

[1209] Changing notification methods based on emotions

[1210] Based on the information from the emotion engine, the server adjusts the notification method.

[1211] Input: Emotion data from the emotion engine

[1212] Data processing: Emotional data is used to change the notification method (volume, display format, etc.).

[1213] Output: Adjusted notification method

[1214] Specific behavior:

[1215] If the emotion engine determines that the user is in a "concentrated state," it will mute the notification volume.

[1216] Switch to banner display for email and message notifications.

[1217] Step 9:

[1218] Task completion and updates

[1219] After the user completes a task, they mark it as complete on their device, which then syncs it with the server, and the completed task is removed from their to-do list.

[1220] Input: Task information that the user marked as complete

[1221] Data processing: Delete completed tasks from the database and update the to-do list.

[1222] Output: Updated ToDo list

[1223] Specific behavior:

[1224] When the user marks a task as completed, the device sends that information to the server.

[1225] The server removes the completed task from the database and updates the to-do list.

[1226] (Application example 2)

[1227] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1228] Conventional task management systems set task priorities without considering the user's emotional state, which can lead to inefficient task processing. Furthermore, notification methods are uniform, potentially disrupting user concentration. In crowded environments, such as brick-and-mortar stores, employees are prone to stress and fatigue, making efficient task management difficult. Under these circumstances, a flexible task management method that responds to employees' emotional state is needed.

[1229] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1230] In this invention, the server includes means for periodically retrieving new messages from an information transmission medium, means for analyzing the content of the retrieved messages and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them into tasks, means for integrating the organized tasks into a single ToDo list and displaying them with priority, means for marking or coloring tasks with an alert mark or color that are approaching their deadline, means for notifying the user of the ToDo list on their device, means for analyzing the user's emotional state using an emotion engine and dynamically changing task priorities based on the emotional state, and means for adjusting the notification method according to the user's emotional state. This enables efficient task management based on the user's emotional state, reducing employee stress and maintaining their concentration.

[1231] An "information transmission medium" is a medium that provides communication means such as an email server, a chat tool server, or a web chat server.

[1232] A "new message" is a new message obtained from a predetermined information transmission medium.

[1233] "Analysis" is the process of analyzing the content of the acquired message using natural language processing technology and extracting specified elements.

[1234] A "specified keyword" is a keyword for identifying a specific action or information from the message content.

[1235] A "ToDo list" is a list that organizes extracted tasks and allows users to manage them.

[1236] "Alert marks and colors" are visual display means for visually indicating the urgency and importance of a task.

[1237] A "terminal" is a device that is directly operated by a user, and includes a smartphone, smart glasses, etc.

[1238] "Emotional state" refers to the user's emotional level or type, such as stress, fatigue, concentration, or joy.

[1239] An "emotion engine" is a system that assesses a user's emotional state using technologies such as voice recognition, facial expression analysis, and text analysis.

[1240] "Dynamic change" refers to adjusting task priorities in real time based on the user's emotional state.

[1241] "Notification method" refers to the means by which a user is notified of a task or alert, and includes adjusting the notification sound and visual display.

[1242] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions.

[1243] Server Roles

[1244] The server plays a central role in this system and implements the following functions:

[1245] 1. Data Acquisition:

[1246] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[1247] 2. Message analysis:

[1248] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract specified keywords, the user's name, address, mentions, and deadline information, thereby understanding the meaning of the message and extracting important information.

[1249] 3. Task summary and organization:

[1250] Based on the extracted information, each message is summarized and organized into a meaningful task. For example, a task such as "Report submission" (deadline: next Monday) is generated.

[1251] 4. ToDo List Generation:

[1252] Organize your tasks into a single to-do list, stored in a database and displayed based on your priorities, which are automatically set based on the importance and urgency of the tasks.

[1253] 5. Alert Settings:

[1254] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1255] 6. Emotion engine integration:

[1256] It receives input from the emotion engine to recognize the user's current emotional state, which uses techniques such as voice recognition, facial expression analysis, and text analysis to assess emotions.

[1257] 7. Changing task priorities based on emotions:

[1258] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[1259] 8. Change notification methods based on emotions:

[1260] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[1261] Device Role

[1262] The terminal is a device that is directly operated by the user and implements the following functions:

[1263] 1. Data display:

[1264] The terminal displays the ToDo list received from the server to the user.

[1265] 2. Alert display:

[1266] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[1267] 3. Collecting Emotional Data:

[1268] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[1269] 4. Completing and updating tasks:

[1270] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[1271] The role of the emotional engine

[1272] The emotion engine analyzes the user's emotional state and implements the following functions:

[1273] 1. Recognition of emotional states:

[1274] It uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[1275] 2. Data transmission:

[1276] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[1277] User Roles

[1278] The user is the ultimate beneficiary of this system and performs the following actions:

[1279] 1. Check the list:

[1280] Users can check their to-do list on their device and understand the details and deadlines of each task.

[1281] 2. Task processing:

[1282] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[1283] 3. Providing Emotion Data:

[1284] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[1285] Examples and prompts

[1286] For example, if an employee says they're "very stressed" in a busy store, the emotion engine will recognize this increased stress and move lower priority tasks higher on the list.

[1287] An example of a prompt to pass to a generative AI model:

[1288] Emotion engine emotion assessment: "Evaluate the user's emotion from this message: 'Very stressed'"

[1289] Task Management: "Here's your current task list. Reprioritize it based on the sentiment score."

[1290] This enables efficient task management based on emotional state, reducing stress and helping employees maintain focus.

[1291] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1292] Step 1:

[1293] The server periodically retrieves new messages from information transmission media. In this process, messages are collected from email servers, chat tool servers, and web chat servers using APIs. The input is message data retrieved from the APIs of each media, and the output is an unanalyzed message list stored in the server.

[1294] Step 2:

[1295] The server analyzes the contents of the received messages and extracts the specified keywords, the user's name, address, mentions, and deadline information. This process uses natural language processing (NLP). The input is the unanalyzed message data, and the output is a dataset containing the extracted important information (keywords, name, address, mentions, deadline).

[1296] Step 3:

[1297] The server summarizes the extracted information and organizes it into meaningful tasks. This process uses a summarization algorithm to convert messages into concise tasks. The input is a dataset containing important information, and the output is an organized list of tasks to display to the user.

[1298] Step 4:

[1299] The server then consolidates the organized tasks into a single to-do list and displays them prioritized. This process uses an algorithm that automatically prioritizes tasks based on their importance and urgency. The input is an organized task list, and the output is a prioritized to-do list.

[1300] Step 5:

[1301] The server assigns alert marks and colors to tasks with upcoming deadlines. This process uses logic that assigns a red alert mark to tasks with due dates within two days. The input is a to-do list, and the output is an updated to-do list with alert marks and colors.

[1302] Step 6:

[1303] The terminal displays the ToDo list received from the server to the user. This process visually displays the list using a UI (user interface). The input is a prioritized ToDo list, and the output is a task list displayed to the user.

[1304] Step 7:

[1305] The device collects emotion data from the user's voice, facial expressions, input text, etc., and sends it to the emotion engine. This process uses various sensors and analysis technologies to obtain emotion data. The input is the user's voice, facial expressions, and text data, and the output is emotion data sent to the emotion engine.

[1306] Step 8:

[1307] The emotion engine analyzes the user's emotional state and sends data to the server based on the emotional state. This process uses speech recognition, facial expression analysis, and text analysis to evaluate emotions. The input is emotion data, and the output is analyzed emotional state data.

[1308] Step 9:

[1309] The server receives input from the emotion engine and dynamically reprioritizes tasks based on the emotional state. This process uses an algorithm that reprioritizes tasks based on the emotional state score. The input is the parsed emotional state data, and the output is an updated to-do list that has been reprioritized based on the emotion.

[1310] Step 10:

[1311] The server adjusts the notification method according to the user's emotional state. This process involves making adjustments such as reducing the notification sound so as not to disturb the user's concentration. The input is the emotional state data, and the output is the adjusted notification method.

[1312] The following are examples of prompts that can be passed to a generative AI model:

[1313] Emotion engine emotion assessment: "Evaluate the user's emotion from this message: 'Very stressed'"

[1314] Task Management: "Here's your current task list. Reprioritize it based on the sentiment score."

[1315] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1316] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1317] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1318] [Third embodiment]

[1319] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1320] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1321] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1322] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1323] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1324] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1325] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1326] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1327] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1328] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1329] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1330] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1331] The system for implementing this invention efficiently manages tasks through the mutual cooperation of three elements: a server, a terminal, and a user. The specific roles of each element and the processing contents of the program are explained below.

[1332] Server Roles

[1333] The server plays a central role in this system and implements the following functions:

[1334] 1. Data Acquisition

[1335] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[1336] 2. Message Analysis

[1337] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract your name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as time and date information, which is then converted into a standard format.

[1338] 3. Task summary and organization

[1339] The server then summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, a sentence in an email saying "Yamada-san, please submit your report by next Monday" will generate a task called "Submit report" (deadline: next Monday).

[1340] 4. ToDo list generation

[1341] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[1342] 5. Alert Settings

[1343] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1344] 6. Notification function

[1345] The server notifies the user of the created to-do list via email or a dedicated notification API.

[1346] Device Role

[1347] The terminal is a device that is directly operated by the user and implements the following functions:

[1348] 1. Data display

[1349] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[1350] 2. Alert display

[1351] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[1352] 3. Completing and updating tasks

[1353] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[1354] User Roles

[1355] The user is the ultimate beneficiary of this system and performs the following actions:

[1356] 1. Check the list

[1357] Users can check their to-do list on their device and understand the details and deadlines of each task.

[1358] 2. Task Processing

[1359] The user can then work on the displayed tasks in order of priority. After completing a task, they mark it as completed on their device and update the list.

[1360] Specific examples

[1361] An example of the system's operation is shown below.

[1362] Example 1: Creating a task from an email

[1363] Email content received by the server: "Yamada-san, please submit your report by next Monday. Thank you."

[1364] Information extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[1365] Server-generated task: "Report submission" (Deadline: next Monday)

[1366] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[1367] Example 2: Creating a task from Google Chat

[1368] Chat contents retrieved by the server: "@Yamada-san, please check the materials before tomorrow's meeting."

[1369] Information extracted by the server: mention (@Yamada), request (document confirmation), deadline (tomorrow's meeting)

[1370] Server-generated task: "Check materials" (Deadline: tomorrow's meeting)

[1371] To-do list notified to the device: "Check materials" (deadline: tomorrow's meeting), with attention alerts

[1372] This system allows users to efficiently manage tasks and prevent oversight.

[1373] The processing flow will be explained below.

[1374] Step 1:

[1375] The server periodically (for example, every 5 minutes) launches a task and accesses an information transmission medium (email server, chat tool server, web chat server).

[1376] Specifically, it logs in to the mail server using the IMAP or POP3 protocol, and retrieves new messages from the chat tool using a dedicated API.

[1377] Step 2:

[1378] The server analyzes new messages received from each information carrier.

[1379] The analysis uses natural language processing (NLP) technology to extract information such as the message body, the date and time of sending, and the sender, with particular attention paid to your name, address, mentions, and designated keywords (such as "request," "confirmation," and "question").

[1380] Step 3:

[1381] The server generates a summarized task content based on the information extracted from the parsed message.

[1382] For example, from an email that says "Yamada, please submit your report by next Monday," you can create a task called "Report submission" (deadline: next Monday). The date and time information is also converted to the standard format (YYYY-MM-DD).

[1383] Step 4:

[1384] The server stores the generated tasks in a database and integrates the to-do list.

[1385] Tasks are saved in a database, taking care to avoid duplication. Saved tasks are organized by user and managed centrally.

[1386] Step 5:

[1387] The server organizes the tasks on the to-do list based on priority.

[1388] The order of the list is determined by the urgency of the task and its deadline. Tasks with upcoming deadlines are given high priority.

[1389] Step 6:

[1390] The server monitors the deadlines of tasks and displays alert marks and colors on tasks whose deadlines are approaching.

[1391] For example, tasks with a deadline within two days will be marked with a red alert mark.

[1392] Step 7:

[1393] The server notifies the user's device of the generated ToDo list.

[1394] Delivery to devices is via email, dedicated app notification APIs, and web app update functions.

[1395] Step 8:

[1396] The terminal displays the ToDo list received from the server to the user.

[1397] The list is displayed in a table or calendar format, with tasks with alerts specifically highlighted.

[1398] Step 9:

[1399] Users can check their to-do list on their device and process tasks based on their content and priority.

[1400] When the task is complete, the user marks the task as complete on the terminal.

[1401] Step 10:

[1402] The terminal synchronizes the information of the user's task completion with the server.

[1403] The server removes the completed tasks from the database and updates the current to-do list.

[1404] The above is the specific processing flow of this system. This system efficiently manages each task in a centralized manner, preventing users from overlooking something and enabling them to respond quickly.

[1405] Example 1

[1406] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1407] In conventional task management systems, the process of effectively retrieving new messages from information transmission media and organizing them into tasks is cumbersome and not fully automated. Furthermore, the system lacks the ability to prioritize tasks based on their importance and deadlines, increasing the risk of users overlooking tasks. Furthermore, the system lacks alert functions for task deadlines and update processes after task completion, making it difficult for users to efficiently manage tasks.

[1408] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1409] In this invention, the server includes: means for periodically acquiring new data from an information transmission medium; means for analyzing the acquired data and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information; means for summarizing the extracted information and organizing it into tasks; means for integrating the organized tasks into a single to-do list and automatically prioritizing and displaying them; means for marking or coloring tasks with an alert mark if the deadline is approaching; means for monitoring task deadlines and notifying users of tasks with an approaching deadline; means for notifying the user of the to-do list on the user's device; and means for synchronizing the information with the server and updating the to-do list when the user marks a task as completed. This improves task management automation, allowing users to efficiently manage tasks without missing any tasks.

[1410] An "information transmission medium" is a communication means for exchanging information, such as an email server, a chat tool server, or a web chat server.

[1411] "Periodic acquisition" means that data is automatically acquired at preset time intervals.

[1412] "Analyzing" means understanding the content of the acquired data and classifying and extracting information according to certain rules.

[1413] "Specified keywords" are words that indicate specific conditions or content, and are set so that the system can extract them preferentially.

[1414] "Extract" means to extract specific information from within a message.

[1415] "Summarizing" means to briefly summarize extracted information.

[1416] A "task" is a specific task or action that a user must perform.

[1417] "Organizing" means arranging summarized information systematically and making it easy to manage.

[1418] A "ToDo list" is a list that displays tasks that need to be completed in a list format.

[1419] "Integrating" means combining multiple tasks into one list.

[1420] "Priority" is a criterion for determining the display order and execution order based on the importance and urgency of tasks.

[1421] A "deadline" is the deadline for completing a task.

[1422] An "alert mark" is a sign or icon that calls attention to specific information.

[1423] "Coloring" means adding color to information to draw visual attention.

[1424] "Notifying" means informing the user of information.

[1425] "Synchronizing" means keeping data consistent across different devices and systems.

[1426] "To update" means to bring existing information or data up to date.

[1427] The system for implementing this invention efficiently manages tasks through collaboration between the server, the terminal, and the user. The specific roles of each element and the processing contents of the program are explained below.

[1428] Server Roles

[1429] The server plays a central role in the system and implements the following functions:

[1430] 1. Get new messages

[1431] The server periodically retrieves new messages from the information transmission medium (e.g., email server, chat tool server, web chat server) using APIs and protocols (IMAP, POP3) specific to each medium. For example, the server retrieves new emails from the email server using the IMAP protocol.

[1432] 2. Message Content Analysis

[1433] The server analyzes the received message using natural language processing technology (for example, Google Cloud Natural Language API or spaCy) and extracts the specified keywords, the user's name, address, mentions, and deadline information. For example, from an email that says "Yamada-san, please submit your report by next Monday," the server extracts "Name: Yamada," "Request: Report submission," and "Deadline: next Monday."

[1434] 3. Summarize and organize your tasks

[1435] The server summarizes the message based on the extracted information and organizes it into meaningful tasks, such as generating a task called "Report Submission" (Deadline: Next Monday).

[1436] 4. Creating and updating a to-do list

[1437] The server consolidates the organized tasks into a single to-do list and stores it in a database, where task priorities are automatically set and adjusted based on deadlines and importance.

[1438] 5. Setting up alerts

[1439] The server monitors task deadlines and assigns alert marks and colors to tasks that are approaching their deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1440] 6. Notification function

[1441] The server notifies the user of the created to-do list via email or a dedicated notification API. For example, the server uses the Gmail API to send the new to-do list to the user's Gmail account via email.

[1442] Device Role

[1443] A terminal is a device that is directly operated by a user and has the following functions:

[1444] 1. Data display

[1445] The device receives the to-do list from the server and displays it to the user. For example, the list can be presented visually using a web app or mobile app.

[1446] 2. Alert display

[1447] The device displays tasks with alert marks and color coding to indicate which tasks the user should prioritize. Tasks with red marks are displayed at the top of the list.

[1448] 3. Completing and updating tasks

[1449] When a user completes a task, the device synchronizes the information with the server and updates the to-do list. The completed task is removed from the list. For example, when a user marks the "Submit Report" task as completed, the device sends the information to the server and the list is updated.

[1450] User Roles

[1451] The user is the ultimate beneficiary of this system and performs the following operations:

[1452] 1. Check the list

[1453] The user checks the to-do list on the device and understands the content and deadline of each task. For example, the user checks the task "Check materials" (deadline: tomorrow's meeting).

[1454] 2. Task Processing

[1455] The user will then work on the displayed tasks in order of priority. After completing a task, they will mark it as completed on their device and update the list. For example, the user will work on the "Document Review" task and mark it as completed.

[1456] Example operation

[1457] 1. Creating a task from email

[1458] The server retrieves the email from the email server using the IMAP protocol.

[1459] Email content: "Yamada-san, please submit your report by next Monday."

[1460] Information analyzed and extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[1461] Server-generated task: "Report submission" (Deadline: next Monday)

[1462] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[1463] When a user marks a task as complete, the device sends the information to the server and the task is removed from the list.

[1464] 2. Task creation from chat tools

[1465] The server uses the API to retrieve messages from the chat tool server.

[1466] Message: "@Yamada-san, please check the materials before tomorrow's meeting."

[1467] Information analyzed and extracted by the server: mention (@Yamada), request (document confirmation), deadline (tomorrow's meeting)

[1468] Server-generated task: "Check materials" (Deadline: tomorrow's meeting)

[1469] To-do list notified to the device: "Check materials" (deadline: tomorrow's meeting), with alert mark

[1470] When a user marks a task as complete, the device sends the information to the server and the task is removed from the list.

[1471] This allows users to efficiently manage tasks and prevent oversight.

[1472] Generative AI model used and example prompts

[1473] Generative AI model: OpenAI GPT-3

[1474] Example prompt

[1475] Please create a task from the email below.

[1476] Email content: Yamada-san, please submit your report by next Monday.

[1477] Expected output: Task "Report submission" (Deadline: next Monday)

[1478] Using these prompts, the AI ​​model extracts the specified information and generates a task.

[1479] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1480] Step 1:

[1481] Get new messages

[1482] Input: Information transmission medium (email server, chat tool server, web chat server)

[1483] The server periodically retrieves new messages using the API or protocol of each medium (IMAP, POP3).

[1484] Specific operation: The server retrieves new mail from the email server using the IMAP protocol.

[1485] Output: Retrieved message data

[1486] Step 2:

[1487] Message content analysis

[1488] Input: Retrieved message data

[1489] The server uses natural language processing technology (e.g., Google Cloud Natural Language API or spaCy) to analyze the content of the message and extract specified keywords, your name, address, mentions, and deadline date and time information.

[1490] Specific operation: The server extracts "Name: Yamada", "Request: Report submission", and "Deadline: Next Monday" from an email that says "Yamada-san, please submit your report by next Monday."

[1491] Output: Extracted information (name, request details, deadline, etc.)

[1492] Step 3:

[1493] Summarizing and organizing tasks

[1494] Input: Extracted information

[1495] The server uses the extracted information to summarize the messages and organize them into meaningful tasks.

[1496] Specific operation: The server generates the extracted task "Report submission" (deadline: next Monday).

[1497] Output: Organized tasks

[1498] Step 4:

[1499] Creating and updating to-do lists

[1500] Input: Organized tasks

[1501] The server consolidates the organized tasks into a single to-do list and stores it in a database, with task priorities automatically set.

[1502] Specific operation: The server integrates a new task "Report submission" (deadline: next Monday) into the ToDo list.

[1503] Output: ToDo list

[1504] Step 5:

[1505] Configuring Alerts

[1506] Input: Tasks in your ToDo list

[1507] The server monitors task deadlines and displays alert marks and colors for tasks whose deadlines are approaching.

[1508] Specific behavior: The server marks tasks with deadlines within two days with a red alert mark.

[1509] Output: ToDo list with alerts

[1510] Step 6:

[1511] Notification function

[1512] Input: To-do list with alerts

[1513] The server notifies the user of the generated to-do list via email or a dedicated notification API.

[1514] What happens: The server uses the Gmail API to email the new to-do list to the user's Gmail account.

[1515] Output: ToDo list notified to the user's device

[1516] Step 7:

[1517] Data Display

[1518] Input: ToDo list notified to the user's device

[1519] The terminal displays the received ToDo list to the user.

[1520] Specific operation: The device uses a web app or mobile app to display the task "Submit report" (deadline: next Monday).

[1521] Output: Displayed ToDo list

[1522] Step 8:

[1523] Alert display

[1524] Input: To-do list on device

[1525] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[1526] What it does: The device will display tasks marked in red at the top of the list.

[1527] Output: Highlighted tasks

[1528] Step 9:

[1529] Completing and updating tasks

[1530] Input: Tasks marked as completed by user action

[1531] When a user completes a task, the device syncs the information with the server, updates the to-do list, and removes the completed task from the list.

[1532] Specific behavior: The user marks the "report submission" task as completed, the device sends the information to the server, and the list is updated.

[1533] Output: Updated ToDo list

[1534] This allows users to efficiently manage tasks and prevent oversight.

[1535] (Application example 1)

[1536] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1537] Conventional task management systems relied on email and chat messages, often requiring manual input from humans. Furthermore, they lacked a mechanism for efficiently managing error messages and operational status information from industrial robots. This resulted in the time-consuming task of responding to errors and managing tasks, resulting in reduced productivity.

[1538] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1539] In this invention, the server includes means for periodically acquiring new messages from an information transmission medium, means for analyzing the contents of the acquired messages and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them as tasks, means for integrating the organized tasks into a single ToDo list and displaying them in order of priority, means for marking or coloring tasks with an alert mark or color, means for analyzing messages from industrial robots and adding abnormality messages and operating statuses as tasks, and means for notifying the user of the ToDo list on their terminal. This makes it possible to improve the efficiency of task management and prevent oversights.

[1540] An "information transmission medium" is a communication means such as an email server, a chat tool server, or a web chat server, and is a medium for exchanging messages.

[1541] A "new message" is a message that is received for the first time over the communication medium and has not yet been processed.

[1542] A "specified keyword" is a specific word or phrase that is set in advance by the user and is used for automatic task extraction.

[1543] "My Name" refers to the user's own name or identification information, and is used to extract related tasks from the message content.

[1544] An "address" is an email address or a user's identifier in a chat tool.

[1545] "Mention" is a mechanism for notifying users who are specifically mentioned in chat tools or messages, and is implemented using the @ symbol, etc.

[1546] "Deadline date and time information" refers to a specific date and time set as the deadline for completing a task, and is used to make the user aware of the urgency of the task.

[1547] A "task" is an activity or work performed to achieve a specific purpose or goal.

[1548] A "ToDo list" is an organized collection of tasks that allows a user to view all the tasks that need to be done.

[1549] "Alert marks and colors" are symbols and colors that visually emphasize the urgency and importance of a task, and serve to draw the user's attention.

[1550] An "industrial robot" is an automated machine used in industrial fields such as manufacturing, and is used to efficiently perform specific tasks or processes.

[1551] An "abnormality message" is a message that an industrial robot sends when it detects some kind of error or abnormality.

[1552] "Operation status" refers to information that indicates the current operating state and work progress of an industrial robot.

[1553] The system for implementing this invention involves three elements, a server, a terminal, and a user, working together to efficiently manage tasks, including the operational status and error messages of industrial robots. The specific roles of each element and the processing contents of the program are explained below.

[1554] Server Roles

[1555] The server plays a central role in this system and implements the following functions:

[1556] 1. Data Acquisition

[1557] The server periodically retrieves new messages from information transmission media (email server, chat tool server, web chat server). This uses the API of each media. For example, IMAP or POP3 protocols are used to retrieve emails from the mail server, and a dedicated API is used to retrieve messages from the chat tool. In addition, data from industrial robots is also retrieved using a dedicated communication protocol.

[1558] 2. Message Analysis

[1559] The server analyzes the contents of the received messages using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"). Information about the time and date is also extracted and converted into a standard format. Furthermore, the server analyzes abnormal messages and operating status from industrial robots and recognizes them as tasks.

[1560] 3. Task summary and organization

[1561] The server summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, a sentence in an email saying, "Person in charge, please check the machine's operation by next Monday," generates a task called "Check the machine's operation" (deadline: next Monday). Similarly, a message from a robot saying, "Error code 1234: An abnormal operation has occurred," generates a task called "Error response" (immediate response).

[1562] 4. ToDo list generation

[1563] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[1564] 5. Alert Settings

[1565] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1566] 6. Notification function

[1567] The server notifies the user of the created to-do list via email or a dedicated notification API.

[1568] Device Role

[1569] The terminal is a device that is directly operated by the user and implements the following functions:

[1570] 1. Data display

[1571] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[1572] 2. Alert display

[1573] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[1574] 3. Completing and updating tasks

[1575] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[1576] User Roles

[1577] The user is the ultimate beneficiary of this system and performs the following actions:

[1578] 1. Check the list

[1579] Users can check their to-do list on their device and understand the details and deadlines of each task.

[1580] 2. Task Processing

[1581] The user can then work on the displayed tasks in order of priority. After completing a task, they mark it as completed on their device and update the list.

[1582] Specific examples

[1583] An example of the system's operation is shown below.

[1584] Example 1: Creating a task from an email

[1585] Email content received by the server: "To the person in charge, please check that the machine is working properly by next Monday. Thank you."

[1586] Information extracted by the server: Name (person in charge), request details (machine operation check), deadline (next Monday)

[1587] Server-generated task: "Check machine operation" (Deadline: next Monday)

[1588] To-do list displayed on the device: "Check machine operation" (Deadline: next Monday)

[1589] Example 2: Task generation from an industrial robot

[1590] Message received from the server: "Error code 1234: Abnormal operation"

[1591] Information extracted by the server: Abnormality details (operational abnormality), priority (high)

[1592] Server-generated tasks: "Error handling" (immediate response)

[1593] To-do list notified on the device: "Error handling" (immediate response)

[1594] Prompt Sentence Examples

[1595] text

[1596] The body of the email should follow the format below. Please extract the task details and deadline. For example:

[1597] Email body: "Person in charge, please check the operation by next Monday."

[1598] Task: "Check operation"

[1599] Due: "Next Monday"

[1600] This system allows users to efficiently manage tasks and respond to abnormalities in industrial robots in real time.

[1601] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1602] Step 1:

[1603] The server periodically retrieves new messages from the information carrier. This includes retrieving emails using the IMAP and POP3 protocols, and retrieving messages from chat tools using a dedicated API. The input is the new messages, and the output is the retrieved raw data. This data is used in the subsequent analysis steps.

[1604] Step 2:

[1605] The server analyzes the acquired raw data using natural language processing (NLP) technology. This analysis extracts the user's name, address, mentions, specified keywords (such as "request," "confirmation," or "question"), and deadline date and time information. It also analyzes abnormality messages and operating status from the industrial robot to extract information about the abnormality and operating status. The input is raw data, and the output is extracted metadata.

[1606] Step 3:

[1607] The server summarizes each message based on the extracted metadata and organizes it into meaningful tasks. For example, a sentence in an email saying, "Person in charge, please check the machine's operation by next Monday," generates a task called "Check the machine's operation" (deadline: next Monday). Similarly, a message from a robot saying, "Error code 1234: An abnormal operation has occurred," generates a task called "Error response" (immediate response). The input is metadata, and the output is the generated task.

[1608] Step 4:

[1609] The server consolidates the organized tasks into a single to-do list. This list is stored in a database and displayed based on task priority. Priorities are automatically set based on the importance and urgency of the tasks. The input is the generated tasks, and the output is the consolidated to-do list.

[1610] Step 5:

[1611] The server monitors the deadlines of tasks in the to-do list and assigns alert marks and colors to tasks with upcoming deadlines. For example, a red alert mark is assigned to a task with a deadline within two days. The input is a to-do list, and the output is a visually emphasized to-do list.

[1612] Step 6:

[1613] The server notifies the user of the generated ToDo list via email or a dedicated notification API. The input is a visually enhanced ToDo list, and the output is the notified ToDo list.

[1614] Step 7:

[1615] The terminal displays the ToDo list received from the server to the user. The user manages tasks based on this list. The input is the notified ToDo list, and the output is the list displayed on the terminal.

[1616] Step 8:

[1617] The terminal highlights tasks with alert marks and colors to indicate which tasks the user should prioritize. The input is a highlighted to-do list, and the output is the priority tasks recognized by the user.

[1618] Step 9:

[1619] When a user completes a task, the device synchronizes the information with the server and updates the to-do list. At the same time, the completed task is removed from the list. The input is the task that was notified as completed, and the output is the updated to-do list.

[1620] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1621] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions. The specific roles of each element and the processing content of the program are explained below.

[1622] Server Roles

[1623] The server plays a central role in this system and implements the following functions:

[1624] 1. Data Acquisition

[1625] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[1626] 2. Message Analysis

[1627] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as information about the time and date, and converts it into a standard format.

[1628] 3. Task summary and organization

[1629] The server then summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, an email message saying "Yamada-san, please submit your report by next Monday" generates a task called "Submit report" (deadline: next Monday).

[1630] 4. ToDo list generation

[1631] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[1632] 5. Alert Settings

[1633] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1634] 6. Emotion engine integration

[1635] The server receives input from the emotion engine to recognize the user's current emotional state, which evaluates emotions using techniques such as voice recognition, facial expression analysis, and text analysis.

[1636] 7. Changing task priorities based on emotions

[1637] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[1638] 8. Change notification methods based on emotions

[1639] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[1640] Device Role

[1641] The terminal is a device that is directly operated by the user and implements the following functions:

[1642] 1. Data display

[1643] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[1644] 2. Alert display

[1645] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[1646] 3. Collecting Emotional Data

[1647] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[1648] 4. Completing and updating tasks

[1649] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[1650] The role of the emotional engine

[1651] The emotion engine analyzes the user's emotional state and implements the following functions:

[1652] 1. Recognizing emotional states

[1653] The emotion engine uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[1654] 2. Data transmission

[1655] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[1656] User Roles

[1657] The user is the ultimate beneficiary of this system and performs the following actions:

[1658] 1. Check the list

[1659] Users can check their to-do list on their device and understand the details and deadlines of each task.

[1660] 2. Task Processing

[1661] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[1662] 3. Providing Emotion Data

[1663] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[1664] Specific examples

[1665] An example of the system's operation is shown below.

[1666] Example 1: Creating a task from an email

[1667] Email content received by the server: "Yamada-san, please submit your report by next Monday. Thank you."

[1668] Information extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[1669] Server-generated task: "Report submission" (Deadline: next Monday)

[1670] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[1671] Example 2: Changing task priorities with an emotion engine

[1672] The emotion engine analyzes the user's voice and facial expressions to recognize stress levels

[1673] The server dynamically changes the priority of tasks based on emotional information, and displays tasks that are easy to handle at the top.

[1674] Updated ToDo list notified on device: Priorities have been changed, with "Check documents" (deadline: tomorrow's meeting, light task) at the top

[1675] This system not only allows users to efficiently manage tasks and prevent oversights, but also allows them to respond flexibly according to their emotional state.

[1676] The processing flow will be explained below.

[1677] Step 1:

[1678] The server periodically (for example, every 5 minutes) launches a task and accesses an information transmission medium (email server, chat tool server, web chat server).

[1679] Specifically, it logs in to the mail server using the IMAP or POP3 protocol, and retrieves new messages from the chat tool using a dedicated API.

[1680] Step 2:

[1681] The server analyzes new messages received from each information carrier.

[1682] The analysis uses natural language processing (NLP) technology to extract information such as the message body, the date and time of sending, and the sender, with particular attention paid to your name, address, mentions, and designated keywords (such as "request," "confirmation," and "question").

[1683] Step 3:

[1684] The server generates a summarized task content based on the information extracted from the parsed message.

[1685] For example, from an email that says, "Yamada, please submit your report by next Monday," you can extract the name "Yamada," the request to "submit the report," and the deadline "next Monday," and create a task called "Submit report" (deadline: next Monday).

[1686] Step 4:

[1687] The server stores the generated tasks in a database and integrates the to-do list.

[1688] Tasks are saved in a database, taking care to avoid duplication. Saved tasks are organized by user and managed centrally.

[1689] Step 5:

[1690] The server organizes the tasks on the to-do list based on priority.

[1691] The order of the list is determined by the urgency of the task and its deadline. Tasks with upcoming deadlines are given high priority.

[1692] Step 6:

[1693] The server monitors the deadlines of tasks and displays alert marks and colors on tasks whose deadlines are approaching.

[1694] For example, tasks with a deadline within two days will be marked with a red alert mark.

[1695] Step 7:

[1696] The emotion engine collects data such as the user's voice, facial expressions, and input text to analyze the user's emotional state.

[1697] Using voice analysis and facial expression analysis technology, the system assesses the user's stress level, fatigue state, concentration, etc.

[1698] Step 8:

[1699] Based on the emotion data from the emotion engine, the server dynamically changes the priority of tasks.

[1700] For example, if the emotion engine detects that the user is in a high stress state, the server will prioritize and display relatively light tasks.

[1701] Step 9:

[1702] Based on the emotion data from the emotion engine, the server adjusts how the task is notified.

[1703] For example, if the server detects that the user is concentrating, it will make adjustments such as reducing notification sounds and reducing the number of pop-up notifications.

[1704] Step 10:

[1705] The server notifies the user's device of the generated ToDo list.

[1706] Delivery to devices is via email, dedicated app notification APIs, and web app update functions.

[1707] Step 11:

[1708] The terminal displays the ToDo list received from the server to the user.

[1709] The list is displayed in a table or calendar format, with tasks with alerts highlighted, and the list view is based on an emotion engine.

[1710] Step 12:

[1711] Users can check their to-do list on their device and process tasks based on their content and priority.

[1712] When the task is complete, the user marks the task as complete on the terminal.

[1713] Step 13:

[1714] The terminal synchronizes the information of the user's task completion with the server.

[1715] The server removes the completed tasks from the database and updates the current to-do list.

[1716] The above is the specific processing flow of this system. By combining it with an emotion engine, flexible task management that responds to the user's emotional state is realized, enabling efficient work execution.

[1717] Example 2

[1718] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1719] In modern society, with the massive volume of messages received daily from various information transmission media, it is difficult for users to efficiently manage their tasks. Simply listing tasks without considering the user's emotions and state increases stress and burden, and reduces efficiency. Furthermore, there is a lack of methods for properly communicating task deadlines and importance. This leads to problems such as users overlooking tasks or failing to respond appropriately to important tasks.

[1720] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for periodically acquiring new messages from an information transmission medium, means for analyzing the content of the acquired messages and extracting specified keywords, the user's name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them as tasks, means for integrating the organized tasks into a single to-do list and displaying them with priority, means for marking or coloring tasks with an alert mark when the deadline is approaching, means for collecting user emotion data from the terminal and transmitting it to the emotion engine, means for receiving input from the emotion engine and dynamically changing task priorities in consideration of the user's emotional state, means for adjusting the notification method based on the input from the emotion engine, and means for notifying the user of the to-do list on the terminal. This allows the user to efficiently manage tasks and flexibly respond to their emotional state.

[1721] "Information transmission medium" refers to systems and services for transmitting information and sending and receiving messages and data, such as email servers, chat tool servers, and web chat servers.

[1722] A "new message" refers to a message that is received for the first time through an information transmission medium and has not yet been processed by a user.

[1723] "Analysis" refers to the process of analyzing the content of captured messages and extracting specific information or patterns.

[1724] "Keywords" are words or phrases with a specific meaning or purpose that are used to extract important information from the content of a message.

[1725] An "emotion engine" is a system that recognizes the user's emotional state and collects and analyzes data from voice, facial expressions, input text, etc.

[1726] A "task" refers to a specific action or task that a user needs to perform, and is organized and managed as a to-do list.

[1727] A "ToDo list" is a list that lists a user's tasks and indicates information such as their priority and deadlines.

[1728] An "alert mark" refers to a mark or color that is displayed to warn or draw attention to a specific situation or task that is approaching a deadline.

[1729] "Priority" refers to a ranking that determines which tasks should be processed first based on the importance and urgency of the tasks.

[1730] "Emotional state" refers to the user's current psychological state or feelings, and includes specific sensations and emotions such as stress, fatigue, and concentration.

[1731] "Notification method" refers to the means or format for notifying the user of a task or alert, and includes audio, visual, banner display, etc.

[1732] "Device" refers to a device that is directly operated by a user, including, for example, a computer, a smartphone, or a tablet.

[1733] "Integration" refers to the process of combining multiple elements or pieces of information into one system or list.

[1734] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions. The specific roles of each element and the processing content of the program are explained below.

[1735] Server Roles

[1736] 1. Data Acquisition

[1737] The server periodically retrieves new messages from the email server, chat tool server, and web chat server, using the API of each medium. For example, IMAP or POP3 protocols are used to retrieve emails from the mail server, and a dedicated API is used to retrieve messages from the chat tool.

[1738] 2. Message Analysis

[1739] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as information about the time and date, and converts it into a standard format.

[1740] 3. Task summary and organization

[1741] The server summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, an email message saying "Customer, please submit the report by next Monday" generates a task called "Submit report" (deadline: next Monday).

[1742] 4. ToDo list generation

[1743] The server consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[1744] 5. Alert Settings

[1745] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1746] 6. Emotion engine integration

[1747] The server receives input from the emotion engine to recognize the user's current emotional state, which evaluates emotions using techniques such as voice recognition, facial expression analysis, and text analysis.

[1748] 7. Changing task priorities based on emotions

[1749] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[1750] 8. Change notification methods based on emotions

[1751] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[1752] Device Role

[1753] The terminal is a device that is directly operated by the user and implements the following functions:

[1754] 1. Data display

[1755] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[1756] 2. Alert display

[1757] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[1758] 3. Collecting Emotional Data

[1759] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[1760] 4. Completing and updating tasks

[1761] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[1762] The role of the emotional engine

[1763] The emotion engine analyzes the user's emotional state and implements the following functions:

[1764] 1. Recognizing emotional states

[1765] The emotion engine uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[1766] 2. Data transmission

[1767] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[1768] User Roles

[1769] The user is the ultimate beneficiary of this system and performs the following actions:

[1770] 1. Check the list

[1771] Users can check their to-do list on their device and understand the details and deadlines of each task.

[1772] 2. Task Processing

[1773] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[1774] 3. Providing Emotion Data

[1775] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[1776] Specific examples

[1777] An example of the system's operation is shown below.

[1778] Example 1: Creating a task from an email

[1779] The server receives the following email: "Dear customer, please submit your report by next Monday."

[1780] Information extracted by the server: Name (customer), request (report submission), deadline (next Monday)

[1781] Server-generated task: "Report submission" (Deadline: next Monday)

[1782] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[1783] Example 2: Changing task priorities with an emotion engine

[1784] The emotion engine analyzes the user's voice and facial expressions to recognize stress levels

[1785] The server dynamically changes the priority of tasks based on emotional information, and displays tasks that are easy to handle at the top.

[1786] Updated ToDo list notified on device: Priorities have been changed, with "Check documents" (deadline: tomorrow's meeting, light task) at the top

[1787] This system not only allows users to efficiently manage tasks and prevent oversights, but also allows them to respond flexibly according to their emotional state.

[1788] Prompt Sentence Examples

[1789] Here are some examples of prompts for generative AI models:

[1790] The user is in a stressful state and is suddenly given another task: "Customer, please prepare materials for the meeting by the end of this week." In this case, how can the system change the task priority and adjust the notification method? Please explain the specific steps.

[1791] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1792] Processing Steps

[1793] Step 1:

[1794] Data Acquisition

[1795] The server periodically retrieves new messages from the email server, chat tool server, and web chat server.

[1796] Input: New messages on email servers, chat tool servers, and web chat servers

[1797] Data processing: Retrieve messages using APIs, such as Gmail's IMAP protocol or Slack's API.

[1798] Output: A list of retrieved message data

[1799] Specific behavior:

[1800] Retrieves unread emails from the inbox using Gmail's IMAP protocol.

[1801] Use the Slack API to get the latest messages in a specific channel.

[1802] Step 2:

[1803] Message Parsing

[1804] The server analyzes the content of the received message using natural language processing (NLP) technology.

[1805] Input: List of retrieved message data

[1806] Data processing: Use an NLP library (e.g., Spacy or NLTK) to parse messages and extract names, addresses, mentions, keywords, and date information.

[1807] Output: A list of parsed information (names, keywords, date information, etc.)

[1808] Specific behavior:

[1809] Convert the phrase "next Monday" to standard format (YYYY-MM-DD).

[1810] Detects keywords such as "request," "confirmation," and "question" to determine the type of task.

[1811] Step 3:

[1812] Task summary and organization

[1813] The server summarizes the messages based on the extracted information and organizes them into meaningful tasks.

[1814] Input: A list of parsed information

[1815] Data processing: Extract important information from messages (e.g., "report submission" and "deadline: next Monday") and generate tasks.

[1816] Output: A list of generated tasks

[1817] Specific behavior:

[1818] Extract "Name," "Report submission," and "Deadline" from the message and save them as task data.

[1819] Step 4:

[1820] ToDo list generation

[1821] The server combines the generated tasks into a single ToDo list and saves it in a database.

[1822] Input: A list of generated tasks

[1823] Data processing: Sort tasks based on priority and combine them into one to-do list.

[1824] Output: Consolidated ToDo list

[1825] Specific behavior:

[1826] Calculate the importance and urgency score for each task and place them at the top of the list.

[1827] Save the integrated to-do list in a database.

[1828] Step 5:

[1829] Alert Settings

[1830] The server monitors task deadlines and sets alerts for upcoming deadlines.

[1831] Input: Integrated ToDo List

[1832] Data processing: Compare the current time with the task deadline and mark tasks with an alert if the deadline is approaching.

[1833] Output: ToDo list with alerts

[1834] Specific behavior:

[1835] Detects tasks with deadlines within two days and marks them with a red alert.

[1836] Reflect tasks with alert marks in your ToDo list.

[1837] Step 6:

[1838] Collecting Emotional Data

[1839] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[1840] Input: User's voice, facial expressions, input text

[1841] Data processing: Convert analog data into digital data and send it to the emotion engine.

[1842] Output: Emotion data sent to the emotion engine

[1843] Specific behavior:

[1844] The user's facial expressions are captured by a camera and analyzed using a facial expression analysis algorithm (such as OpenCV).

[1845] The voice recorded by the microphone is input into the emotion engine.

[1846] Step 7:

[1847] Changing task priorities based on emotions

[1848] The emotion engine recognizes the user's emotional state (e.g., stress or fatigue) and transmits it to the server, which then dynamically changes task priorities based on this information.

[1849] Input: emotion data from emotion engine, integrated to-do list

[1850] Data processing: Analyze sentiment data and reprioritize tasks.

[1851] Output: ToDo list with changed priorities

[1852] Specific behavior:

[1853] If the emotion engine determines that the user's stress level is "high," it temporarily lowers the priority of the current high-load task.

[1854] Move lighter tasks to the top of the list and notify the user.

[1855] Step 8:

[1856] Changing notification methods based on emotions

[1857] Based on the information from the emotion engine, the server adjusts the notification method.

[1858] Input: Emotion data from the emotion engine

[1859] Data processing: Emotional data is used to change the notification method (volume, display format, etc.).

[1860] Output: Adjusted notification method

[1861] Specific behavior:

[1862] If the emotion engine determines that the user is in a "concentrated state," it will mute the notification volume.

[1863] Switch to banner display for email and message notifications.

[1864] Step 9:

[1865] Task completion and updates

[1866] After the user completes a task, they mark it as complete on their device, which then syncs it with the server, and the completed task is removed from their to-do list.

[1867] Input: Task information that the user marked as complete

[1868] Data processing: Delete completed tasks from the database and update the to-do list.

[1869] Output: Updated ToDo list

[1870] Specific behavior:

[1871] When the user marks a task as completed, the device sends that information to the server.

[1872] The server removes the completed task from the database and updates the to-do list.

[1873] (Application example 2)

[1874] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1875] Conventional task management systems set task priorities without considering the user's emotional state, which can lead to inefficient task processing. Furthermore, notification methods are uniform, potentially disrupting user concentration. In crowded environments, such as brick-and-mortar stores, employees are prone to stress and fatigue, making efficient task management difficult. Under these circumstances, a flexible task management method that responds to employees' emotional state is needed.

[1876] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1877] In this invention, the server includes means for periodically retrieving new messages from an information transmission medium, means for analyzing the content of the retrieved messages and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them into tasks, means for integrating the organized tasks into a single ToDo list and displaying them with priority, means for marking or coloring tasks with an alert mark or color that are approaching their deadline, means for notifying the user of the ToDo list on their device, means for analyzing the user's emotional state using an emotion engine and dynamically changing task priorities based on the emotional state, and means for adjusting the notification method according to the user's emotional state. This enables efficient task management based on the user's emotional state, reducing employee stress and maintaining their concentration.

[1878] An "information transmission medium" is a medium that provides communication means such as an email server, a chat tool server, or a web chat server.

[1879] A "new message" is a new message obtained from a predetermined information transmission medium.

[1880] "Analysis" is the process of analyzing the content of the acquired message using natural language processing technology and extracting specified elements.

[1881] A "specified keyword" is a keyword for identifying a specific action or information from the message content.

[1882] A "ToDo list" is a list that organizes extracted tasks and allows users to manage them.

[1883] "Alert marks and colors" are visual display means for visually indicating the urgency and importance of a task.

[1884] A "terminal" is a device that is directly operated by a user, and includes a smartphone, smart glasses, etc.

[1885] "Emotional state" refers to the user's emotional level or type, such as stress, fatigue, concentration, or joy.

[1886] An "emotion engine" is a system that assesses a user's emotional state using technologies such as voice recognition, facial expression analysis, and text analysis.

[1887] "Dynamic change" refers to adjusting task priorities in real time based on the user's emotional state.

[1888] "Notification method" refers to the means by which a user is notified of a task or alert, and includes adjusting the notification sound and visual display.

[1889] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions.

[1890] Server Roles

[1891] The server plays a central role in this system and implements the following functions:

[1892] 1. Data Acquisition:

[1893] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[1894] 2. Message analysis:

[1895] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract specified keywords, the user's name, address, mentions, and deadline information, thereby understanding the meaning of the message and extracting important information.

[1896] 3. Task summary and organization:

[1897] Based on the extracted information, each message is summarized and organized into a meaningful task. For example, a task such as "Report submission" (deadline: next Monday) is generated.

[1898] 4. ToDo List Generation:

[1899] Organize your tasks into a single to-do list, stored in a database and displayed based on your priorities, which are automatically set based on the importance and urgency of the tasks.

[1900] 5. Alert Settings:

[1901] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1902] 6. Emotion engine integration:

[1903] It receives input from the emotion engine to recognize the user's current emotional state, which uses techniques such as voice recognition, facial expression analysis, and text analysis to assess emotions.

[1904] 7. Changing task priorities based on emotions:

[1905] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[1906] 8. Change notification methods based on emotions:

[1907] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[1908] Device Role

[1909] The terminal is a device that is directly operated by the user and implements the following functions:

[1910] 1. Data display:

[1911] The terminal displays the ToDo list received from the server to the user.

[1912] 2. Alert display:

[1913] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[1914] 3. Collecting Emotional Data:

[1915] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[1916] 4. Completing and updating tasks:

[1917] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[1918] The role of the emotional engine

[1919] The emotion engine analyzes the user's emotional state and implements the following functions:

[1920] 1. Recognition of emotional states:

[1921] It uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[1922] 2. Data transmission:

[1923] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[1924] User Roles

[1925] The user is the ultimate beneficiary of this system and performs the following actions:

[1926] 1. Check the list:

[1927] Users can check their to-do list on their device and understand the details and deadlines of each task.

[1928] 2. Task processing:

[1929] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[1930] 3. Providing Emotion Data:

[1931] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[1932] Examples and prompts

[1933] For example, if an employee says they're "very stressed" in a busy store, the emotion engine will recognize this increased stress and move lower priority tasks higher on the list.

[1934] An example of a prompt to pass to a generative AI model:

[1935] Emotion engine emotion assessment: "Evaluate the user's emotion from this message: 'Very stressed'"

[1936] Task Management: "Here's your current task list. Reprioritize it based on the sentiment score."

[1937] This enables efficient task management based on emotional state, reducing stress and helping employees maintain focus.

[1938] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1939] Step 1:

[1940] The server periodically retrieves new messages from information transmission media. In this process, messages are collected from email servers, chat tool servers, and web chat servers using APIs. The input is message data retrieved from the APIs of each media, and the output is an unanalyzed message list stored in the server.

[1941] Step 2:

[1942] The server analyzes the contents of the received messages and extracts the specified keywords, the user's name, address, mentions, and deadline information. This process uses natural language processing (NLP). The input is the unanalyzed message data, and the output is a dataset containing the extracted important information (keywords, name, address, mentions, deadline).

[1943] Step 3:

[1944] The server summarizes the extracted information and organizes it into meaningful tasks. This process uses a summarization algorithm to convert messages into concise tasks. The input is a dataset containing important information, and the output is an organized list of tasks to display to the user.

[1945] Step 4:

[1946] The server then consolidates the organized tasks into a single to-do list and displays them prioritized. This process uses an algorithm that automatically prioritizes tasks based on their importance and urgency. The input is an organized task list, and the output is a prioritized to-do list.

[1947] Step 5:

[1948] The server assigns alert marks and colors to tasks with upcoming deadlines. This process uses logic that assigns a red alert mark to tasks with due dates within two days. The input is a to-do list, and the output is an updated to-do list with alert marks and colors.

[1949] Step 6:

[1950] The terminal displays the ToDo list received from the server to the user. This process visually displays the list using a UI (user interface). The input is a prioritized ToDo list, and the output is a task list displayed to the user.

[1951] Step 7:

[1952] The device collects emotion data from the user's voice, facial expressions, input text, etc., and sends it to the emotion engine. This process uses various sensors and analysis technologies to obtain emotion data. The input is the user's voice, facial expressions, and text data, and the output is emotion data sent to the emotion engine.

[1953] Step 8:

[1954] The emotion engine analyzes the user's emotional state and sends data to the server based on the emotional state. This process uses speech recognition, facial expression analysis, and text analysis to evaluate emotions. The input is emotion data, and the output is analyzed emotional state data.

[1955] Step 9:

[1956] The server receives input from the emotion engine and dynamically reprioritizes tasks based on the emotional state. This process uses an algorithm that reprioritizes tasks based on the emotional state score. The input is the parsed emotional state data, and the output is an updated to-do list that has been reprioritized based on the emotion.

[1957] Step 10:

[1958] The server adjusts the notification method according to the user's emotional state. This process involves making adjustments such as reducing the notification sound so as not to disturb the user's concentration. The input is the emotional state data, and the output is the adjusted notification method.

[1959] The following are examples of prompts that can be passed to a generative AI model:

[1960] Emotion engine emotion assessment: "Evaluate the user's emotion from this message: 'Very stressed'"

[1961] Task Management: "Here's your current task list. Reprioritize it based on the sentiment score."

[1962] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1963] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1964] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1965] [Fourth embodiment]

[1966] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1967] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1968] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1969] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1970] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1971] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1972] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1973] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1974] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1975] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1976] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1977] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1978] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1979] The system for implementing this invention efficiently manages tasks through the mutual cooperation of three elements: a server, a terminal, and a user. The specific roles of each element and the processing contents of the program are explained below.

[1980] Server Roles

[1981] The server plays a central role in this system and implements the following functions:

[1982] 1. Data Acquisition

[1983] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[1984] 2. Message Analysis

[1985] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract your name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as time and date information, which is then converted into a standard format.

[1986] 3. Task summary and organization

[1987] The server then summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, a sentence in an email saying "Yamada-san, please submit your report by next Monday" will generate a task called "Submit report" (deadline: next Monday).

[1988] 4. ToDo list generation

[1989] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[1990] 5. Alert Settings

[1991] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[1992] 6. Notification function

[1993] The server notifies the user of the created to-do list via email or a dedicated notification API.

[1994] Device Role

[1995] The terminal is a device that is directly operated by the user and implements the following functions:

[1996] 1. Data display

[1997] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[1998] 2. Alert display

[1999] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[2000] 3. Completing and updating tasks

[2001] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[2002] User Roles

[2003] The user is the ultimate beneficiary of this system and performs the following actions:

[2004] 1. Check the list

[2005] Users can check their to-do list on their device and understand the details and deadlines of each task.

[2006] 2. Task Processing

[2007] The user can then work on the displayed tasks in order of priority. After completing a task, they mark it as completed on their device and update the list.

[2008] Specific examples

[2009] An example of the system's operation is shown below.

[2010] Example 1: Creating a task from an email

[2011] Email content received by the server: "Yamada-san, please submit your report by next Monday. Thank you."

[2012] Information extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[2013] Server-generated task: "Report submission" (Deadline: next Monday)

[2014] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[2015] Example 2: Creating a task from Google Chat

[2016] Chat contents retrieved by the server: "@Yamada-san, please check the materials before tomorrow's meeting."

[2017] Information extracted by the server: mention (@Yamada), request (document confirmation), deadline (tomorrow's meeting)

[2018] Server-generated task: "Check materials" (Deadline: tomorrow's meeting)

[2019] To-do list notified to the device: "Check materials" (deadline: tomorrow's meeting), with attention alerts

[2020] This system allows users to efficiently manage tasks and prevent oversight.

[2021] The processing flow will be explained below.

[2022] Step 1:

[2023] The server periodically (for example, every 5 minutes) launches a task and accesses an information transmission medium (email server, chat tool server, web chat server).

[2024] Specifically, it logs in to the mail server using the IMAP or POP3 protocol, and retrieves new messages from the chat tool using a dedicated API.

[2025] Step 2:

[2026] The server analyzes new messages received from each information carrier.

[2027] The analysis uses natural language processing (NLP) technology to extract information such as the message body, the date and time of sending, and the sender, with particular attention paid to your name, address, mentions, and designated keywords (such as "request," "confirmation," and "question").

[2028] Step 3:

[2029] The server generates a summarized task content based on the information extracted from the parsed message.

[2030] For example, from an email that says "Yamada, please submit your report by next Monday," you can create a task called "Report submission" (deadline: next Monday). The date and time information is also converted to the standard format (YYYY-MM-DD).

[2031] Step 4:

[2032] The server stores the generated tasks in a database and integrates the to-do list.

[2033] Tasks are saved in a database, taking care to avoid duplication. Saved tasks are organized by user and managed centrally.

[2034] Step 5:

[2035] The server organizes the tasks on the to-do list based on priority.

[2036] The order of the list is determined by the urgency of the task and its deadline. Tasks with upcoming deadlines are given high priority.

[2037] Step 6:

[2038] The server monitors the deadlines of tasks and displays alert marks and colors on tasks whose deadlines are approaching.

[2039] For example, tasks with a deadline within two days will be marked with a red alert mark.

[2040] Step 7:

[2041] The server notifies the user's device of the generated ToDo list.

[2042] Delivery to devices is via email, dedicated app notification APIs, and web app update functions.

[2043] Step 8:

[2044] The terminal displays the ToDo list received from the server to the user.

[2045] The list is displayed in a table or calendar format, with tasks with alerts specifically highlighted.

[2046] Step 9:

[2047] Users can check their to-do list on their device and process tasks based on their content and priority.

[2048] When the task is complete, the user marks the task as complete on the terminal.

[2049] Step 10:

[2050] The terminal synchronizes the information of the user's task completion with the server.

[2051] The server removes the completed tasks from the database and updates the current to-do list.

[2052] The above is the specific processing flow of this system. This system efficiently manages each task in a centralized manner, preventing users from overlooking something and enabling them to respond quickly.

[2053] Example 1

[2054] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2055] In conventional task management systems, the process of effectively retrieving new messages from information transmission media and organizing them into tasks is cumbersome and not fully automated. Furthermore, the system lacks the ability to prioritize tasks based on their importance and deadlines, increasing the risk of users overlooking tasks. Furthermore, the system lacks alert functions for task deadlines and update processes after task completion, making it difficult for users to efficiently manage tasks.

[2056] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2057] In this invention, the server includes: means for periodically acquiring new data from an information transmission medium; means for analyzing the acquired data and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information; means for summarizing the extracted information and organizing it into tasks; means for integrating the organized tasks into a single to-do list and automatically prioritizing and displaying them; means for marking or coloring tasks with an alert mark if the deadline is approaching; means for monitoring task deadlines and notifying users of tasks with an approaching deadline; means for notifying the user of the to-do list on the user's device; and means for synchronizing the information with the server and updating the to-do list when the user marks a task as completed. This improves task management automation, allowing users to efficiently manage tasks without missing any tasks.

[2058] An "information transmission medium" is a communication means for exchanging information, such as an email server, a chat tool server, or a web chat server.

[2059] "Periodic acquisition" means that data is automatically acquired at preset time intervals.

[2060] "Analyzing" means understanding the content of the acquired data and classifying and extracting information according to certain rules.

[2061] "Specified keywords" are words that indicate specific conditions or content, and are set so that the system can extract them preferentially.

[2062] "Extract" means to extract specific information from within a message.

[2063] "Summarizing" means to briefly summarize extracted information.

[2064] A "task" is a specific task or action that a user must perform.

[2065] "Organizing" means arranging summarized information systematically and making it easy to manage.

[2066] A "ToDo list" is a list that displays tasks that need to be completed in a list format.

[2067] "Integrating" means combining multiple tasks into one list.

[2068] "Priority" is a criterion for determining the display order and execution order based on the importance and urgency of tasks.

[2069] A "deadline" is the deadline for completing a task.

[2070] An "alert mark" is a sign or icon that calls attention to specific information.

[2071] "Coloring" means adding color to information to draw visual attention.

[2072] "Notifying" means informing the user of information.

[2073] "Synchronizing" means keeping data consistent across different devices and systems.

[2074] "To update" means to bring existing information or data up to date.

[2075] The system for implementing this invention efficiently manages tasks through collaboration between the server, the terminal, and the user. The specific roles of each element and the processing contents of the program are explained below.

[2076] Server Roles

[2077] The server plays a central role in the system and implements the following functions:

[2078] 1. Get new messages

[2079] The server periodically retrieves new messages from the information transmission medium (e.g., email server, chat tool server, web chat server) using APIs and protocols (IMAP, POP3) specific to each medium. For example, the server retrieves new emails from the email server using the IMAP protocol.

[2080] 2. Message Content Analysis

[2081] The server analyzes the received message using natural language processing technology (for example, Google Cloud Natural Language API or spaCy) and extracts the specified keywords, the user's name, address, mentions, and deadline information. For example, from an email that says "Yamada-san, please submit your report by next Monday," the server extracts "Name: Yamada," "Request: Report submission," and "Deadline: next Monday."

[2082] 3. Summarize and organize your tasks

[2083] The server summarizes the message based on the extracted information and organizes it into meaningful tasks, such as generating a task called "Report Submission" (Deadline: Next Monday).

[2084] 4. Creating and updating a to-do list

[2085] The server consolidates the organized tasks into a single to-do list and stores it in a database, where task priorities are automatically set and adjusted based on deadlines and importance.

[2086] 5. Setting up alerts

[2087] The server monitors task deadlines and assigns alert marks and colors to tasks that are approaching their deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[2088] 6. Notification function

[2089] The server notifies the user of the created to-do list via email or a dedicated notification API. For example, the server uses the Gmail API to send the new to-do list to the user's Gmail account via email.

[2090] Device Role

[2091] A terminal is a device that is directly operated by a user and has the following functions:

[2092] 1. Data display

[2093] The device receives the to-do list from the server and displays it to the user. For example, the list can be presented visually using a web app or mobile app.

[2094] 2. Alert display

[2095] The device displays tasks with alert marks and color coding to indicate which tasks the user should prioritize. Tasks with red marks are displayed at the top of the list.

[2096] 3. Completing and updating tasks

[2097] When a user completes a task, the device synchronizes the information with the server and updates the to-do list. The completed task is removed from the list. For example, when a user marks the "Submit Report" task as completed, the device sends the information to the server and the list is updated.

[2098] User Roles

[2099] The user is the ultimate beneficiary of this system and performs the following operations:

[2100] 1. Check the list

[2101] The user checks the to-do list on the device and understands the content and deadline of each task. For example, the user checks the task "Check materials" (deadline: tomorrow's meeting).

[2102] 2. Task Processing

[2103] The user will then work on the displayed tasks in order of priority. After completing a task, they will mark it as completed on their device and update the list. For example, the user will work on the "Document Review" task and mark it as completed.

[2104] Example operation

[2105] 1. Creating a task from email

[2106] The server retrieves the email from the email server using the IMAP protocol.

[2107] Email content: "Yamada-san, please submit your report by next Monday."

[2108] Information analyzed and extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[2109] Server-generated task: "Report submission" (Deadline: next Monday)

[2110] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[2111] When a user marks a task as complete, the device sends the information to the server and the task is removed from the list.

[2112] 2. Task creation from chat tools

[2113] The server uses the API to retrieve messages from the chat tool server.

[2114] Message: "@Yamada-san, please check the materials before tomorrow's meeting."

[2115] Information analyzed and extracted by the server: mention (@Yamada), request (document confirmation), deadline (tomorrow's meeting)

[2116] Server-generated task: "Check materials" (Deadline: tomorrow's meeting)

[2117] To-do list notified to the device: "Check materials" (deadline: tomorrow's meeting), with alert mark

[2118] When a user marks a task as complete, the device sends the information to the server and the task is removed from the list.

[2119] This allows users to efficiently manage tasks and prevent oversight.

[2120] Generative AI model used and example prompts

[2121] Generative AI model: OpenAI GPT-3

[2122] Example prompt

[2123] Please create a task from the email below.

[2124] Email content: Yamada-san, please submit your report by next Monday.

[2125] Expected output: Task "Report submission" (Deadline: next Monday)

[2126] Using these prompts, the AI ​​model extracts the specified information and generates a task.

[2127] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2128] Step 1:

[2129] Get new messages

[2130] Input: Information transmission medium (email server, chat tool server, web chat server)

[2131] The server periodically retrieves new messages using the API or protocol of each medium (IMAP, POP3).

[2132] Specific operation: The server retrieves new mail from the email server using the IMAP protocol.

[2133] Output: Retrieved message data

[2134] Step 2:

[2135] Message content analysis

[2136] Input: Retrieved message data

[2137] The server uses natural language processing technology (e.g., Google Cloud Natural Language API or spaCy) to analyze the content of the message and extract specified keywords, your name, address, mentions, and deadline date and time information.

[2138] Specific operation: The server extracts "Name: Yamada", "Request: Report submission", and "Deadline: Next Monday" from an email that says "Yamada-san, please submit your report by next Monday."

[2139] Output: Extracted information (name, request details, deadline, etc.)

[2140] Step 3:

[2141] Summarizing and organizing tasks

[2142] Input: Extracted information

[2143] The server uses the extracted information to summarize the messages and organize them into meaningful tasks.

[2144] Specific operation: The server generates the extracted task "Report submission" (deadline: next Monday).

[2145] Output: Organized tasks

[2146] Step 4:

[2147] Creating and updating to-do lists

[2148] Input: Organized tasks

[2149] The server consolidates the organized tasks into a single to-do list and stores it in a database, with task priorities automatically set.

[2150] Specific operation: The server integrates a new task "Report submission" (deadline: next Monday) into the ToDo list.

[2151] Output: ToDo list

[2152] Step 5:

[2153] Configuring Alerts

[2154] Input: Tasks in your ToDo list

[2155] The server monitors task deadlines and displays alert marks and colors for tasks whose deadlines are approaching.

[2156] Specific behavior: The server marks tasks with deadlines within two days with a red alert mark.

[2157] Output: ToDo list with alerts

[2158] Step 6:

[2159] Notification function

[2160] Input: To-do list with alerts

[2161] The server notifies the user of the generated to-do list via email or a dedicated notification API.

[2162] What happens: The server uses the Gmail API to email the new to-do list to the user's Gmail account.

[2163] Output: ToDo list notified to the user's device

[2164] Step 7:

[2165] Data Display

[2166] Input: ToDo list notified to the user's device

[2167] The terminal displays the received ToDo list to the user.

[2168] Specific operation: The device uses a web app or mobile app to display the task "Submit report" (deadline: next Monday).

[2169] Output: Displayed ToDo list

[2170] Step 8:

[2171] Alert display

[2172] Input: To-do list on device

[2173] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[2174] What it does: The device will display tasks marked in red at the top of the list.

[2175] Output: Highlighted tasks

[2176] Step 9:

[2177] Completing and updating tasks

[2178] Input: Tasks marked as completed by user action

[2179] When a user completes a task, the device syncs the information with the server, updates the to-do list, and removes the completed task from the list.

[2180] Specific behavior: The user marks the "report submission" task as completed, the device sends the information to the server, and the list is updated.

[2181] Output: Updated ToDo list

[2182] This allows users to efficiently manage tasks and prevent oversight.

[2183] (Application example 1)

[2184] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2185] Conventional task management systems relied on email and chat messages, often requiring manual input from humans. Furthermore, they lacked a mechanism for efficiently managing error messages and operational status information from industrial robots. This resulted in the time-consuming task of responding to errors and managing tasks, resulting in reduced productivity.

[2186] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2187] In this invention, the server includes means for periodically acquiring new messages from an information transmission medium, means for analyzing the contents of the acquired messages and extracting specified keywords, the user's own name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them as tasks, means for integrating the organized tasks into a single ToDo list and displaying them in order of priority, means for marking or coloring tasks with an alert mark or color, means for analyzing messages from industrial robots and adding abnormality messages and operating statuses as tasks, and means for notifying the user of the ToDo list on their terminal. This makes it possible to improve the efficiency of task management and prevent oversights.

[2188] An "information transmission medium" is a communication means such as an email server, a chat tool server, or a web chat server, and is a medium for exchanging messages.

[2189] A "new message" is a message that is received for the first time over the communication medium and has not yet been processed.

[2190] A "specified keyword" is a specific word or phrase that is set in advance by the user and is used for automatic task extraction.

[2191] "My Name" refers to the user's own name or identification information, and is used to extract related tasks from the message content.

[2192] An "address" is an email address or a user's identifier in a chat tool.

[2193] "Mention" is a mechanism for notifying users who are specifically mentioned in chat tools or messages, and is implemented using the @ symbol, etc.

[2194] "Deadline date and time information" refers to a specific date and time set as the deadline for completing a task, and is used to make the user aware of the urgency of the task.

[2195] A "task" is an activity or work performed to achieve a specific purpose or goal.

[2196] A "ToDo list" is an organized collection of tasks that allows a user to view all the tasks that need to be done.

[2197] "Alert marks and colors" are symbols and colors that visually emphasize the urgency and importance of a task, and serve to draw the user's attention.

[2198] An "industrial robot" is an automated machine used in industrial fields such as manufacturing, and is used to efficiently perform specific tasks or processes.

[2199] An "abnormality message" is a message that an industrial robot sends when it detects some kind of error or abnormality.

[2200] "Operation status" refers to information that indicates the current operating state and work progress of an industrial robot.

[2201] The system for implementing this invention involves three elements, a server, a terminal, and a user, working together to efficiently manage tasks, including the operational status and error messages of industrial robots. The specific roles of each element and the processing contents of the program are explained below.

[2202] Server Roles

[2203] The server plays a central role in this system and implements the following functions:

[2204] 1. Data Acquisition

[2205] The server periodically retrieves new messages from information transmission media (email server, chat tool server, web chat server). This uses the API of each media. For example, IMAP or POP3 protocols are used to retrieve emails from the mail server, and a dedicated API is used to retrieve messages from the chat tool. In addition, data from industrial robots is also retrieved using a dedicated communication protocol.

[2206] 2. Message Analysis

[2207] The server analyzes the contents of the received messages using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"). Information about the time and date is also extracted and converted into a standard format. Furthermore, the server analyzes abnormal messages and operating status from industrial robots and recognizes them as tasks.

[2208] 3. Task summary and organization

[2209] The server summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, a sentence in an email saying, "Person in charge, please check the machine's operation by next Monday," generates a task called "Check the machine's operation" (deadline: next Monday). Similarly, a message from a robot saying, "Error code 1234: An abnormal operation has occurred," generates a task called "Error response" (immediate response).

[2210] 4. ToDo list generation

[2211] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[2212] 5. Alert Settings

[2213] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[2214] 6. Notification function

[2215] The server notifies the user of the created to-do list via email or a dedicated notification API.

[2216] Device Role

[2217] The terminal is a device that is directly operated by the user and implements the following functions:

[2218] 1. Data display

[2219] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[2220] 2. Alert display

[2221] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[2222] 3. Completing and updating tasks

[2223] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[2224] User Roles

[2225] The user is the ultimate beneficiary of this system and performs the following actions:

[2226] 1. Check the list

[2227] Users can check their to-do list on their device and understand the details and deadlines of each task.

[2228] 2. Task Processing

[2229] The user can then work on the displayed tasks in order of priority. After completing a task, they mark it as completed on their device and update the list.

[2230] Specific examples

[2231] An example of the system's operation is shown below.

[2232] Example 1: Creating a task from an email

[2233] Email content received by the server: "To the person in charge, please check that the machine is working properly by next Monday. Thank you."

[2234] Information extracted by the server: Name (person in charge), request details (machine operation check), deadline (next Monday)

[2235] Server-generated task: "Check machine operation" (Deadline: next Monday)

[2236] To-do list displayed on the device: "Check machine operation" (Deadline: next Monday)

[2237] Example 2: Task generation from an industrial robot

[2238] Message received from the server: "Error code 1234: Abnormal operation"

[2239] Information extracted by the server: Abnormality details (operational abnormality), priority (high)

[2240] Server-generated tasks: "Error handling" (immediate response)

[2241] To-do list notified on the device: "Error handling" (immediate response)

[2242] Prompt Sentence Examples

[2243] text

[2244] The body of the email should follow the format below. Please extract the task details and deadline. For example:

[2245] Email body: "Person in charge, please check the operation by next Monday."

[2246] Task: "Check operation"

[2247] Due: "Next Monday"

[2248] This system allows users to efficiently manage tasks and respond to abnormalities in industrial robots in real time.

[2249] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2250] Step 1:

[2251] The server periodically retrieves new messages from the information carrier. This includes retrieving emails using the IMAP and POP3 protocols, and retrieving messages from chat tools using a dedicated API. The input is the new messages, and the output is the retrieved raw data. This data is used in the subsequent analysis steps.

[2252] Step 2:

[2253] The server analyzes the acquired raw data using natural language processing (NLP) technology. This analysis extracts the user's name, address, mentions, specified keywords (such as "request," "confirmation," or "question"), and deadline date and time information. It also analyzes abnormality messages and operating status from the industrial robot to extract information about the abnormality and operating status. The input is raw data, and the output is extracted metadata.

[2254] Step 3:

[2255] The server summarizes each message based on the extracted metadata and organizes it into meaningful tasks. For example, a sentence in an email saying, "Person in charge, please check the machine's operation by next Monday," generates a task called "Check the machine's operation" (deadline: next Monday). Similarly, a message from a robot saying, "Error code 1234: An abnormal operation has occurred," generates a task called "Error response" (immediate response). The input is metadata, and the output is the generated task.

[2256] Step 4:

[2257] The server consolidates the organized tasks into a single to-do list. This list is stored in a database and displayed based on task priority. Priorities are automatically set based on the importance and urgency of the tasks. The input is the generated tasks, and the output is the consolidated to-do list.

[2258] Step 5:

[2259] The server monitors the deadlines of tasks in the to-do list and assigns alert marks and colors to tasks with upcoming deadlines. For example, a red alert mark is assigned to a task with a deadline within two days. The input is a to-do list, and the output is a visually emphasized to-do list.

[2260] Step 6:

[2261] The server notifies the user of the generated ToDo list via email or a dedicated notification API. The input is a visually enhanced ToDo list, and the output is the notified ToDo list.

[2262] Step 7:

[2263] The terminal displays the ToDo list received from the server to the user. The user manages tasks based on this list. The input is the notified ToDo list, and the output is the list displayed on the terminal.

[2264] Step 8:

[2265] The terminal highlights tasks with alert marks and colors to indicate which tasks the user should prioritize. The input is a highlighted to-do list, and the output is the priority tasks recognized by the user.

[2266] Step 9:

[2267] When a user completes a task, the device synchronizes the information with the server and updates the to-do list. At the same time, the completed task is removed from the list. The input is the task that was notified as completed, and the output is the updated to-do list.

[2268] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2269] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions. The specific roles of each element and the processing content of the program are explained below.

[2270] Server Roles

[2271] The server plays a central role in this system and implements the following functions:

[2272] 1. Data Acquisition

[2273] The server periodically retrieves new messages from the information transmission medium (email server, chat tool server, web chat server). This is done using the API of each medium. For example, to retrieve emails from a mail server, the IMAP or POP3 protocol is used, and to retrieve messages from a chat tool, a dedicated API is used.

[2274] 2. Message Analysis

[2275] The server analyzes the contents of the received message using natural language processing (NLP) technology to extract the user's name, address, mentions, and specified keywords (such as "request," "confirmation," or "question"), as well as information about the time and date, and converts it into a standard format.

[2276] 3. Task summary and organization

[2277] The server then summarizes each message based on the extracted information and organizes it into meaningful tasks. For example, an email message saying "Yamada-san, please submit your report by next Monday" generates a task called "Submit report" (deadline: next Monday).

[2278] 4. ToDo list generation

[2279] The server then consolidates the organized tasks into a single to-do list, which is stored in a database and displayed based on the user's priorities, which are automatically set based on the importance and urgency of the tasks.

[2280] 5. Alert Settings

[2281] The server monitors task deadlines and assigns alert marks and colors to tasks with upcoming deadlines. For example, a task with a deadline within two days will be assigned a red alert mark.

[2282] 6. Emotion engine integration

[2283] The server receives input from the emotion engine to recognize the user's current emotional state, which evaluates emotions using techniques such as voice recognition, facial expression analysis, and text analysis.

[2284] 7. Changing task priorities based on emotions

[2285] If the emotion engine detects stress or fatigue in the user, the server dynamically changes the priority of tasks and displays relatively light tasks first.

[2286] 8. Change notification methods based on emotions

[2287] The emotion engine changes the notification method so as not to disturb the user's concentration. For example, if it recognizes that the user is concentrating, it will adjust the notification sound by reducing it.

[2288] Device Role

[2289] The terminal is a device that is directly operated by the user and implements the following functions:

[2290] 1. Data display

[2291] The device displays the ToDo list received from the server to the user, who then manages tasks based on this list.

[2292] 2. Alert display

[2293] The terminal highlights tasks with alert marks and colored highlighting, clearly indicating which tasks the user should prioritize.

[2294] 3. Collecting Emotional Data

[2295] The terminal collects emotion data from the user's voice, facial expressions, input text, etc. and sends it to the emotion engine.

[2296] 4. Completing and updating tasks

[2297] When a user completes a task, the device synchronizes the information with the server and updates the to-do list, removing the completed task from the list at the same time.

[2298] The role of the emotional engine

[2299] The emotion engine analyzes the user's emotional state and implements the following functions:

[2300] 1. Recognizing emotional states

[2301] The emotion engine uses technologies such as voice recognition, facial expression analysis, and text analysis to recognize the user's emotional state (e.g., stress, fatigue, concentration, joy).

[2302] 2. Data transmission

[2303] The recognized emotional state is transmitted to the server in real time and used as reference information for task management.

[2304] User Roles

[2305] The user is the ultimate beneficiary of this system and performs the following actions:

[2306] 1. Check the list

[2307] Users can check their to-do list on their device and understand the details and deadlines of each task.

[2308] 2. Task Processing

[2309] The user can then work on the displayed tasks in order of priority, marking them as completed on the device and updating the list.

[2310] 3. Providing Emotion Data

[2311] The user expresses voice and facial expressions in a natural way to the terminal, and provides them to the emotion engine as emotion data.

[2312] Specific examples

[2313] An example of the system's operation is shown below.

[2314] Example 1: Creating a task from an email

[2315] Email content received by the server: "Yamada-san, please submit your report by next Monday. Thank you."

[2316] Information extracted by the server: Name (Yamada), Request (report submission), Deadline (next Monday)

[2317] Server-generated task: "Report submission" (Deadline: next Monday)

[2318] To-do list notified on the device: "Report submission" (Deadline: next Monday)

[2319] Example 2: Changing task priorities with an emotion engine

[2320] The emotion engine analyzes the user's voice and facial expressions to recognize stress levels

[2321] The server dynamically changes the priority of tasks based on emotional information, and displays tasks that are easy to handle at the top.

[2322] Updated ToDo list notified on device: Priorities have been changed, with "Check documents" (deadline: tomorrow's meeting, light task) at the top

[2323] This system not only allows users to efficiently manage tasks and prevent oversights, but also allows them to respond flexibly according to their emotional state.

[2324] The processing flow will be explained below.

[2325] Step 1:

[2326] The server periodically (for example, every 5 minutes) launches a task and accesses an information transmission medium (email server, chat tool server, web chat server).

[2327] Specifically, it logs in to the mail server using the IMAP or POP3 protocol, and retrieves new messages from the chat tool using a dedicated API.

[2328] Step 2:

[2329] The server analyzes new messages received from each information carrier.

[2330] The analysis uses natural language processing (NLP) technology to extract information such as the message body, the date and time of sending, and the sender, with particular attention paid to your name, address, mentions, and designated keywords (such as "request," "confirmation," and "question").

[2331] Step 3:

[2332] The server generates a summarized task content based on the information extracted from the parsed message.

[2333] For example, from an email that says, "Yamada, please submit your report by next Monday," you can extract the name "Yamada," the request to "submit the report," and the deadline "next Monday," and create a task called "Submit report" (deadline: next Monday).

[2334] Step 4:

[2335] The server stores the generated tasks in a database and integrates the to-do list.

[2336] Tasks are saved in a database, taking care to avoid duplication. Saved tasks are organized by user and managed centrally.

[2337] Step 5:

[2338] The server organizes the tasks on the to-do list based on priority.

[2339] The order of the list is determined by the urgency of the task and its deadline. Tasks with upcoming deadlines are given high priority.

[2340] Step 6:

[2341] The server monitors the deadlines of tasks and displays alert marks and colors on tasks whose deadlines are approaching.

[2342] For example, tasks with a deadline within two days will be marked with a red alert mark.

[2343] Step 7:

[2344] The emotion engine collects data such as the user's voice, facial expressions, and input text to analyze the user's emotional state.

[2345] Using voice analysis and facial expression analysis technology, the system assesses the user's stress level, fatigue state, concentration, etc.

[2346] Step 8:

[2347] Based on the emotion data from the emotion engine, the server dynamically changes the priority of tasks.

[2348] For example, if the emotion engine detects that the user is in a high stress state, the server will prioritize and display relatively light tasks.

[2349] Step 9:

[2350] Based on the emotion data from the emotion engine, the server adjusts how the task is notified.

[2351] For example, if the server detects that the user is concentrating, it will make adjustments such as reducing notification sounds and reducing the number of pop-up notifications.

[2352] Step 10:

[2353] The server notifies the user's device of the generated ToDo list.

[2354] Delivery to devices is via email, dedicated app notification APIs, and web app update functions.

[2355] Step 11:

[2356] The terminal displays the ToDo list received from the server to the user.

[2357] The list is displayed in a table or calendar format, with tasks with alerts highlighted, and the list view is based on an emotion engine.

[2358] Step 12:

[2359] Users can check their to-do list on their device and process tasks based on their content and priority.

[2360] When the task is complete, the user marks the task as complete on the terminal.

[2361] Step 13:

[2362] The terminal synchronizes the information of the user's task completion with the server.

[2363] The server removes the completed tasks from the database and updates the current to-do list.

[2364] The above is the specific processing flow of this system. By combining it with an emotion engine, flexible task management that responds to the user's emotional state is realized, enabling efficient work execution.

[2365] Example 2

[2366] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2367] In modern society, with the massive volume of messages received daily from various information transmission media, it is difficult for users to efficiently manage their tasks. Simply listing tasks without considering the user's emotions and state increases stress and burden, and reduces efficiency. Furthermore, there is a lack of methods for properly communicating task deadlines and importance. This leads to problems such as users overlooking tasks or failing to respond appropriately to important tasks.

[2368] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for periodically acquiring new messages from an information transmission medium, means for analyzing the content of the acquired messages and extracting specified keywords, the user's name, address, mentions, and deadline date and time information, means for summarizing the extracted information and organizing them as tasks, means for integrating the organized tasks into a single to-do list and displaying them with priority, means for marking or coloring tasks with an alert mark when the deadline is approaching, means for collecting user emotion data from the terminal and transmitting it to the emotion engine, means for receiving input from the emotion engine and dynamically changing task priorities in consideration of the user's emotional state, means for adjusting the notification method based on the input from the emotion engine, and means for notifying the user of the to-do list on the terminal. This allows the user to efficiently manage tasks and flexibly respond to their emotional state.

[2369] "Information transmission medium" refers to systems and services for transmitting information and sending and receiving messages and data, such as email servers, chat tool servers, and web chat servers.

[2370] A "new message" refers to a message that is received for the first time through an information transmission medium and has not yet been processed by a user.

[2371] "Analysis" refers to the process of analyzing the content of captured messages and extracting specific information or patterns.

[2372] "Keywords" are words or phrases with a specific meaning or purpose that are used to extract important information from the content of a message.

[2373] An "emotion engine" is a system that recognizes the user's emotional state and collects and analyzes data from voice, facial expressions, input text, etc.

[2374] A "task" refers to a specific action or task that a user needs to perform, and is organized and managed as a to-do list.

[2375] A "ToDo list" is a list that lists a user's tasks and indicates information such as their priority and deadlines.

[2376] An "alert mark" refers to a mark or color that is displayed to warn or draw attention to a specific situation or task that is approaching a deadline.

[2377] "Priority" refers to a ranking that determines which tasks should be processed first based on the importance and urgency of the tasks.

[2378] "Emotional state" refers to the user's current psychological state or feelings, and includes specific sensations and emotions such as stress, fatigue, and concentration.

[2379] "Notification method" refers to the means or format for notifying the user of a task or alert, and includes audio, visual, banner display, etc.

[2380] "Device" refers to a device that is directly operated by a user, including, for example, a computer, a smartphone, or a tablet.

[2381] "Integration" refers to the process of combining multiple elements or pieces of information into one system or list.

[2382] The system for implementing this invention involves four elements - a server, a terminal, a user, and an emotion engine - working together to achieve task management that takes into account the user's emotions. The specific roles of each element and the processing content of the program are explained below.

[2383] Server Roles

[2384] 1. Data Acquisition

[2385] The server periodically retrieves new messages from the email server, chat tool server, and web chat server, using the API of each medium. For example, IMAP or POP3 protocols are used to retrieve emails from the mail server, and a dedicated API is used to retrieve messages from the chat tool.

[2386] 2. Message Analysis

[2387] ...

Claims

1. means for periodically retrieving new messages from the information carrier; A means for analyzing the contents of the acquired message and extracting specified keywords, one's own name, address, mentions, and deadline date and time information; A means for summarizing the extracted information and organizing it into tasks; A way to organize and prioritize your tasks in a single to-do list. A way to mark and color tasks with alerts when the deadline is approaching, A means for notifying the user of the ToDo list on their device; A system including:

2. 2. The system according to claim 1, wherein the information transmission medium is an email server, a chat tool server, or a web chat server.

3. 2. The system of claim 1, wherein the analyzing means analyzes message content using natural language processing techniques.

Images