System

Abstract

A system is provided.SOLUTION: The system includes a means for collecting information, a means for extracting an action item from the collected information, a means for listing the action item, a means for setting a prediction time limit for each action item, and a means for presenting the set action item list to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern business people communicate using multiple tools (email, chat, telephone, etc.) in their daily work, making it difficult to keep track of all of the content and respond appropriately. Overlooking or failing to respond to these business communications can cause serious business problems. Furthermore, manually aggregating information from each tool and creating a task list is extremely time-consuming. Therefore, there is a need for a system that can efficiently manage business communications and prevent missed responses. [Means for solving the problem]

[0005] The present invention provides a system that automatically collects information from business communication tools and extracts and lists action items. Specifically, the system includes a means for collecting information, a means for extracting action items from the collected information, a means for listing the action items, a means for setting a predicted deadline for each action item, and a means for presenting the list of set action items to the user. This system collects information from email servers, chat servers, and telephone record servers and extracts action items using natural language processing technology. It also uses predictive AI to automatically set a predicted deadline for each action item and present it to the user in an easy-to-understand manner, thereby improving work efficiency and preventing missed responses.

[0006] "Means for collecting information" refers to devices or software functions for automatically obtaining new information from email servers, chat servers, and telephone record servers.

[0007] An "action item extraction means" is a device or software function that provides natural language processing techniques and algorithms to identify important tasks or instructions from collected information and distinguish them from other information.

[0008] The "means for listing action items" is a device or software function for organizing the extracted action items and displaying them as a list.

[0009] A "means for setting a predicted deadline" is a device or software function that provides predictive AI or algorithms for predicting and automatically setting deadlines for action items.

[0010] A "means for presenting to a user" is a device or software function for visually displaying the action item list and projected deadlines to a user.

[0011] "Natural language processing technology" is a computer technology for understanding and analyzing human language, and is a means of extracting meaning and structure from text.

[0012] "Predictive AI" is an artificial intelligence technology that forecasts future events and outcomes based on past data and patterns.

[0013] An "email server" is a computer system that manages and stores the sending and receiving of e-mail.

[0014] A "chat server" is a computer system for providing and storing real-time messaging services.

[0015] A "telephone record server" is a computer system for storing and managing telephone call records. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that automatically extracts information from various business communication tools, lists action items, and sets expected deadlines. This system is designed to enable business people to prevent oversight of business tasks and manage tasks efficiently.

[0038] Form of information collection

[0039] The server periodically collects information from the email server, chat server, and telephone record server. From the email server, it retrieves unread emails and all emails within a specified period, from the chat server it retrieves new messages, and from the telephone record server it retrieves new call logs. This allows you to centralize information from various business communication tools.

[0040] Examples:

[0041] The server calls the API of each tool at a fixed time (for example, every morning at 9:00) and obtains new information using the necessary authentication information (user name, password, API key, etc.). This information is temporarily stored in storage.

[0042] Forms of information extraction and analysis

[0043] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts.

[0044] Examples:

[0045] If an email from a boss includes instructions such as "Please submit a report by next week," the server will analyze it and extract the action item "Submit report." Similarly, if a project leader sends a chat message with instructions such as "Please prepare for a meeting," this will also be extracted.

[0046] How to generate an action item list

[0047] The server lists the extracted action items, including details of each task, its origin (email, chat, call log, etc.), and associated context information.

[0048] Examples:

[0049] The server generates the following list:

[0050] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0051] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0052] Form of forecast deadline setting

[0053] The server automatically sets a predicted deadline for each action item using predictive AI, which takes into account past task history, task importance, sender priority, and other factors to set an appropriate deadline for each task.

[0054] Examples:

[0055] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past, and a deadline for the "meeting preparation" task the next day. In this way, the expected deadlines set for each task are reflected in the list.

[0056] List presentation format

[0057] The terminal presents the user with a list of the generated action items, allowing the user to quickly respond to each action item by referring to the list.

[0058] Examples:

[0059] The user sees the following list on their PC or smartphone screen:

[0060] 1. "Report submission" - Deadline: 3 days later, Original email: Instructions from boss

[0061] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0062] In this way, the system helps business people to understand the details of business communications thoroughly and efficiently manage and complete tasks.

[0063] The processing flow will be explained below.

[0064] Step 1: Gather information

[0065] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information. It downloads unread emails from the email server via IMAP or POP3 protocol, retrieves new messages from the chat server via API, and retrieves new call logs from the telephone record server. This information is then stored in temporary data storage.

[0066] Step 2: Preprocessing the information

[0067] The server preprocesses the collected raw data, which includes cleaning the data, extracting metadata, and formatting the text. For emails, basic information such as the body of the message, sender, and date and time of receipt is extracted. For chat messages, the server converts the sender and time of the message into text, and for phone logs, the server converts the duration and content of the call into text.

[0068] Step 3: Extract action items

[0069] The server then applies natural language processing (NLP) to the preprocessed data to extract action items. The NLP module performs contextual analysis to identify tasks, instructions, and important information, and then identifies action items based on that. This process extracts and lists task names and related keywords.

[0070] Step 4: Generate an action item list

[0071] The server generates an action item list based on the extracted action items. This list includes details of each action item (e.g., task name, source, related information). The action items are stored in a database for further processing.

[0072] Step 5: Set the forecast deadline

[0073] The server uses predictive AI to set deadlines for each action item. The predictive AI analyzes past task history, task importance, and sender priority to estimate the optimal deadline for each action item. The set deadline is added to the action item list.

[0074] Step 6: Present the list to the user

[0075] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list displays task details and set deadlines, allowing the user to visually manage tasks.

[0076] In this way, the system can centrally collect business communication information from multiple tools, efficiently manage action items, and prevent missed responses.

[0077] Example 1

[0078] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0079] Modern business people use a wide variety of business communication tools, and because information is scattered across them, there is a high risk of missing important tasks or instructions. Furthermore, manually retrieving information from these communication tools and managing tasks is extremely time-consuming and inefficient. It is also difficult to appropriately set task importance and deadlines, leading to missed or delayed responses. Therefore, centralizing information management from business communication tools and efficient task management are key challenges.

[0080] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0081] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for presenting the set action item list to the user, means for periodically calling the API of each information source and obtaining information using authentication information, means for analyzing the collected information and identifying important tasks using natural language processing technology, and means for setting appropriate deadlines using predictive AI based on past task history and importance.This enables centralized management of information from various business communication tools, automatic extraction of tasks, and setting predicted deadlines, enabling efficient task management.

[0082] The "means for collecting information" refers to a function for acquiring information from each information source such as an email server, a chat server, and a telephone record server.

[0083] "Means for extracting action items" is a function for identifying and extracting important tasks and instructions from collected information.

[0084] "A means of listing action items" is a function for organizing extracted tasks and instructions and compiling them into a single list.

[0085] The "means for setting a predicted deadline" is a function for predicting and setting an appropriate completion deadline for each action item.

[0086] The "means for presenting the set action item list to the user" is a function for displaying the generated list so that the user can check it.

[0087] "Means of periodically calling the API of each information source and obtaining information using authentication information" refers to a function that periodically calls the API of an information source and obtains the necessary information using authentication information such as a username, password, or API key.

[0088] "Means for analyzing collected information and identifying important tasks using natural language processing technology" is a function for analyzing collected information using natural language processing technology and identifying important tasks and instructions from text data.

[0089] "Means for setting appropriate deadlines using predictive AI based on past task history and importance" is a function that uses predictive AI based on past task history and the importance of each task to automatically set appropriate deadlines for each task.

[0090] This invention is a system that automatically collects information from various business communication tools, extracts action items, sets predicted deadlines, and creates a list. Detailed embodiments for implementing this system are described below.

[0091] The system mainly includes three main components: a server, a terminal, and a user.

[0092] Form of information collection

[0093] The server is responsible for collecting information. Specifically, it periodically collects information from email servers, chat servers, and telephone record servers. To do this, it uses the APIs of each server. The server runs a scheduled job at a regular time, calling the APIs of each information source and retrieving the latest information using the necessary authentication information (username, password, API key, etc.). This information is then temporarily stored in storage.

[0094] Examples:

[0095] Every morning at 9:00, the server retrieves unread emails and all emails within a specified period from the email server API, retrieves new messages from the chat server API, and retrieves new call logs from the phone record server API.

[0096] Forms of information extraction and analysis

[0097] The server analyzes the collected information and extracts action items using natural language processing (NLP) technology. The server applies NLP algorithms to the collected emails, chat messages, and call logs, dividing each message and log into sentences and phrases, and extracting important keywords and phrases.

[0098] Examples:

[0099] If an email from a boss says, "Please submit a report by next week," the server extracts the action item "Submit report." Similarly, if a chat message from a project leader says, "Please prepare for a meeting," the server extracts the action item "Prepare for a meeting."

[0100] How to generate an action item list

[0101] The server generates a list based on the extracted action items. This list includes details of each action item, its origin (email, chat, phone log, etc.), and related context information. For each action item, the server adds the task name, origin, context information, etc. to the list.

[0102] Examples:

[0103] The server generates the following list:

[0104] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0105] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0106] Form of forecast deadline setting

[0107] The server uses predictive AI to set a predicted deadline for each action item. The predictive AI takes into account past task history, task importance, sender priority, etc. to set an appropriate deadline. The server retrieves and analyzes the history of similar past tasks from a database.

[0108] Examples:

[0109] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. It sets a deadline for the "meeting preparation" task the next day. In this way, the appropriate deadlines are reflected for each task.

[0110] List presentation format

[0111] The device presents the generated list of action items to the user. The server sends the generated list to the device, which then displays the received list on its screen (e.g., a PC or smartphone).

[0112] Examples:

[0113] A user checks the following list on their computer or smartphone:

[0114] 1. "Report submission" - Deadline: 3 days later, Original email: Instructions from boss

[0115] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0116] Specific prompt examples

[0117] Here are some examples of prompts for generative AI models:

[0118] Please design a system that extracts important tasks and instructions from my emails and chat messages. For example, from an email that says "Please turn in a report next week," it could extract the task "Submit report" and set a due date three days later using predictive AI. Also, please make this list available on my smartphone.

[0119] In this way, the system helps business people to efficiently manage the contents of business contacts and respond to them without any omissions.

[0120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0121] Step 1: Gather information

[0122] The server periodically executes a job to collect information. This job calls the APIs of the email server, chat server, and call record server using authentication information (username, password, API key, etc.) to obtain the latest information. The API endpoint and authentication information of each server are used as input, and the obtained data (unread emails, chat messages, new call logs) is saved in storage as output.

[0123] Specific behavior:

[0124] The scheduled job runs every morning at 9am.

[0125] Get unread emails and all emails within a specified period from the email server API.

[0126] Get new messages from the chat server API.

[0127] Get new call logs from the call log server API.

[0128] Step 2: Information extraction and analysis

[0129] The server analyzes the collected data and extracts action items. It uses natural language processing (NLP) techniques to analyze collected emails, chat messages, and call logs to extract important keywords and phrases. It uses the collected data as input and lists important tasks and instructions as output.

[0130] Specific behavior:

[0131] Divide the saved data into sentences or phrases.

[0132] Apply NLP algorithms to identify important keywords and phrases.

[0133] Extract the task "Submit report" from an email from your boss saying "Please submit the report strictly next week."

[0134] Extract the task "Prepare for the meeting" from the chat message from the project leader saying "Please prepare for the meeting."

[0135] Step 3: Generate a list of action items

[0136] The server generates a list from the extracted action items, including details of each task, its origin (email, chat, call log, etc.), and associated context information. It uses the extracted action items as input and generates a formatted list as output.

[0137] Specific behavior:

[0138] Add task name, origin, and context information for each action item.

[0139] The list includes tasks such as "Submit report" and "Prepare for meeting," along with their origin and related information.

[0140] Step 4: Set the forecast deadline

[0141] The server uses predictive AI to set a predicted deadline for each action item. It automatically sets an appropriate deadline by taking into account past task history, importance, sender priority, etc. It uses the history and importance of past similar tasks as input and sets a predicted deadline for each action item as output.

[0142] Specific behavior:

[0143] Retrieve historical data of past similar tasks from a database.

[0144] Predictive AI calculates an appropriate deadline based on importance and sender priority.

[0145] For example, you can set a deadline for "report submission" three days later and a deadline for "meeting preparation" the next day.

[0146] Step 5: Present the list

[0147] The terminal presents the generated list of action items to the user, receives the list sent from the server and displays it on the screen, and uses the list sent from the server as input and displays it in a format that can be viewed by the user as output.

[0148] Specific behavior:

[0149] The terminal receives the list from the server.

[0150] The user checks the list on their PC or smartphone screen.

[0151] The list will show items such as "Submit report" - due date: 3 days later, original email: instructions from boss, "Prepare for meeting" - due date: next day, original chat: instructions from project leader.

[0152] The above is the specific flow and operation of each processing step of this system. At each step, input is received, appropriate data processing and analysis is performed, and output is generated. This allows business people to efficiently manage business communications and respond quickly to required tasks.

[0153] (Application example 1)

[0154] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0155] The complexity of business communications in logistics facilities and its management issues are a problem. Business communications from many information sources are scattered, and there is a risk that workers may miss important tasks. Therefore, a system that can efficiently collect information and manage tasks is required.

[0156] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0157] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for presenting the list of set action items to a user, and means for collecting information from a business communication tool at the logistics facility and generating action items, thereby enabling personnel at the logistics facility to efficiently manage tasks and respond without overlooking important matters.

[0158] "Means for collecting information" refers to the function of periodically obtaining information from a specific server (such as an email server, chat server, or telephone record server).

[0159] "Means of extracting action items from collected information" refers to the ability to identify important tasks or instructions from the acquired information and identify them as individual action items.

[0160] "Means for listing action items" refers to a function for organizing and displaying extracted action items in a list format.

[0161] "Means for setting predicted deadlines" refers to the function of predicting and setting appropriate deadlines for each action item based on its importance and past history.

[0162] The "means for presenting the set action item list to the user" refers to a function for displaying the action items, which have been listed and have their predicted deadlines set, on the user's terminal.

[0163] "A means of collecting information from business communication tools at logistics facilities and generating action items" refers to the function of obtaining information from emails, chat messages, call records, etc. related to logistics operations, analyzing them, and listing them as action items.

[0164] The present invention is a system for automating and efficiently managing information collection and tasks from business communication tools in a logistics facility. Specific embodiments of the system are described below.

[0165] Form of information collection

[0166] The server has a mechanism for periodically collecting information from the email server, chat server, and telephone record server. It retrieves unread emails and all emails within a specified period from the email server, new messages from the chat server, and new call logs from the telephone record server. This information collection is generally performed using the API of each server. The server retrieves new information using the necessary authentication information (username, password, API key, etc.), and this information is temporarily stored in storage.

[0167] Forms of information extraction and analysis

[0168] The server analyzes the collected information and uses natural language processing (NLP) to extract action items. Specifically, it uses Hugging Face's transformers library to automatically identify important tasks and instructions from email text, chat messages, and call logs. For example, if an email from a boss contains the instruction "Please submit your report punctually next week," the server analyzes it and extracts the action item "Submit report."

[0169] How to generate an action item list

[0170] The server generates a list of extracted action items, including details of each task, its origin (email, chat, phone log, etc.), and any associated context information. An example listing might look like this:

[0171] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0172] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0173] Form of forecast deadline setting

[0174] The server automatically sets predicted deadlines based on the collected information. Specifically, it sets an appropriate deadline for each task, taking into account past task history, task importance, sender priority, etc. For example, for the task "Submit a report," it sets a deadline of three days in the future based on the history of similar past tasks.

[0175] List presentation format

[0176] The device presents the generated list of action items to the user. The user can refer to this list and quickly respond to each action item. An example of how a user can view the list of action items on a PC or smartphone screen is shown below.

[0177] 1. "Submit report" - Deadline: 3 days later, Original email: Instructions from boss

[0178] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0179] This system helps logistics facility staff centrally manage information from multiple business communication tools, helping them manage and complete tasks efficiently and without omissions.

[0180] Specific examples and prompts for the generative AI model

[0181] For example, a logistics facility operator might receive:

[0182] "Please check the parts availability by next week" (email)

[0183] "Please prepare for the meeting starting at 2pm" (chat)

[0184] "Please complete the packaging for the new product by tomorrow." (Call notes)

[0185] Example prompt sentence:

[0186] "Identify action items in emails and set appropriate deadlines."

[0187] "Identify the important tasks from the text below and set expected deadlines."

[0188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0189] Step 1: Gather information

[0190] The server obtains information from the email server, chat server, and telephone record server. It uses each server's API to collect new information using the necessary authentication information (username, password, API key, etc.). The collected information is temporarily stored in storage. This allows all contact information related to the user's work to be managed centrally. The input is new message data from each server, and the output is unanalyzed information in temporary storage.

[0191] Step 2: Analyze the information and extract action items

[0192] The server analyzes the collected information using natural language processing (NLP) technology. Specifically, it uses Hugging Face's transformers library to automatically identify important tasks and instructions from email text, chat messages, and call records. It extracts action items from the analysis results and stores them in temporary storage. The input is unanalyzed information, and the output is the extracted action items.

[0193] Step 3: Generate an action item list

[0194] The server lists the extracted action items, including details of each task, its origin (email, chat, phone log, etc.), and related context information. The listed action items are used as data to set the expected deadline in a later step. The input is the extracted action items, and the output is the listed action items.

[0195] Step 4: Set the forecast deadline

[0196] The server sets predicted deadlines for the listed action items. It sets an appropriate deadline for each task, taking into account factors such as past task history, task importance, and sender priority. This prediction is performed using a generative AI model. The input is the listed action items and past task history, and the output is the action items with predicted deadlines.

[0197] Step 5: View the Action Items List

[0198] The terminal presents the user with a list of action items with predicted deadlines sent from the server. The user can refer to this list and quickly respond to each action item. For example, the user checks the list on a smartphone or PC screen. The input is the action items with predicted deadlines, and the output is the list of action items displayed to the user.

[0199] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0200] This invention is a system that automatically extracts information from various business communication tools, lists action items, sets expected deadlines, and optimizes task management using an emotion engine that recognizes the user's emotions. This system is designed to enable business people to prevent oversight of work-related actions, efficiently manage tasks, and flexibly adjust responses based on the user's emotions.

[0201] Form of information collection

[0202] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information, including unread emails from the email server, new messages from chat, and new call logs from the telephone record server, and temporarily stores this information in storage.

[0203] Examples:

[0204] Every morning at 9am, the server calls each tool's API and retrieves new information using the necessary authentication information. This information is stored in a temporary database.

[0205] Forms of information extraction and analysis

[0206] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts and categorizing them into action items.

[0207] Examples:

[0208] If an email from a boss includes the instruction "Please submit a report by next week," the server will parse it and extract the action item "Submit report." Similarly, if a chat message from a project leader includes the instruction "Please prepare for a meeting," this will also be extracted.

[0209] How to generate an action item list

[0210] The server lists the extracted action items, including details of each action item (e.g., task name, source, related information), and stores the list in a database.

[0211] Examples:

[0212] The server generates the following list:

[0213] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0214] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0215] Form of forecast deadline setting

[0216] The server automatically sets the expected deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline for each action item.

[0217] Examples:

[0218] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. For the "meeting preparation" task, it sets a deadline for the next day. The deadlines set in this way are reflected in the list.

[0219] Forms of emotion recognition

[0220] The server combines an emotion engine that recognizes emotions from the user's facial expressions, voice, input data, etc. If it determines that the user is feeling stressed, it automatically adjusts the priority and deadlines of action items.

[0221] Examples:

[0222] While the user is reviewing the action item list, the server analyzes the user's facial expressions and voice tone via the camera and microphone, and the emotion engine determines the user's stress level. If stress is determined to be high, the server extends the deadline for "report submission" by one day and moves "meeting preparation" up the list as a high-priority task.

[0223] How the list is presented to the user

[0224] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list, adjusted by the emotion engine, helps the user effectively manage tasks while reducing stress.

[0225] Examples:

[0226] The user sees the following list on their PC screen:

[0227] 1. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0228] 2. "Report submission" - Deadline: 4 days later (extended), Original email: Instructions from boss

[0229] In this way, the system can centrally collect business communication information from multiple tools, efficiently manage action items, prevent missed responses, and flexibly adjust responses according to the user's emotions.

[0230] The processing flow will be explained below.

[0231] Step 1: Gather information

[0232] The server periodically accesses the email server, chat server, and telephone record server to collect new information. It downloads unread emails from the email server using IMAP or POP3 protocol, retrieves the latest messages from the chat server using API, and retrieves new call logs from the telephone record server. This information is temporarily stored in a database.

[0233] Step 2: Preprocessing the information

[0234] The server preprocesses the collected raw data, which includes cleaning the data, extracting metadata, and formatting the text. For emails, basic information such as the body of the email, sender, and date and time of receipt is extracted, while chat messages are converted into text containing the sender and time of the call, and call logs are converted into text containing the duration and content of the call.

[0235] Step 3: Extract action items

[0236] The server applies natural language processing (NLP) to the preprocessed data to extract action items. The NLP engine identifies tasks, instructions, and important information in the text and identifies them as action items. Specifically, it extracts keywords related to instructions and tasks from each email, chat message, and call log.

[0237] Step 4: Generate an action item list

[0238] The server compiles the extracted action items into a list, which includes details of each action item (e.g., task name, source, related information), and stores the action items in a database for further processing.

[0239] Step 5: Set the forecast deadline

[0240] The server automatically sets a predicted deadline for each action item using predictive AI. The predictive AI analyzes past task history, task importance, and sender priority to estimate the optimal deadline for each action item. The set deadline is added to the action item list.

[0241] Step 6: Emotion Recognition

[0242] The server uses an emotion engine that recognizes emotions from the user's facial expressions, voice, and input data. The emotion engine detects when the user is stressed or tired and reflects this in the management of action items.

[0243] Step 7: Adjusting tasks based on emotions

[0244] The server adjusts the priority and deadline of action items based on the user's emotional data obtained by the emotion engine. For example, if it detects that the user is feeling high stress, it extends the deadline or changes the priority of some action items.

[0245] Step 8: Present the list to the user

[0246] The device presents the final action item list to the user. The user can check the list on a device such as a PC or smartphone and work on the tasks efficiently. In addition, by checking the list adjusted by the emotion engine, the user can manage tasks in a less burdensome way.

[0247] By executing the above processing steps, this system can centrally manage business contact information and optimize task management while taking into consideration the user's feelings. This system allows business people to avoid overlooking work-related tasks and perform their work efficiently while reducing stress.

[0248] Example 2

[0249] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0250] Modern business people use multiple business communication tools, such as email, chat, and phone records, and are required to efficiently manage information from these tools. However, manually collecting this information, extracting important tasks, setting deadlines, and adjusting priorities based on emotions is extremely time-consuming and labor-intensive. Furthermore, task management based on stress and emotional states is often not possible, making it difficult to improve work efficiency and prevent missed responses.

[0251] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting information, a means for extracting action items from the collected information, a means for listing the action items, a means for setting a predicted deadline for each action item, a means for recognizing the user's emotions and adjusting the priority and deadline of the action items, and a means for presenting the set action item list to the user. This makes it possible to centrally manage information from business communication tools and efficiently extract, list, set deadlines for, and adjust tasks according to emotions.

[0252] The "means for collecting information" is a mechanism for automatically obtaining information at specified intervals from multiple information sources such as an email server, a chat server, and a telephone record server.

[0253] The "means for extracting action items from collected information" refers to a mechanism that analyzes the acquired information, identifies important tasks and instructions using natural language processing technology, and extracts them as action items.

[0254] The "means for listing action items" is a mechanism for listing extracted action items including task names, sources, and related information, and storing the list in a database.

[0255] The "means of setting predicted deadlines for each action item" is a mechanism that uses predictive AI to analyze past task history, task importance, and sender priority to estimate appropriate deadline dates.

[0256] The "means for recognizing the user's emotions and adjusting the priority and deadlines of action items" is a mechanism that automatically adjusts the priority and deadlines of action items according to the user's stress level using an emotion engine that identifies emotions from the user's facial expressions, voice, input data, etc.

[0257] The "means for presenting the set action item list to the user" is a mechanism for visually presenting the generated and adjusted action item list to the user via a terminal such as a PC or smartphone.

[0258] This invention is a system that automatically extracts information from various business communication tools, lists action items, sets expected deadlines, and optimizes task management using an emotion engine that recognizes the user's emotions. This system is designed to enable business people to prevent oversight of work-related actions, efficiently manage tasks, and flexibly adjust responses based on the user's emotions.

[0259] A specific embodiment for carrying out the present invention will be described.

[0260] Form of information collection

[0261] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information, including unread emails from the email server, new messages from chat, and new call logs from the telephone record server, and temporarily stores this information in storage.

[0262] Examples:

[0263] Every morning at 9am, the server calls each tool's API and retrieves new information using the necessary authentication information. This information is stored in a temporary database.

[0264] Forms of information extraction and analysis

[0265] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts and categorizing them into action items.

[0266] Examples:

[0267] If an email from a boss includes the instruction "Please submit a report by next week," the server will parse it and extract the action item "Submit report." Similarly, if a chat message from a project leader includes the instruction "Please prepare for a meeting," this will also be extracted.

[0268] How to generate an action item list

[0269] The server lists the extracted action items, including details of each action item (e.g., task name, source, related information), and stores the list in a database.

[0270] Examples:

[0271] The server generates the following list:

[0272] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0273] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0274] Form of forecast deadline setting

[0275] The server automatically sets the expected deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline for each action item.

[0276] Examples:

[0277] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. For the "meeting preparation" task, it sets a deadline for the next day. The deadlines set in this way are reflected in the list.

[0278] Forms of emotion recognition

[0279] The server combines an emotion engine that recognizes emotions from the user's facial expressions, voice, input data, etc. If it determines that the user is feeling stressed, it automatically adjusts the priority and deadlines of action items.

[0280] Examples:

[0281] While the user is reviewing the action item list, the server analyzes the user's facial expressions and voice tone via the camera and microphone, and the emotion engine determines the user's stress level. If stress is determined to be high, the server extends the deadline for "report submission" by one day and moves "meeting preparation" up the list as a high-priority task.

[0282] How the list is presented to the user

[0283] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list, adjusted by the emotion engine, helps the user effectively manage tasks while reducing stress.

[0284] Examples:

[0285] The user sees the following list on their PC screen:

[0286] 1. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0287] 2. "Report submission" - Deadline: 4 days later (extended), Original email: Instructions from boss

[0288] Example prompts for generative AI models

[0289] "How do you operate a system that extracts important tasks from emails, chats, and phone records, automatically sets deadlines, performs sentiment analysis, and adjusts priorities?"

[0290] This allows the system to centrally collect business communication information from multiple tools, efficiently manage action items, prevent missed responses, and flexibly adjust responses based on the user's emotions.

[0291] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0292] Step 1:

[0293] Information gathering

[0294] The server accesses the email server, chat server, and phone record server at a specified interval (for example, every morning at 9:00) to collect new information. This includes unread emails from email, new messages from chat, and new call logs from phone records. The server retrieves this information via API and stores it in a temporary database.

[0295] Input: New information (unread emails, new chat messages, new call logs) retrieved from the APIs of the email, chat, and call log servers.

[0296] Data processing: Collecting new information from each server and formatting it for temporary storage

[0297] Output: New information stored in a temporary database

[0298] Specific operation: The server calls each tool's API with the authentication information, obtains new information, and saves it.

[0299] Step 2:

[0300] Information analysis and action item extraction

[0301] The server uses natural language processing techniques to analyze the collected information and extract action items, identifying important tasks and instructions from email text, chat messages, and call transcripts.

[0302] Input: New information stored in a temporary database

[0303] Data processing: Using natural language processing techniques, we analyze important tasks and instructions and categorize them as action items.

[0304] Output: Extracted action items

[0305] What it does: The server uses natural language processing algorithms to analyze and identify imperative and task-related phrases in a sentence.

[0306] Step 3:

[0307] List of action items

[0308] The server lists the extracted action items and stores them in a database, including details of each action item (e.g., task name, source, related information).

[0309] Input: Extracted action items

[0310] Data processing: Format and list the details of each action item

[0311] Output: List of action items stored in the database

[0312] What it does: The server stores each action item in a database along with the task name, source, and other relevant information.

[0313] Step 4:

[0314] Setting a forecast deadline

[0315] The server sets a predicted deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline.

[0316] Input: A list of action items stored in a database

[0317] Data processing: Use predictive AI to estimate deadlines for each action item

[0318] Output: A list of action items with deadlines

[0319] Specific operation: The server analyzes past task history data and sets a predicted deadline based on the execution period of similar tasks.

[0320] Step 5:

[0321] Emotion recognition and action item adjustment

[0322] The server uses an emotion engine that recognizes emotions from the user's facial expressions, voice, and input data, and adjusts the priority and deadlines of action items if the user is feeling stressed.

[0323] Input: Action item list with deadlines, user facial expression data, voice data, and input data

[0324] Data processing: Analyze user emotions using an emotion engine and adjust the priority and deadlines of action items according to stress levels

[0325] Output: A tailored list of action items

[0326] How it works: The server uses a camera and microphone to analyze the user's facial expressions and voice tone in real time, and the emotion engine determines the user's stress level and adjusts action items.

[0327] Step 6:

[0328] Presenting the list to the user

[0329] The device presents the generated action item list to the user, who can then check the action item list through a device such as a PC or smartphone and quickly respond to the tasks.

[0330] Input: Adjusted action item list

[0331] Output: A list of action items presented to the user

[0332] Specific behavior: The device displays the action item list in the user interface so that the user can easily check it.

[0333] In this way, the system can optimize task management by consistently collecting information, extracting action items, listing them, setting deadlines, recognizing emotions, and presenting them to the user.

[0334] (Application example 2)

[0335] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0336] Conventional task management systems have difficulty centrally collecting information from multiple business communication tools and managing important tasks. Furthermore, they are unable to adjust task priorities and deadlines based on user emotions, which can lead to the accumulation of stress and fatigue. Furthermore, in busy environments like factories, information is often overlooked or missed, often resulting in reduced work efficiency. To solve these problems, the present invention proposes a system that provides centralized information management and task optimization based on user emotions.

[0337] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0338] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for recognizing the user's emotions and adjusting the priority and deadline of the action items, and means for presenting the set action item list to the user. This enables centralized management of important tasks from multiple information sources and efficient task management that takes the user's emotional state into consideration.

[0339] The "means for collecting information" is a system that has the function of automatically acquiring necessary data from multiple information sources such as an email server, a chat server, and a communication log server.

[0340] The "means for extracting action items" is a system that utilizes natural language processing technology to identify and separate important tasks and instructions from collected information.

[0341] The "means for creating a list of action items" is a system that has the function of organizing and displaying the extracted action items in a list format and generating a list including detailed information about each item.

[0342] The "means for setting estimated deadlines" is a system that has the ability to automatically estimate appropriate deadlines for action items based on past task history, task importance, and sender priority.

[0343] "Means for recognizing user emotions and adjusting the priority and deadlines of action items" refers to a system that uses a camera and microphone to analyze the user's facial expressions and voice, determine the user's emotional state, and dynamically change task priorities and deadlines.

[0344] "Means for presenting the set action item list to the user" refers to a system that has the function of displaying the generated action item list on the user's device (e.g., smartphone, tablet, PC) and enabling a quick response.

[0345] A system for carrying out the present invention is configured as follows: A server has means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for recognizing a user's emotions and adjusting the priority and deadlines of the action items, and means for presenting the set action item list to the user.

[0346] The server first collects information from email servers, chat servers, communication log servers, etc. During this process, for example, every morning at 9:00, it calls the API of each tool and retrieves new information using the necessary authentication information. This information is then temporarily stored in a database.

[0347] The collected information is analyzed using natural language processing technology (e.g., libraries such as NLTK and spaCy) to extract necessary action items. For example, if an email from a boss contains the instruction "Please submit the report by next week," the server analyzes it and extracts the action item "Submit the report."

[0348] The extracted action items are then compiled into a list, which includes details about each action item (e.g., task name, source, related information), and is displayed in a user-friendly interface and stored in a database.

[0349] Additionally, a predictive AI model (e.g., a model using Tensorflow® or PyTorch) is used to set an appropriate deadline for each action item. The predictive AI estimates deadlines based on past task history, task importance, and the sender's priority.

[0350] The system also incorporates an emotion engine (e.g., Affectiva SDK) that uses the smartphone's camera and microphone to recognize the user's emotions. If the engine determines that the user is feeling stressed, it automatically adjusts the priority and deadline of action items. For example, if the server determines that the user is feeling very stressed, it will extend the deadline for "report submission" by one day and move "meeting preparation" up the list as a high-priority task.

[0351] Finally, the device (smartphone, tablet, or PC) presents the generated action item list to the user, allowing them to efficiently manage their tasks and accomplish their work while reducing stress.

[0352] As a concrete example, consider an environment where many work instructions are generated within a factory. Using a factory task manager app, all work instructions can be centrally managed, natural language processing technology can be used to automatically extract tasks from the instructions, an emotion recognition engine can be used to check the stress levels of workers, and predictive AI can be used to set appropriate deadlines. In this way, work instructions will not be overlooked and tasks will be carried out efficiently.

[0353] An example of a prompt would be:

[0354] "Develop an application that collects work instructions in a factory, analyzes them, and uses emotion recognition to prioritize and efficiently manage tasks."

[0355] In this way, the present invention can provide unified management of information and task optimization based on user sentiment.

[0356] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0357] Step 1:

[0358] The server uses the means of collecting information to call the API of each communication tool (email server, chat server, communication log server) at specified intervals (e.g., every morning at 9:00) to obtain new information. The input is the authentication information and API endpoint of each tool, and the output is the newly obtained emails, chat messages, and call records. The information is temporarily stored in a database.

[0359] Step 2:

[0360] The server uses natural language processing technology to analyze the information collected in step 1 and extract action items. Specifically, it tokenizes the email body, chat messages, and call records and performs grammatical analysis. The input is the unanalyzed information collected in step 1, and the output is the extracted action items (e.g., "Submit a report" or "Prepare for a meeting").

[0361] Step 3:

[0362] The server lists the extracted action items. The input is the action items extracted in step 2, and the output is a list of their details (e.g., task name, source, related information). This list is stored in a database in a user-friendly format.

[0363] Step 4:

[0364] The server uses a predictive AI model to set a predicted deadline for each action item. The input is the action item list generated in step 3 and past task history data, and the output is a list of action items with predicted deadlines. The AI ​​model is built using TensorFlow and PyTorch.

[0365] Step 5:

[0366] The server recognizes the user's emotions using the smartphone's camera and microphone. The emotion engine (e.g., Affectiva SDK) analyzes the user's facial expressions and voice to determine their stress and fatigue levels. The input is real-time data from the camera and microphone, and the output is the user's emotional state (e.g., high stress, low stress).

[0367] Step 6:

[0368] The server automatically adjusts the priority and deadlines of action items based on the emotion recognition results in step 5. The input is the action item list from step 4 and the emotion recognition results from step 5, and the output is the adjusted action item list. For example, if the user is in a high stress state, the deadline for "submitting a report" is extended and "preparing for a meeting" is moved up in the list.

[0369] Step 7:

[0370] The device presents the configured action item list to the user. The input is the action item list adjusted in step 6, and the output is the action item list displayed on the device (smartphone, tablet, PC) screen. The user can check this and manage tasks efficiently.

[0371] In this way, the present invention unifies data from multiple sources and provides optimized task management through natural language processing, emotion recognition, and predictive AI models.

[0372] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0373] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0374] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0375] [Second embodiment]

[0376] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0377] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0378] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0379] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0380] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0381] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0382] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0383] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0384] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0385] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0386] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0387] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0388] This invention is a system that automatically extracts information from various business communication tools, lists action items, and sets expected deadlines. This system is designed to enable business people to prevent oversight of business tasks and manage tasks efficiently.

[0389] Form of information collection

[0390] The server periodically collects information from the email server, chat server, and telephone record server. From the email server, it retrieves unread emails and all emails within a specified period, from the chat server it retrieves new messages, and from the telephone record server it retrieves new call logs. This allows you to centralize information from various business communication tools.

[0391] Examples:

[0392] The server calls the API of each tool at a fixed time (for example, every morning at 9:00) and obtains new information using the necessary authentication information (user name, password, API key, etc.). This information is temporarily stored in storage.

[0393] Forms of information extraction and analysis

[0394] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts.

[0395] Examples:

[0396] If an email from a boss includes instructions such as "Please submit a report by next week," the server will analyze it and extract the action item "Submit report." Similarly, if a project leader sends a chat message with instructions such as "Please prepare for a meeting," this will also be extracted.

[0397] How to generate an action item list

[0398] The server lists the extracted action items, including details of each task, its origin (email, chat, call log, etc.), and associated context information.

[0399] Examples:

[0400] The server generates the following list:

[0401] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0402] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0403] Form of forecast deadline setting

[0404] The server automatically sets a predicted deadline for each action item using predictive AI, which takes into account past task history, task importance, sender priority, and other factors to set an appropriate deadline for each task.

[0405] Examples:

[0406] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past, and a deadline for the "meeting preparation" task the next day. In this way, the expected deadlines set for each task are reflected in the list.

[0407] List presentation format

[0408] The terminal presents the user with a list of the generated action items, allowing the user to quickly respond to each action item by referring to the list.

[0409] Examples:

[0410] The user sees the following list on their PC or smartphone screen:

[0411] 1. "Report submission" - Deadline: 3 days later, Original email: Instructions from boss

[0412] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0413] In this way, the system helps business people to understand the details of business communications thoroughly and efficiently manage and complete tasks.

[0414] The processing flow will be explained below.

[0415] Step 1: Gather information

[0416] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information. It downloads unread emails from the email server via IMAP or POP3 protocol, retrieves new messages from the chat server via API, and retrieves new call logs from the telephone record server. This information is then stored in temporary data storage.

[0417] Step 2: Preprocessing the information

[0418] The server preprocesses the collected raw data, which includes cleaning the data, extracting metadata, and formatting the text. For emails, basic information such as the body of the message, sender, and date and time of receipt is extracted. For chat messages, the server converts the sender and time of the message into text, and for phone logs, the server converts the duration and content of the call into text.

[0419] Step 3: Extract action items

[0420] The server then applies natural language processing (NLP) to the preprocessed data to extract action items. The NLP module performs contextual analysis to identify tasks, instructions, and important information, and then identifies action items based on that. This process extracts and lists task names and related keywords.

[0421] Step 4: Generate an action item list

[0422] The server generates an action item list based on the extracted action items. This list includes details of each action item (e.g., task name, source, related information). The action items are stored in a database for further processing.

[0423] Step 5: Set the forecast deadline

[0424] The server uses predictive AI to set deadlines for each action item. The predictive AI analyzes past task history, task importance, and sender priority to estimate the optimal deadline for each action item. The set deadline is added to the action item list.

[0425] Step 6: Present the list to the user

[0426] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list displays task details and set deadlines, allowing the user to visually manage tasks.

[0427] In this way, the system can centrally collect business communication information from multiple tools, efficiently manage action items, and prevent missed responses.

[0428] Example 1

[0429] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0430] Modern business people use a wide variety of business communication tools, and because information is scattered across them, there is a high risk of missing important tasks or instructions. Furthermore, manually retrieving information from these communication tools and managing tasks is extremely time-consuming and inefficient. It is also difficult to appropriately set task importance and deadlines, leading to missed or delayed responses. Therefore, centralizing information management from business communication tools and efficient task management are key challenges.

[0431] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0432] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for presenting the set action item list to the user, means for periodically calling the API of each information source and obtaining information using authentication information, means for analyzing the collected information and identifying important tasks using natural language processing technology, and means for setting appropriate deadlines using predictive AI based on past task history and importance.This enables centralized management of information from various business communication tools, automatic extraction of tasks, and setting predicted deadlines, enabling efficient task management.

[0433] The "means for collecting information" refers to a function for acquiring information from each information source such as an email server, a chat server, and a telephone record server.

[0434] "Means for extracting action items" is a function for identifying and extracting important tasks and instructions from collected information.

[0435] "A means of listing action items" is a function for organizing extracted tasks and instructions and compiling them into a single list.

[0436] The "means for setting a predicted deadline" is a function for predicting and setting an appropriate completion deadline for each action item.

[0437] The "means for presenting the set action item list to the user" is a function for displaying the generated list so that the user can check it.

[0438] "Means of periodically calling the API of each information source and obtaining information using authentication information" refers to a function that periodically calls the API of an information source and obtains the necessary information using authentication information such as a username, password, or API key.

[0439] "Means for analyzing collected information and identifying important tasks using natural language processing technology" is a function for analyzing collected information using natural language processing technology and identifying important tasks and instructions from text data.

[0440] "Means for setting appropriate deadlines using predictive AI based on past task history and importance" is a function that uses predictive AI based on past task history and the importance of each task to automatically set appropriate deadlines for each task.

[0441] This invention is a system that automatically collects information from various business communication tools, extracts action items, sets predicted deadlines, and creates a list. Detailed embodiments for implementing this system are described below.

[0442] The system mainly includes three main components: a server, a terminal, and a user.

[0443] Form of information collection

[0444] The server is responsible for collecting information. Specifically, it periodically collects information from email servers, chat servers, and telephone record servers. To do this, it uses the APIs of each server. The server runs a scheduled job at a regular time, calling the APIs of each information source and retrieving the latest information using the necessary authentication information (username, password, API key, etc.). This information is then temporarily stored in storage.

[0445] Examples:

[0446] Every morning at 9:00, the server retrieves unread emails and all emails within a specified period from the email server API, retrieves new messages from the chat server API, and retrieves new call logs from the phone record server API.

[0447] Forms of information extraction and analysis

[0448] The server analyzes the collected information and extracts action items using natural language processing (NLP) technology. The server applies NLP algorithms to the collected emails, chat messages, and call logs, dividing each message and log into sentences and phrases, and extracting important keywords and phrases.

[0449] Examples:

[0450] If an email from a boss says, "Please submit a report by next week," the server extracts the action item "Submit report." Similarly, if a chat message from a project leader says, "Please prepare for a meeting," the server extracts the action item "Prepare for a meeting."

[0451] How to generate an action item list

[0452] The server generates a list based on the extracted action items. This list includes details of each action item, its origin (email, chat, phone log, etc.), and related context information. For each action item, the server adds the task name, origin, context information, etc. to the list.

[0453] Examples:

[0454] The server generates the following list:

[0455] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0456] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0457] Form of forecast deadline setting

[0458] The server uses predictive AI to set a predicted deadline for each action item. The predictive AI takes into account past task history, task importance, sender priority, etc. to set an appropriate deadline. The server retrieves and analyzes the history of similar past tasks from a database.

[0459] Examples:

[0460] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. It sets a deadline for the "meeting preparation" task the next day. In this way, the appropriate deadlines are reflected for each task.

[0461] List presentation format

[0462] The device presents the generated list of action items to the user. The server sends the generated list to the device, which then displays the received list on its screen (e.g., a PC or smartphone).

[0463] Examples:

[0464] A user checks the following list on their computer or smartphone:

[0465] 1. "Report submission" - Deadline: 3 days later, Original email: Instructions from boss

[0466] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0467] Specific prompt examples

[0468] Here are some examples of prompts for generative AI models:

[0469] Please design a system that extracts important tasks and instructions from my emails and chat messages. For example, from an email that says "Please turn in a report next week," it could extract the task "Submit report" and set a due date three days later using predictive AI. Also, please make this list available on my smartphone.

[0470] In this way, the system helps business people to efficiently manage the contents of business contacts and respond to them without any omissions.

[0471] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0472] Step 1: Gather information

[0473] The server periodically executes a job to collect information. This job calls the APIs of the email server, chat server, and call record server using authentication information (username, password, API key, etc.) to obtain the latest information. The API endpoint and authentication information of each server are used as input, and the obtained data (unread emails, chat messages, new call logs) is saved in storage as output.

[0474] Specific behavior:

[0475] The scheduled job runs every morning at 9am.

[0476] Get unread emails and all emails within a specified period from the email server API.

[0477] Get new messages from the chat server API.

[0478] Get new call logs from the call log server API.

[0479] Step 2: Information extraction and analysis

[0480] The server analyzes the collected data and extracts action items. It uses natural language processing (NLP) techniques to analyze collected emails, chat messages, and call logs to extract important keywords and phrases. It uses the collected data as input and lists important tasks and instructions as output.

[0481] Specific behavior:

[0482] Divide the saved data into sentences or phrases.

[0483] Apply NLP algorithms to identify important keywords and phrases.

[0484] Extract the task "Submit report" from an email from your boss saying "Please submit the report strictly next week."

[0485] Extract the task "Prepare for the meeting" from the chat message from the project leader saying "Please prepare for the meeting."

[0486] Step 3: Generate a list of action items

[0487] The server generates a list from the extracted action items, including details of each task, its origin (email, chat, call log, etc.), and associated context information. It uses the extracted action items as input and generates a formatted list as output.

[0488] Specific behavior:

[0489] Add task name, origin, and context information for each action item.

[0490] The list includes tasks such as "Submit report" and "Prepare for meeting," along with their origin and related information.

[0491] Step 4: Set the forecast deadline

[0492] The server uses predictive AI to set a predicted deadline for each action item. It automatically sets an appropriate deadline by taking into account past task history, importance, sender priority, etc. It uses the history and importance of past similar tasks as input and sets a predicted deadline for each action item as output.

[0493] Specific behavior:

[0494] Retrieve historical data of past similar tasks from a database.

[0495] Predictive AI calculates an appropriate deadline based on importance and sender priority.

[0496] For example, you can set a deadline for "report submission" three days later and a deadline for "meeting preparation" the next day.

[0497] Step 5: Present the list

[0498] The terminal presents the generated list of action items to the user, receives the list sent from the server and displays it on the screen, and uses the list sent from the server as input and displays it in a format that can be viewed by the user as output.

[0499] Specific behavior:

[0500] The terminal receives the list from the server.

[0501] The user checks the list on their PC or smartphone screen.

[0502] The list will show items such as "Submit report" - due date: 3 days later, original email: instructions from boss, "Prepare for meeting" - due date: next day, original chat: instructions from project leader.

[0503] The above is the specific flow and operation of each processing step of this system. At each step, input is received, appropriate data processing and analysis is performed, and output is generated. This allows business people to efficiently manage business communications and respond quickly to required tasks.

[0504] (Application example 1)

[0505] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0506] The complexity of business communications in logistics facilities and its management issues are a problem. Business communications from many information sources are scattered, and there is a risk that workers may miss important tasks. Therefore, a system that can efficiently collect information and manage tasks is required.

[0507] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0508] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for presenting the list of set action items to a user, and means for collecting information from a business communication tool at the logistics facility and generating action items, thereby enabling personnel at the logistics facility to efficiently manage tasks and respond without overlooking important matters.

[0509] "Means for collecting information" refers to the function of periodically obtaining information from a specific server (such as an email server, chat server, or telephone record server).

[0510] "Means of extracting action items from collected information" refers to the ability to identify important tasks or instructions from the acquired information and identify them as individual action items.

[0511] "Means for listing action items" refers to a function for organizing and displaying extracted action items in a list format.

[0512] "Means for setting predicted deadlines" refers to the function of predicting and setting appropriate deadlines for each action item based on its importance and past history.

[0513] The "means for presenting the set action item list to the user" refers to a function for displaying the action items, which have been listed and have their predicted deadlines set, on the user's terminal.

[0514] "A means of collecting information from business communication tools at logistics facilities and generating action items" refers to the function of obtaining information from emails, chat messages, call records, etc. related to logistics operations, analyzing them, and listing them as action items.

[0515] The present invention is a system for automating and efficiently managing information collection and tasks from business communication tools in a logistics facility. Specific embodiments of the system are described below.

[0516] Form of information collection

[0517] The server has a mechanism for periodically collecting information from the email server, chat server, and telephone record server. It retrieves unread emails and all emails within a specified period from the email server, new messages from the chat server, and new call logs from the telephone record server. This information collection is generally performed using the API of each server. The server retrieves new information using the necessary authentication information (username, password, API key, etc.), and this information is temporarily stored in storage.

[0518] Forms of information extraction and analysis

[0519] The server analyzes the collected information and uses natural language processing (NLP) to extract action items. Specifically, it uses Hugging Face's transformers library to automatically identify important tasks and instructions from email text, chat messages, and call logs. For example, if an email from a boss contains the instruction "Please submit your report punctually next week," the server analyzes it and extracts the action item "Submit report."

[0520] How to generate an action item list

[0521] The server generates a list of extracted action items, including details of each task, its origin (email, chat, phone log, etc.), and any associated context information. An example listing might look like this:

[0522] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0523] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0524] Form of forecast deadline setting

[0525] The server automatically sets predicted deadlines based on the collected information. Specifically, it sets an appropriate deadline for each task, taking into account past task history, task importance, sender priority, etc. For example, for the task "Submit a report," it sets a deadline of three days in the future based on the history of similar past tasks.

[0526] List presentation format

[0527] The device presents the generated list of action items to the user. The user can refer to this list and quickly respond to each action item. An example of how a user can view the list of action items on a PC or smartphone screen is shown below.

[0528] 1. "Submit report" - Deadline: 3 days later, Original email: Instructions from boss

[0529] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0530] This system helps logistics facility staff centrally manage information from multiple business communication tools, helping them manage and complete tasks efficiently and without omissions.

[0531] Specific examples and prompts for the generative AI model

[0532] For example, a logistics facility operator might receive:

[0533] "Please check the parts availability by next week" (email)

[0534] "Please prepare for the meeting starting at 2pm" (chat)

[0535] "Please complete the packaging for the new product by tomorrow." (Call notes)

[0536] Example prompt sentence:

[0537] "Identify action items in emails and set appropriate deadlines."

[0538] "Identify the important tasks from the text below and set expected deadlines."

[0539] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0540] Step 1: Gather information

[0541] The server obtains information from the email server, chat server, and telephone record server. It uses each server's API to collect new information using the necessary authentication information (username, password, API key, etc.). The collected information is temporarily stored in storage. This allows all contact information related to the user's work to be managed centrally. The input is new message data from each server, and the output is unanalyzed information in temporary storage.

[0542] Step 2: Analyze the information and extract action items

[0543] The server analyzes the collected information using natural language processing (NLP) technology. Specifically, it uses Hugging Face's transformers library to automatically identify important tasks and instructions from email text, chat messages, and call records. It extracts action items from the analysis results and stores them in temporary storage. The input is unanalyzed information, and the output is the extracted action items.

[0544] Step 3: Generate an action item list

[0545] The server lists the extracted action items, including details of each task, its origin (email, chat, phone log, etc.), and related context information. The listed action items are used as data to set the expected deadline in a later step. The input is the extracted action items, and the output is the listed action items.

[0546] Step 4: Set the forecast deadline

[0547] The server sets predicted deadlines for the listed action items. It sets an appropriate deadline for each task, taking into account factors such as past task history, task importance, and sender priority. This prediction is performed using a generative AI model. The input is the listed action items and past task history, and the output is the action items with predicted deadlines.

[0548] Step 5: View the Action Items List

[0549] The terminal presents the user with a list of action items with predicted deadlines sent from the server. The user can refer to this list and quickly respond to each action item. For example, the user checks the list on a smartphone or PC screen. The input is the action items with predicted deadlines, and the output is the list of action items displayed to the user.

[0550] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0551] This invention is a system that automatically extracts information from various business communication tools, lists action items, sets expected deadlines, and optimizes task management using an emotion engine that recognizes the user's emotions. This system is designed to enable business people to prevent oversight of work-related actions, efficiently manage tasks, and flexibly adjust responses based on the user's emotions.

[0552] Form of information collection

[0553] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information, including unread emails from the email server, new messages from chat, and new call logs from the telephone record server, and temporarily stores this information in storage.

[0554] Examples:

[0555] Every morning at 9am, the server calls each tool's API and retrieves new information using the necessary authentication information. This information is stored in a temporary database.

[0556] Forms of information extraction and analysis

[0557] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts and categorizing them into action items.

[0558] Examples:

[0559] If an email from a boss includes the instruction "Please submit a report by next week," the server will parse it and extract the action item "Submit report." Similarly, if a chat message from a project leader includes the instruction "Please prepare for a meeting," this will also be extracted.

[0560] How to generate an action item list

[0561] The server lists the extracted action items, including details of each action item (e.g., task name, source, related information), and stores the list in a database.

[0562] Examples:

[0563] The server generates the following list:

[0564] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0565] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0566] Form of forecast deadline setting

[0567] The server automatically sets the expected deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline for each action item.

[0568] Examples:

[0569] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. For the "meeting preparation" task, it sets a deadline for the next day. The deadlines set in this way are reflected in the list.

[0570] Forms of emotion recognition

[0571] The server combines an emotion engine that recognizes emotions from the user's facial expressions, voice, input data, etc. If it determines that the user is feeling stressed, it automatically adjusts the priority and deadlines of action items.

[0572] Examples:

[0573] While the user is reviewing the action item list, the server analyzes the user's facial expressions and voice tone via the camera and microphone, and the emotion engine determines the user's stress level. If stress is determined to be high, the server extends the deadline for "report submission" by one day and moves "meeting preparation" up the list as a high-priority task.

[0574] How the list is presented to the user

[0575] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list, adjusted by the emotion engine, helps the user effectively manage tasks while reducing stress.

[0576] Examples:

[0577] The user sees the following list on their PC screen:

[0578] 1. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0579] 2. "Report submission" - Deadline: 4 days later (extended), Original email: Instructions from boss

[0580] In this way, the system can centrally collect business communication information from multiple tools, efficiently manage action items, prevent missed responses, and flexibly adjust responses according to the user's emotions.

[0581] The processing flow will be explained below.

[0582] Step 1: Gather information

[0583] The server periodically accesses the email server, chat server, and telephone record server to collect new information. It downloads unread emails from the email server using IMAP or POP3 protocol, retrieves the latest messages from the chat server using API, and retrieves new call logs from the telephone record server. This information is temporarily stored in a database.

[0584] Step 2: Preprocessing the information

[0585] The server preprocesses the collected raw data, which includes cleaning the data, extracting metadata, and formatting the text. For emails, basic information such as the body of the email, sender, and date and time of receipt is extracted, while chat messages are converted into text containing the sender and time of the call, and call logs are converted into text containing the duration and content of the call.

[0586] Step 3: Extract action items

[0587] The server applies natural language processing (NLP) to the preprocessed data to extract action items. The NLP engine identifies tasks, instructions, and important information in the text and identifies them as action items. Specifically, it extracts keywords related to instructions and tasks from each email, chat message, and call log.

[0588] Step 4: Generate an action item list

[0589] The server compiles the extracted action items into a list, which includes details of each action item (e.g., task name, source, related information), and stores the action items in a database for further processing.

[0590] Step 5: Set the forecast deadline

[0591] The server automatically sets a predicted deadline for each action item using predictive AI. The predictive AI analyzes past task history, task importance, and sender priority to estimate the optimal deadline for each action item. The set deadline is added to the action item list.

[0592] Step 6: Emotion Recognition

[0593] The server uses an emotion engine that recognizes emotions from the user's facial expressions, voice, and input data. The emotion engine detects when the user is stressed or tired and reflects this in the management of action items.

[0594] Step 7: Adjusting tasks based on emotions

[0595] The server adjusts the priority and deadline of action items based on the user's emotional data obtained by the emotion engine. For example, if it detects that the user is feeling high stress, it extends the deadline or changes the priority of some action items.

[0596] Step 8: Present the list to the user

[0597] The device presents the final action item list to the user. The user can check the list on a device such as a PC or smartphone and work on the tasks efficiently. In addition, by checking the list adjusted by the emotion engine, the user can manage tasks in a less burdensome way.

[0598] By executing the above processing steps, this system can centrally manage business contact information and optimize task management while taking into consideration the user's feelings. This system allows business people to avoid overlooking work-related tasks and perform their work efficiently while reducing stress.

[0599] Example 2

[0600] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0601] Modern business people use multiple business communication tools, such as email, chat, and phone records, and are required to efficiently manage information from these tools. However, manually collecting this information, extracting important tasks, setting deadlines, and adjusting priorities based on emotions is extremely time-consuming and labor-intensive. Furthermore, task management based on stress and emotional states is often not possible, making it difficult to improve work efficiency and prevent missed responses.

[0602] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting information, a means for extracting action items from the collected information, a means for listing the action items, a means for setting a predicted deadline for each action item, a means for recognizing the user's emotions and adjusting the priority and deadline of the action items, and a means for presenting the set action item list to the user. This makes it possible to centrally manage information from business communication tools and efficiently extract, list, set deadlines for, and adjust tasks according to emotions.

[0603] The "means for collecting information" is a mechanism for automatically obtaining information at specified intervals from multiple information sources such as an email server, a chat server, and a telephone record server.

[0604] The "means for extracting action items from collected information" refers to a mechanism that analyzes the acquired information, identifies important tasks and instructions using natural language processing technology, and extracts them as action items.

[0605] The "means for listing action items" is a mechanism for listing extracted action items including task names, sources, and related information, and storing the list in a database.

[0606] The "means of setting predicted deadlines for each action item" is a mechanism that uses predictive AI to analyze past task history, task importance, and sender priority to estimate appropriate deadline dates.

[0607] The "means for recognizing the user's emotions and adjusting the priority and deadlines of action items" is a mechanism that automatically adjusts the priority and deadlines of action items according to the user's stress level using an emotion engine that identifies emotions from the user's facial expressions, voice, input data, etc.

[0608] The "means for presenting the set action item list to the user" is a mechanism for visually presenting the generated and adjusted action item list to the user via a terminal such as a PC or smartphone.

[0609] This invention is a system that automatically extracts information from various business communication tools, lists action items, sets expected deadlines, and optimizes task management using an emotion engine that recognizes the user's emotions. This system is designed to enable business people to prevent oversight of work-related actions, efficiently manage tasks, and flexibly adjust responses based on the user's emotions.

[0610] A specific embodiment for carrying out the present invention will be described.

[0611] Form of information collection

[0612] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information, including unread emails from the email server, new messages from chat, and new call logs from the telephone record server, and temporarily stores this information in storage.

[0613] Examples:

[0614] Every morning at 9am, the server calls each tool's API and retrieves new information using the necessary authentication information. This information is stored in a temporary database.

[0615] Forms of information extraction and analysis

[0616] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts and categorizing them into action items.

[0617] Examples:

[0618] If an email from a boss includes the instruction "Please submit a report by next week," the server will parse it and extract the action item "Submit report." Similarly, if a chat message from a project leader includes the instruction "Please prepare for a meeting," this will also be extracted.

[0619] How to generate an action item list

[0620] The server lists the extracted action items, including details of each action item (e.g., task name, source, related information), and stores the list in a database.

[0621] Examples:

[0622] The server generates the following list:

[0623] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0624] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0625] Form of forecast deadline setting

[0626] The server automatically sets the expected deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline for each action item.

[0627] Examples:

[0628] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. For the "meeting preparation" task, it sets a deadline for the next day. The deadlines set in this way are reflected in the list.

[0629] Forms of emotion recognition

[0630] The server combines an emotion engine that recognizes emotions from the user's facial expressions, voice, input data, etc. If it determines that the user is feeling stressed, it automatically adjusts the priority and deadlines of action items.

[0631] Examples:

[0632] While the user is reviewing the action item list, the server analyzes the user's facial expressions and voice tone via the camera and microphone, and the emotion engine determines the user's stress level. If stress is determined to be high, the server extends the deadline for "report submission" by one day and moves "meeting preparation" up the list as a high-priority task.

[0633] How the list is presented to the user

[0634] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list, adjusted by the emotion engine, helps the user effectively manage tasks while reducing stress.

[0635] Examples:

[0636] The user sees the following list on their PC screen:

[0637] 1. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0638] 2. "Report submission" - Deadline: 4 days later (extended), Original email: Instructions from boss

[0639] Example prompts for generative AI models

[0640] "How do you operate a system that extracts important tasks from emails, chats, and phone records, automatically sets deadlines, performs sentiment analysis, and adjusts priorities?"

[0641] This allows the system to centrally collect business communication information from multiple tools, efficiently manage action items, prevent missed responses, and flexibly adjust responses based on the user's emotions.

[0642] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0643] Step 1:

[0644] Information gathering

[0645] The server accesses the email server, chat server, and phone record server at a specified interval (for example, every morning at 9:00) to collect new information. This includes unread emails from email, new messages from chat, and new call logs from phone records. The server retrieves this information via API and stores it in a temporary database.

[0646] Input: New information (unread emails, new chat messages, new call logs) retrieved from the APIs of the email, chat, and call log servers.

[0647] Data processing: Collecting new information from each server and formatting it for temporary storage

[0648] Output: New information stored in a temporary database

[0649] Specific operation: The server calls each tool's API with the authentication information, obtains new information, and saves it.

[0650] Step 2:

[0651] Information analysis and action item extraction

[0652] The server uses natural language processing techniques to analyze the collected information and extract action items, identifying important tasks and instructions from email text, chat messages, and call transcripts.

[0653] Input: New information stored in a temporary database

[0654] Data processing: Using natural language processing techniques, we analyze important tasks and instructions and categorize them as action items.

[0655] Output: Extracted action items

[0656] What it does: The server uses natural language processing algorithms to analyze and identify imperative and task-related phrases in a sentence.

[0657] Step 3:

[0658] List of action items

[0659] The server lists the extracted action items and stores them in a database, including details of each action item (e.g., task name, source, related information).

[0660] Input: Extracted action items

[0661] Data processing: Format and list the details of each action item

[0662] Output: List of action items stored in the database

[0663] What it does: The server stores each action item in a database along with the task name, source, and other relevant information.

[0664] Step 4:

[0665] Setting a forecast deadline

[0666] The server sets a predicted deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline.

[0667] Input: A list of action items stored in a database

[0668] Data processing: Use predictive AI to estimate deadlines for each action item

[0669] Output: A list of action items with deadlines

[0670] Specific operation: The server analyzes past task history data and sets a predicted deadline based on the execution period of similar tasks.

[0671] Step 5:

[0672] Emotion recognition and action item adjustment

[0673] The server uses an emotion engine that recognizes emotions from the user's facial expressions, voice, and input data, and adjusts the priority and deadlines of action items if the user is feeling stressed.

[0674] Input: Action item list with deadlines, user facial expression data, voice data, and input data

[0675] Data processing: Analyze user emotions using an emotion engine and adjust the priority and deadlines of action items according to stress levels

[0676] Output: A tailored list of action items

[0677] How it works: The server uses a camera and microphone to analyze the user's facial expressions and voice tone in real time, and the emotion engine determines the user's stress level and adjusts action items.

[0678] Step 6:

[0679] Presenting the list to the user

[0680] The device presents the generated action item list to the user, who can then check the action item list through a device such as a PC or smartphone and quickly respond to the tasks.

[0681] Input: Adjusted action item list

[0682] Output: A list of action items presented to the user

[0683] Specific behavior: The device displays the action item list in the user interface so that the user can easily check it.

[0684] In this way, the system can optimize task management by consistently collecting information, extracting action items, listing them, setting deadlines, recognizing emotions, and presenting them to the user.

[0685] (Application example 2)

[0686] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0687] Conventional task management systems have difficulty centrally collecting information from multiple business communication tools and managing important tasks. Furthermore, they are unable to adjust task priorities and deadlines based on user emotions, which can lead to the accumulation of stress and fatigue. Furthermore, in busy environments like factories, information is often overlooked or missed, often resulting in reduced work efficiency. To solve these problems, the present invention proposes a system that provides centralized information management and task optimization based on user emotions.

[0688] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0689] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for recognizing the user's emotions and adjusting the priority and deadline of the action items, and means for presenting the set action item list to the user. This enables centralized management of important tasks from multiple information sources and efficient task management that takes the user's emotional state into consideration.

[0690] The "means for collecting information" is a system that has the function of automatically acquiring necessary data from multiple information sources such as an email server, a chat server, and a communication log server.

[0691] The "means for extracting action items" is a system that utilizes natural language processing technology to identify and separate important tasks and instructions from collected information.

[0692] The "means for creating a list of action items" is a system that has the function of organizing and displaying the extracted action items in a list format and generating a list including detailed information about each item.

[0693] The "means for setting estimated deadlines" is a system that has the ability to automatically estimate appropriate deadlines for action items based on past task history, task importance, and sender priority.

[0694] "Means for recognizing user emotions and adjusting the priority and deadlines of action items" refers to a system that uses a camera and microphone to analyze the user's facial expressions and voice, determine the user's emotional state, and dynamically change task priorities and deadlines.

[0695] "Means for presenting the set action item list to the user" refers to a system that has the function of displaying the generated action item list on the user's device (e.g., smartphone, tablet, PC) and enabling a quick response.

[0696] A system for carrying out the present invention is configured as follows: A server has means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for recognizing a user's emotions and adjusting the priority and deadlines of the action items, and means for presenting the set action item list to the user.

[0697] The server first collects information from email servers, chat servers, communication log servers, etc. During this process, for example, every morning at 9:00, it calls the API of each tool and retrieves new information using the necessary authentication information. This information is then temporarily stored in a database.

[0698] The collected information is analyzed using natural language processing technology (e.g., libraries such as NLTK and spaCy) to extract necessary action items. For example, if an email from a boss contains the instruction "Please submit the report by next week," the server analyzes it and extracts the action item "Submit the report."

[0699] The extracted action items are then compiled into a list, which includes details about each action item (e.g., task name, source, related information), and is displayed in a user-friendly interface and stored in a database.

[0700] Additionally, a predictive AI model (e.g., one using TensorFlow or PyTorch) is used to set an appropriate expected deadline for each action item, based on past task history, task importance, and sender priority.

[0701] The system also incorporates an emotion engine (e.g., Affectiva SDK) that uses the smartphone's camera and microphone to recognize the user's emotions. If the engine determines that the user is feeling stressed, it automatically adjusts the priority and deadline of action items. For example, if the server determines that the user is feeling very stressed, it will extend the deadline for "report submission" by one day and move "meeting preparation" up the list as a high-priority task.

[0702] Finally, the device (smartphone, tablet, or PC) presents the generated action item list to the user, allowing them to efficiently manage their tasks and accomplish their work while reducing stress.

[0703] As a concrete example, consider an environment where many work instructions are generated within a factory. Using a factory task manager app, all work instructions can be centrally managed, natural language processing technology can be used to automatically extract tasks from the instructions, an emotion recognition engine can be used to check the stress levels of workers, and predictive AI can be used to set appropriate deadlines. In this way, work instructions will not be overlooked and tasks will be carried out efficiently.

[0704] An example of a prompt would be:

[0705] "Develop an application that collects work instructions in a factory, analyzes them, and uses emotion recognition to prioritize and efficiently manage tasks."

[0706] In this way, the present invention can provide unified management of information and task optimization based on user sentiment.

[0707] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0708] Step 1:

[0709] The server uses the means of collecting information to call the API of each communication tool (email server, chat server, communication log server) at specified intervals (e.g., every morning at 9:00) to obtain new information. The input is the authentication information and API endpoint of each tool, and the output is the newly obtained emails, chat messages, and call records. The information is temporarily stored in a database.

[0710] Step 2:

[0711] The server uses natural language processing technology to analyze the information collected in step 1 and extract action items. Specifically, it tokenizes the email body, chat messages, and call records and performs grammatical analysis. The input is the unanalyzed information collected in step 1, and the output is the extracted action items (e.g., "Submit a report" or "Prepare for a meeting").

[0712] Step 3:

[0713] The server lists the extracted action items. The input is the action items extracted in step 2, and the output is a list of their details (e.g., task name, source, related information). This list is stored in a database in a user-friendly format.

[0714] Step 4:

[0715] The server uses a predictive AI model to set a predicted deadline for each action item. The input is the action item list generated in step 3 and past task history data, and the output is a list of action items with predicted deadlines. The AI ​​model is built using TensorFlow and PyTorch.

[0716] Step 5:

[0717] The server recognizes the user's emotions using the smartphone's camera and microphone. The emotion engine (e.g., Affectiva SDK) analyzes the user's facial expressions and voice to determine their stress and fatigue levels. The input is real-time data from the camera and microphone, and the output is the user's emotional state (e.g., high stress, low stress).

[0718] Step 6:

[0719] The server automatically adjusts the priority and deadlines of action items based on the emotion recognition results in step 5. The input is the action item list from step 4 and the emotion recognition results from step 5, and the output is the adjusted action item list. For example, if the user is in a high stress state, the deadline for "submitting a report" is extended and "preparing for a meeting" is moved up in the list.

[0720] Step 7:

[0721] The device presents the configured action item list to the user. The input is the action item list adjusted in step 6, and the output is the action item list displayed on the device (smartphone, tablet, PC) screen. The user can check this and manage tasks efficiently.

[0722] In this way, the present invention unifies data from multiple sources and provides optimized task management through natural language processing, emotion recognition, and predictive AI models.

[0723] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0724] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0725] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0726] [Third embodiment]

[0727] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0728] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0729] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0730] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0731] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0732] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0733] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0734] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0735] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0736] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0737] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0738] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0739] This invention is a system that automatically extracts information from various business communication tools, lists action items, and sets expected deadlines. This system is designed to enable business people to prevent oversight of business tasks and manage tasks efficiently.

[0740] Form of information collection

[0741] The server periodically collects information from the email server, chat server, and telephone record server. From the email server, it retrieves unread emails and all emails within a specified period, from the chat server it retrieves new messages, and from the telephone record server it retrieves new call logs. This allows you to centralize information from various business communication tools.

[0742] Examples:

[0743] The server calls the API of each tool at a fixed time (for example, every morning at 9:00) and obtains new information using the necessary authentication information (user name, password, API key, etc.). This information is temporarily stored in storage.

[0744] Forms of information extraction and analysis

[0745] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts.

[0746] Examples:

[0747] If an email from a boss includes instructions such as "Please submit a report by next week," the server will analyze it and extract the action item "Submit report." Similarly, if a project leader sends a chat message with instructions such as "Please prepare for a meeting," this will also be extracted.

[0748] How to generate an action item list

[0749] The server lists the extracted action items, including details of each task, its origin (email, chat, call log, etc.), and associated context information.

[0750] Examples:

[0751] The server generates the following list:

[0752] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0753] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0754] Form of forecast deadline setting

[0755] The server automatically sets a predicted deadline for each action item using predictive AI, which takes into account past task history, task importance, sender priority, and other factors to set an appropriate deadline for each task.

[0756] Examples:

[0757] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past, and a deadline for the "meeting preparation" task the next day. In this way, the expected deadlines set for each task are reflected in the list.

[0758] List presentation format

[0759] The terminal presents the user with a list of the generated action items, allowing the user to quickly respond to each action item by referring to the list.

[0760] Examples:

[0761] The user sees the following list on their PC or smartphone screen:

[0762] 1. "Report submission" - Deadline: 3 days later, Original email: Instructions from boss

[0763] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0764] In this way, the system helps business people to understand the details of business communications thoroughly and efficiently manage and complete tasks.

[0765] The processing flow will be explained below.

[0766] Step 1: Gather information

[0767] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information. It downloads unread emails from the email server via IMAP or POP3 protocol, retrieves new messages from the chat server via API, and retrieves new call logs from the telephone record server. This information is then stored in temporary data storage.

[0768] Step 2: Preprocessing the information

[0769] The server preprocesses the collected raw data, which includes cleaning the data, extracting metadata, and formatting the text. For emails, basic information such as the body of the message, sender, and date and time of receipt is extracted. For chat messages, the server converts the sender and time of the message into text, and for phone logs, the server converts the duration and content of the call into text.

[0770] Step 3: Extract action items

[0771] The server then applies natural language processing (NLP) to the preprocessed data to extract action items. The NLP module performs contextual analysis to identify tasks, instructions, and important information, and then identifies action items based on that. This process extracts and lists task names and related keywords.

[0772] Step 4: Generate an action item list

[0773] The server generates an action item list based on the extracted action items. This list includes details of each action item (e.g., task name, source, related information). The action items are stored in a database for further processing.

[0774] Step 5: Set the forecast deadline

[0775] The server uses predictive AI to set deadlines for each action item. The predictive AI analyzes past task history, task importance, and sender priority to estimate the optimal deadline for each action item. The set deadline is added to the action item list.

[0776] Step 6: Present the list to the user

[0777] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list displays task details and set deadlines, allowing the user to visually manage tasks.

[0778] In this way, the system can centrally collect business communication information from multiple tools, efficiently manage action items, and prevent missed responses.

[0779] Example 1

[0780] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0781] Modern business people use a wide variety of business communication tools, and because information is scattered across them, there is a high risk of missing important tasks or instructions. Furthermore, manually retrieving information from these communication tools and managing tasks is extremely time-consuming and inefficient. It is also difficult to appropriately set task importance and deadlines, leading to missed or delayed responses. Therefore, centralizing information management from business communication tools and efficient task management are key challenges.

[0782] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0783] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for presenting the set action item list to the user, means for periodically calling the API of each information source and obtaining information using authentication information, means for analyzing the collected information and identifying important tasks using natural language processing technology, and means for setting appropriate deadlines using predictive AI based on past task history and importance.This enables centralized management of information from various business communication tools, automatic extraction of tasks, and setting predicted deadlines, enabling efficient task management.

[0784] The "means for collecting information" refers to a function for acquiring information from each information source such as an email server, a chat server, and a telephone record server.

[0785] "Means for extracting action items" is a function for identifying and extracting important tasks and instructions from collected information.

[0786] "A means of listing action items" is a function for organizing extracted tasks and instructions and compiling them into a single list.

[0787] The "means for setting a predicted deadline" is a function for predicting and setting an appropriate completion deadline for each action item.

[0788] The "means for presenting the set action item list to the user" is a function for displaying the generated list so that the user can check it.

[0789] "Means of periodically calling the API of each information source and obtaining information using authentication information" refers to a function that periodically calls the API of an information source and obtains the necessary information using authentication information such as a username, password, or API key.

[0790] "Means for analyzing collected information and identifying important tasks using natural language processing technology" is a function for analyzing collected information using natural language processing technology and identifying important tasks and instructions from text data.

[0791] "Means for setting appropriate deadlines using predictive AI based on past task history and importance" is a function that uses predictive AI based on past task history and the importance of each task to automatically set appropriate deadlines for each task.

[0792] This invention is a system that automatically collects information from various business communication tools, extracts action items, sets predicted deadlines, and creates a list. Detailed embodiments for implementing this system are described below.

[0793] The system mainly includes three main components: a server, a terminal, and a user.

[0794] Form of information collection

[0795] The server is responsible for collecting information. Specifically, it periodically collects information from email servers, chat servers, and telephone record servers. To do this, it uses the APIs of each server. The server runs a scheduled job at a regular time, calling the APIs of each information source and retrieving the latest information using the necessary authentication information (username, password, API key, etc.). This information is then temporarily stored in storage.

[0796] Examples:

[0797] Every morning at 9:00, the server retrieves unread emails and all emails within a specified period from the email server API, retrieves new messages from the chat server API, and retrieves new call logs from the phone record server API.

[0798] Forms of information extraction and analysis

[0799] The server analyzes the collected information and extracts action items using natural language processing (NLP) technology. The server applies NLP algorithms to the collected emails, chat messages, and call logs, dividing each message and log into sentences and phrases, and extracting important keywords and phrases.

[0800] Examples:

[0801] If an email from a boss says, "Please submit a report by next week," the server extracts the action item "Submit report." Similarly, if a chat message from a project leader says, "Please prepare for a meeting," the server extracts the action item "Prepare for a meeting."

[0802] How to generate an action item list

[0803] The server generates a list based on the extracted action items. This list includes details of each action item, its origin (email, chat, phone log, etc.), and related context information. For each action item, the server adds the task name, origin, context information, etc. to the list.

[0804] Examples:

[0805] The server generates the following list:

[0806] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0807] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0808] Form of forecast deadline setting

[0809] The server uses predictive AI to set a predicted deadline for each action item. The predictive AI takes into account past task history, task importance, sender priority, etc. to set an appropriate deadline. The server retrieves and analyzes the history of similar past tasks from a database.

[0810] Examples:

[0811] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. It sets a deadline for the "meeting preparation" task the next day. In this way, the appropriate deadlines are reflected for each task.

[0812] List presentation format

[0813] The device presents the generated list of action items to the user. The server sends the generated list to the device, which then displays the received list on its screen (e.g., a PC or smartphone).

[0814] Examples:

[0815] A user checks the following list on their computer or smartphone:

[0816] 1. "Report submission" - Deadline: 3 days later, Original email: Instructions from boss

[0817] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0818] Specific prompt examples

[0819] Here are some examples of prompts for generative AI models:

[0820] Please design a system that extracts important tasks and instructions from my emails and chat messages. For example, from an email that says "Please turn in a report next week," it could extract the task "Submit report" and set a due date three days later using predictive AI. Also, please make this list available on my smartphone.

[0821] In this way, the system helps business people to efficiently manage the contents of business contacts and respond to them without any omissions.

[0822] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0823] Step 1: Gather information

[0824] The server periodically executes a job to collect information. This job calls the APIs of the email server, chat server, and call record server using authentication information (username, password, API key, etc.) to obtain the latest information. The API endpoint and authentication information of each server are used as input, and the obtained data (unread emails, chat messages, new call logs) is saved in storage as output.

[0825] Specific behavior:

[0826] The scheduled job runs every morning at 9am.

[0827] Get unread emails and all emails within a specified period from the email server API.

[0828] Get new messages from the chat server API.

[0829] Get new call logs from the call log server API.

[0830] Step 2: Information extraction and analysis

[0831] The server analyzes the collected data and extracts action items. It uses natural language processing (NLP) techniques to analyze collected emails, chat messages, and call logs to extract important keywords and phrases. It uses the collected data as input and lists important tasks and instructions as output.

[0832] Specific behavior:

[0833] Divide the saved data into sentences or phrases.

[0834] Apply NLP algorithms to identify important keywords and phrases.

[0835] Extract the task "Submit report" from an email from your boss saying "Please submit the report strictly next week."

[0836] Extract the task "Prepare for the meeting" from the chat message from the project leader saying "Please prepare for the meeting."

[0837] Step 3: Generate a list of action items

[0838] The server generates a list from the extracted action items, including details of each task, its origin (email, chat, call log, etc.), and associated context information. It uses the extracted action items as input and generates a formatted list as output.

[0839] Specific behavior:

[0840] Add task name, origin, and context information for each action item.

[0841] The list includes tasks such as "Submit report" and "Prepare for meeting," along with their origin and related information.

[0842] Step 4: Set the forecast deadline

[0843] The server uses predictive AI to set a predicted deadline for each action item. It automatically sets an appropriate deadline by taking into account past task history, importance, sender priority, etc. It uses the history and importance of past similar tasks as input and sets a predicted deadline for each action item as output.

[0844] Specific behavior:

[0845] Retrieve historical data of past similar tasks from a database.

[0846] Predictive AI calculates an appropriate deadline based on importance and sender priority.

[0847] For example, you can set a deadline for "report submission" three days later and a deadline for "meeting preparation" the next day.

[0848] Step 5: Present the list

[0849] The terminal presents the generated list of action items to the user, receives the list sent from the server and displays it on the screen, and uses the list sent from the server as input and displays it in a format that can be viewed by the user as output.

[0850] Specific behavior:

[0851] The terminal receives the list from the server.

[0852] The user checks the list on their PC or smartphone screen.

[0853] The list will show items such as "Submit report" - due date: 3 days later, original email: instructions from boss, "Prepare for meeting" - due date: next day, original chat: instructions from project leader.

[0854] The above is the specific flow and operation of each processing step of this system. At each step, input is received, appropriate data processing and analysis is performed, and output is generated. This allows business people to efficiently manage business communications and respond quickly to required tasks.

[0855] (Application example 1)

[0856] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0857] The complexity of business communications in logistics facilities and its management issues are a problem. Business communications from many information sources are scattered, and there is a risk that workers may miss important tasks. Therefore, a system that can efficiently collect information and manage tasks is required.

[0858] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0859] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for presenting the list of set action items to a user, and means for collecting information from a business communication tool at the logistics facility and generating action items, thereby enabling personnel at the logistics facility to efficiently manage tasks and respond without overlooking important matters.

[0860] "Means for collecting information" refers to the function of periodically obtaining information from a specific server (such as an email server, chat server, or telephone record server).

[0861] "Means of extracting action items from collected information" refers to the ability to identify important tasks or instructions from the acquired information and identify them as individual action items.

[0862] "Means for listing action items" refers to a function for organizing and displaying extracted action items in a list format.

[0863] "Means for setting predicted deadlines" refers to the function of predicting and setting appropriate deadlines for each action item based on its importance and past history.

[0864] The "means for presenting the set action item list to the user" refers to a function for displaying the action items, which have been listed and have their predicted deadlines set, on the user's terminal.

[0865] "A means of collecting information from business communication tools at logistics facilities and generating action items" refers to the function of obtaining information from emails, chat messages, call records, etc. related to logistics operations, analyzing them, and listing them as action items.

[0866] The present invention is a system for automating and efficiently managing information collection and tasks from business communication tools in a logistics facility. Specific embodiments of the system are described below.

[0867] Form of information collection

[0868] The server has a mechanism for periodically collecting information from the email server, chat server, and telephone record server. It retrieves unread emails and all emails within a specified period from the email server, new messages from the chat server, and new call logs from the telephone record server. This information collection is generally performed using the API of each server. The server retrieves new information using the necessary authentication information (username, password, API key, etc.), and this information is temporarily stored in storage.

[0869] Forms of information extraction and analysis

[0870] The server analyzes the collected information and uses natural language processing (NLP) to extract action items. Specifically, it uses Hugging Face's transformers library to automatically identify important tasks and instructions from email text, chat messages, and call logs. For example, if an email from a boss contains the instruction "Please submit your report punctually next week," the server analyzes it and extracts the action item "Submit report."

[0871] How to generate an action item list

[0872] The server generates a list of extracted action items, including details of each task, its origin (email, chat, phone log, etc.), and any associated context information. An example listing might look like this:

[0873] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0874] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0875] Form of forecast deadline setting

[0876] The server automatically sets predicted deadlines based on the collected information. Specifically, it sets an appropriate deadline for each task, taking into account past task history, task importance, sender priority, etc. For example, for the task "Submit a report," it sets a deadline of three days in the future based on the history of similar past tasks.

[0877] List presentation format

[0878] The device presents the generated list of action items to the user. The user can refer to this list and quickly respond to each action item. An example of how a user can view the list of action items on a PC or smartphone screen is shown below.

[0879] 1. "Submit report" - Deadline: 3 days later, Original email: Instructions from boss

[0880] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0881] This system helps logistics facility staff centrally manage information from multiple business communication tools, helping them manage and complete tasks efficiently and without omissions.

[0882] Specific examples and prompts for the generative AI model

[0883] For example, a logistics facility operator might receive:

[0884] "Please check the parts availability by next week" (email)

[0885] "Please prepare for the meeting starting at 2pm" (chat)

[0886] "Please complete the packaging for the new product by tomorrow." (Call notes)

[0887] Example prompt sentence:

[0888] "Identify action items in emails and set appropriate deadlines."

[0889] "Identify the important tasks from the text below and set expected deadlines."

[0890] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0891] Step 1: Gather information

[0892] The server obtains information from the email server, chat server, and telephone record server. It uses each server's API to collect new information using the necessary authentication information (username, password, API key, etc.). The collected information is temporarily stored in storage. This allows all contact information related to the user's work to be managed centrally. The input is new message data from each server, and the output is unanalyzed information in temporary storage.

[0893] Step 2: Analyze the information and extract action items

[0894] The server analyzes the collected information using natural language processing (NLP) technology. Specifically, it uses Hugging Face's transformers library to automatically identify important tasks and instructions from email text, chat messages, and call records. It extracts action items from the analysis results and stores them in temporary storage. The input is unanalyzed information, and the output is the extracted action items.

[0895] Step 3: Generate an action item list

[0896] The server lists the extracted action items, including details of each task, its origin (email, chat, phone log, etc.), and related context information. The listed action items are used as data to set the expected deadline in a later step. The input is the extracted action items, and the output is the listed action items.

[0897] Step 4: Set the forecast deadline

[0898] The server sets predicted deadlines for the listed action items. It sets an appropriate deadline for each task, taking into account factors such as past task history, task importance, and sender priority. This prediction is performed using a generative AI model. The input is the listed action items and past task history, and the output is the action items with predicted deadlines.

[0899] Step 5: View the Action Items List

[0900] The terminal presents the user with a list of action items with predicted deadlines sent from the server. The user can refer to this list and quickly respond to each action item. For example, the user checks the list on a smartphone or PC screen. The input is the action items with predicted deadlines, and the output is the list of action items displayed to the user.

[0901] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0902] This invention is a system that automatically extracts information from various business communication tools, lists action items, sets expected deadlines, and optimizes task management using an emotion engine that recognizes the user's emotions. This system is designed to enable business people to prevent oversight of work-related actions, efficiently manage tasks, and flexibly adjust responses based on the user's emotions.

[0903] Form of information collection

[0904] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information, including unread emails from the email server, new messages from chat, and new call logs from the telephone record server, and temporarily stores this information in storage.

[0905] Examples:

[0906] Every morning at 9am, the server calls each tool's API and retrieves new information using the necessary authentication information. This information is stored in a temporary database.

[0907] Forms of information extraction and analysis

[0908] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts and categorizing them into action items.

[0909] Examples:

[0910] If an email from a boss includes the instruction "Please submit a report by next week," the server will parse it and extract the action item "Submit report." Similarly, if a chat message from a project leader includes the instruction "Please prepare for a meeting," this will also be extracted.

[0911] How to generate an action item list

[0912] The server lists the extracted action items, including details of each action item (e.g., task name, source, related information), and stores the list in a database.

[0913] Examples:

[0914] The server generates the following list:

[0915] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0916] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0917] Form of forecast deadline setting

[0918] The server automatically sets the expected deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline for each action item.

[0919] Examples:

[0920] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. For the "meeting preparation" task, it sets a deadline for the next day. The deadlines set in this way are reflected in the list.

[0921] Forms of emotion recognition

[0922] The server combines an emotion engine that recognizes emotions from the user's facial expressions, voice, input data, etc. If it determines that the user is feeling stressed, it automatically adjusts the priority and deadlines of action items.

[0923] Examples:

[0924] While the user is reviewing the action item list, the server analyzes the user's facial expressions and voice tone via the camera and microphone, and the emotion engine determines the user's stress level. If stress is determined to be high, the server extends the deadline for "report submission" by one day and moves "meeting preparation" up the list as a high-priority task.

[0925] How the list is presented to the user

[0926] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list, adjusted by the emotion engine, helps the user effectively manage tasks while reducing stress.

[0927] Examples:

[0928] The user sees the following list on their PC screen:

[0929] 1. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0930] 2. "Report submission" - Deadline: 4 days later (extended), Original email: Instructions from boss

[0931] In this way, the system can centrally collect business communication information from multiple tools, efficiently manage action items, prevent missed responses, and flexibly adjust responses according to the user's emotions.

[0932] The processing flow will be explained below.

[0933] Step 1: Gather information

[0934] The server periodically accesses the email server, chat server, and telephone record server to collect new information. It downloads unread emails from the email server using IMAP or POP3 protocol, retrieves the latest messages from the chat server using API, and retrieves new call logs from the telephone record server. This information is temporarily stored in a database.

[0935] Step 2: Preprocessing the information

[0936] The server preprocesses the collected raw data, which includes cleaning the data, extracting metadata, and formatting the text. For emails, basic information such as the body of the email, sender, and date and time of receipt is extracted, while chat messages are converted into text containing the sender and time of the call, and call logs are converted into text containing the duration and content of the call.

[0937] Step 3: Extract action items

[0938] The server applies natural language processing (NLP) to the preprocessed data to extract action items. The NLP engine identifies tasks, instructions, and important information in the text and identifies them as action items. Specifically, it extracts keywords related to instructions and tasks from each email, chat message, and call log.

[0939] Step 4: Generate an action item list

[0940] The server compiles the extracted action items into a list, which includes details of each action item (e.g., task name, source, related information), and stores the action items in a database for further processing.

[0941] Step 5: Set the forecast deadline

[0942] The server automatically sets a predicted deadline for each action item using predictive AI. The predictive AI analyzes past task history, task importance, and sender priority to estimate the optimal deadline for each action item. The set deadline is added to the action item list.

[0943] Step 6: Emotion Recognition

[0944] The server uses an emotion engine that recognizes emotions from the user's facial expressions, voice, and input data. The emotion engine detects when the user is stressed or tired and reflects this in the management of action items.

[0945] Step 7: Adjusting tasks based on emotions

[0946] The server adjusts the priority and deadline of action items based on the user's emotional data obtained by the emotion engine. For example, if it detects that the user is feeling high stress, it extends the deadline or changes the priority of some action items.

[0947] Step 8: Present the list to the user

[0948] The device presents the final action item list to the user. The user can check the list on a device such as a PC or smartphone and work on the tasks efficiently. In addition, by checking the list adjusted by the emotion engine, the user can manage tasks in a less burdensome way.

[0949] By executing the above processing steps, this system can centrally manage business contact information and optimize task management while taking into consideration the user's feelings. This system allows business people to avoid overlooking work-related tasks and perform their work efficiently while reducing stress.

[0950] Example 2

[0951] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0952] Modern business people use multiple business communication tools, such as email, chat, and phone records, and are required to efficiently manage information from these tools. However, manually collecting this information, extracting important tasks, setting deadlines, and adjusting priorities based on emotions is extremely time-consuming and labor-intensive. Furthermore, task management based on stress and emotional states is often not possible, making it difficult to improve work efficiency and prevent missed responses.

[0953] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting information, a means for extracting action items from the collected information, a means for listing the action items, a means for setting a predicted deadline for each action item, a means for recognizing the user's emotions and adjusting the priority and deadline of the action items, and a means for presenting the set action item list to the user. This makes it possible to centrally manage information from business communication tools and efficiently extract, list, set deadlines for, and adjust tasks according to emotions.

[0954] The "means for collecting information" is a mechanism for automatically obtaining information at specified intervals from multiple information sources such as an email server, a chat server, and a telephone record server.

[0955] The "means for extracting action items from collected information" refers to a mechanism that analyzes the acquired information, identifies important tasks and instructions using natural language processing technology, and extracts them as action items.

[0956] The "means for listing action items" is a mechanism for listing extracted action items including task names, sources, and related information, and storing the list in a database.

[0957] The "means of setting predicted deadlines for each action item" is a mechanism that uses predictive AI to analyze past task history, task importance, and sender priority to estimate appropriate deadline dates.

[0958] The "means for recognizing the user's emotions and adjusting the priority and deadlines of action items" is a mechanism that automatically adjusts the priority and deadlines of action items according to the user's stress level using an emotion engine that identifies emotions from the user's facial expressions, voice, input data, etc.

[0959] The "means for presenting the set action item list to the user" is a mechanism for visually presenting the generated and adjusted action item list to the user via a terminal such as a PC or smartphone.

[0960] This invention is a system that automatically extracts information from various business communication tools, lists action items, sets expected deadlines, and optimizes task management using an emotion engine that recognizes the user's emotions. This system is designed to enable business people to prevent oversight of work-related actions, efficiently manage tasks, and flexibly adjust responses based on the user's emotions.

[0961] A specific embodiment for carrying out the present invention will be described.

[0962] Form of information collection

[0963] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information, including unread emails from the email server, new messages from chat, and new call logs from the telephone record server, and temporarily stores this information in storage.

[0964] Examples:

[0965] Every morning at 9am, the server calls each tool's API and retrieves new information using the necessary authentication information. This information is stored in a temporary database.

[0966] Forms of information extraction and analysis

[0967] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts and categorizing them into action items.

[0968] Examples:

[0969] If an email from a boss includes the instruction "Please submit a report by next week," the server will parse it and extract the action item "Submit report." Similarly, if a chat message from a project leader includes the instruction "Please prepare for a meeting," this will also be extracted.

[0970] How to generate an action item list

[0971] The server lists the extracted action items, including details of each action item (e.g., task name, source, related information), and stores the list in a database.

[0972] Examples:

[0973] The server generates the following list:

[0974] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[0975] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[0976] Form of forecast deadline setting

[0977] The server automatically sets the expected deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline for each action item.

[0978] Examples:

[0979] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. For the "meeting preparation" task, it sets a deadline for the next day. The deadlines set in this way are reflected in the list.

[0980] Forms of emotion recognition

[0981] The server combines an emotion engine that recognizes emotions from the user's facial expressions, voice, input data, etc. If it determines that the user is feeling stressed, it automatically adjusts the priority and deadlines of action items.

[0982] Examples:

[0983] While the user is reviewing the action item list, the server analyzes the user's facial expressions and voice tone via the camera and microphone, and the emotion engine determines the user's stress level. If stress is determined to be high, the server extends the deadline for "report submission" by one day and moves "meeting preparation" up the list as a high-priority task.

[0984] How the list is presented to the user

[0985] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list, adjusted by the emotion engine, helps the user effectively manage tasks while reducing stress.

[0986] Examples:

[0987] The user sees the following list on their PC screen:

[0988] 1. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[0989] 2. "Report submission" - Deadline: 4 days later (extended), Original email: Instructions from boss

[0990] Example prompts for generative AI models

[0991] "How do you operate a system that extracts important tasks from emails, chats, and phone records, automatically sets deadlines, performs sentiment analysis, and adjusts priorities?"

[0992] This allows the system to centrally collect business communication information from multiple tools, efficiently manage action items, prevent missed responses, and flexibly adjust responses based on the user's emotions.

[0993] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0994] Step 1:

[0995] Information gathering

[0996] The server accesses the email server, chat server, and phone record server at a specified interval (for example, every morning at 9:00) to collect new information. This includes unread emails from email, new messages from chat, and new call logs from phone records. The server retrieves this information via API and stores it in a temporary database.

[0997] Input: New information (unread emails, new chat messages, new call logs) retrieved from the APIs of the email, chat, and call log servers.

[0998] Data processing: Collecting new information from each server and formatting it for temporary storage

[0999] Output: New information stored in a temporary database

[1000] Specific operation: The server calls each tool's API with the authentication information, obtains new information, and saves it.

[1001] Step 2:

[1002] Information analysis and action item extraction

[1003] The server uses natural language processing techniques to analyze the collected information and extract action items, identifying important tasks and instructions from email text, chat messages, and call transcripts.

[1004] Input: New information stored in a temporary database

[1005] Data processing: Using natural language processing techniques, we analyze important tasks and instructions and categorize them as action items.

[1006] Output: Extracted action items

[1007] What it does: The server uses natural language processing algorithms to analyze and identify imperative and task-related phrases in a sentence.

[1008] Step 3:

[1009] List of action items

[1010] The server lists the extracted action items and stores them in a database, including details of each action item (e.g., task name, source, related information).

[1011] Input: Extracted action items

[1012] Data processing: Format and list the details of each action item

[1013] Output: List of action items stored in the database

[1014] What it does: The server stores each action item in a database along with the task name, source, and other relevant information.

[1015] Step 4:

[1016] Setting a forecast deadline

[1017] The server sets a predicted deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline.

[1018] Input: A list of action items stored in a database

[1019] Data processing: Use predictive AI to estimate deadlines for each action item

[1020] Output: A list of action items with deadlines

[1021] Specific operation: The server analyzes past task history data and sets a predicted deadline based on the execution period of similar tasks.

[1022] Step 5:

[1023] Emotion recognition and action item adjustment

[1024] The server uses an emotion engine that recognizes emotions from the user's facial expressions, voice, and input data, and adjusts the priority and deadlines of action items if the user is feeling stressed.

[1025] Input: Action item list with deadlines, user facial expression data, voice data, and input data

[1026] Data processing: Analyze user emotions using an emotion engine and adjust the priority and deadlines of action items according to stress levels

[1027] Output: A tailored list of action items

[1028] How it works: The server uses a camera and microphone to analyze the user's facial expressions and voice tone in real time, and the emotion engine determines the user's stress level and adjusts action items.

[1029] Step 6:

[1030] Presenting the list to the user

[1031] The device presents the generated action item list to the user, who can then check the action item list through a device such as a PC or smartphone and quickly respond to the tasks.

[1032] Input: Adjusted action item list

[1033] Output: A list of action items presented to the user

[1034] Specific behavior: The device displays the action item list in the user interface so that the user can easily check it.

[1035] In this way, the system can optimize task management by consistently collecting information, extracting action items, listing them, setting deadlines, recognizing emotions, and presenting them to the user.

[1036] (Application example 2)

[1037] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1038] Conventional task management systems have difficulty centrally collecting information from multiple business communication tools and managing important tasks. Furthermore, they are unable to adjust task priorities and deadlines based on user emotions, which can lead to the accumulation of stress and fatigue. Furthermore, in busy environments like factories, information is often overlooked or missed, often resulting in reduced work efficiency. To solve these problems, the present invention proposes a system that provides centralized information management and task optimization based on user emotions.

[1039] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1040] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for recognizing the user's emotions and adjusting the priority and deadline of the action items, and means for presenting the set action item list to the user. This enables centralized management of important tasks from multiple information sources and efficient task management that takes the user's emotional state into consideration.

[1041] The "means for collecting information" is a system that has the function of automatically acquiring necessary data from multiple information sources such as an email server, a chat server, and a communication log server.

[1042] The "means for extracting action items" is a system that utilizes natural language processing technology to identify and separate important tasks and instructions from collected information.

[1043] The "means for creating a list of action items" is a system that has the function of organizing and displaying the extracted action items in a list format and generating a list including detailed information about each item.

[1044] The "means for setting estimated deadlines" is a system that has the ability to automatically estimate appropriate deadlines for action items based on past task history, task importance, and sender priority.

[1045] "Means for recognizing user emotions and adjusting the priority and deadlines of action items" refers to a system that uses a camera and microphone to analyze the user's facial expressions and voice, determine the user's emotional state, and dynamically change task priorities and deadlines.

[1046] "Means for presenting the set action item list to the user" refers to a system that has the function of displaying the generated action item list on the user's device (e.g., smartphone, tablet, PC) and enabling a quick response.

[1047] A system for carrying out the present invention is configured as follows: A server has means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for recognizing a user's emotions and adjusting the priority and deadlines of the action items, and means for presenting the set action item list to the user.

[1048] The server first collects information from email servers, chat servers, communication log servers, etc. During this process, for example, every morning at 9:00, it calls the API of each tool and retrieves new information using the necessary authentication information. This information is then temporarily stored in a database.

[1049] The collected information is analyzed using natural language processing technology (e.g., libraries such as NLTK and spaCy) to extract necessary action items. For example, if an email from a boss contains the instruction "Please submit the report by next week," the server analyzes it and extracts the action item "Submit the report."

[1050] The extracted action items are then compiled into a list, which includes details about each action item (e.g., task name, source, related information), and is displayed in a user-friendly interface and stored in a database.

[1051] Additionally, a predictive AI model (e.g., one using TensorFlow or PyTorch) is used to set an appropriate expected deadline for each action item, based on past task history, task importance, and sender priority.

[1052] The system also incorporates an emotion engine (e.g., Affectiva SDK) that uses the smartphone's camera and microphone to recognize the user's emotions. If the engine determines that the user is feeling stressed, it automatically adjusts the priority and deadline of action items. For example, if the server determines that the user is feeling very stressed, it will extend the deadline for "report submission" by one day and move "meeting preparation" up the list as a high-priority task.

[1053] Finally, the device (smartphone, tablet, or PC) presents the generated action item list to the user, allowing them to efficiently manage their tasks and accomplish their work while reducing stress.

[1054] As a concrete example, consider an environment where many work instructions are generated within a factory. Using a factory task manager app, all work instructions can be centrally managed, natural language processing technology can be used to automatically extract tasks from the instructions, an emotion recognition engine can be used to check the stress levels of workers, and predictive AI can be used to set appropriate deadlines. In this way, work instructions will not be overlooked and tasks will be carried out efficiently.

[1055] An example of a prompt would be:

[1056] "Develop an application that collects work instructions in a factory, analyzes them, and uses emotion recognition to prioritize and efficiently manage tasks."

[1057] In this way, the present invention can provide unified management of information and task optimization based on user sentiment.

[1058] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1059] Step 1:

[1060] The server uses the means of collecting information to call the API of each communication tool (email server, chat server, communication log server) at specified intervals (e.g., every morning at 9:00) to obtain new information. The input is the authentication information and API endpoint of each tool, and the output is the newly obtained emails, chat messages, and call records. The information is temporarily stored in a database.

[1061] Step 2:

[1062] The server uses natural language processing technology to analyze the information collected in step 1 and extract action items. Specifically, it tokenizes the email body, chat messages, and call records and performs grammatical analysis. The input is the unanalyzed information collected in step 1, and the output is the extracted action items (e.g., "Submit a report" or "Prepare for a meeting").

[1063] Step 3:

[1064] The server lists the extracted action items. The input is the action items extracted in step 2, and the output is a list of their details (e.g., task name, source, related information). This list is stored in a database in a user-friendly format.

[1065] Step 4:

[1066] The server uses a predictive AI model to set a predicted deadline for each action item. The input is the action item list generated in step 3 and past task history data, and the output is a list of action items with predicted deadlines. The AI ​​model is built using TensorFlow and PyTorch.

[1067] Step 5:

[1068] The server recognizes the user's emotions using the smartphone's camera and microphone. The emotion engine (e.g., Affectiva SDK) analyzes the user's facial expressions and voice to determine their stress and fatigue levels. The input is real-time data from the camera and microphone, and the output is the user's emotional state (e.g., high stress, low stress).

[1069] Step 6:

[1070] The server automatically adjusts the priority and deadlines of action items based on the emotion recognition results in step 5. The input is the action item list from step 4 and the emotion recognition results from step 5, and the output is the adjusted action item list. For example, if the user is in a high stress state, the deadline for "submitting a report" is extended and "preparing for a meeting" is moved up in the list.

[1071] Step 7:

[1072] The device presents the configured action item list to the user. The input is the action item list adjusted in step 6, and the output is the action item list displayed on the device (smartphone, tablet, PC) screen. The user can check this and manage tasks efficiently.

[1073] In this way, the present invention unifies data from multiple sources and provides optimized task management through natural language processing, emotion recognition, and predictive AI models.

[1074] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1075] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1076] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1077] [Fourth embodiment]

[1078] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1079] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1080] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1081] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1082] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1083] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1084] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1085] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1086] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1087] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1088] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1089] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1090] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1091] This invention is a system that automatically extracts information from various business communication tools, lists action items, and sets expected deadlines. This system is designed to enable business people to prevent oversight of business tasks and manage tasks efficiently.

[1092] Form of information collection

[1093] The server periodically collects information from the email server, chat server, and telephone record server. From the email server, it retrieves unread emails and all emails within a specified period, from the chat server it retrieves new messages, and from the telephone record server it retrieves new call logs. This allows you to centralize information from various business communication tools.

[1094] Examples:

[1095] The server calls the API of each tool at a fixed time (for example, every morning at 9:00) and obtains new information using the necessary authentication information (user name, password, API key, etc.). This information is temporarily stored in storage.

[1096] Forms of information extraction and analysis

[1097] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts.

[1098] Examples:

[1099] If an email from a boss includes instructions such as "Please submit a report by next week," the server will analyze it and extract the action item "Submit report." Similarly, if a project leader sends a chat message with instructions such as "Please prepare for a meeting," this will also be extracted.

[1100] How to generate an action item list

[1101] The server lists the extracted action items, including details of each task, its origin (email, chat, call log, etc.), and associated context information.

[1102] Examples:

[1103] The server generates the following list:

[1104] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[1105] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[1106] Form of forecast deadline setting

[1107] The server automatically sets a predicted deadline for each action item using predictive AI, which takes into account past task history, task importance, sender priority, and other factors to set an appropriate deadline for each task.

[1108] Examples:

[1109] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past, and a deadline for the "meeting preparation" task the next day. In this way, the expected deadlines set for each task are reflected in the list.

[1110] List presentation format

[1111] The terminal presents the user with a list of the generated action items, allowing the user to quickly respond to each action item by referring to the list.

[1112] Examples:

[1113] The user sees the following list on their PC or smartphone screen:

[1114] 1. "Report submission" - Deadline: 3 days later, Original email: Instructions from boss

[1115] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[1116] In this way, the system helps business people to understand the details of business communications thoroughly and efficiently manage and complete tasks.

[1117] The processing flow will be explained below.

[1118] Step 1: Gather information

[1119] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information. It downloads unread emails from the email server via IMAP or POP3 protocol, retrieves new messages from the chat server via API, and retrieves new call logs from the telephone record server. This information is then stored in temporary data storage.

[1120] Step 2: Preprocessing the information

[1121] The server preprocesses the collected raw data, which includes cleaning the data, extracting metadata, and formatting the text. For emails, basic information such as the body of the message, sender, and date and time of receipt is extracted. For chat messages, the server converts the sender and time of the message into text, and for phone logs, the server converts the duration and content of the call into text.

[1122] Step 3: Extract action items

[1123] The server then applies natural language processing (NLP) to the preprocessed data to extract action items. The NLP module performs contextual analysis to identify tasks, instructions, and important information, and then identifies action items based on that. This process extracts and lists task names and related keywords.

[1124] Step 4: Generate an action item list

[1125] The server generates an action item list based on the extracted action items. This list includes details of each action item (e.g., task name, source, related information). The action items are stored in a database for further processing.

[1126] Step 5: Set the forecast deadline

[1127] The server uses predictive AI to set deadlines for each action item. The predictive AI analyzes past task history, task importance, and sender priority to estimate the optimal deadline for each action item. The set deadline is added to the action item list.

[1128] Step 6: Present the list to the user

[1129] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list displays task details and set deadlines, allowing the user to visually manage tasks.

[1130] In this way, the system can centrally collect business communication information from multiple tools, efficiently manage action items, and prevent missed responses.

[1131] Example 1

[1132] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1133] Modern business people use a wide variety of business communication tools, and because information is scattered across them, there is a high risk of missing important tasks or instructions. Furthermore, manually retrieving information from these communication tools and managing tasks is extremely time-consuming and inefficient. It is also difficult to appropriately set task importance and deadlines, leading to missed or delayed responses. Therefore, centralizing information management from business communication tools and efficient task management are key challenges.

[1134] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1135] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for presenting the set action item list to the user, means for periodically calling the API of each information source and obtaining information using authentication information, means for analyzing the collected information and identifying important tasks using natural language processing technology, and means for setting appropriate deadlines using predictive AI based on past task history and importance.This enables centralized management of information from various business communication tools, automatic extraction of tasks, and setting predicted deadlines, enabling efficient task management.

[1136] The "means for collecting information" refers to a function for acquiring information from each information source such as an email server, a chat server, and a telephone record server.

[1137] "Means for extracting action items" is a function for identifying and extracting important tasks and instructions from collected information.

[1138] "A means of listing action items" is a function for organizing extracted tasks and instructions and compiling them into a single list.

[1139] The "means for setting a predicted deadline" is a function for predicting and setting an appropriate completion deadline for each action item.

[1140] The "means for presenting the set action item list to the user" is a function for displaying the generated list so that the user can check it.

[1141] "Means of periodically calling the API of each information source and obtaining information using authentication information" refers to a function that periodically calls the API of an information source and obtains the necessary information using authentication information such as a username, password, or API key.

[1142] "Means for analyzing collected information and identifying important tasks using natural language processing technology" is a function for analyzing collected information using natural language processing technology and identifying important tasks and instructions from text data.

[1143] "Means for setting appropriate deadlines using predictive AI based on past task history and importance" is a function that uses predictive AI based on past task history and the importance of each task to automatically set appropriate deadlines for each task.

[1144] This invention is a system that automatically collects information from various business communication tools, extracts action items, sets predicted deadlines, and creates a list. Detailed embodiments for implementing this system are described below.

[1145] The system mainly includes three main components: a server, a terminal, and a user.

[1146] Form of information collection

[1147] The server is responsible for collecting information. Specifically, it periodically collects information from email servers, chat servers, and telephone record servers. To do this, it uses the APIs of each server. The server runs a scheduled job at a regular time, calling the APIs of each information source and retrieving the latest information using the necessary authentication information (username, password, API key, etc.). This information is then temporarily stored in storage.

[1148] Examples:

[1149] Every morning at 9:00, the server retrieves unread emails and all emails within a specified period from the email server API, retrieves new messages from the chat server API, and retrieves new call logs from the phone record server API.

[1150] Forms of information extraction and analysis

[1151] The server analyzes the collected information and extracts action items using natural language processing (NLP) technology. The server applies NLP algorithms to the collected emails, chat messages, and call logs, dividing each message and log into sentences and phrases, and extracting important keywords and phrases.

[1152] Examples:

[1153] If an email from a boss says, "Please submit a report by next week," the server extracts the action item "Submit report." Similarly, if a chat message from a project leader says, "Please prepare for a meeting," the server extracts the action item "Prepare for a meeting."

[1154] How to generate an action item list

[1155] The server generates a list based on the extracted action items. This list includes details of each action item, its origin (email, chat, phone log, etc.), and related context information. For each action item, the server adds the task name, origin, context information, etc. to the list.

[1156] Examples:

[1157] The server generates the following list:

[1158] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[1159] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[1160] Form of forecast deadline setting

[1161] The server uses predictive AI to set a predicted deadline for each action item. The predictive AI takes into account past task history, task importance, sender priority, etc. to set an appropriate deadline. The server retrieves and analyzes the history of similar past tasks from a database.

[1162] Examples:

[1163] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. It sets a deadline for the "meeting preparation" task the next day. In this way, the appropriate deadlines are reflected for each task.

[1164] List presentation format

[1165] The device presents the generated list of action items to the user. The server sends the generated list to the device, which then displays the received list on its screen (e.g., a PC or smartphone).

[1166] Examples:

[1167] A user checks the following list on their computer or smartphone:

[1168] 1. "Report submission" - Deadline: 3 days later, Original email: Instructions from boss

[1169] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[1170] Specific prompt examples

[1171] Here are some examples of prompts for generative AI models:

[1172] Please design a system that extracts important tasks and instructions from my emails and chat messages. For example, from an email that says "Please turn in a report next week," it could extract the task "Submit report" and set a due date three days later using predictive AI. Also, please make this list available on my smartphone.

[1173] In this way, the system helps business people to efficiently manage the contents of business contacts and respond to them without any omissions.

[1174] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1175] Step 1: Gather information

[1176] The server periodically executes a job to collect information. This job calls the APIs of the email server, chat server, and call record server using authentication information (username, password, API key, etc.) to obtain the latest information. The API endpoint and authentication information of each server are used as input, and the obtained data (unread emails, chat messages, new call logs) is saved in storage as output.

[1177] Specific behavior:

[1178] The scheduled job runs every morning at 9am.

[1179] Get unread emails and all emails within a specified period from the email server API.

[1180] Get new messages from the chat server API.

[1181] Get new call logs from the call log server API.

[1182] Step 2: Information extraction and analysis

[1183] The server analyzes the collected data and extracts action items. It uses natural language processing (NLP) techniques to analyze collected emails, chat messages, and call logs to extract important keywords and phrases. It uses the collected data as input and lists important tasks and instructions as output.

[1184] Specific behavior:

[1185] Divide the saved data into sentences or phrases.

[1186] Apply NLP algorithms to identify important keywords and phrases.

[1187] Extract the task "Submit report" from an email from your boss saying "Please submit the report strictly next week."

[1188] Extract the task "Prepare for the meeting" from the chat message from the project leader saying "Please prepare for the meeting."

[1189] Step 3: Generate a list of action items

[1190] The server generates a list from the extracted action items, including details of each task, its origin (email, chat, call log, etc.), and associated context information. It uses the extracted action items as input and generates a formatted list as output.

[1191] Specific behavior:

[1192] Add task name, origin, and context information for each action item.

[1193] The list includes tasks such as "Submit report" and "Prepare for meeting," along with their origin and related information.

[1194] Step 4: Set the forecast deadline

[1195] The server uses predictive AI to set a predicted deadline for each action item. It automatically sets an appropriate deadline by taking into account past task history, importance, sender priority, etc. It uses the history and importance of past similar tasks as input and sets a predicted deadline for each action item as output.

[1196] Specific behavior:

[1197] Retrieve historical data of past similar tasks from a database.

[1198] Predictive AI calculates an appropriate deadline based on importance and sender priority.

[1199] For example, you can set a deadline for "report submission" three days later and a deadline for "meeting preparation" the next day.

[1200] Step 5: Present the list

[1201] The terminal presents the generated list of action items to the user, receives the list sent from the server and displays it on the screen, and uses the list sent from the server as input and displays it in a format that can be viewed by the user as output.

[1202] Specific behavior:

[1203] The terminal receives the list from the server.

[1204] The user checks the list on their PC or smartphone screen.

[1205] The list will show items such as "Submit report" - due date: 3 days later, original email: instructions from boss, "Prepare for meeting" - due date: next day, original chat: instructions from project leader.

[1206] The above is the specific flow and operation of each processing step of this system. At each step, input is received, appropriate data processing and analysis is performed, and output is generated. This allows business people to efficiently manage business communications and respond quickly to required tasks.

[1207] (Application example 1)

[1208] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1209] The complexity of business communications in logistics facilities and its management issues are a problem. Business communications from many information sources are scattered, and there is a risk that workers may miss important tasks. Therefore, a system that can efficiently collect information and manage tasks is required.

[1210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1211] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for presenting the list of set action items to a user, and means for collecting information from a business communication tool at the logistics facility and generating action items, thereby enabling personnel at the logistics facility to efficiently manage tasks and respond without overlooking important matters.

[1212] "Means for collecting information" refers to the function of periodically obtaining information from a specific server (such as an email server, chat server, or telephone record server).

[1213] "Means of extracting action items from collected information" refers to the ability to identify important tasks or instructions from the acquired information and identify them as individual action items.

[1214] "Means for listing action items" refers to a function for organizing and displaying extracted action items in a list format.

[1215] "Means for setting predicted deadlines" refers to the function of predicting and setting appropriate deadlines for each action item based on its importance and past history.

[1216] The "means for presenting the set action item list to the user" refers to a function for displaying the action items, which have been listed and have their predicted deadlines set, on the user's terminal.

[1217] "A means of collecting information from business communication tools at logistics facilities and generating action items" refers to the function of obtaining information from emails, chat messages, call records, etc. related to logistics operations, analyzing them, and listing them as action items.

[1218] The present invention is a system for automating and efficiently managing information collection and tasks from business communication tools in a logistics facility. Specific embodiments of the system are described below.

[1219] Form of information collection

[1220] The server has a mechanism for periodically collecting information from the email server, chat server, and telephone record server. It retrieves unread emails and all emails within a specified period from the email server, new messages from the chat server, and new call logs from the telephone record server. This information collection is generally performed using the API of each server. The server retrieves new information using the necessary authentication information (username, password, API key, etc.), and this information is temporarily stored in storage.

[1221] Forms of information extraction and analysis

[1222] The server analyzes the collected information and uses natural language processing (NLP) to extract action items. Specifically, it uses Hugging Face's transformers library to automatically identify important tasks and instructions from email text, chat messages, and call logs. For example, if an email from a boss contains the instruction "Please submit your report punctually next week," the server analyzes it and extracts the action item "Submit report."

[1223] How to generate an action item list

[1224] The server generates a list of extracted action items, including details of each task, its origin (email, chat, phone log, etc.), and any associated context information. An example listing might look like this:

[1225] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[1226] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[1227] Form of forecast deadline setting

[1228] The server automatically sets predicted deadlines based on the collected information. Specifically, it sets an appropriate deadline for each task, taking into account past task history, task importance, sender priority, etc. For example, for the task "Submit a report," it sets a deadline of three days in the future based on the history of similar past tasks.

[1229] List presentation format

[1230] The device presents the generated list of action items to the user. The user can refer to this list and quickly respond to each action item. An example of how a user can view the list of action items on a PC or smartphone screen is shown below.

[1231] 1. "Submit report" - Deadline: 3 days later, Original email: Instructions from boss

[1232] 2. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[1233] This system helps logistics facility staff centrally manage information from multiple business communication tools, helping them manage and complete tasks efficiently and without omissions.

[1234] Specific examples and prompts for the generative AI model

[1235] For example, a logistics facility operator might receive:

[1236] "Please check the parts availability by next week" (email)

[1237] "Please prepare for the meeting starting at 2pm" (chat)

[1238] "Please complete the packaging for the new product by tomorrow." (Call notes)

[1239] Example prompt sentence:

[1240] "Identify action items in emails and set appropriate deadlines."

[1241] "Identify the important tasks from the text below and set expected deadlines."

[1242] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1243] Step 1: Gather information

[1244] The server obtains information from the email server, chat server, and telephone record server. It uses each server's API to collect new information using the necessary authentication information (username, password, API key, etc.). The collected information is temporarily stored in storage. This allows all contact information related to the user's work to be managed centrally. The input is new message data from each server, and the output is unanalyzed information in temporary storage.

[1245] Step 2: Analyze the information and extract action items

[1246] The server analyzes the collected information using natural language processing (NLP) technology. Specifically, it uses Hugging Face's transformers library to automatically identify important tasks and instructions from email text, chat messages, and call records. It extracts action items from the analysis results and stores them in temporary storage. The input is unanalyzed information, and the output is the extracted action items.

[1247] Step 3: Generate an action item list

[1248] The server lists the extracted action items, including details of each task, its origin (email, chat, phone log, etc.), and related context information. The listed action items are used as data to set the expected deadline in a later step. The input is the extracted action items, and the output is the listed action items.

[1249] Step 4: Set the forecast deadline

[1250] The server sets predicted deadlines for the listed action items. It sets an appropriate deadline for each task, taking into account factors such as past task history, task importance, and sender priority. This prediction is performed using a generative AI model. The input is the listed action items and past task history, and the output is the action items with predicted deadlines.

[1251] Step 5: View the Action Items List

[1252] The terminal presents the user with a list of action items with predicted deadlines sent from the server. The user can refer to this list and quickly respond to each action item. For example, the user checks the list on a smartphone or PC screen. The input is the action items with predicted deadlines, and the output is the list of action items displayed to the user.

[1253] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1254] This invention is a system that automatically extracts information from various business communication tools, lists action items, sets expected deadlines, and optimizes task management using an emotion engine that recognizes the user's emotions. This system is designed to enable business people to prevent oversight of work-related actions, efficiently manage tasks, and flexibly adjust responses based on the user's emotions.

[1255] Form of information collection

[1256] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information, including unread emails from the email server, new messages from chat, and new call logs from the telephone record server, and temporarily stores this information in storage.

[1257] Examples:

[1258] Every morning at 9am, the server calls each tool's API and retrieves new information using the necessary authentication information. This information is stored in a temporary database.

[1259] Forms of information extraction and analysis

[1260] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts and categorizing them into action items.

[1261] Examples:

[1262] If an email from a boss includes the instruction "Please submit a report by next week," the server will parse it and extract the action item "Submit report." Similarly, if a chat message from a project leader includes the instruction "Please prepare for a meeting," this will also be extracted.

[1263] How to generate an action item list

[1264] The server lists the extracted action items, including details of each action item (e.g., task name, source, related information), and stores the list in a database.

[1265] Examples:

[1266] The server generates the following list:

[1267] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[1268] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[1269] Form of forecast deadline setting

[1270] The server automatically sets the expected deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline for each action item.

[1271] Examples:

[1272] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. For the "meeting preparation" task, it sets a deadline for the next day. The deadlines set in this way are reflected in the list.

[1273] Forms of emotion recognition

[1274] The server combines an emotion engine that recognizes emotions from the user's facial expressions, voice, input data, etc. If it determines that the user is feeling stressed, it automatically adjusts the priority and deadlines of action items.

[1275] Examples:

[1276] While the user is reviewing the action item list, the server analyzes the user's facial expressions and voice tone via the camera and microphone, and the emotion engine determines the user's stress level. If stress is determined to be high, the server extends the deadline for "report submission" by one day and moves "meeting preparation" up the list as a high-priority task.

[1277] How the list is presented to the user

[1278] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list, adjusted by the emotion engine, helps the user effectively manage tasks while reducing stress.

[1279] Examples:

[1280] The user sees the following list on their PC screen:

[1281] 1. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[1282] 2. "Report submission" - Deadline: 4 days later (extended), Original email: Instructions from boss

[1283] In this way, the system can centrally collect business communication information from multiple tools, efficiently manage action items, prevent missed responses, and flexibly adjust responses according to the user's emotions.

[1284] The processing flow will be explained below.

[1285] Step 1: Gather information

[1286] The server periodically accesses the email server, chat server, and telephone record server to collect new information. It downloads unread emails from the email server using IMAP or POP3 protocol, retrieves the latest messages from the chat server using API, and retrieves new call logs from the telephone record server. This information is temporarily stored in a database.

[1287] Step 2: Preprocessing the information

[1288] The server preprocesses the collected raw data, which includes cleaning the data, extracting metadata, and formatting the text. For emails, basic information such as the body of the email, sender, and date and time of receipt is extracted, while chat messages are converted into text containing the sender and time of the call, and call logs are converted into text containing the duration and content of the call.

[1289] Step 3: Extract action items

[1290] The server applies natural language processing (NLP) to the preprocessed data to extract action items. The NLP engine identifies tasks, instructions, and important information in the text and identifies them as action items. Specifically, it extracts keywords related to instructions and tasks from each email, chat message, and call log.

[1291] Step 4: Generate an action item list

[1292] The server compiles the extracted action items into a list, which includes details of each action item (e.g., task name, source, related information), and stores the action items in a database for further processing.

[1293] Step 5: Set the forecast deadline

[1294] The server automatically sets a predicted deadline for each action item using predictive AI. The predictive AI analyzes past task history, task importance, and sender priority to estimate the optimal deadline for each action item. The set deadline is added to the action item list.

[1295] Step 6: Emotion Recognition

[1296] The server uses an emotion engine that recognizes emotions from the user's facial expressions, voice, and input data. The emotion engine detects when the user is stressed or tired and reflects this in the management of action items.

[1297] Step 7: Adjusting tasks based on emotions

[1298] The server adjusts the priority and deadline of action items based on the user's emotional data obtained by the emotion engine. For example, if it detects that the user is feeling high stress, it extends the deadline or changes the priority of some action items.

[1299] Step 8: Present the list to the user

[1300] The device presents the final action item list to the user. The user can check the list on a device such as a PC or smartphone and work on the tasks efficiently. In addition, by checking the list adjusted by the emotion engine, the user can manage tasks in a less burdensome way.

[1301] By executing the above processing steps, this system can centrally manage business contact information and optimize task management while taking into consideration the user's feelings. This system allows business people to avoid overlooking work-related tasks and perform their work efficiently while reducing stress.

[1302] Example 2

[1303] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1304] Modern business people use multiple business communication tools, such as email, chat, and phone records, and are required to efficiently manage information from these tools. However, manually collecting this information, extracting important tasks, setting deadlines, and adjusting priorities based on emotions is extremely time-consuming and labor-intensive. Furthermore, task management based on stress and emotional states is often not possible, making it difficult to improve work efficiency and prevent missed responses.

[1305] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting information, a means for extracting action items from the collected information, a means for listing the action items, a means for setting a predicted deadline for each action item, a means for recognizing the user's emotions and adjusting the priority and deadline of the action items, and a means for presenting the set action item list to the user. This makes it possible to centrally manage information from business communication tools and efficiently extract, list, set deadlines for, and adjust tasks according to emotions.

[1306] The "means for collecting information" is a mechanism for automatically obtaining information at specified intervals from multiple information sources such as an email server, a chat server, and a telephone record server.

[1307] The "means for extracting action items from collected information" refers to a mechanism that analyzes the acquired information, identifies important tasks and instructions using natural language processing technology, and extracts them as action items.

[1308] The "means for listing action items" is a mechanism for listing extracted action items including task names, sources, and related information, and storing the list in a database.

[1309] The "means of setting predicted deadlines for each action item" is a mechanism that uses predictive AI to analyze past task history, task importance, and sender priority to estimate appropriate deadline dates.

[1310] The "means for recognizing the user's emotions and adjusting the priority and deadlines of action items" is a mechanism that automatically adjusts the priority and deadlines of action items according to the user's stress level using an emotion engine that identifies emotions from the user's facial expressions, voice, input data, etc.

[1311] The "means for presenting the set action item list to the user" is a mechanism for visually presenting the generated and adjusted action item list to the user via a terminal such as a PC or smartphone.

[1312] This invention is a system that automatically extracts information from various business communication tools, lists action items, sets expected deadlines, and optimizes task management using an emotion engine that recognizes the user's emotions. This system is designed to enable business people to prevent oversight of work-related actions, efficiently manage tasks, and flexibly adjust responses based on the user's emotions.

[1313] A specific embodiment for carrying out the present invention will be described.

[1314] Form of information collection

[1315] The server accesses the email server, chat server, and telephone record server at specified intervals to collect new information, including unread emails from the email server, new messages from chat, and new call logs from the telephone record server, and temporarily stores this information in storage.

[1316] Examples:

[1317] Every morning at 9am, the server calls each tool's API and retrieves new information using the necessary authentication information. This information is stored in a temporary database.

[1318] Forms of information extraction and analysis

[1319] The server uses natural language processing technology to analyze the collected information and extract action items, automatically identifying important tasks and instructions from emails, chat messages, and call transcripts and categorizing them into action items.

[1320] Examples:

[1321] If an email from a boss includes the instruction "Please submit a report by next week," the server will parse it and extract the action item "Submit report." Similarly, if a chat message from a project leader includes the instruction "Please prepare for a meeting," this will also be extracted.

[1322] How to generate an action item list

[1323] The server lists the extracted action items, including details of each action item (e.g., task name, source, related information), and stores the list in a database.

[1324] Examples:

[1325] The server generates the following list:

[1326] 1. "Report submission" - Original email: Email from boss, Deadline: Not set

[1327] 2. "Meeting Preparation" - Original chat: Message from project leader, Deadline: Not set

[1328] Form of forecast deadline setting

[1329] The server automatically sets the expected deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline for each action item.

[1330] Examples:

[1331] The server sets a deadline for the "report submission" task three days in the future based on the history of similar tasks in the past. For the "meeting preparation" task, it sets a deadline for the next day. The deadlines set in this way are reflected in the list.

[1332] Forms of emotion recognition

[1333] The server combines an emotion engine that recognizes emotions from the user's facial expressions, voice, input data, etc. If it determines that the user is feeling stressed, it automatically adjusts the priority and deadlines of action items.

[1334] Examples:

[1335] While the user is reviewing the action item list, the server analyzes the user's facial expressions and voice tone via the camera and microphone, and the emotion engine determines the user's stress level. If stress is determined to be high, the server extends the deadline for "report submission" by one day and moves "meeting preparation" up the list as a high-priority task.

[1336] How the list is presented to the user

[1337] The device presents the generated action item list to the user. The user can check the action item list through a device such as a PC or smartphone and quickly respond to each task. The list, adjusted by the emotion engine, helps the user effectively manage tasks while reducing stress.

[1338] Examples:

[1339] The user sees the following list on their PC screen:

[1340] 1. "Prepare for the meeting" - Deadline: Next day, Original chat: Instructions from the project leader

[1341] 2. "Report submission" - Deadline: 4 days later (extended), Original email: Instructions from boss

[1342] Example prompts for generative AI models

[1343] "How do you operate a system that extracts important tasks from emails, chats, and phone records, automatically sets deadlines, performs sentiment analysis, and adjusts priorities?"

[1344] This allows the system to centrally collect business communication information from multiple tools, efficiently manage action items, prevent missed responses, and flexibly adjust responses based on the user's emotions.

[1345] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1346] Step 1:

[1347] Information gathering

[1348] The server accesses the email server, chat server, and phone record server at a specified interval (for example, every morning at 9:00) to collect new information. This includes unread emails from email, new messages from chat, and new call logs from phone records. The server retrieves this information via API and stores it in a temporary database.

[1349] Input: New information (unread emails, new chat messages, new call logs) retrieved from the APIs of the email, chat, and call log servers.

[1350] Data processing: Collecting new information from each server and formatting it for temporary storage

[1351] Output: New information stored in a temporary database

[1352] Specific operation: The server calls each tool's API with the authentication information, obtains new information, and saves it.

[1353] Step 2:

[1354] Information analysis and action item extraction

[1355] The server uses natural language processing techniques to analyze the collected information and extract action items, identifying important tasks and instructions from email text, chat messages, and call transcripts.

[1356] Input: New information stored in a temporary database

[1357] Data processing: Using natural language processing techniques, we analyze important tasks and instructions and categorize them as action items.

[1358] Output: Extracted action items

[1359] What it does: The server uses natural language processing algorithms to analyze and identify imperative and task-related phrases in a sentence.

[1360] Step 3:

[1361] List of action items

[1362] The server lists the extracted action items and stores them in a database, including details of each action item (e.g., task name, source, related information).

[1363] Input: Extracted action items

[1364] Data processing: Format and list the details of each action item

[1365] Output: List of action items stored in the database

[1366] What it does: The server stores each action item in a database along with the task name, source, and other relevant information.

[1367] Step 4:

[1368] Setting a forecast deadline

[1369] The server sets a predicted deadline for each action item using predictive AI, which analyzes past task history, task importance, and sender priority to estimate an appropriate deadline.

[1370] Input: A list of action items stored in a database

[1371] Data processing: Use predictive AI to estimate deadlines for each action item

[1372] Output: A list of action items with deadlines

[1373] Specific operation: The server analyzes past task history data and sets a predicted deadline based on the execution period of similar tasks.

[1374] Step 5:

[1375] Emotion recognition and action item adjustment

[1376] The server uses an emotion engine that recognizes emotions from the user's facial expressions, voice, and input data, and adjusts the priority and deadlines of action items if the user is feeling stressed.

[1377] Input: Action item list with deadlines, user facial expression data, voice data, and input data

[1378] Data processing: Analyze user emotions using an emotion engine and adjust the priority and deadlines of action items according to stress levels

[1379] Output: A tailored list of action items

[1380] How it works: The server uses a camera and microphone to analyze the user's facial expressions and voice tone in real time, and the emotion engine determines the user's stress level and adjusts action items.

[1381] Step 6:

[1382] Presenting the list to the user

[1383] The device presents the generated action item list to the user, who can then check the action item list through a device such as a PC or smartphone and quickly respond to the tasks.

[1384] Input: Adjusted action item list

[1385] Output: A list of action items presented to the user

[1386] Specific behavior: The device displays the action item list in the user interface so that the user can easily check it.

[1387] In this way, the system can optimize task management by consistently collecting information, extracting action items, listing them, setting deadlines, recognizing emotions, and presenting them to the user.

[1388] (Application example 2)

[1389] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1390] Conventional task management systems have difficulty centrally collecting information from multiple business communication tools and managing important tasks. Furthermore, they are unable to adjust task priorities and deadlines based on user emotions, which can lead to the accumulation of stress and fatigue. Furthermore, in busy environments like factories, information is often overlooked or missed, often resulting in reduced work efficiency. To solve these problems, the present invention proposes a system that provides centralized information management and task optimization based on user emotions.

[1391] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1392] In this invention, the server includes means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for recognizing the user's emotions and adjusting the priority and deadline of the action items, and means for presenting the set action item list to the user. This enables centralized management of important tasks from multiple information sources and efficient task management that takes the user's emotional state into consideration.

[1393] The "means for collecting information" is a system that has the function of automatically acquiring necessary data from multiple information sources such as an email server, a chat server, and a communication log server.

[1394] The "means for extracting action items" is a system that utilizes natural language processing technology to identify and separate important tasks and instructions from collected information.

[1395] The "means for creating a list of action items" is a system that has the function of organizing and displaying the extracted action items in a list format and generating a list including detailed information about each item.

[1396] The "means for setting estimated deadlines" is a system that has the ability to automatically estimate appropriate deadlines for action items based on past task history, task importance, and sender priority.

[1397] "Means for recognizing user emotions and adjusting the priority and deadlines of action items" refers to a system that uses a camera and microphone to analyze the user's facial expressions and voice, determine the user's emotional state, and dynamically change task priorities and deadlines.

[1398] "Means for presenting the set action item list to the user" refers to a system that has the function of displaying the generated action item list on the user's device (e.g., smartphone, tablet, PC) and enabling a quick response.

[1399] A system for carrying out the present invention is configured as follows: A server has means for collecting information, means for extracting action items from the collected information, means for listing the action items, means for setting a predicted deadline for each action item, means for recognizing a user's emotions and adjusting the priority and deadlines of the action items, and means for presenting the set action item list to the user.

[1400] The server first collects information from email servers, chat servers, communication log servers, etc. During this process, for example, every morning at 9:00, it calls the API of each tool and retrieves new information using the necessary authentication information. This information is then temporarily stored in a database.

[1401] The collected information is analyzed using natural language processing technology (e.g., libraries such as NLTK and spaCy) to extract necessary action items. For example, if an email from a boss contains the instruction "Please submit the report by next week," the server analyzes it and extracts the action item "Submit the report."

[1402] The extracted action items are then compiled into a list, which includes details about each action item (e.g., task name, source, related information), and is displayed in a user-friendly interface and stored in a database.

[1403] Additionally, a predictive AI model (e.g., one using TensorFlow or PyTorch) is used to set an appropriate expected deadline for each action item, based on past task history, task importance, and sender priority.

[1404] The system also incorporates an emotion engine (e.g., Affectiva SDK) that uses the smartphone's camera and microphone to recognize the user's emotions. If the engine determines that the user is feeling stressed, it automatically adjusts the priority and deadline of action items. For example, if the server determines that the user is feeling very stressed, it will extend the deadline for "report submission" by one day and move "meeting preparation" up the list as a high-priority task.

[1405] Finally, the device (smartphone, tablet, or PC) presents the generated action item list to the user, allowing them to efficiently manage their tasks and accomplish their work while reducing stress.

[1406] As a concrete example, consider an environment where many work instructions are generated within a factory. Using a factory task manager app, all work instructions can be centrally managed, natural language processing technology can be used to automatically extract tasks from the instructions, an emotion recognition engine can be used to check the stress levels of workers, and predictive AI can be used to set appropriate deadlines. In this way, work instructions will not be overlooked and tasks will be carried out efficiently.

[1407] An example of a prompt would be:

[1408] "Develop an application that collects work instructions in a factory, analyzes them, and uses emotion recognition to prioritize and efficiently manage tasks."

[1409] In this way, the present invention can provide unified management of information and task optimization based on user sentiment.

[1410] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1411] Step 1:

[1412] The server uses the means of collecting information to call the API of each communication tool (email server, chat server, communication log server) at specified intervals (e.g., every morning at 9:00) to obtain new information. The input is the authentication information and API endpoint of each tool, and the output is the newly obtained emails, chat messages, and call records. The information is temporarily stored in a database.

[1413] Step 2:

[1414] The server uses natural language processing technology to analyze the information collected in step 1 and extract action items. Specifically, it tokenizes the email body, chat messages, and call records and performs grammatical analysis. The input is the unanalyzed information collected in step 1, and the output is the extracted action items (e.g., "Submit a report" or "Prepare for a meeting").

[1415] Step 3:

[1416] The server lists the extracted action items. The input is the action items extracted in step 2, and the output is a list of their details (e.g., task name, source, related information). This list is stored in a database in a user-friendly format.

[1417] Step 4:

[1418] The server uses a predictive AI model to set a predicted deadline for each action item. The input is the action item list generated in step 3 and past task history data, and the output is a list of action items with predicted deadlines. The AI ​​model is built using TensorFlow and PyTorch.

[1419] Step 5:

[1420] The server recognizes the user's emotions using the smartphone's camera and microphone. The emotion engine (e.g., Affectiva SDK) analyzes the user's facial expressions and voice to determine their stress and fatigue levels. The input is real-time data from the camera and microphone, and the output is the user's emotional state (e.g., high stress, low stress).

[1421] Step 6:

[1422] The server automatically adjusts the priority and deadlines of action items based on the emotion recognition results in step 5. The input is the action item list from step 4 and the emotion recognition results from step 5, and the output is the adjusted action item list. For example, if the user is in a high stress state, the deadline for "submitting a report" is extended and "preparing for a meeting" is moved up in the list.

[1423] Step 7:

[1424] The device presents the configured action item list to the user. The input is the action item list adjusted in step 6, and the output is the action item list displayed on the device (smartphone, tablet, PC) screen. The user can check this and manage tasks efficiently.

[1425] In this way, the present invention unifies data from multiple sources and provides optimized task management through natural language processing, emotion recognition, and predictive AI models.

[1426] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1428] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1429] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1430] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1431] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1432] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1433] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1434] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1435] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1436] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1437] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1438] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1439] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1440] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1441] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1442] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1443] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1444] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1445] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1446] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1447] The following is further disclosed regarding the above embodiment.

[1448] (Claim 1)

[1449] means of collecting information;

[1450] A means for extracting action items from the collected information;

[1451] A way to list action items,

[1452] a means for setting a projected deadline for each action item;

[1453] means for presenting the configured action item list to the user;

[1454] A system including:

[1455] (Claim 2)

[1456] means for collecting information from an email server, means for collecting information from a chat server, and means for collecting information from a telephone record server;

[1457] 10. The system of claim 1.

[1458] (Claim 3)

[1459] means for extracting action items from the collected information using natural language processing techniques;

[1460] 10. The system of claim 1.

[1461] (Claim 4)

[1462] including using predictive AI to set projected deadlines for each action item;

[1463] 10. The system of claim 1.

[1464] (Claim 5)

[1465] a means for listing the extracted action items and providing an input field for including a predicted deadline in the list;

[1466] 10. The system of claim 1.

[1467] "Example 1"

[1468] (Claim 1)

[1469] means of collecting information;

[1470] A means for extracting action items from the collected information;

[1471] A way to list action items,

[1472] a means for setting a projected deadline for each action item;

[1473] means for presenting the configured action item list to the user;

[1474] A means of periodically calling the API of each information source and retrieving information using authentication information;

[1475] A means of analyzing the collected information and identifying important tasks using natural language processing technology;

[1476] A method to set appropriate deadlines using predictive AI based on past task history and importance, and

[1477] A system including:

[1478] (Claim 2)

[1479] means for collecting information from an email server, means for collecting information from a chat server, and means for collecting information from a telephone record server;

[1480] 10. The system of claim 1.

[1481] (Claim 3)

[1482] means for extracting action items from the collected information using natural language processing techniques;

[1483] 10. The system of claim 1.

[1484] "Application Example 1"

[1485] (Claim 1)

[1486] means of collecting information;

[1487] A means for extracting action items from the collected information;

[1488] A way to list action items,

[1489] a means for setting a projected deadline for each action item;

[1490] means for presenting the configured action item list to the user;

[1491] a means for collecting information from business communication tools at a logistics facility and generating action items;

[1492] A system including:

[1493] (Claim 2)

[1494] means for collecting information from an email server, means for collecting information from a chat server, and means for collecting information from a telephone record server;

[1495] 10. The system of claim 1.

[1496] (Claim 3)

[1497] means for extracting action items from the collected information using natural language processing techniques;

[1498] 10. The system of claim 1.

[1499] "Example 2: Combining Emotion Engines"

[1500] (Claim 1)

[1501] means of collecting information;

[1502] A means for extracting action items from the collected information;

[1503] A way to list action items,

[1504] a means for setting a projected deadline for each action item;

[1505] A way to recognize user emotions and adjust the priorities and deadlines of action items;

[1506] means for presenting the configured action item list to the user;

[1507] A system including:

[1508] (Claim 2)

[1509] means for collecting information from an email server, means for collecting information from a chat server, and means for collecting information from a telephone record server;

[1510] 10. The system of claim 1.

[1511] (Claim 3)

[1512] means for extracting action items from the collected information using natural language processing techniques;

[1513] 10. The system of claim 1.

[1514] "Application example 2 when combining emotion engines"

[1515] (Claim 1)

[1516] means of collecting information;

[1517] A means for extracting action items from the collected information;

[1518] A way to list action items,

[1519] a means for setting a projected deadline for each action item;

[1520] A way to recognize user emotions and adjust the priorities and deadlines of action items;

[1521] means for presenting the configured action item list to the user;

[1522] A system including:

[1523] (Claim 2)

[1524] The method includes collecting information from an email server, collecting information from a chat server, and collecting information from a communication log server.

[1525] 10. The system of claim 1.

[1526] (Claim 3)

[1527] means for extracting action items from the collected information using natural language processing techniques;

[1528] 10. The system of claim 1. [Explanation of symbols]

[1529] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means of collecting information; A means for extracting action items from the collected information; A way to list action items, a means for setting a projected deadline for each action item; means for presenting the configured action item list to the user; A system including:

2. means for collecting information from an email server, means for collecting information from a chat server, and means for collecting information from a telephone record server; The system of claim 1 .

3. means for extracting action items from the collected information using natural language processing techniques; The system of claim 1 .

4. including using predictive AI to set projected deadlines for each action item; The system of claim 1 .

5. a means for listing the extracted action items and providing an input field for including a predicted deadline in the list; The system of claim 1 .

Images