system

Abstract

We provide the system. [Solution] A means of acquiring information through multiple communication methods, A means of analyzing acquired information using natural language processing technology and extracting important information, A method for automatically incorporating extracted important information into the user's schedule, A means of providing timely notices for scheduled matters, A system that includes a means of updating the schedule based on user instructions.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a modern business environment, a large amount of information flows in daily through multiple communication means, and the risk of overlooking important information and requests is increasing. Due to such omission of information, it becomes difficult to manage requests and deadlines, and there is a problem that business efficiency decreases.

Means for Solving the Problems

[0005] This invention solves the above problems by providing a system that analyzes information acquired through multiple communication means using natural language processing technology and automatically extracts important information. Furthermore, by integrating the extracted important information into the user's schedule and providing notifications according to deadlines, it prevents tasks from being overlooked and improves work efficiency. In addition, it enables users to easily grasp information from multiple communication means through a centralized information display means.

[0006] "Communication methods" refer to technical interfaces for sending and receiving information, such as email, chat applications, and messaging services.

[0007] "Means of acquiring information" refers to the process or tools for collecting data from communication sources and importing it into the user's device.

[0008] "Natural language processing technology" refers to algorithms and methodologies that enable computers to understand and analyze human language.

[0009] "Important matters" refer to information that users should prioritize addressing from a business, daily life, or other perspective.

[0010] "Methods for incorporating into a schedule" refers to methods or systems for integrating extracted key items into existing schedules or task lists.

[0011] "Means of notification" refer to alert and message sending functions that inform users of task deadlines and important matters.

[0012] "Methods for updating schedules" refers to the process of modifying or changing schedules and task lists in response to user instructions. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a tagged processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General - Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a tagged RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a tagged storage is one or more non - volatile storage devices that store various programs and various parameters, etc. Examples of non - volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, a tagged communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi - Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention provides a system and method for efficiently managing information obtained by users through multiple communication means and preventing the omission of important information. This system acquires, analyzes, manages, and notifies information through the interaction of a server, terminals, and users.

[0035] First, the server automatically retrieves new messages from users using multiple communication methods, such as email servers and chat application APIs. At this stage, the server stores the collected unprocessed messages for later analysis.

[0036] Next, the server uses natural language processing techniques to analyze the retrieved messages. In this analysis process, the server extracts information related to important items, such as "deadline" and "request details." This helps identify which messages should be prioritized for the user.

[0037] Next, the server automatically incorporates the extracted information into the user's schedule. Considering existing appointments and meeting dates, the server can perform optimal schedule adjustments. For tasks added to the schedule, the server sets deadlines as needed.

[0038] Furthermore, the device receives integrated schedules and tasks from the server, providing users with a centralized view. This view is designed to allow users to intuitively see all tasks and related information.

[0039] Finally, the device will issue an alarm notification based on the deadline set for each task. This alarm function allows users to efficiently manage tasks without missing important deadlines.

[0040] For example, if a user misses checking an important email, the server automatically analyzes the email and adds a task to their schedule, such as "Complete preparations for next week's meeting." The device then issues an alarm about this task as the deadline approaches, supporting the user in taking proactive action.

[0041] This system aims to prevent errors caused by the sheer volume of information and maximize the efficiency of users' work.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server connects to multiple communication methods (e.g., mail servers and chat applications) and uses APIs to retrieve new messages from users. The server then stores these messages in a database.

[0045] Step 2:

[0046] The server sequentially sends the stored messages to the natural language processing (NLP) engine. The NLP engine analyzes the message content and extracts important information and tasks. Specifically, it identifies keywords such as "deadline" and "request" and extracts related information.

[0047] Step 3:

[0048] The server uses the analysis results obtained from the NLP engine to compare the extracted key information with the user's schedule data. The server then adds new tasks to the schedule and coordinates them with the user's meetings and other appointments.

[0049] Step 4:

[0050] The terminal receives schedule information sent from the server and displays an integrated view to the user. This view includes all tasks, due dates, and associated notes.

[0051] Step 5:

[0052] The server tracks the deadlines for scheduled tasks and sends notifications to the device as the deadline approaches. The device displays an alarm to the user, reminding them of any incomplete tasks.

[0053] Step 6:

[0054] When users need to add new tasks or modify existing ones, they send instructions to the server via their terminal. The server then updates the schedule based on these instructions, and the changes are immediately reflected in the unified view.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] In today's information society, users receive a massive volume of messages daily from multiple communication channels, making it difficult to properly manage important information and incorporate it into action plans in a timely manner. This often leads to overlooking important information and delays in schedules, resulting in decreased work efficiency.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes means for acquiring data via multiple communication means, means for analyzing the acquired data using natural language processing technology and extracting important items, and means for automatically incorporating the extracted important items into the user's action plan. This enables the user to efficiently manage an action plan based on the importance of the information, ensuring that important tasks are not overlooked and supporting the smooth execution of the schedule.

[0060] "Communication means" refers to technical devices or protocols for sending and receiving information, including, for example, email servers and APIs for online chat platforms.

[0061] "Data" refers to a collection of information obtained through means of communication, and includes text, audio, images, or a combination thereof.

[0062] "Natural language processing technology" is a computational technique that uses computers to analyze, understand, and manipulate human language, and is used to extract important information from text.

[0063] "Important items" refer to information that needs to be addressed as a priority in the user's work or action plan, and include deadlines and requests.

[0064] An "action plan" is a compilation of a user's schedule and appointments, and is part of a timetable used for managing daily tasks.

[0065] A "warning" is a means of notifying a user that a deadline for an appointment or task is approaching, and can be provided visually or audibly.

[0066] This invention is a system that efficiently manages information and supports users' work. The server acquires information using multiple communication methods. This includes APIs for mail servers and chat applications, specifically automatically acquiring emails and messages through the APIs. For example, messages are acquired from mail servers using the IMAP protocol, and data is retrieved from chat applications using the provided APIs.

[0067] The server then analyzes the acquired data using natural language processing techniques. This analysis utilizes text analysis libraries, such as Python's NLTK or SpaCy. The purpose of the analysis is to identify important items within the text, such as "deadline" and "request details." These extracted items are then automatically incorporated into the user's action plan by the server.

[0068] Next, the device receives an integrated action plan provided by the server. This plan is displayed on a screen that centrally displays the user's planned activities. The device's alert function notifies the user when the deadline for each task is approaching, preventing them from missing tasks.

[0069] For example, if a user misses an important email, the server automatically analyzes the email and incorporates a task such as "complete preparations for next week's meeting" into the user's action plan. This allows the device to notify the user of an alarm as the deadline approaches, enabling the user to take action in advance.

[0070] A possible prompt for explaining this system using a generative AI model might be: "Please explain the function that incorporates important email tasks for next week's meeting into the action plan and notifies users before the deadline."

[0071] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0072] Step 1:

[0073] The server retrieves user messages through multiple communication methods. It uses APIs from mail servers and chat applications as input. The server retrieves unread and new messages from these APIs and stores them in a database. Specifically, this involves accessing mail servers using the IMAP protocol to collect unprocessed messages. The retrieved message data is provided as output.

[0074] Step 2:

[0075] The server analyzes the acquired message data using natural language processing techniques. The message data saved in step 1 is used as input. The server uses a text analysis library (e.g., Python's NLTK or SpaCy) to extract important items from the message, such as "deadline" and "request details." Specifically, it may identify keywords within the text and extract related information. The output is a list of the extracted important items.

[0076] Step 3:

[0077] The server automatically incorporates the extracted key items into the user's action plan. It uses the list of key items obtained in step 2 as input. The server manipulates scheduling software (e.g., Google Calendar or Outlook) to add these as new tasks to the action plan. Specifically, it places tasks at the optimal time, taking existing appointments into consideration. An updated action plan is generated as output.

[0078] Step 4:

[0079] The terminal receives the integrated action plan from the server and displays it to the user. It takes the updated action plan from step 3 as input. The terminal visually displays this on a dedicated application, providing it to the user in an easy-to-understand manner. Specifically, it uses calendar and list view functions to allow the user to grasp the status of all tasks. The output is a visualized action plan.

[0080] Step 5:

[0081] The device alerts users to tasks with approaching deadlines based on the action plan. It uses the updated action plan received in step 4 as input. The device utilizes a notification function linked to task deadlines to alert the user. Specifically, it displays a pop-up or audio notification at a specified time to inform the user of the task deadline. An alert notification is generated for the user as output.

[0082] (Application Example 1)

[0083] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0084] In modern times, many users access various information and conduct transactions through multiple communication methods and platforms. However, efficiently managing important information and payment deadlines obtained from these diverse sources and taking appropriate action based on that information is difficult. As a result, important tasks may be overlooked or payment deadlines missed, disrupting users' lives and work. This invention aims to solve such problems and realize efficient information management and notification.

[0085] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0086] In this invention, the server includes means for acquiring data via multiple communication means, means for analyzing the acquired data using natural language processing technology and extracting important information, and means for automatically incorporating the extracted important information into the user's activity plan. This enables the user to properly manage important information and payment deadlines obtained from various sources and to receive necessary notifications.

[0087] "Communication methods" refer to methods and systems for transmitting information, and include email and chat applications.

[0088] "Data" refers to various types of information obtained through communication methods, including textual and numerical information.

[0089] "Natural language processing technology" is a technology that enables computers to understand and process human language, and is used for text analysis and extraction of important information.

[0090] "Important matters" refer to information that should be prioritized and managed in the user's activities, and include, for example, payment deadlines and necessary tasks.

[0091] An "activity plan" is a schedule that lists the tasks and appointments that users need to complete, and it is managed in a simplified manner.

[0092] "Deadline-based notifications" is a function that issues warnings or reminders to users based on a set date or time.

[0093] "Transaction information" refers to data related to buying, selling, and payments conducted across multiple platforms.

[0094] "Payment deadline" refers to the final date and time by which payment for a particular transaction or invoice must be completed.

[0095] "Priority" is a criterion for determining the order in which multiple tasks or pieces of information should be addressed.

[0096] The system that implements this application example is configured as follows:

[0097] First, the server obtains data from users using multiple communication methods, such as email and chat application APIs. This data includes transaction information and important schedule information. The server then analyzes this obtained data using natural language processing techniques, such as NLTK and spaCy, to extract important information. This important information includes payment deadlines and important tasks.

[0098] Next, the server uses a database management system (e.g., SQLite) to manage the extracted important information so that it can be automatically incorporated into the user's activity plan. It can be linked with the user's existing schedule and deadlines can be set for each important item.

[0099] The user's device receives integrated activity plan information and displays it centrally through a smartphone application. This display allows users to intuitively check their schedule and prioritize important tasks.

[0100] Furthermore, the device notifies users based on the deadlines for each task, issuing timely warnings. This helps prevent payment delays and ensures that important tasks are completed without fail.

[0101] For example, if a user enters the prompt "When is my next credit card payment due?" into a smartphone app, the app analyzes the deadline from the user's schedule and displays a notification on the screen saying, "Your credit card payment due is this Friday."

[0102] This system allows users to efficiently manage data from various sources and perform their daily tasks without missing important deadlines.

[0103] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0104] Step 1:

[0105] The server automatically retrieves new user data using APIs from email servers and chat applications. This process extracts text data from the APIs and stores it on the server. The input is the data request from the API, and the output is the stored, unprocessed text data.

[0106] Step 2:

[0107] The server applies natural language processing techniques to the acquired text data. For example, it uses NLTK or spaCy to analyze the text and extract important information. The input is raw text data, and the output is a list of important items. This list includes things like payment deadlines and important tasks.

[0108] Step 3:

[0109] The server stores the extracted important information in a database and incorporates it into the user's activity plan. Specifically, it inserts the data into SQLite, adjusts it with the existing schedule, and sets deadlines for the important items. The input is a list of important items, and the output is the updated schedule.

[0110] Step 4:

[0111] The terminal displays integrated activity plan information received from the server in a user interface. This display allows users to intuitively check their schedules and tasks. The input is the updated schedule, and the output is a visually represented activity plan.

[0112] Step 5:

[0113] The terminal sends notifications based on the deadlines of each task. The program uses an internal clock to monitor deadlines and issues alerts at specific times. The input is a list of tasks with deadlines, and the output is a notification to the user.

[0114] Step 6:

[0115] The user enters prompt messages via a terminal to request specific information. The system analyzes the data based on the queries sent to the server and provides the user with appropriate information. The input is the user's prompt message, and the output is the information provided based on that message.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] This invention provides a system that takes into account the emotional state of the user and enables more personalized processing of important information and task management. This system achieves highly accurate schedule management through interaction between the server, terminal, and user.

[0118] First, the server acquires information through multiple communication methods, including messaging services such as email and chat. The acquired information is stored in a database in preparation for subsequent processing.

[0119] The server uses natural language processing technology to analyze these messages. The analysis process extracts important information and identifies it as a priority task. At this stage, the server uses an emotion engine to recognize the user's emotional state. This emotional state is inferred from factors such as the words the user uses and their typing speed.

[0120] After the emotion engine understands the user's emotional state, the server dynamically adjusts the priority of the acquired tasks based on this information. For example, if the user is feeling stressed, the server will take measures such as reducing urgent tasks.

[0121] The device receives information from the server and provides the user with a centralized interface. Through this interface, the user can view all schedules and access tasks prioritized according to their emotional state. The device can change the timing and expression of notifications based on the results of the emotion engine, and is designed to reduce the user's workload.

[0122] For example, if the emotion engine detects that a user is experiencing high levels of stress, the server will send a meeting reminder slightly earlier, allowing the user to prepare with ample time. In this way, work is facilitated according to the user's emotions, providing an environment where users can work more comfortably.

[0123] Through this system, users can manage tasks optimally according to their emotional state, improving their daily work efficiency. Thus, this invention is not merely an information management system, but is innovative in that it enables flexible responses tailored to the user's situation.

[0124] The following describes the processing flow.

[0125] Step 1:

[0126] The server retrieves new user messages in real time through APIs for multiple communication methods (email, chat, messaging services, etc.). To centrally manage these messages, the server stores them in a dedicated database.

[0127] Step 2:

[0128] The server analyzes stored messages using natural language processing techniques. During the analysis process, the server detects keywords in the messages (e.g., "urgent," "deadline") and extracts important information and tasks. This extracted information is later used for prioritizing.

[0129] Step 3:

[0130] The server uses an emotion engine to assess the user's current emotional state. This assessment is performed by analyzing factors such as the user's input speed, selected words, and device operation history. This emotional state data is essential for task management in the next step.

[0131] Step 4:

[0132] The server dynamically adjusts the priority of tasks extracted based on the user's emotional state. For example, if a user is stressed, the schedule is reorganized to prioritize less demanding tasks. This adjustment reduces the user's psychological burden.

[0133] Step 5:

[0134] The terminal receives integrated task information and a summary of the user's emotional state from the server, providing a schedule view optimized for the user. This view organizes tasks by priority and is designed to make it easy for the user to access the information they need most.

[0135] Step 6:

[0136] The device sends notifications to the user at times that correspond to their emotional state. For example, it provides flexible responses, such as sending notifications later than usual when the user is relaxed, and setting alerts in advance when the user is stressed.

[0137] Step 7:

[0138] Users can check tasks, add new tasks, and modify existing tasks via their devices. If changes are made, the server immediately updates the schedule and reflects them in the next task priority settings. However, the changes are always made while taking the user's emotional state into consideration.

[0139] (Example 2)

[0140] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0141] In today's digital environment, users are often overwhelmed by the sheer volume of data from diverse information sources, making it difficult to properly prioritize important tasks. Furthermore, because each user has different emotional states, standard methods struggle to provide flexible responses tailored to individual needs.

[0142] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0143] In this invention, the server includes means for collecting information via multiple digital communication means, means for analyzing and extracting important data using natural language processing technology, and means for estimating the user's emotional state. This enables flexible task management and information processing in accordance with the user's emotional state.

[0144] "Digital communication methods" refer to technical means for exchanging information via computer networks, such as email and chat services.

[0145] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate natural human language, and has the ability to extract meaning and important information from text data.

[0146] "Important data" refers to information that has a high priority regarding the user's task management and timetable, and that requires prompt attention.

[0147] A "timetable" is a digital organizational method in the form of a list or calendar used to organize and manage users' schedules and tasks.

[0148] "Emotional state" refers to the user's mental state or mood, and is a factor that influences daily work and task management.

[0149] "Dynamic configuration" refers to the act of making settings and adjustments in real time according to the situation and conditions, and means taking flexible measures that respond to change rather than following a fixed procedure.

[0150] This invention is a system that takes into account the emotional state of the user and enables more personalized processing of important information and task management. This system achieves highly accurate schedule management through interaction between the server, terminal, and user.

[0151] The server collects information using multiple digital communication methods, such as email and chat services. This information is stored in databases such as MongoDB and MySQL®. The server uses software such as spaCy and TENSORFLOW® to analyze the collected information using natural language processing technology and extract important data.

[0152] Furthermore, the server uses an emotion engine to estimate the user's emotional state. This emotion recognition is achieved by analyzing the results of natural language processing and the tone and input speed of the words the user uses. Based on the emotion engine, the server dynamically sets task priorities and takes measures to reduce the user's psychological burden.

[0153] The device provides the user with an intuitive and unified interface based on information from the server. Through this interface, the user can view all schedules and access priority tasks. For example, the device can adjust the timing and wording of notifications to present information in a way that minimizes user stress.

[0154] For example, if the emotion engine detects that a user is experiencing high levels of stress, the server will notify the user of high-priority tasks earlier than usual, allowing the user to respond with ample time. In this way, by adjusting tasks according to emotions, the system provides an environment where users can perform their work more efficiently.

[0155] As an example of inputting a prompt into the generating AI model, you can use a sentence like, "Suggest effective ways for a user experiencing stress to manage their tasks." This allows the AI ​​model to generate the optimal task management method tailored to the user's emotional state.

[0156] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0157] Step 1:

[0158] The server collects information using multiple digital communication methods, such as email and chat. The input at this stage is raw digital data obtained from multiple messaging platforms. The server structures this data and stores it in a database. For example, it retrieves email bodies from mail servers and messages from chat APIs and registers them in the database in text format.

[0159] Step 2:

[0160] The server analyzes the collected data using natural language processing (NLP) techniques. This process uses text data stored in a database as input. The server extracts important keywords and phrases from the text using NLP tools such as spaCy and TensorFlow, and outputs a list of potential tasks. Specifically, it identifies meeting dates and tasks with deadlines.

[0161] Step 3:

[0162] The server uses an emotion engine to estimate the user's emotional state. The input for this step includes the results of NLP analysis, as well as data on the user's past input patterns and language usage characteristics. The data processing performed by the server combines this information to generate an emotion score that recognizes the user's current psychological state. For example, if a hastily typed message contains many negative words, the server might determine that the user is stressed.

[0163] Step 4:

[0164] The server dynamically adjusts task priorities based on estimated emotional states. This step takes emotional scores and a list of potential tasks as input, re-evaluates the importance and urgency of tasks, and generates an adjusted task list as output. Specifically, for users experiencing stress, it reduces the number of tasks with approaching deadlines and provides advance notice of long-term plans.

[0165] Step 5:

[0166] The terminal provides information to the user through an optimized interface, based on the adjusted task information received from the server. The input is task list data from the server, and the output is the task management interface displayed on the user's screen. The terminal adjusts the timing and content of notifications and performs specific actions to visually communicate the details and priority of each task to the user.

[0167] Step 6:

[0168] Users check their schedules and manage tasks using an interface displayed on their terminal. Input is the information displayed on the terminal, and user actions and feedback are returned to the server as output. Specifically, data is reflected in the system when users complete tasks or change their schedules. Based on this feedback, the server re-evaluates the tasks in the next step.

[0169] (Application Example 2)

[0170] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0171] Modern information processing systems do not take into account users' emotional states when managing schedules or prioritizing tasks, making it difficult to respond flexibly to individual user needs. Furthermore, retail stores face the challenge of being unable to provide customer service that aligns with customers' emotions, thus failing to achieve sufficient customer satisfaction.

[0172] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0173] In this invention, the server includes means for acquiring information via multiple communication means, means for analyzing the user's emotional state and dynamically adjusting notifications based on that emotional state, and means for supporting customer service tailored to the user based on the acquired information. This enables personalized schedule management according to the user's emotional state, thereby improving the quality of customer service in physical stores.

[0174] "Communication methods" refer to various methods and technical means used to obtain information, including email and chat services.

[0175] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and process human language, and includes the process of extracting meaning from text.

[0176] "Important matters" refer to items from the acquired information that should be processed with particular priority, and are information that is judged to have high priority in the user's schedule management.

[0177] "Emotional state" refers to the psychological or emotional state exhibited by the user, and is the mental state inferred from their language and behavior.

[0178] "Dynamic adjustment" refers to making changes flexibly according to the situation, rather than being fixed, and specifically includes operations that change notifications and other information according to the individual user's status.

[0179] "Supporting customer service" means providing support to help stores handle customer interactions more effectively, including providing service tailored to the customer's emotional state.

[0180] This invention relates to a system that processes information while considering the emotional state of the user to perform task management and customer service support. The system consists of a server, terminals, and users. The server acquires information through multiple communication methods. It is responsible for collecting data from information sources such as email and messaging services. This information is analyzed using natural language processing technology to extract important information. The analysis utilizes the Vision API and Speech-to-Text technology provided by Google Cloud.

[0181] The server uses an emotion engine to recognize the user's emotional state. This emotional state is inferred from the user's speech patterns and typing speed. Task priorities are dynamically adjusted according to the user's emotions. For example, if the server detects that the user is stressed, it reduces the number of urgent tasks. Furthermore, to support customer service in physical stores, information is displayed on smart glasses. This allows store employees to understand the customer's emotional state in real time and provide appropriate service.

[0182] As a concrete example, if a user is under stress, a response could be given to slightly advance the notification of a high-priority meeting to allow them time to prepare. In retail customer service, if a customer appears happy, new products may be recommended, while if they appear stressed, a quick response may be prioritized. Using a generative AI model (e.g., GPT-4(registered trademark)), prompt statements like the following can be used.

[0183] "A customer has entered the store and is asking about special offers. They seem excited. What kind of customer service phrases should I use?"

[0184] By implementing this system, users can manage tasks optimally according to their emotional state, thereby improving customer satisfaction in physical stores.

[0185] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0186] Step 1:

[0187] The server collects information from multiple communication methods. It receives data from email and messaging services as input. It stores information in natural language format in a database as output. This step involves performing specific actions to retrieve information via the APIs of each communication method.

[0188] Step 2:

[0189] The server analyzes the acquired information using natural language processing technology. The input is the natural language data collected in step 1. The output is summarized information extracted as important points. A text analysis engine is used for the analysis, performing keyword extraction and sentiment analysis.

[0190] Step 3:

[0191] The server uses an emotion engine to recognize the user's emotional state. Input consists of user text data and information obtained from input speed. Output is a score representing the user's emotional state. Sentiment analysis is performed by combining the natural language processing results with behavioral patterns.

[0192] Step 4:

[0193] The server dynamically adjusts task priorities based on the user's emotional state. Inputs are the user's emotional score and extracted key points. Output is a list of prioritized tasks. A scheduling algorithm is used to rearrange the tasks.

[0194] Step 5:

[0195] The terminal presents the user with a task list with adjusted priorities. The input is the task list generated in step 4. The output is a user-readable task list display, provided as visual information through the user interface.

[0196] Step 6:

[0197] The server generates prompts using a generative AI model and suggests user-specific actions. The input is the user's emotional state and task. The output is suggested actions and customer service phrases. The generative AI model is used to create prompts and provide the user with guidance on how to respond.

[0198] Step 7:

[0199] The user adjusts their actions based on the presented tasks and suggestions. The inputs are the task list in step 5 and the suggestions in step 6. The output is the user's actual actions and choices. This allows the user to take the optimal action according to their emotional state.

[0200] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0201] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0202] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0203] [Second Embodiment]

[0204] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0205] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0206] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0207] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0208] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0209] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0210] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0211] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0212] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0213] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0214] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0215] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0216] This invention provides a system and method for efficiently managing information obtained by users through multiple communication means and preventing the omission of important information. This system acquires, analyzes, manages, and notifies information through the interaction of a server, terminals, and users.

[0217] First, the server automatically retrieves new messages from users using multiple communication methods, such as email servers and chat application APIs. At this stage, the server stores the collected unprocessed messages for later analysis.

[0218] Next, the server uses natural language processing techniques to analyze the retrieved messages. In this analysis process, the server extracts information related to important items, such as "deadline" and "request details." This helps identify which messages should be prioritized for the user.

[0219] Next, the server automatically incorporates the extracted information into the user's schedule. Considering existing appointments and meeting dates, the server can perform optimal schedule adjustments. For tasks added to the schedule, the server sets deadlines as needed.

[0220] Furthermore, the device receives integrated schedules and tasks from the server, providing users with a centralized view. This view is designed to allow users to intuitively see all tasks and related information.

[0221] Finally, the device will issue an alarm notification based on the deadline set for each task. This alarm function allows users to efficiently manage tasks without missing important deadlines.

[0222] For example, if a user misses checking an important email, the server automatically analyzes the email and adds a task to their schedule, such as "Complete preparations for next week's meeting." The device then issues an alarm about this task as the deadline approaches, supporting the user in taking proactive action.

[0223] This system aims to prevent errors caused by the sheer volume of information and maximize the efficiency of users' work.

[0224] The following describes the processing flow.

[0225] Step 1:

[0226] The server connects to multiple communication methods (e.g., mail servers and chat applications) and uses APIs to retrieve new messages from users. The server then stores these messages in a database.

[0227] Step 2:

[0228] The server sequentially sends the stored messages to the natural language processing (NLP) engine. The NLP engine analyzes the message content and extracts important information and tasks. Specifically, it identifies keywords such as "deadline" and "request" and extracts related information.

[0229] Step 3:

[0230] The server uses the analysis results obtained from the NLP engine to compare the extracted key information with the user's schedule data. The server then adds new tasks to the schedule and coordinates them with the user's meetings and other appointments.

[0231] Step 4:

[0232] The terminal receives schedule information sent from the server and displays an integrated view to the user. This view includes all tasks, due dates, and associated notes.

[0233] Step 5:

[0234] The server tracks the deadlines for scheduled tasks and sends notifications to the device as the deadline approaches. The device displays an alarm to the user, reminding them of any incomplete tasks.

[0235] Step 6:

[0236] When users need to add new tasks or modify existing ones, they send instructions to the server via their terminal. The server then updates the schedule based on these instructions, and the changes are immediately reflected in the unified view.

[0237] (Example 1)

[0238] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0239] In today's information society, users receive a massive volume of messages daily from multiple communication channels, making it difficult to properly manage important information and incorporate it into action plans in a timely manner. This often leads to overlooking important information and delays in schedules, resulting in decreased work efficiency.

[0240] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0241] In this invention, the server includes means for acquiring data via multiple communication means, means for analyzing the acquired data using natural language processing technology and extracting important items, and means for automatically incorporating the extracted important items into the user's action plan. This enables the user to efficiently manage an action plan based on the importance of the information, ensuring that important tasks are not overlooked and supporting the smooth execution of the schedule.

[0242] "Communication means" refers to technical devices or protocols for sending and receiving information, including, for example, email servers and APIs for online chat platforms.

[0243] "Data" refers to a collection of information obtained through means of communication, and includes text, audio, images, or a combination thereof.

[0244] "Natural language processing technology" is a computational technique that uses computers to analyze, understand, and manipulate human language, and is used to extract important information from text.

[0245] "Important items" refer to information that needs to be addressed as a priority in the user's work or action plan, and include deadlines and requests.

[0246] An "action plan" is a compilation of a user's schedule and appointments, and is part of a timetable used for managing daily tasks.

[0247] A "warning" is a means of notifying a user that a deadline for an appointment or task is approaching, and can be provided visually or audibly.

[0248] This invention is a system that efficiently manages information and supports users' work. The server acquires information using multiple communication methods. This includes APIs for mail servers and chat applications, specifically automatically acquiring emails and messages through the APIs. For example, messages are acquired from mail servers using the IMAP protocol, and data is retrieved from chat applications using the provided APIs.

[0249] The server then analyzes the acquired data using natural language processing techniques. This analysis utilizes text analysis libraries, such as Python's NLTK or SpaCy. The purpose of the analysis is to identify important items within the text, such as "deadline" and "request details." These extracted items are then automatically incorporated into the user's action plan by the server.

[0250] Next, the device receives an integrated action plan provided by the server. This plan is displayed on a screen that centrally displays the user's planned activities. The device's alert function notifies the user when the deadline for each task is approaching, preventing them from missing tasks.

[0251] For example, if a user misses an important email, the server automatically analyzes the email and incorporates a task such as "complete preparations for next week's meeting" into the user's action plan. This allows the device to notify the user of an alarm as the deadline approaches, enabling the user to take action in advance.

[0252] A possible prompt for explaining this system using a generative AI model might be: "Please explain the function that incorporates important email tasks for next week's meeting into the action plan and notifies users before the deadline."

[0253] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0254] Step 1:

[0255] The server retrieves user messages through multiple communication methods. It uses APIs from mail servers and chat applications as input. The server retrieves unread and new messages from these APIs and stores them in a database. Specifically, this involves accessing mail servers using the IMAP protocol to collect unprocessed messages. The retrieved message data is provided as output.

[0256] Step 2:

[0257] The server analyzes the acquired message data using natural language processing techniques. The message data saved in step 1 is used as input. The server uses a text analysis library (e.g., Python's NLTK or SpaCy) to extract important items from the message, such as "deadline" and "request details." Specifically, it may identify keywords within the text and extract related information. The output is a list of the extracted important items.

[0258] Step 3:

[0259] The server automatically incorporates the extracted key items into the user's action plan. It uses the list of key items obtained in step 2 as input. The server manipulates scheduling software (e.g., Google Calendar or Outlook) to add these items as new tasks to the action plan. Specifically, it places tasks at the optimal time, taking existing appointments into consideration. An updated action plan is generated as output.

[0260] Step 4:

[0261] The terminal receives the integrated action plan from the server and displays it to the user. It takes the updated action plan from step 3 as input. The terminal visually displays this on a dedicated application, providing it to the user in an easy-to-understand manner. Specifically, it uses calendar and list view functions to allow the user to grasp the status of all tasks. The output is a visualized action plan.

[0262] Step 5:

[0263] The device alerts users to tasks with approaching deadlines based on the action plan. It uses the updated action plan received in step 4 as input. The device utilizes a notification function linked to task deadlines to alert the user. Specifically, it displays a pop-up or audio notification at a specified time to inform the user of the task deadline. An alert notification is generated for the user as output.

[0264] (Application Example 1)

[0265] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] In modern times, many users access various information and conduct transactions through multiple communication methods and platforms. However, efficiently managing important information and payment deadlines obtained from these diverse sources and taking appropriate action based on that information is difficult. As a result, important tasks may be overlooked or payment deadlines missed, disrupting users' lives and work. This invention aims to solve such problems and realize efficient information management and notification.

[0267] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0268] In this invention, the server includes means for acquiring data via multiple communication means, means for analyzing the acquired data using natural language processing technology and extracting important information, and means for automatically incorporating the extracted important information into the user's activity plan. This enables the user to properly manage important information and payment deadlines obtained from various sources and to receive necessary notifications.

[0269] "Communication methods" refer to methods and systems for transmitting information, and include email and chat applications.

[0270] "Data" refers to various types of information obtained through communication methods, including textual and numerical information.

[0271] "Natural language processing technology" is a technology that enables computers to understand and process human language, and is used for text analysis and extraction of important information.

[0272] "Important matters" refer to information that should be prioritized and managed in the user's activities, and include, for example, payment deadlines and necessary tasks.

[0273] An "activity plan" is a schedule that lists the tasks and appointments that users need to complete, and it is managed in a simplified manner.

[0274] "Deadline-based notifications" is a function that issues warnings or reminders to users based on a set date or time.

[0275] "Transaction information" refers to data related to buying, selling, and payments conducted across multiple platforms.

[0276] "Payment deadline" refers to the final date and time by which payment for a particular transaction or invoice must be completed.

[0277] "Priority" is a criterion for determining the order in which multiple tasks or pieces of information should be addressed.

[0278] The system that implements this application example is configured as follows:

[0279] First, the server obtains data from users using multiple communication methods, such as email and chat application APIs. This data includes transaction information and important schedule information. The server then analyzes this obtained data using natural language processing techniques, such as NLTK and spaCy, to extract important information. This important information includes payment deadlines and important tasks.

[0280] Next, the server uses a database management system (e.g., SQLite) to manage the extracted important information so that it can be automatically incorporated into the user's activity plan. It can be linked with the user's existing schedule and deadlines can be set for each important item.

[0281] The user's device receives integrated activity plan information and displays it centrally through a smartphone application. This display allows users to intuitively check their schedule and prioritize important tasks.

[0282] Furthermore, the terminal makes notifications based on the deadlines of each task and issues warnings to the user in a timely manner. This prevents payment delays and enables important tasks to be processed without omission.

[0283] As a specific example, when the user enters the prompt sentence "What is the payment date of the next credit card?" into the smartphone app, the app analyzes the deadline from the user's schedule and displays a notification on the screen saying "The payment deadline for the credit card is this Friday."

[0284] With this system, users can efficiently manage data obtained from various information sources and carry out their daily operations without missing important deadlines.

[0285] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0286] Step 1:

[0287] <000,0908>The server automatically obtains the user's new data using the APIs of an email server or a chat application. In this process, text data is extracted from the API and saved on the server. The input is a data request from the API, and the output is the saved unprocessed text data.

[0288] Step 2:

[0289] The server applies natural language processing technology to the obtained text data. For example, NLTK or spaCy is used to analyze the text and extract important matters. The input is the unprocessed text data, and the output is a list of important items. This list includes payment deadlines and important tasks, etc.

[0290] Step 3:

[0291] The server stores the extracted important information in a database and incorporates it into the user's activity plan. Specifically, it inserts the data into SQLite, adjusts it with the existing schedule, and sets deadlines for the important items. The input is a list of important items, and the output is the updated schedule.

[0292] Step 4:

[0293] The terminal displays integrated activity plan information received from the server in a user interface. This display allows users to intuitively check their schedules and tasks. The input is the updated schedule, and the output is a visually represented activity plan.

[0294] Step 5:

[0295] The terminal sends notifications based on the deadlines of each task. The program uses an internal clock to monitor deadlines and issues alerts at specific times. The input is a list of tasks with deadlines, and the output is a notification to the user.

[0296] Step 6:

[0297] The user enters prompt messages via a terminal to request specific information. The system analyzes the data based on the queries sent to the server and provides the user with appropriate information. The input is the user's prompt message, and the output is the information provided based on that message.

[0298] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0299] This invention provides a system that takes into account the emotional state of the user and enables more personalized processing of important information and task management. This system achieves highly accurate schedule management through interaction between the server, terminal, and user.

[0300] First, the server acquires information through multiple communication methods, including messaging services such as email and chat. The acquired information is stored in a database in preparation for subsequent processing.

[0301] The server uses natural language processing technology to analyze these messages. The analysis process extracts important information and identifies it as a priority task. At this stage, the server uses an emotion engine to recognize the user's emotional state. This emotional state is inferred from factors such as the words the user uses and their typing speed.

[0302] After the emotion engine understands the user's emotional state, the server dynamically adjusts the priority of the acquired tasks based on this information. For example, if the user is feeling stressed, the server will take measures such as reducing urgent tasks.

[0303] The device receives information from the server and provides the user with a centralized interface. Through this interface, the user can view all schedules and access tasks prioritized according to their emotional state. The device can change the timing and expression of notifications based on the results of the emotion engine, and is designed to reduce the user's workload.

[0304] For example, if the emotion engine detects that a user is experiencing high levels of stress, the server will send a meeting reminder slightly earlier, allowing the user to prepare with ample time. In this way, work is facilitated according to the user's emotions, providing an environment where users can work more comfortably.

[0305] Through this system, users can manage tasks optimally according to their emotional state, improving their daily work efficiency. Thus, this invention is not merely an information management system, but is innovative in that it enables flexible responses tailored to the user's situation.

[0306] The following describes the process flow.

[0307] Step 1:

[0308] The server retrieves new messages from the user in real time through the APIs of multiple communication means (such as email, chat, messaging services, etc.). Since the server manages these messages centrally, it stores them in a dedicated database.

[0309] Step 2:

[0310] The server analyzes the stored messages using natural language processing technology. In the analysis process, the server detects keywords (e.g., "urgent", "deadline") in the messages and extracts important matters and tasks. The extracted information is later used for priority setting.

[0311] Step 3:

[0312] The server evaluates the current emotional state of the user by leveraging an emotion engine. The evaluation is performed by analyzing the user's input speed, selected words, terminal operation history, etc. This emotional state data is essential for task management in the next step.

[0313] Step 4:

[0314] The server dynamically adjusts the priority of the tasks extracted based on the emotional state. For example, if the user is in a stressed state, the schedule is reorganized so that tasks with less load are prioritized. This adjustment reduces the user's psychological burden.

[0315] Step 5:

[0316] The terminal receives integrated task information and a summary of the user's emotional state from the server, providing a schedule view optimized for the user. This view organizes tasks by priority and is designed to make it easy for the user to access the information they need most.

[0317] Step 6:

[0318] The device sends notifications to the user at times that correspond to their emotional state. For example, it provides flexible responses, such as sending notifications later than usual when the user is relaxed, and setting alerts in advance when the user is stressed.

[0319] Step 7:

[0320] Users can check tasks, add new tasks, and modify existing tasks via their devices. If changes are made, the server immediately updates the schedule and reflects them in the next task priority settings. However, the changes are always made while taking the user's emotional state into consideration.

[0321] (Example 2)

[0322] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0323] In today's digital environment, users are often overwhelmed by the sheer volume of data from diverse information sources, making it difficult to properly prioritize important tasks. Furthermore, because each user has different emotional states, standard methods struggle to provide flexible responses tailored to individual needs.

[0324] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0325] In this invention, the server includes means for collecting information via multiple digital communication means, means for analyzing and extracting important data using natural language processing technology, and means for estimating the user's emotional state. This enables flexible task management and information processing in accordance with the user's emotional state.

[0326] "Digital communication methods" refer to technical means for exchanging information via computer networks, such as email and chat services.

[0327] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate natural human language, and has the ability to extract meaning and important information from text data.

[0328] "Important data" refers to information that has a high priority regarding the user's task management and timetable, and that requires prompt attention.

[0329] A "timetable" is a digital organizational method in the form of a list or calendar used to organize and manage users' schedules and tasks.

[0330] "Emotional state" refers to the user's mental state or mood, and is a factor that influences daily work and task management.

[0331] "Dynamic configuration" refers to the act of making settings and adjustments in real time according to the situation and conditions, and means taking flexible measures that respond to change rather than following a fixed procedure.

[0332] This invention is a system that takes into account the emotional state of the user and enables more personalized processing of important information and task management. This system achieves highly accurate schedule management through interaction between the server, terminal, and user.

[0333] The server collects information using multiple digital communication methods, such as email and chat services. This information is stored in databases such as MongoDB and MySQL. The server uses software such as spaCy and TensorFlow to analyze the collected information using natural language processing techniques and extract important data.

[0334] Furthermore, the server uses an emotion engine to estimate the user's emotional state. This emotion recognition is achieved by analyzing the results of natural language processing and the tone and input speed of the words the user uses. Based on the emotion engine, the server dynamically sets task priorities and takes measures to reduce the user's psychological burden.

[0335] The device provides the user with an intuitive and unified interface based on information from the server. Through this interface, the user can view all schedules and access priority tasks. For example, the device can adjust the timing and wording of notifications to present information in a way that minimizes user stress.

[0336] For example, if the emotion engine detects that a user is experiencing high levels of stress, the server will notify the user of high-priority tasks earlier than usual, allowing the user to respond with ample time. In this way, by adjusting tasks according to emotions, the system provides an environment where users can perform their work more efficiently.

[0337] As an example of inputting a prompt into the generating AI model, you can use a sentence like, "Suggest effective ways for a user experiencing stress to manage their tasks." This allows the AI ​​model to generate the optimal task management method tailored to the user's emotional state.

[0338] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0339] Step 1:

[0340] The server collects information using multiple digital communication methods, such as email and chat. The input at this stage is raw digital data obtained from multiple messaging platforms. The server structures this data and stores it in a database. For example, it retrieves email bodies from mail servers and messages from chat APIs and registers them in the database in text format.

[0341] Step 2:

[0342] The server analyzes the collected data using natural language processing (NLP) techniques. This process uses text data stored in a database as input. The server extracts important keywords and phrases from the text using NLP tools such as spaCy and TensorFlow, and outputs a list of potential tasks. Specifically, it identifies meeting dates and tasks with deadlines.

[0343] Step 3:

[0344] The server uses an emotion engine to estimate the user's emotional state. The input for this step includes the results of NLP analysis, as well as data on the user's past input patterns and language usage characteristics. The data processing performed by the server combines this information to generate an emotion score that recognizes the user's current psychological state. For example, if a hastily typed message contains many negative words, the server might determine that the user is stressed.

[0345] Step 4:

[0346] The server dynamically adjusts task priorities based on estimated emotional states. This step takes emotional scores and a list of potential tasks as input, re-evaluates the importance and urgency of tasks, and generates an adjusted task list as output. Specifically, for users experiencing stress, it reduces the number of tasks with approaching deadlines and provides advance notice of long-term plans.

[0347] Step 5:

[0348] The terminal provides information to the user through an optimized interface, based on the adjusted task information received from the server. The input is task list data from the server, and the output is the task management interface displayed on the user's screen. The terminal adjusts the timing and content of notifications and performs specific actions to visually communicate the details and priority of each task to the user.

[0349] Step 6:

[0350] Users check their schedules and manage tasks using an interface displayed on their terminal. Input is the information displayed on the terminal, and user actions and feedback are returned to the server as output. Specifically, data is reflected in the system when users complete tasks or change their schedules. Based on this feedback, the server re-evaluates the tasks in the next step.

[0351] (Application Example 2)

[0352] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0353] Modern information processing systems do not take into account users' emotional states when managing schedules or prioritizing tasks, making it difficult to respond flexibly to individual user needs. Furthermore, retail stores face the challenge of being unable to provide customer service that aligns with customers' emotions, thus failing to achieve sufficient customer satisfaction.

[0354] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0355] In this invention, the server includes means for acquiring information via multiple communication means, means for analyzing the user's emotional state and dynamically adjusting notifications based on that emotional state, and means for supporting customer service tailored to the user based on the acquired information. This enables personalized schedule management according to the user's emotional state, thereby improving the quality of customer service in physical stores.

[0356] "Communication methods" refer to various methods and technical means used to obtain information, including email and chat services.

[0357] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and process human language, and includes the process of extracting meaning from text.

[0358] "Important matters" refer to items from the acquired information that should be processed with particular priority, and are information that is judged to have high priority in the user's schedule management.

[0359] "Emotional state" refers to the psychological or emotional state exhibited by the user, and is the mental state inferred from their language and behavior.

[0360] "Dynamic adjustment" refers to making changes flexibly according to the situation, rather than being fixed, and specifically includes operations that change notifications and other information according to the individual user's status.

[0361] "Supporting customer service" means providing support to help stores handle customer interactions more effectively, including providing service tailored to the customer's emotional state.

[0362] This invention relates to a system that processes information while considering the emotional state of the user to perform task management and customer service support. The system consists of a server, terminals, and users. The server acquires information through multiple communication methods. It is responsible for collecting data from information sources such as email and messaging services. This information is analyzed using natural language processing technology to extract important information. The analysis utilizes the Vision API and Speech-to-Text technology provided by Google Cloud.

[0363] The server uses an emotion engine to recognize the user's emotional state. This emotional state is inferred from the user's speech patterns and typing speed. Task priorities are dynamically adjusted according to the user's emotions. For example, if the server detects that the user is stressed, it reduces the number of urgent tasks. Furthermore, to support customer service in physical stores, information is displayed on smart glasses. This allows store employees to understand the customer's emotional state in real time and provide appropriate service.

[0364] As a concrete example, if a user is under stress, a high-priority meeting notification can be moved forward slightly to allow them time to prepare. In retail customer service, if a customer appears happy, new products may be promoted; if they appear stressed, a quick response may be prioritized. Using a generative AI model (e.g., GPT-4), prompt statements like the following can be used.

[0365] "A customer has entered the store and is asking about special offers. They seem excited. What kind of customer service phrases should I use?"

[0366] By implementing this system, users can manage tasks optimally according to their emotional state, thereby improving customer satisfaction in physical stores.

[0367] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0368] Step 1:

[0369] The server collects information from multiple communication methods. It receives data from email and messaging services as input. It stores information in natural language format in a database as output. This step involves performing specific actions to retrieve information via the APIs of each communication method.

[0370] Step 2:

[0371] The server analyzes the acquired information using natural language processing technology. The input is the natural language data collected in step 1. The output is summarized information extracted as important points. A text analysis engine is used for the analysis, performing keyword extraction and sentiment analysis.

[0372] Step 3:

[0373] The server uses an emotion engine to recognize the user's emotional state. Input consists of user text data and information obtained from input speed. Output is a score representing the user's emotional state. Sentiment analysis is performed by combining the natural language processing results with behavioral patterns.

[0374] Step 4:

[0375] The server dynamically adjusts task priorities based on the user's emotional state. Inputs are the user's emotional score and extracted key points. Output is a list of prioritized tasks. A scheduling algorithm is used to rearrange the tasks.

[0376] Step 5:

[0377] The terminal presents the user with a task list with adjusted priorities. The input is the task list generated in step 4. The output is a user-readable task list display, provided as visual information through the user interface.

[0378] Step 6:

[0379] The server generates prompts using a generative AI model and suggests user-specific actions. The input is the user's emotional state and task. The output is suggested actions and customer service phrases. The generative AI model is used to create prompts and provide the user with guidance on how to respond.

[0380] Step 7:

[0381] The user adjusts their actions based on the presented tasks and suggestions. The inputs are the task list in step 5 and the suggestions in step 6. The output is the user's actual actions and choices. This allows the user to take the optimal action according to their emotional state.

[0382] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0383] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0384] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0385] [Third Embodiment]

[0386] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0387] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0388] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0389] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0390] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0391] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0392] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0393] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0394] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0395] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0396] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0397] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0398] This invention provides a system and method for efficiently managing information obtained by users through multiple communication means and preventing the omission of important information. This system acquires, analyzes, manages, and notifies information through the interaction of a server, terminals, and users.

[0399] First, the server automatically retrieves new messages from users using multiple communication methods, such as email servers and chat application APIs. At this stage, the server stores the collected unprocessed messages for later analysis.

[0400] Next, the server uses natural language processing techniques to analyze the retrieved messages. In this analysis process, the server extracts information related to important items, such as "deadline" and "request details." This helps identify which messages should be prioritized for the user.

[0401] Next, the server automatically incorporates the extracted information into the user's schedule. Considering existing appointments and meeting dates, the server can perform optimal schedule adjustments. For tasks added to the schedule, the server sets deadlines as needed.

[0402] Furthermore, the device receives integrated schedules and tasks from the server, providing users with a centralized view. This view is designed to allow users to intuitively see all tasks and related information.

[0403] Finally, the device will issue an alarm notification based on the deadline set for each task. This alarm function allows users to efficiently manage tasks without missing important deadlines.

[0404] For example, if a user misses checking an important email, the server automatically analyzes the email and adds a task to their schedule, such as "Complete preparations for next week's meeting." The device then issues an alarm about this task as the deadline approaches, supporting the user in taking proactive action.

[0405] This system aims to prevent errors caused by the sheer volume of information and maximize the efficiency of users' work.

[0406] The following describes the processing flow.

[0407] Step 1:

[0408] The server connects to multiple communication methods (e.g., mail servers and chat applications) and uses APIs to retrieve new messages from users. The server then stores these messages in a database.

[0409] Step 2:

[0410] The server sequentially sends the stored messages to the natural language processing (NLP) engine. The NLP engine analyzes the message content and extracts important information and tasks. Specifically, it identifies keywords such as "deadline" and "request" and extracts related information.

[0411] Step 3:

[0412] The server uses the analysis results obtained from the NLP engine to compare the extracted key information with the user's schedule data. The server then adds new tasks to the schedule and coordinates them with the user's meetings and other appointments.

[0413] Step 4:

[0414] The terminal receives schedule information sent from the server and displays an integrated view to the user. This view includes all tasks, due dates, and associated notes.

[0415] Step 5:

[0416] The server tracks the deadlines for scheduled tasks and sends notifications to the device as the deadline approaches. The device displays an alarm to the user, reminding them of any incomplete tasks.

[0417] Step 6:

[0418] When users need to add new tasks or modify existing ones, they send instructions to the server via their terminal. The server then updates the schedule based on these instructions, and the changes are immediately reflected in the unified view.

[0419] (Example 1)

[0420] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0421] In today's information society, users receive a massive volume of messages daily from multiple communication channels, making it difficult to properly manage important information and incorporate it into action plans in a timely manner. This often leads to overlooking important information and delays in schedules, resulting in decreased work efficiency.

[0422] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0423] In this invention, the server includes means for acquiring data via multiple communication means, means for analyzing the acquired data using natural language processing technology and extracting important items, and means for automatically incorporating the extracted important items into the user's action plan. This enables the user to efficiently manage an action plan based on the importance of the information, ensuring that important tasks are not overlooked and supporting the smooth execution of the schedule.

[0424] "Communication means" refers to technical devices or protocols for sending and receiving information, including, for example, email servers and APIs for online chat platforms.

[0425] "Data" refers to a collection of information obtained through means of communication, and includes text, audio, images, or a combination thereof.

[0426] "Natural language processing technology" is a computational technique that uses computers to analyze, understand, and manipulate human language, and is used to extract important information from text.

[0427] "Important items" refer to information that needs to be addressed as a priority in the user's work or action plan, and include deadlines and requests.

[0428] An "action plan" is a compilation of a user's schedule and appointments, and is part of a timetable used for managing daily tasks.

[0429] A "warning" is a means of notifying a user that a deadline for an appointment or task is approaching, and can be provided visually or audibly.

[0430] This invention is a system that efficiently manages information and supports users' work. The server acquires information using multiple communication methods. This includes APIs for mail servers and chat applications, specifically automatically acquiring emails and messages through the APIs. For example, messages are acquired from mail servers using the IMAP protocol, and data is retrieved from chat applications using the provided APIs.

[0431] The server then analyzes the acquired data using natural language processing techniques. This analysis utilizes text analysis libraries, such as Python's NLTK or SpaCy. The purpose of the analysis is to identify important items within the text, such as "deadline" and "request details." These extracted items are then automatically incorporated into the user's action plan by the server.

[0432] Next, the device receives an integrated action plan provided by the server. This plan is displayed on a screen that centrally displays the user's planned activities. The device's alert function notifies the user when the deadline for each task is approaching, preventing them from missing tasks.

[0433] For example, if a user misses an important email, the server automatically analyzes the email and incorporates a task such as "complete preparations for next week's meeting" into the user's action plan. This allows the device to notify the user of an alarm as the deadline approaches, enabling the user to take action in advance.

[0434] A possible prompt for explaining this system using a generative AI model might be: "Please explain the function that incorporates important email tasks for next week's meeting into the action plan and notifies users before the deadline."

[0435] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0436] Step 1:

[0437] The server retrieves user messages through multiple communication methods. It uses APIs from mail servers and chat applications as input. The server retrieves unread and new messages from these APIs and stores them in a database. Specifically, this involves accessing mail servers using the IMAP protocol to collect unprocessed messages. The retrieved message data is provided as output.

[0438] Step 2:

[0439] The server analyzes the acquired message data using natural language processing techniques. The message data saved in step 1 is used as input. The server uses a text analysis library (e.g., Python's NLTK or SpaCy) to extract important items from the message, such as "deadline" and "request details." Specifically, it may identify keywords within the text and extract related information. The output is a list of the extracted important items.

[0440] Step 3:

[0441] The server automatically incorporates the extracted key items into the user's action plan. It uses the list of key items obtained in step 2 as input. The server manipulates scheduling software (e.g., Google Calendar or Outlook) to add these items as new tasks to the action plan. Specifically, it places tasks at the optimal time, taking existing appointments into consideration. An updated action plan is generated as output.

[0442] Step 4:

[0443] The terminal receives the integrated action plan from the server and displays it to the user. It takes the updated action plan from step 3 as input. The terminal visually displays this on a dedicated application, providing it to the user in an easy-to-understand manner. Specifically, it uses calendar and list view functions to allow the user to grasp the status of all tasks. The output is a visualized action plan.

[0444] Step 5:

[0445] The device alerts users to tasks with approaching deadlines based on the action plan. It uses the updated action plan received in step 4 as input. The device utilizes a notification function linked to task deadlines to alert the user. Specifically, it displays a pop-up or audio notification at a specified time to inform the user of the task deadline. An alert notification is generated for the user as output.

[0446] (Application Example 1)

[0447] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0448] In modern times, many users access various information and conduct transactions through multiple communication methods and platforms. However, efficiently managing important information and payment deadlines obtained from these diverse sources and taking appropriate action based on that information is difficult. As a result, important tasks may be overlooked or payment deadlines missed, disrupting users' lives and work. This invention aims to solve such problems and realize efficient information management and notification.

[0449] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0450] In this invention, the server includes means for acquiring data via multiple communication means, means for analyzing the acquired data using natural language processing technology and extracting important information, and means for automatically incorporating the extracted important information into the user's activity plan. This enables the user to properly manage important information and payment deadlines obtained from various sources and to receive necessary notifications.

[0451] "Communication methods" refer to methods and systems for transmitting information, and include email and chat applications.

[0452] "Data" refers to various types of information obtained through communication methods, including textual and numerical information.

[0453] "Natural language processing technology" is a technology that enables computers to understand and process human language, and is used for text analysis and extraction of important information.

[0454] "Important matters" refer to information that should be prioritized and managed in the user's activities, and include, for example, payment deadlines and necessary tasks.

[0455] An "activity plan" is a schedule that lists the tasks and appointments that users need to complete, and it is managed in a simplified manner.

[0456] "Deadline-based notifications" is a function that issues warnings or reminders to users based on a set date or time.

[0457] "Transaction information" refers to data related to buying, selling, and payments conducted across multiple platforms.

[0458] "Payment deadline" refers to the final date and time by which payment for a particular transaction or invoice must be completed.

[0459] "Priority" is a criterion for determining the order in which multiple tasks or pieces of information should be addressed.

[0460] The system that implements this application example is configured as follows:

[0461] First, the server obtains data from users using multiple communication methods, such as email and chat application APIs. This data includes transaction information and important schedule information. The server then analyzes this obtained data using natural language processing techniques, such as NLTK and spaCy, to extract important information. This important information includes payment deadlines and important tasks.

[0462] Next, the server uses a database management system (e.g., SQLite) to manage the extracted important information so that it can be automatically incorporated into the user's activity plan. It can be linked with the user's existing schedule and deadlines can be set for each important item.

[0463] The user's device receives integrated activity plan information and displays it centrally through a smartphone application. This display allows users to intuitively check their schedule and prioritize important tasks.

[0464] Furthermore, the device notifies users based on the deadlines for each task, issuing timely warnings. This helps prevent payment delays and ensures that important tasks are completed without fail.

[0465] For example, if a user enters the prompt "When is my next credit card payment due?" into a smartphone app, the app analyzes the deadline from the user's schedule and displays a notification on the screen saying, "Your credit card payment due is this Friday."

[0466] This system allows users to efficiently manage data from various sources and perform their daily tasks without missing important deadlines.

[0467] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0468] Step 1:

[0469] The server automatically retrieves new user data using APIs from email servers and chat applications. This process extracts text data from the APIs and stores it on the server. The input is the data request from the API, and the output is the stored, unprocessed text data.

[0470] Step 2:

[0471] The server applies natural language processing techniques to the acquired text data. For example, it uses NLTK or spaCy to analyze the text and extract important information. The input is raw text data, and the output is a list of important items. This list includes things like payment deadlines and important tasks.

[0472] Step 3:

[0473] The server stores the extracted important information in a database and incorporates it into the user's activity plan. Specifically, it inserts the data into SQLite, adjusts it with the existing schedule, and sets deadlines for the important items. The input is a list of important items, and the output is the updated schedule.

[0474] Step 4:

[0475] The terminal displays integrated activity plan information received from the server in a user interface. This display allows users to intuitively check their schedules and tasks. The input is the updated schedule, and the output is a visually represented activity plan.

[0476] Step 5:

[0477] The terminal sends notifications based on the deadlines of each task. The program uses an internal clock to monitor deadlines and issues alerts at specific times. The input is a list of tasks with deadlines, and the output is a notification to the user.

[0478] Step 6:

[0479] The user enters prompt messages via a terminal to request specific information. The system analyzes the data based on the queries sent to the server and provides the user with appropriate information. The input is the user's prompt message, and the output is the information provided based on that message.

[0480] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0481] This invention provides a system that takes into account the emotional state of the user and enables more personalized processing of important information and task management. This system achieves highly accurate schedule management through interaction between the server, terminal, and user.

[0482] First, the server acquires information through multiple communication methods, including messaging services such as email and chat. The acquired information is stored in a database in preparation for subsequent processing.

[0483] The server uses natural language processing technology to analyze these messages. The analysis process extracts important information and identifies it as a priority task. At this stage, the server uses an emotion engine to recognize the user's emotional state. This emotional state is inferred from factors such as the words the user uses and their typing speed.

[0484] After the emotion engine understands the user's emotional state, the server dynamically adjusts the priority of the acquired tasks based on this information. For example, if the user is feeling stressed, the server will take measures such as reducing urgent tasks.

[0485] The device receives information from the server and provides the user with a centralized interface. Through this interface, the user can view all schedules and access tasks prioritized according to their emotional state. The device can change the timing and expression of notifications based on the results of the emotion engine, and is designed to reduce the user's workload.

[0486] For example, if the emotion engine detects that a user is experiencing high levels of stress, the server will send a meeting reminder slightly earlier, allowing the user to prepare with ample time. In this way, work is facilitated according to the user's emotions, providing an environment where users can work more comfortably.

[0487] Through this system, users can manage tasks optimally according to their emotional state, improving their daily work efficiency. Thus, this invention is not merely an information management system, but is innovative in that it enables flexible responses tailored to the user's situation.

[0488] The following describes the processing flow.

[0489] Step 1:

[0490] The server retrieves new user messages in real time through APIs for multiple communication methods (email, chat, messaging services, etc.). To centrally manage these messages, the server stores them in a dedicated database.

[0491] Step 2:

[0492] The server analyzes stored messages using natural language processing techniques. During the analysis process, the server detects keywords in the messages (e.g., "urgent," "deadline") and extracts important information and tasks. This extracted information is later used for prioritizing.

[0493] Step 3:

[0494] The server uses an emotion engine to assess the user's current emotional state. This assessment is performed by analyzing factors such as the user's input speed, selected words, and device operation history. This emotional state data is essential for task management in the next step.

[0495] Step 4:

[0496] The server dynamically adjusts the priority of tasks extracted based on the user's emotional state. For example, if a user is stressed, the schedule is reorganized to prioritize less demanding tasks. This adjustment reduces the user's psychological burden.

[0497] Step 5:

[0498] The terminal receives integrated task information and a summary of the user's emotional state from the server, providing a schedule view optimized for the user. This view organizes tasks by priority and is designed to make it easy for the user to access the information they need most.

[0499] Step 6:

[0500] The device sends notifications to the user at times that correspond to their emotional state. For example, it provides flexible responses, such as sending notifications later than usual when the user is relaxed, and setting alerts in advance when the user is stressed.

[0501] Step 7:

[0502] Users can check tasks, add new tasks, and modify existing tasks via their devices. If changes are made, the server immediately updates the schedule and reflects them in the next task priority settings. However, the changes are always made while taking the user's emotional state into consideration.

[0503] (Example 2)

[0504] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0505] In today's digital environment, users are often overwhelmed by the sheer volume of data from diverse information sources, making it difficult to properly prioritize important tasks. Furthermore, because each user has different emotional states, standard methods struggle to provide flexible responses tailored to individual needs.

[0506] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0507] In this invention, the server includes means for collecting information via multiple digital communication means, means for analyzing and extracting important data using natural language processing technology, and means for estimating the user's emotional state. This enables flexible task management and information processing in accordance with the user's emotional state.

[0508] "Digital communication methods" refer to technical means for exchanging information via computer networks, such as email and chat services.

[0509] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate natural human language, and has the ability to extract meaning and important information from text data.

[0510] "Important data" refers to information that has a high priority regarding the user's task management and timetable, and that requires prompt attention.

[0511] A "timetable" is a digital organizational method in the form of a list or calendar used to organize and manage users' schedules and tasks.

[0512] "Emotional state" refers to the user's mental state or mood, and is a factor that influences daily work and task management.

[0513] "Dynamic configuration" refers to the act of making settings and adjustments in real time according to the situation and conditions, and means taking flexible measures that respond to change rather than following a fixed procedure.

[0514] This invention is a system that takes into account the emotional state of the user and enables more personalized processing of important information and task management. This system achieves highly accurate schedule management through interaction between the server, terminal, and user.

[0515] The server collects information using multiple digital communication methods, such as email and chat services. This information is stored in databases such as MongoDB and MySQL. The server uses software such as spaCy and TensorFlow to analyze the collected information using natural language processing techniques and extract important data.

[0516] Furthermore, the server uses an emotion engine to estimate the user's emotional state. This emotion recognition is achieved by analyzing the results of natural language processing and the tone and input speed of the words the user uses. Based on the emotion engine, the server dynamically sets task priorities and takes measures to reduce the user's psychological burden.

[0517] The device provides the user with an intuitive and unified interface based on information from the server. Through this interface, the user can view all schedules and access priority tasks. For example, the device can adjust the timing and wording of notifications to present information in a way that minimizes user stress.

[0518] For example, if the emotion engine detects that a user is experiencing high levels of stress, the server will notify the user of high-priority tasks earlier than usual, allowing the user to respond with ample time. In this way, by adjusting tasks according to emotions, the system provides an environment where users can perform their work more efficiently.

[0519] As an example of inputting a prompt into the generating AI model, you can use a sentence like, "Suggest effective ways for a user experiencing stress to manage their tasks." This allows the AI ​​model to generate the optimal task management method tailored to the user's emotional state.

[0520] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0521] Step 1:

[0522] The server collects information using multiple digital communication methods, such as email and chat. The input at this stage is raw digital data obtained from multiple messaging platforms. The server structures this data and stores it in a database. For example, it retrieves email bodies from mail servers and messages from chat APIs and registers them in the database in text format.

[0523] Step 2:

[0524] The server analyzes the collected data using natural language processing (NLP) techniques. This process uses text data stored in a database as input. The server extracts important keywords and phrases from the text using NLP tools such as spaCy and TensorFlow, and outputs a list of potential tasks. Specifically, it identifies meeting dates and tasks with deadlines.

[0525] Step 3:

[0526] The server uses an emotion engine to estimate the user's emotional state. The input for this step includes the results of NLP analysis, as well as data on the user's past input patterns and language usage characteristics. The data processing performed by the server combines this information to generate an emotion score that recognizes the user's current psychological state. For example, if a hastily typed message contains many negative words, the server might determine that the user is stressed.

[0527] Step 4:

[0528] The server dynamically adjusts task priorities based on estimated emotional states. This step takes emotional scores and a list of potential tasks as input, re-evaluates the importance and urgency of tasks, and generates an adjusted task list as output. Specifically, for users experiencing stress, it reduces the number of tasks with approaching deadlines and provides advance notice of long-term plans.

[0529] Step 5:

[0530] The terminal provides information to the user through an optimized interface, based on the adjusted task information received from the server. The input is task list data from the server, and the output is the task management interface displayed on the user's screen. The terminal adjusts the timing and content of notifications and performs specific actions to visually communicate the details and priority of each task to the user.

[0531] Step 6:

[0532] Users check their schedules and manage tasks using an interface displayed on their terminal. Input is the information displayed on the terminal, and user actions and feedback are returned to the server as output. Specifically, data is reflected in the system when users complete tasks or change their schedules. Based on this feedback, the server re-evaluates the tasks in the next step.

[0533] (Application Example 2)

[0534] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0535] Modern information processing systems do not take into account users' emotional states when managing schedules or prioritizing tasks, making it difficult to respond flexibly to individual user needs. Furthermore, retail stores face the challenge of being unable to provide customer service that aligns with customers' emotions, thus failing to achieve sufficient customer satisfaction.

[0536] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0537] In this invention, the server includes means for acquiring information via multiple communication means, means for analyzing the user's emotional state and dynamically adjusting notifications based on that emotional state, and means for supporting customer service tailored to the user based on the acquired information. This enables personalized schedule management according to the user's emotional state, thereby improving the quality of customer service in physical stores.

[0538] "Communication methods" refer to various methods and technical means used to obtain information, including email and chat services.

[0539] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and process human language, and includes the process of extracting meaning from text.

[0540] "Important matters" refer to items from the acquired information that should be processed with particular priority, and are information that is judged to have high priority in the user's schedule management.

[0541] "Emotional state" refers to the psychological or emotional state exhibited by the user, and is the mental state inferred from their language and behavior.

[0542] "Dynamic adjustment" refers to making changes flexibly according to the situation, rather than being fixed, and specifically includes operations that change notifications and other information according to the individual user's status.

[0543] "Supporting customer service" means providing support to help stores handle customer interactions more effectively, including providing service tailored to the customer's emotional state.

[0544] This invention relates to a system that processes information while considering the emotional state of the user to perform task management and customer service support. The system consists of a server, terminals, and users. The server acquires information through multiple communication methods. It is responsible for collecting data from information sources such as email and messaging services. This information is analyzed using natural language processing technology to extract important information. The analysis utilizes the Vision API and Speech-to-Text technology provided by Google Cloud.

[0545] The server uses an emotion engine to recognize the user's emotional state. This emotional state is inferred from the user's speech patterns and typing speed. Task priorities are dynamically adjusted according to the user's emotions. For example, if the server detects that the user is stressed, it reduces the number of urgent tasks. Furthermore, to support customer service in physical stores, information is displayed on smart glasses. This allows store employees to understand the customer's emotional state in real time and provide appropriate service.

[0546] As a concrete example, if a user is under stress, a high-priority meeting notification can be moved forward slightly to allow them time to prepare. In retail customer service, if a customer appears happy, new products may be promoted; if they appear stressed, a quick response may be prioritized. Using a generative AI model (e.g., GPT-4), prompt statements like the following can be used.

[0547] "A customer has entered the store and is asking about special offers. They seem excited. What kind of customer service phrases should I use?"

[0548] By implementing this system, users can manage tasks optimally according to their emotional state, thereby improving customer satisfaction in physical stores.

[0549] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0550] Step 1:

[0551] The server collects information from multiple communication methods. It receives data from email and messaging services as input. It stores information in natural language format in a database as output. This step involves performing specific actions to retrieve information via the APIs of each communication method.

[0552] Step 2:

[0553] The server analyzes the acquired information using natural language processing technology. The input is the natural language data collected in step 1. The output is summarized information extracted as important points. A text analysis engine is used for the analysis, performing keyword extraction and sentiment analysis.

[0554] Step 3:

[0555] The server uses an emotion engine to recognize the user's emotional state. Input consists of user text data and information obtained from input speed. Output is a score representing the user's emotional state. Sentiment analysis is performed by combining the natural language processing results with behavioral patterns.

[0556] Step 4:

[0557] The server dynamically adjusts task priorities based on the user's emotional state. Inputs are the user's emotional score and extracted key points. Output is a list of prioritized tasks. A scheduling algorithm is used to rearrange the tasks.

[0558] Step 5:

[0559] The terminal presents the user with a task list with adjusted priorities. The input is the task list generated in step 4. The output is a user-readable task list display, provided as visual information through the user interface.

[0560] Step 6:

[0561] The server generates prompts using a generative AI model and suggests user-specific actions. The input is the user's emotional state and task. The output is suggested actions and customer service phrases. The generative AI model is used to create prompts and provide the user with guidance on how to respond.

[0562] Step 7:

[0563] The user adjusts their actions based on the presented tasks and suggestions. The inputs are the task list in step 5 and the suggestions in step 6. The output is the user's actual actions and choices. This allows the user to take the optimal action according to their emotional state.

[0564] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0565] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0566] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0567] [Fourth Embodiment]

[0568] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0569] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0570] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0571] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0572] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0573] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0574] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0575] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0576] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0577] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0578] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0579] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0580] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0581] This invention provides a system and method for efficiently managing information obtained by users through multiple communication means and preventing the omission of important information. This system acquires, analyzes, manages, and notifies information through the interaction of a server, terminals, and users.

[0582] First, the server automatically retrieves new messages from users using multiple communication methods, such as email servers and chat application APIs. At this stage, the server stores the collected unprocessed messages for later analysis.

[0583] Next, the server uses natural language processing techniques to analyze the retrieved messages. In this analysis process, the server extracts information related to important items, such as "deadline" and "request details." This helps identify which messages should be prioritized for the user.

[0584] Next, the server automatically incorporates the extracted information into the user's schedule. Considering existing appointments and meeting dates, the server can perform optimal schedule adjustments. For tasks added to the schedule, the server sets deadlines as needed.

[0585] Furthermore, the device receives integrated schedules and tasks from the server, providing users with a centralized view. This view is designed to allow users to intuitively see all tasks and related information.

[0586] Finally, the device will issue an alarm notification based on the deadline set for each task. This alarm function allows users to efficiently manage tasks without missing important deadlines.

[0587] For example, if a user misses checking an important email, the server automatically analyzes the email and adds a task to their schedule, such as "Complete preparations for next week's meeting." The device then issues an alarm about this task as the deadline approaches, supporting the user in taking proactive action.

[0588] This system aims to prevent errors caused by the sheer volume of information and maximize the efficiency of users' work.

[0589] The following describes the processing flow.

[0590] Step 1:

[0591] The server connects to multiple communication methods (e.g., mail servers and chat applications) and uses APIs to retrieve new messages from users. The server then stores these messages in a database.

[0592] Step 2:

[0593] The server sequentially sends the stored messages to the natural language processing (NLP) engine. The NLP engine analyzes the message content and extracts important information and tasks. Specifically, it identifies keywords such as "deadline" and "request" and extracts related information.

[0594] Step 3:

[0595] The server uses the analysis results obtained from the NLP engine to compare the extracted key information with the user's schedule data. The server then adds new tasks to the schedule and coordinates them with the user's meetings and other appointments.

[0596] Step 4:

[0597] The terminal receives schedule information sent from the server and displays an integrated view to the user. This view includes all tasks, due dates, and associated notes.

[0598] Step 5:

[0599] The server tracks the deadlines for scheduled tasks and sends notifications to the device as the deadline approaches. The device displays an alarm to the user, reminding them of any incomplete tasks.

[0600] Step 6:

[0601] When users need to add new tasks or modify existing ones, they send instructions to the server via their terminal. The server then updates the schedule based on these instructions, and the changes are immediately reflected in the unified view.

[0602] (Example 1)

[0603] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0604] In today's information society, users receive a massive volume of messages daily from multiple communication channels, making it difficult to properly manage important information and incorporate it into action plans in a timely manner. This often leads to overlooking important information and delays in schedules, resulting in decreased work efficiency.

[0605] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0606] In this invention, the server includes means for acquiring data via multiple communication means, means for analyzing the acquired data using natural language processing technology and extracting important items, and means for automatically incorporating the extracted important items into the user's action plan. This enables the user to efficiently manage an action plan based on the importance of the information, ensuring that important tasks are not overlooked and supporting the smooth execution of the schedule.

[0607] "Communication means" refers to technical devices or protocols for sending and receiving information, including, for example, email servers and APIs for online chat platforms.

[0608] "Data" refers to a collection of information obtained through means of communication, and includes text, audio, images, or a combination thereof.

[0609] "Natural language processing technology" is a computational technique that uses computers to analyze, understand, and manipulate human language, and is used to extract important information from text.

[0610] "Important items" refer to information that needs to be addressed as a priority in the user's work or action plan, and include deadlines and requests.

[0611] An "action plan" is a compilation of a user's schedule and appointments, and is part of a timetable used for managing daily tasks.

[0612] A "warning" is a means of notifying a user that a deadline for an appointment or task is approaching, and can be provided visually or audibly.

[0613] This invention is a system that efficiently manages information and supports users' work. The server acquires information using multiple communication methods. This includes APIs for mail servers and chat applications, specifically automatically acquiring emails and messages through the APIs. For example, messages are acquired from mail servers using the IMAP protocol, and data is retrieved from chat applications using the provided APIs.

[0614] The server then analyzes the acquired data using natural language processing techniques. This analysis utilizes text analysis libraries, such as Python's NLTK or SpaCy. The purpose of the analysis is to identify important items within the text, such as "deadline" and "request details." These extracted items are then automatically incorporated into the user's action plan by the server.

[0615] Next, the device receives an integrated action plan provided by the server. This plan is displayed on a screen that centrally displays the user's planned activities. The device's alert function notifies the user when the deadline for each task is approaching, preventing them from missing tasks.

[0616] For example, if a user misses an important email, the server automatically analyzes the email and incorporates a task such as "complete preparations for next week's meeting" into the user's action plan. This allows the device to notify the user of an alarm as the deadline approaches, enabling the user to take action in advance.

[0617] A possible prompt for explaining this system using a generative AI model might be: "Please explain the function that incorporates important email tasks for next week's meeting into the action plan and notifies users before the deadline."

[0618] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0619] Step 1:

[0620] The server retrieves user messages through multiple communication methods. It uses APIs from mail servers and chat applications as input. The server retrieves unread and new messages from these APIs and stores them in a database. Specifically, this involves accessing mail servers using the IMAP protocol to collect unprocessed messages. The retrieved message data is provided as output.

[0621] Step 2:

[0622] The server analyzes the acquired message data using natural language processing techniques. The message data saved in step 1 is used as input. The server uses a text analysis library (e.g., Python's NLTK or SpaCy) to extract important items from the message, such as "deadline" and "request details." Specifically, it may identify keywords within the text and extract related information. The output is a list of the extracted important items.

[0623] Step 3:

[0624] The server automatically incorporates the extracted key items into the user's action plan. It uses the list of key items obtained in step 2 as input. The server manipulates scheduling software (e.g., Google Calendar or Outlook) to add these items as new tasks to the action plan. Specifically, it places tasks at the optimal time, taking existing appointments into consideration. An updated action plan is generated as output.

[0625] Step 4:

[0626] The terminal receives the integrated action plan from the server and displays it to the user. It takes the updated action plan from step 3 as input. The terminal visually displays this on a dedicated application, providing it to the user in an easy-to-understand manner. Specifically, it uses calendar and list view functions to allow the user to grasp the status of all tasks. The output is a visualized action plan.

[0627] Step 5:

[0628] The device alerts users to tasks with approaching deadlines based on the action plan. It uses the updated action plan received in step 4 as input. The device utilizes a notification function linked to task deadlines to alert the user. Specifically, it displays a pop-up or audio notification at a specified time to inform the user of the task deadline. An alert notification is generated for the user as output.

[0629] (Application Example 1)

[0630] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0631] In modern times, many users access various information and conduct transactions through multiple communication methods and platforms. However, efficiently managing important information and payment deadlines obtained from these diverse sources and taking appropriate action based on that information is difficult. As a result, important tasks may be overlooked or payment deadlines missed, disrupting users' lives and work. This invention aims to solve such problems and realize efficient information management and notification.

[0632] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0633] In this invention, the server includes means for acquiring data via multiple communication means, means for analyzing the acquired data using natural language processing technology and extracting important information, and means for automatically incorporating the extracted important information into the user's activity plan. This enables the user to properly manage important information and payment deadlines obtained from various sources and to receive necessary notifications.

[0634] "Communication methods" refer to methods and systems for transmitting information, and include email and chat applications.

[0635] "Data" refers to various types of information obtained through communication methods, including textual and numerical information.

[0636] "Natural language processing technology" is a technology that enables computers to understand and process human language, and is used for text analysis and extraction of important information.

[0637] "Important matters" refer to information that should be prioritized and managed in the user's activities, and include, for example, payment deadlines and necessary tasks.

[0638] An "activity plan" is a schedule that lists the tasks and appointments that users need to complete, and it is managed in a simplified manner.

[0639] "Deadline-based notifications" is a function that issues warnings or reminders to users based on a set date or time.

[0640] "Transaction information" refers to data related to buying, selling, and payments conducted across multiple platforms.

[0641] "Payment deadline" refers to the final date and time by which payment for a particular transaction or invoice must be completed.

[0642] "Priority" is a criterion for determining the order in which multiple tasks or pieces of information should be addressed.

[0643] The system that implements this application example is configured as follows:

[0644] First, the server obtains data from users using multiple communication methods, such as email and chat application APIs. This data includes transaction information and important schedule information. The server then analyzes this obtained data using natural language processing techniques, such as NLTK and spaCy, to extract important information. This important information includes payment deadlines and important tasks.

[0645] Next, the server uses a database management system (e.g., SQLite) to manage the extracted important information so that it can be automatically incorporated into the user's activity plan. It can be linked with the user's existing schedule and deadlines can be set for each important item.

[0646] The user's device receives integrated activity plan information and displays it centrally through a smartphone application. This display allows users to intuitively check their schedule and prioritize important tasks.

[0647] Furthermore, the device notifies users based on the deadlines for each task, issuing timely warnings. This helps prevent payment delays and ensures that important tasks are completed without fail.

[0648] For example, if a user enters the prompt "When is my next credit card payment due?" into a smartphone app, the app analyzes the deadline from the user's schedule and displays a notification on the screen saying, "Your credit card payment due is this Friday."

[0649] This system allows users to efficiently manage data from various sources and perform their daily tasks without missing important deadlines.

[0650] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0651] Step 1:

[0652] The server automatically retrieves new user data using APIs from email servers and chat applications. This process extracts text data from the APIs and stores it on the server. The input is the data request from the API, and the output is the stored, unprocessed text data.

[0653] Step 2:

[0654] The server applies natural language processing techniques to the acquired text data. For example, it uses NLTK or spaCy to analyze the text and extract important information. The input is raw text data, and the output is a list of important items. This list includes things like payment deadlines and important tasks.

[0655] Step 3:

[0656] The server stores the extracted important information in a database and incorporates it into the user's activity plan. Specifically, it inserts the data into SQLite, adjusts it with the existing schedule, and sets deadlines for the important items. The input is a list of important items, and the output is the updated schedule.

[0657] Step 4:

[0658] The terminal displays integrated activity plan information received from the server in a user interface. This display allows users to intuitively check their schedules and tasks. The input is the updated schedule, and the output is a visually represented activity plan.

[0659] Step 5:

[0660] The terminal sends notifications based on the deadlines of each task. The program uses an internal clock to monitor deadlines and issues alerts at specific times. The input is a list of tasks with deadlines, and the output is a notification to the user.

[0661] Step 6:

[0662] The user enters prompt messages via a terminal to request specific information. The system analyzes the data based on the queries sent to the server and provides the user with appropriate information. The input is the user's prompt message, and the output is the information provided based on that message.

[0663] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0664] This invention provides a system that takes into account the emotional state of the user and enables more personalized processing of important information and task management. This system achieves highly accurate schedule management through interaction between the server, terminal, and user.

[0665] First, the server acquires information through multiple communication methods, including messaging services such as email and chat. The acquired information is stored in a database in preparation for subsequent processing.

[0666] The server uses natural language processing technology to analyze these messages. The analysis process extracts important information and identifies it as a priority task. At this stage, the server uses an emotion engine to recognize the user's emotional state. This emotional state is inferred from factors such as the words the user uses and their typing speed.

[0667] After the emotion engine understands the user's emotional state, the server dynamically adjusts the priority of the acquired tasks based on this information. For example, if the user is feeling stressed, the server will take measures such as reducing urgent tasks.

[0668] The device receives information from the server and provides the user with a centralized interface. Through this interface, the user can view all schedules and access tasks prioritized according to their emotional state. The device can change the timing and expression of notifications based on the results of the emotion engine, and is designed to reduce the user's workload.

[0669] For example, if the emotion engine detects that a user is experiencing high levels of stress, the server will send a meeting reminder slightly earlier, allowing the user to prepare with ample time. In this way, work is facilitated according to the user's emotions, providing an environment where users can work more comfortably.

[0670] Through this system, users can manage tasks optimally according to their emotional state, improving their daily work efficiency. Thus, this invention is not merely an information management system, but is innovative in that it enables flexible responses tailored to the user's situation.

[0671] The following describes the processing flow.

[0672] Step 1:

[0673] The server retrieves new user messages in real time through APIs for multiple communication methods (email, chat, messaging services, etc.). To centrally manage these messages, the server stores them in a dedicated database.

[0674] Step 2:

[0675] The server analyzes stored messages using natural language processing techniques. During the analysis process, the server detects keywords in the messages (e.g., "urgent," "deadline") and extracts important information and tasks. This extracted information is later used for prioritizing.

[0676] Step 3:

[0677] The server uses an emotion engine to assess the user's current emotional state. This assessment is performed by analyzing factors such as the user's input speed, selected words, and device operation history. This emotional state data is essential for task management in the next step.

[0678] Step 4:

[0679] The server dynamically adjusts the priority of tasks extracted based on the user's emotional state. For example, if a user is stressed, the schedule is reorganized to prioritize less demanding tasks. This adjustment reduces the user's psychological burden.

[0680] Step 5:

[0681] The terminal receives integrated task information and a summary of the user's emotional state from the server, providing a schedule view optimized for the user. This view organizes tasks by priority and is designed to make it easy for the user to access the information they need most.

[0682] Step 6:

[0683] The device sends notifications to the user at times that correspond to their emotional state. For example, it provides flexible responses, such as sending notifications later than usual when the user is relaxed, and setting alerts in advance when the user is stressed.

[0684] Step 7:

[0685] Users can check tasks, add new tasks, and modify existing tasks via their devices. If changes are made, the server immediately updates the schedule and reflects them in the next task priority settings. However, the changes are always made while taking the user's emotional state into consideration.

[0686] (Example 2)

[0687] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0688] In today's digital environment, users are often overwhelmed by the sheer volume of data from diverse information sources, making it difficult to properly prioritize important tasks. Furthermore, because each user has different emotional states, standard methods struggle to provide flexible responses tailored to individual needs.

[0689] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0690] In this invention, the server includes means for collecting information via multiple digital communication means, means for analyzing and extracting important data using natural language processing technology, and means for estimating the user's emotional state. This enables flexible task management and information processing in accordance with the user's emotional state.

[0691] "Digital communication methods" refer to technical means for exchanging information via computer networks, such as email and chat services.

[0692] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate natural human language, and has the ability to extract meaning and important information from text data.

[0693] "Important data" refers to information that has a high priority regarding the user's task management and timetable, and that requires prompt attention.

[0694] A "timetable" is a digital organizational method in the form of a list or calendar used to organize and manage users' schedules and tasks.

[0695] "Emotional state" refers to the user's mental state or mood, and is a factor that influences daily work and task management.

[0696] "Dynamic configuration" refers to the act of making settings and adjustments in real time according to the situation and conditions, and means taking flexible measures that respond to change rather than following a fixed procedure.

[0697] This invention is a system that takes into account the emotional state of the user and enables more personalized processing of important information and task management. This system achieves highly accurate schedule management through interaction between the server, terminal, and user.

[0698] The server collects information using multiple digital communication methods, such as email and chat services. This information is stored in databases such as MongoDB and MySQL. The server uses software such as spaCy and TensorFlow to analyze the collected information using natural language processing techniques and extract important data.

[0699] Furthermore, the server uses an emotion engine to estimate the user's emotional state. This emotion recognition is achieved by analyzing the results of natural language processing and the tone and input speed of the words the user uses. Based on the emotion engine, the server dynamically sets task priorities and takes measures to reduce the user's psychological burden.

[0700] The device provides the user with an intuitive and unified interface based on information from the server. Through this interface, the user can view all schedules and access priority tasks. For example, the device can adjust the timing and wording of notifications to present information in a way that minimizes user stress.

[0701] For example, if the emotion engine detects that a user is experiencing high levels of stress, the server will notify the user of high-priority tasks earlier than usual, allowing the user to respond with ample time. In this way, by adjusting tasks according to emotions, the system provides an environment where users can perform their work more efficiently.

[0702] As an example of inputting a prompt into the generating AI model, you can use a sentence like, "Suggest effective ways for a user experiencing stress to manage their tasks." This allows the AI ​​model to generate the optimal task management method tailored to the user's emotional state.

[0703] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0704] Step 1:

[0705] The server collects information using multiple digital communication methods, such as email and chat. The input at this stage is raw digital data obtained from multiple messaging platforms. The server structures this data and stores it in a database. For example, it retrieves email bodies from mail servers and messages from chat APIs and registers them in the database in text format.

[0706] Step 2:

[0707] The server analyzes the collected data using natural language processing (NLP) techniques. This process uses text data stored in a database as input. The server extracts important keywords and phrases from the text using NLP tools such as spaCy and TensorFlow, and outputs a list of potential tasks. Specifically, it identifies meeting dates and tasks with deadlines.

[0708] Step 3:

[0709] The server uses an emotion engine to estimate the user's emotional state. The input for this step includes the results of NLP analysis, as well as data on the user's past input patterns and language usage characteristics. The data processing performed by the server combines this information to generate an emotion score that recognizes the user's current psychological state. For example, if a hastily typed message contains many negative words, the server might determine that the user is stressed.

[0710] Step 4:

[0711] The server dynamically adjusts task priorities based on estimated emotional states. This step takes emotional scores and a list of potential tasks as input, re-evaluates the importance and urgency of tasks, and generates an adjusted task list as output. Specifically, for users experiencing stress, it reduces the number of tasks with approaching deadlines and provides advance notice of long-term plans.

[0712] Step 5:

[0713] The terminal provides information to the user through an optimized interface, based on the adjusted task information received from the server. The input is task list data from the server, and the output is the task management interface displayed on the user's screen. The terminal adjusts the timing and content of notifications and performs specific actions to visually communicate the details and priority of each task to the user.

[0714] Step 6:

[0715] Users check their schedules and manage tasks using an interface displayed on their terminal. Input is the information displayed on the terminal, and user actions and feedback are returned to the server as output. Specifically, data is reflected in the system when users complete tasks or change their schedules. Based on this feedback, the server re-evaluates the tasks in the next step.

[0716] (Application Example 2)

[0717] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0718] Modern information processing systems do not take into account users' emotional states when managing schedules or prioritizing tasks, making it difficult to respond flexibly to individual user needs. Furthermore, retail stores face the challenge of being unable to provide customer service that aligns with customers' emotions, thus failing to achieve sufficient customer satisfaction.

[0719] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0720] In this invention, the server includes means for acquiring information via multiple communication means, means for analyzing the user's emotional state and dynamically adjusting notifications based on that emotional state, and means for supporting customer service tailored to the user based on the acquired information. This enables personalized schedule management according to the user's emotional state, thereby improving the quality of customer service in physical stores.

[0721] "Communication methods" refer to various methods and technical means used to obtain information, including email and chat services.

[0722] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and process human language, and includes the process of extracting meaning from text.

[0723] "Important matters" refer to items from the acquired information that should be processed with particular priority, and are information that is judged to have high priority in the user's schedule management.

[0724] "Emotional state" refers to the psychological or emotional state exhibited by the user, and is the mental state inferred from their language and behavior.

[0725] "Dynamic adjustment" refers to making changes flexibly according to the situation, rather than being fixed, and specifically includes operations that change notifications and other information according to the individual user's status.

[0726] "Supporting customer service" means providing support to help stores handle customer interactions more effectively, including providing service tailored to the customer's emotional state.

[0727] This invention relates to a system that processes information while considering the emotional state of the user to perform task management and customer service support. The system consists of a server, terminals, and users. The server acquires information through multiple communication methods. It is responsible for collecting data from information sources such as email and messaging services. This information is analyzed using natural language processing technology to extract important information. The analysis utilizes the Vision API and Speech-to-Text technology provided by Google Cloud.

[0728] The server uses an emotion engine to recognize the user's emotional state. This emotional state is inferred from the user's speech patterns and typing speed. Task priorities are dynamically adjusted according to the user's emotions. For example, if the server detects that the user is stressed, it reduces the number of urgent tasks. Furthermore, to support customer service in physical stores, information is displayed on smart glasses. This allows store employees to understand the customer's emotional state in real time and provide appropriate service.

[0729] As a concrete example, if a user is under stress, a high-priority meeting notification can be moved forward slightly to allow them time to prepare. In retail customer service, if a customer appears happy, new products may be promoted; if they appear stressed, a quick response may be prioritized. Using a generative AI model (e.g., GPT-4), prompt statements like the following can be used.

[0730] "A customer has entered the store and is asking about special offers. They seem excited. What kind of customer service phrases should I use?"

[0731] By implementing this system, users can manage tasks optimally according to their emotional state, thereby improving customer satisfaction in physical stores.

[0732] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0733] Step 1:

[0734] The server collects information from multiple communication methods. It receives data from email and messaging services as input. It stores information in natural language format in a database as output. This step involves performing specific actions to retrieve information via the APIs of each communication method.

[0735] Step 2:

[0736] The server analyzes the acquired information using natural language processing technology. The input is the natural language data collected in step 1. The output is summarized information extracted as important points. A text analysis engine is used for the analysis, performing keyword extraction and sentiment analysis.

[0737] Step 3:

[0738] The server uses an emotion engine to recognize the user's emotional state. Input consists of user text data and information obtained from input speed. Output is a score representing the user's emotional state. Sentiment analysis is performed by combining the natural language processing results with behavioral patterns.

[0739] Step 4:

[0740] The server dynamically adjusts task priorities based on the user's emotional state. Inputs are the user's emotional score and extracted key points. Output is a list of prioritized tasks. A scheduling algorithm is used to rearrange the tasks.

[0741] Step 5:

[0742] The terminal presents the user with a task list with adjusted priorities. The input is the task list generated in step 4. The output is a user-readable task list display, provided as visual information through the user interface.

[0743] Step 6:

[0744] The server generates prompts using a generative AI model and suggests user-specific actions. The input is the user's emotional state and task. The output is suggested actions and customer service phrases. The generative AI model is used to create prompts and provide the user with guidance on how to respond.

[0745] Step 7:

[0746] The user adjusts their actions based on the presented tasks and suggestions. The inputs are the task list in step 5 and the suggestions in step 6. The output is the user's actual actions and choices. This allows the user to take the optimal action according to their emotional state.

[0747] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0748] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0749] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0750] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0751] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0752] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0753] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0754] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0755] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0756] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0757] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0758] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0759] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0760] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0761] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0762] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0763] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0764] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0765] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0766] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0767] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0768] The following is further disclosed regarding the embodiments described above.

[0769] (Claim 1)

[0770] A means of acquiring information through multiple communication methods,

[0771] A means of analyzing acquired information using natural language processing technology and extracting important information,

[0772] A method for automatically incorporating extracted important information into the user's schedule,

[0773] A means of providing timely notices for scheduled matters,

[0774] A system that includes a means of updating the schedule based on user instructions.

[0775] (Claim 2)

[0776] The system according to claim 1, further comprising means for centrally displaying information from communication means.

[0777] (Claim 3)

[0778] The system according to claim 1, further comprising means for automatically setting schedule priorities, taking into account the importance of the acquired information.

[0779] "Example 1"

[0780] (Claim 1)

[0781] A means of acquiring data via multiple communication methods,

[0782] A method for analyzing acquired data using natural language processing technology and extracting important items,

[0783] A means to automatically incorporate the extracted key items into the user's action plan,

[0784] A means of issuing timely warnings regarding the action plan,

[0785] An information processing system that includes means for updating action plans based on user instructions.

[0786] (Claim 2)

[0787] The information processing system according to claim 1, further comprising means for integrating and displaying data from communication means.

[0788] (Claim 3)

[0789] The information processing system according to claim 1, further comprising means for automatically setting priority levels for an action plan, taking into account the importance of the acquired data.

[0790] "Application Example 1"

[0791] (Claim 1)

[0792] A means of acquiring data via multiple communication methods,

[0793] A method for analyzing acquired data using natural language processing technology and extracting important information,

[0794] A means to automatically incorporate the extracted key information into the user's activity plan,

[0795] Means of providing timely notices for planned matters,

[0796] A means of managing transaction information from various platforms and analyzing and notifying payment deadlines,

[0797] A system that includes means for updating activity plans based on user instructions.

[0798] (Claim 2)

[0799] The system according to claim 1, further comprising means for centrally displaying data.

[0800] (Claim 3)

[0801] The system according to claim 1, further comprising means for automatically setting priority for activity plans, taking into account the importance of the acquired data.

[0802] "Example 2 of combining an emotion engine"

[0803] (Claim 1)

[0804] A means of collecting information via multiple digital communication methods,

[0805] A method for analyzing collected information using natural language processing technology and extracting important data,

[0806] A method for automatically incorporating extracted important data into the user's timetable,

[0807] A means of providing timely notifications based on a schedule,

[0808] A means of updating the timetable based on user instructions,

[0809] A means of estimating the emotional state of the user,

[0810] A means of dynamically setting the order of the time schedule based on emotional state,

[0811] A system that includes this.

[0812] (Claim 2)

[0813] The system according to claim 1, further comprising means for comprehensively displaying information from communication means.

[0814] (Claim 3)

[0815] The system according to claim 1, further comprising means for dynamically adjusting the ranking of a timetable, taking into account the importance and emotional state of the acquired information.

[0816] "Application example 2 when combining with an emotional engine"

[0817] (Claim 1)

[0818] A means of acquiring information through multiple communication methods,

[0819] A means of analyzing acquired information using natural language processing technology and extracting important information,

[0820] A method for automatically incorporating extracted important information into the user's schedule,

[0821] A means of providing timely notices for scheduled matters,

[0822] A means of updating the schedule based on user instructions,

[0823] A means for analyzing the user's emotional state and dynamically adjusting notifications based on that emotional state,

[0824] A means to support customer service tailored to the user based on the acquired information.

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, further comprising means for centrally displaying information from communication means and visually presenting the user's emotional state in real time.

[0828] (Claim 3)

[0829] The system according to claim 1, further comprising means for automatically setting schedule priorities and suggesting support that is appropriate to the user's emotions, taking into account the importance of the information acquired and the user's emotional state. [Explanation of Symbols]

[0830] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of acquiring information through multiple communication methods, A means of analyzing acquired information using natural language processing technology and extracting important information, A method for automatically incorporating extracted important information into the user's schedule, A means of providing timely notices for scheduled matters, A system that includes a means of updating the schedule based on user instructions.

2. The system according to claim 1, further comprising means for centrally displaying information from communication means.

3. The system according to claim 1, further comprising means for automatically setting schedule priorities, taking into account the importance of the acquired information.

Images