system

Abstract

We provide the system. [Solution] A means for receiving and analyzing communication information, A means for automatically generating work instruction information based on the aforementioned analysis results, A means for recording the generated work instruction information, Means for executing the aforementioned work instruction information, A means of notifying the completion status of the work, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the modern business environment, there are numerous communication tools and digital platforms, but task information and communication content between these tools are scattered, making integrated management difficult. As a result, problems such as information duplication and oversight occur, leading to a decline in business efficiency. Additionally, there is a significant burden of manual task management and reminders on individual platforms, which is time-consuming and labor-intensive.

Means for Solving the Problems

[0005] This invention collects message data from various communication tools by using means to receive and analyze communication data. It then provides a system for centralized task management by automatically generating and recording task information based on the analysis results. Furthermore, by combining this with means for automatic reminders based on task deadlines, it helps users avoid missing important tasks and take action at the appropriate time. Additionally, by notifying users of reminder information through a user interface, users can intuitively receive information, and by adding a task editing function based on input, flexible task management by the user is achieved.

[0006] "Communication data" refers to the content of messages and information sent and received through various means of communication.

[0007] "Analysis" refers to the process of extracting meaning from received communication data and structuring it using algorithms such as language processing.

[0008] "Task information" refers to a description of the actions and related information necessary to achieve business objectives.

[0009] "Automatically generating" refers to a system creating or setting information according to predetermined rules or programs without human intervention.

[0010] "Means of recording" refers to a system for saving generated information and maintaining it in a state where it can be referenced later.

[0011] "Means of providing reminders" refers to functions that notify users of task deadlines and due dates and draw their attention to them.

[0012] A "user interface" refers to the display screen of a system that allows users to visually confirm and interact with information.

[0013] "Notifying" refers to the action of informing users of important information or events.

[0014] "Means of making information editable" refers to features that allow users to modify or update existing information. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disk (e.g., hard disk), or magnetic tape, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention provides a system for efficiently managing communication data received from various communication devices and information platforms, and for automatically generating and reminding business tasks. This system integrates the functions of data reception, analysis, task generation, recording, reminder, and notification, and its specific operation is described below.

[0037] The server receives message data from various communication tools using APIs and webhooks. This makes it possible to centrally collect information from multiple chat services and email accounts used by the user. The received data is analyzed within the server using natural language processing technology to extract information relevant to the task. For example, from a message stating, "The Project X report needs to be submitted by next Tuesday," information such as "Report submission" and "Deadline: Next Tuesday" would be extracted.

[0038] Next, the server uses this extracted information to record it as a new task in the task management database. Each task is automatically assigned attributes such as assignee, deadline, and priority.

[0039] The device displays the latest task information from the server through its user interface. This allows the user to intuitively grasp the list of tasks and check the progress of each task. The device also sends pop-up notifications and email alerts to the user based on set reminder times. In this way, it prevents the user from missing important tasks.

[0040] Users can view and edit task information displayed on their devices. For example, they can freely change task deadlines or adjust assignees. These changes are immediately reflected on the server, and the database is updated accordingly.

[0041] The system of the present invention integrates the above functions to eliminate information inconsistencies and loss that occur between communication devices, thereby improving operational efficiency. Furthermore, it reduces the operational burden of task management for users, enabling more efficient work execution.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server receives message data from various communication tools using APIs and webhooks. This process centrally aggregates data from all chat services and email accounts used by the user.

[0045] Step 2:

[0046] The server sends the received message data to the natural language processing engine. The engine analyzes the message content, recognizes and extracts keywords and phrases relevant to the task.

[0047] Step 3:

[0048] The server formats the extracted task-related information and records it as a new task entry in the task management database. At this stage, basic properties such as the assignee, deadline, and priority are also automatically set.

[0049] Step 4:

[0050] The terminal retrieves task information updated by the server and displays the latest task list in the user interface. It is designed to allow users to visually and intuitively grasp important information.

[0051] Step 5:

[0052] The device sends reminders to the user via pop-up notifications and audio alerts based on the set reminder time. These notifications are sent at the appropriate time to ensure that task deadlines are not missed.

[0053] Step 6:

[0054] Users can view task information displayed on their device and edit tasks as needed. Changes made to task properties are immediately updated in the database via the server.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] There are problems such as information being dispersed from various communication methods and not being properly centrally managed, and productivity decreasing due to manual task management. In addition, deadlines and priorities are often missed, leading to unnecessary delays and errors. There is a need to resolve these situations and achieve efficient information management and task management.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes means for receiving and integrating information from multiple communication means, means for analyzing the received information and extracting task-related information using natural language processing technology, and means for automatically generating tasks based on the extracted information and recording them in a management database. This enables efficient centralization of information and automated task management and recording.

[0060] "Multiple means of communication" refers to methods that allow data to be sent and received using different types of communication technologies and platforms, thereby enabling the aggregation of information.

[0061] "Means for receiving and integrating information" refers to the function of a server that takes in data from various communication sources and consolidates it into a format that can be centrally managed.

[0062] Natural language processing technology is a technique that analyzes text data and converts human language into a format that machines can understand. This makes it possible to accurately extract the meaning and intent of information.

[0063] "Means for extracting task-related information" refers to a function that identifies and extracts data related to specific actions or plans from the received information.

[0064] "Methods for automatically generating tasks" refer to functions that automatically initiate new tasks and activities as plans based on extracted information.

[0065] "Means of recording in a management database" refers to a function that records generated tasks and saves them in a database for later reference.

[0066] A "means of recall" refers to a function that issues notifications or alerts to draw the user's attention based on set time periods or conditions.

[0067] This invention is a system that aggregates information from communication means to efficiently manage tasks. For the system to function, the server, terminals, and users must each fulfill their respective roles.

[0068] The server uses APIs and webhooks to receive information from various communication methods. This allows data to be aggregated and centrally managed from various platforms used by users, such as chat services and email services. The received data is analyzed within the server using natural language processing technology. Possible software to be used includes Apache® OpenNLP and Google® Natural Language API. Through analysis, important task-related information is extracted from the message, such as information about deadlines and assigned personnel.

[0069] Based on the extracted information, the server automatically generates tasks and records them in the management database. Examples of databases used include MySQL® and PostgreSQL. This automatically assigns attributes such as deadlines, assignees, and priorities to each task, improving user work efficiency.

[0070] The terminal intuitively displays task information retrieved from the server to the user through its user interface. To achieve this, the terminal can use a web browser or a dedicated application. Furthermore, the terminal sends pop-up notifications and email alerts to the user based on configured reminder functions, prompting them to remember important task deadlines.

[0071] Users can view task information displayed by this system and edit it as needed. They can change task deadlines, adjust assignees, and perform other actions. These changes are reflected on the server in real time, and the management database is also updated accordingly.

[0072] For example, when a person in charge of a product development project is preparing for the next meeting, they can prompt the AI ​​model with "I want to check the preparation status for the next meeting," and the relevant tasks and schedule will be automatically generated and managed. This allows the user to further improve their work efficiency.

[0073] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0074] Step 1:

[0075] The server aggregates information from various communication platforms. Specifically, it receives messages from chat tools and email services via APIs and webhooks. The input is communication data from each tool, and the output is a unified, unanalyzed dataset. This process makes it possible to aggregate information from all the platforms used by the user in one place.

[0076] Step 2:

[0077] The server analyzes the received data using natural language processing techniques. Technologies used include Apache OpenNLP and Google's Natural Language API. The input is unanalyzed message data, and the output is a dataset containing task-related information. Through this information analysis, for example, information about the task content and deadline can be extracted from a message such as "Submit the Project X report by next Tuesday."

[0078] Step 3:

[0079] The server automatically generates tasks based on the analyzed information and saves them to the management database. The input is the analyzed task information, and the output is the recorded task database entry. At this stage, attributes such as the task assignee, deadline, and priority are set. Possible databases to use include MySQL and PostgreSQL.

[0080] Step 4:

[0081] The terminal displays task information to the user through a user interface. The input is task data received from the server, and the output is task information visually displayed to the user. This allows the user to intuitively check the content and status of each task.

[0082] Step 5:

[0083] The device notifies the user based on the set reminder time. Inputs include task deadlines and reminder settings, while outputs include pop-up notifications and email alerts. This process allows users to manage their schedules effectively without missing important deadlines.

[0084] Step 6:

[0085] Users review the displayed task information and edit it as needed. Input is the editing instructions received through the user interface, and output is the updated task information. For example, users can extend the task deadline or change the assignee, and these changes are reflected on the server in real time, updating the database.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] In modern industrial fields, particularly in automated factory environments, there is a need to efficiently manage information from diverse communication platforms while ensuring that autonomous devices such as robots reliably understand and execute instructions. However, conventional systems have shortcomings in analyzing received communication information and automating work instructions, resulting in decreased work efficiency. This invention aims to solve these information management and automation problems, enabling autonomous devices to reliably perform their tasks.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes means for receiving and analyzing communication information, means for automatically generating work instruction information based on the analysis results, and means for recording the generated work instruction information. This enables autonomous equipment in a factory to efficiently receive and execute work instructions, thereby improving work efficiency and ensuring accurate work execution.

[0091] "Communication information" is a general term for data and messages received from various information platforms and communication devices.

[0092] "Analysis" is the process of deciphering received communication information and extracting the necessary information.

[0093] "Work instruction information" refers to specific tasks and commands that autonomous devices should perform based on the analyzed information.

[0094] "Recording" refers to saving the generated work instruction information to storage such as a database.

[0095] "Execution means" refers to the hardware and software functions that carry out tasks based on received work instruction information.

[0096] "Notification" refers to the act of informing relevant parties about the progress or completion status of a task.

[0097] The system that realizes this invention consists of programs that enable autonomous factory equipment to efficiently perform its tasks. The server receives manufacturing instruction information from various communication platforms using APIs and webhooks. This information is analyzed using natural language processing technology to extract work instruction information. Natural language processing libraries such as Spacy are mainly used.

[0098] The server generates specific work instruction information based on the analyzed data and records this information in a database. At this time, attributes such as the content of the work and the deadline are set for the instruction information. Autonomous machines within the factory receive instructions from the server and execute the tasks. When a task is completed, the machine notifies the server of its completion status, and the information is communicated to the relevant parties.

[0099] As a concrete example, in a factory production line, an instruction to "assemble part A and part B" is sent to a server via email. The server analyzes the instruction and issues a command to the robot to begin assembly. The robot receives the instruction, starts the work, and notifies the server upon completion. The entire system is designed for smooth information flow and efficient work execution.

[0100] When using a generative AI model, it is possible to predict the next task instruction based on past instruction logs. Examples of prompts include: "Predict the next factory tasks required: 1. Assemble parts A and B, 2. Proceed to the testing phase by 5 p.m., 3. Number of workers required." Using this prompt, you can request the AI ​​model to predict task instructions.

[0101] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0102] Step 1:

[0103] The server receives manufacturing instruction information from the communication platform via APIs and webhooks. The input consists of raw data sent from each platform, which is captured in a stream format and stored in the communication log.

[0104] Step 2:

[0105] The server analyzes the received manufacturing instruction information using natural language processing techniques. Specifically, it uses libraries such as Spacy to extract keywords from the text and organize and list the information related to the work instructions. The input is raw text messages, and the output is structured task information.

[0106] Step 3:

[0107] The server generates work instruction information based on the analyzed data and sets attributes such as task content and deadlines. The input is the task information included in the output of step 2, and the output generates work instruction data that includes specific work instructions.

[0108] Step 4:

[0109] The server records the generated work instruction information in the database. This is done by an insertion process using SQL. The input is the work instruction data generated in step 3, and the output is a new record in the database.

[0110] Step 5:

[0111] The terminal receives work instruction information sent from the server and notifies autonomous devices of this information. The input is work instruction data from the server, and the output is commands sent to the devices. A communication protocol is used in this process.

[0112] Step 6:

[0113] Autonomous devices begin work based on received instructions and notify the server of their status upon completion. Input is the work command sent to the device, and output is the work completion notification to the server, which updates the corresponding record in the database.

[0114] Step 7:

[0115] The user inputs prompts using a generative AI model and predicts the next task instruction based on past instruction logs. In this process, the generated prompts and past data are given to the AI ​​as input, and a list of predicted tasks is received as output.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] This invention is a system that recognizes emotions through communication with the user and applies this recognition to task management and reminder functions. This system integrates an emotion engine that analyzes the user's emotions and dynamically adjusts task information based on that information. The specific operation of this system is described below.

[0118] The server receives data that can measure emotions, just like regular communication data. This data includes voice tone, message text codes, and facial expressions extracted from the user's video. The analysis engine processes this data to determine the user's emotional state (e.g., stress, relaxation, concentration). This emotional data is treated as useful metadata for processing tasks.

[0119] The terminal provides a task management utility, integrating and displaying task information and sentiment analysis results sent from the server on the user's face. The sentiment engine allows the terminal to dynamically adjust task management information based on the user's current emotional state. For example, if the system determines that the user is under significant stress, it can postpone lower-priority tasks and shift high-load tasks to lower-load ones.

[0120] Furthermore, reminder messages are customized according to the user's mood. Based on the sentiment analysis results, the server adjusts the wording and tone of the reminders to create more user-friendly communication. For example, it sends friendly messages when the user is relaxed and concise notifications when the user is focused.

[0121] Users can receive emotion-based feedback through their devices and easily edit and manage task information to maximize their work efficiency. This provides task management and communication optimized for each individual user, thereby improving work performance. In this way, the present invention functions as a means to realize user-centered, flexible task management and provides a business support system with a more human-centered approach.

[0122] The following describes the processing flow.

[0123] Step 1:

[0124] The server receives audio, text, and video data sent from the user's various communication tools. This data includes the user's voice tone, facial expressions, and emotional expressions in the text for analysis by the emotion engine.

[0125] Step 2:

[0126] The server's emotion engine analyzes received data to determine the user's current emotional state. For example, it might detect stress from voice data or extract positive intentions from text information.

[0127] Step 3:

[0128] The server dynamically adjusts existing task information based on the results of sentiment analysis. If the user is experiencing stress, changes are made such as lowering the priority of tasks or postponing the execution of tasks that can wait.

[0129] Step 4:

[0130] The terminal receives updates from the server and displays the latest task information and emotional status on the user interface. This allows the user to immediately see optimized task information tailored to their situation.

[0131] Step 5:

[0132] The device sends customized reminder notifications to the user based on the analysis results from the emotion engine. For example, if the user is not feeling anxious, the tone of the notification will be adjusted to use a relaxed tone.

[0133] Step 6:

[0134] Users can further customize their task information using the editing functions provided on their device. This allows users to adjust tasks according to their emotional state at their own discretion.

[0135] Step 7:

[0136] The server updates the task database based on user edits, ensuring the entire system is up-to-date. This provides the necessary information to be prepared for the next task cycle.

[0137] (Example 2)

[0138] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0139] In today's information society, individuals experience various stresses and emotional fluctuations, but traditional task management systems fail to take these emotional factors into account. As a result, people suffer from unnecessary stress and inefficient scheduling, leading to a decline in the quality of management.

[0140] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0141] In this invention, the server includes means for receiving and analyzing biosignals, means for determining emotional states, and means for dynamically adjusting task information. This enables flexible task management and customized reminders tailored to individual emotional states.

[0142] "Communication" is a general term for the technologies and processes used to send and receive data and information.

[0143] "Biosignals" refer to signals emitted from the human body, including data that indicates emotions and states such as voice and facial expressions.

[0144] "Analysis" is the process of breaking down data into smaller parts to verify and understand it, and extracting specific information.

[0145] "Emotional state" refers to a person's psychological and physiological state, including stress, relaxation, and concentration.

[0146] "Task information" refers to detailed data about the tasks and schedules that a user needs to complete.

[0147] "Dynamic adjustment" refers to the process of changing content and settings in real time in response to changes in circumstances and conditions.

[0148] A "reminder" is the act of sending a notification to a user to remind them of specific information or a task.

[0149] A "user interface" refers to the structure of screens and control systems that allow a user to interact with a system.

[0150] "Editable" means that existing information can be changed, deleted, or added.

[0151] This system provides task management and reminder functions based on the user's emotions. A specific implementation is shown below.

[0152] The server receives biometric signals such as voice, text, and video collected from the user. This is done using speech recognition software and image analysis tools (e.g., common speech recognition APIs, image processing libraries). The server preprocesses this data and uses an analysis engine (e.g., a natural language processing (NLP) engine) to determine the user's emotional state. Based on the analysis results, the server generates metadata indicating the emotional state.

[0153] The device dynamically adjusts task information according to the user's emotional state. Specifically, it utilizes task management software (e.g., a typical task management application) to change the schedule and tasks displayed to the user. For example, if the user is under high stress, it will postpone low-priority tasks and recommend lighter tasks that match their relaxed state.

[0154] Furthermore, the reminder function is also customized according to the user's emotional state. The server adjusts the tone and content of reminder messages and sends notifications through the engagement software. Friendly messages are sent when the user is relaxed, and concise messages when the user is focused.

[0155] Furthermore, users can freely edit task information and adjust settings through their devices. This allows users to enjoy task management optimized for their emotional state, enabling them to perform their daily tasks efficiently.

[0156] Examples of prompt messages include the following:

[0157] "Generate task management suggestions suitable for when the user's emotional state is stressed."

[0158] "Please design a reminder message for users who are in a relaxed state."

[0159] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0160] Step 1:

[0161] The server receives input data from the user, such as voice, text, and video. This data is then used to collect biometric signals. Specifically, a speech recognition API is used to convert voice to text, and an image processing library is used to extract facial expression data from video. This data is output as pre-processed input data for determining the user's emotions.

[0162] Step 2:

[0163] The server passes pre-processed data as input to the analysis engine, which then calculates the user's emotional state. The analysis engine determines the emotional label from voice tone, text sentiment analysis, and video facial recognition. As a result, metadata indicating the emotional state is output.

[0164] Step 3:

[0165] The device receives emotion metadata sent from the server and inputs it into the task management application. Based on this information, task information is dynamically adjusted. Specifically, high-priority tasks are displayed first, and if the user is stressed, for example, high-load tasks are postponed. The adjusted task information is output and displayed on the user interface.

[0166] Step 4:

[0167] The server generates reminder messages based on the user's emotional state. These messages are customized according to the emotional state and sent to the user through the engagement platform. For example, a relaxed user will receive a message in a friendly tone, while a focused user will receive concise instructions.

[0168] Step 5:

[0169] Users can view task information and reminder messages provided through their device and make modifications as needed. User changes are saved in real time on the device and referenced during subsequent feedback. This feedback loop improves user productivity.

[0170] (Application Example 2)

[0171] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0172] Traditional task management systems process tasks uniformly without considering the user's emotional state, making it difficult to maximize work efficiency while reducing the psychological burden on users. In particular, there was a need for dynamic prioritization based on emotional information such as stress levels and concentration, as well as optimization of reminder notifications.

[0173] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0174] In this invention, the server includes means for receiving and analyzing communication information, means for automatically generating business information based on the analysis results, means for recording the generated business information, means for dynamically adjusting the business content based on changes in circumstances, means for optimizing the business information based on the user's emotional state, and means for providing notifications based on the deadline for the business. This enables the dynamic optimization of business content in accordance with the user's emotional state, reducing psychological burden and allowing for efficient business execution.

[0175] "Communication information" refers to data and signals transmitted and received via a network, and is the subject of analysis.

[0176] "Analysis results" refer to judgments based on extracted information and data obtained by processing communication information.

[0177] "Business information" refers to information about the user's work and processes, generated based on the analysis results.

[0178] "Recording" means saving generated business information so that it can be searched and referenced later.

[0179] "Changes in circumstances" refers to temporal or conditional changes in the work environment or the user's condition.

[0180] "Dynamic adjustment" means changing the content and priorities of tasks in real time in response to changing circumstances.

[0181] "Emotional state" refers to information that indicates the user's psychological or physiological state, such as stress, concentration, or relaxation.

[0182] "Optimization" means adjusting business information under given conditions to make it efficient and effective.

[0183] "Providing notifications" refers to the act of conveying necessary information to users based on set conditions and timeframes.

[0184] This invention is a system for optimizing business information according to the user's emotional state. The server first receives communication information via the network. This includes voice tone, text messages, and facial expression data from images acquired through a camera. By analyzing this information, the server determines the user's emotional state.

[0185] The server dynamically generates business information based on analysis results and adjusts the business content according to changes in the situation. This business information is transmitted to the terminal and notified through the user interface. The terminal uses a digital display and speaker to notify the user. It can also accept user input and edit the business information.

[0186] The hardware used includes home robots and smartphones, while the software utilizes natural language processing engines and facial recognition algorithms. Specifically, Google Cloud Speech-to-Text is used as the speech analysis engine, and OpenCV is used for facial recognition.

[0187] As a concrete example, a home robot checks the user's voice commands and facial expressions, and if it detects a stressed state, it delays the execution of cleaning tasks and plays music instead to promote relaxation. When the robot is relaxed, it provides a gentle voice notification to encourage task completion.

[0188] Example prompt for a generative AI model: "Suggest the best notification method when the user is relaxed."

[0189] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0190] Step 1:

[0191] The server receives communication information from the user via the network. This information includes voice tone, text messages, and facial expression data from images captured via the camera. The input data is stored as an initial stage.

[0192] Step 2:

[0193] The server applies an emotion analysis engine to the received communication information. It uses Google Cloud Speech-to-Text for speech analysis and OpenCV for facial recognition to classify the user's emotional state into categories such as stress, relaxation, and concentration. Based on this, emotion state data is output as the analysis result.

[0194] Step 3:

[0195] The server dynamically generates business information based on the analysis results. The priority and deadlines of this business information are adjusted according to the user's emotional state. For example, tasks are changed to low-load tasks during a stressful state. After data processing, the adjusted task information is output.

[0196] Step 4:

[0197] The server records the generated business information. This information is stored in a database for later reference and editing. The input in this step is the adjusted business information, and the output is the recorded business information.

[0198] Step 5:

[0199] The terminal notifies the user of business information transmitted from the server through a user interface. Notifications are displayed on a digital screen, and voice guidance is also available. Input is business information from the server, and output is visual and auditory notification to the user.

[0200] Step 6:

[0201] Users view and edit business information via a terminal. Input consists of editing commands based on user instructions, and output is the updated business information. The information is then sent back to the server for further processing based on the user's actions.

[0202] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0203] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0204] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0205] [Second Embodiment]

[0206] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0207] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0208] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0209] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0210] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0211] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0212] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0213] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0214] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0215] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0216] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0217] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0218] This invention provides a system for efficiently managing communication data received from various communication devices and information platforms, and for automatically generating and reminding business tasks. This system integrates the functions of data reception, analysis, task generation, recording, reminder, and notification, and its specific operation is described below.

[0219] The server receives message data from various communication tools using APIs and webhooks. This makes it possible to centrally collect information from multiple chat services and email accounts used by the user. The received data is analyzed within the server using natural language processing technology to extract information relevant to the task. For example, from a message stating, "The Project X report needs to be submitted by next Tuesday," information such as "Report submission" and "Deadline: Next Tuesday" would be extracted.

[0220] Next, the server uses this extracted information to record it as a new task in the task management database. Each task is automatically assigned attributes such as assignee, deadline, and priority.

[0221] The device displays the latest task information from the server through its user interface. This allows the user to intuitively grasp the list of tasks and check the progress of each task. The device also sends pop-up notifications and email alerts to the user based on set reminder times. In this way, it prevents the user from missing important tasks.

[0222] Users can view and edit task information displayed on their devices. For example, they can freely change task deadlines or adjust assignees. These changes are immediately reflected on the server, and the database is updated accordingly.

[0223] The system of the present invention integrates the above functions to eliminate information inconsistencies and loss that occur between communication devices, thereby improving operational efficiency. Furthermore, it reduces the operational burden of task management for users, enabling more efficient work execution.

[0224] The following describes the processing flow.

[0225] Step 1:

[0226] The server receives message data from various communication tools using APIs and webhooks. This process centrally aggregates data from all chat services and email accounts used by the user.

[0227] Step 2:

[0228] The server sends the received message data to the natural language processing engine. The engine analyzes the message content, recognizes and extracts keywords and phrases relevant to the task.

[0229] Step 3:

[0230] The server formats the extracted task-related information and records it as a new task entry in the task management database. At this stage, basic properties such as the assignee, deadline, and priority are also automatically set.

[0231] Step 4:

[0232] The terminal retrieves task information updated by the server and displays the latest task list in the user interface. It is designed to allow users to visually and intuitively grasp important information.

[0233] Step 5:

[0234] The device sends reminders to the user via pop-up notifications and audio alerts based on the set reminder time. These notifications are sent at the appropriate time to ensure that task deadlines are not missed.

[0235] Step 6:

[0236] Users can view task information displayed on their device and edit tasks as needed. Changes made to task properties are immediately updated in the database via the server.

[0237] (Example 1)

[0238] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0239] There are problems such as information being dispersed from various communication methods and not being properly centrally managed, and productivity decreasing due to manual task management. In addition, deadlines and priorities are often missed, leading to unnecessary delays and errors. There is a need to resolve these situations and achieve efficient information management and task management.

[0240] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0241] In this invention, the server includes means for receiving and integrating information from multiple communication means, means for analyzing the received information and extracting task-related information using natural language processing technology, and means for automatically generating tasks based on the extracted information and recording them in a management database. This enables efficient centralization of information and automated task management and recording.

[0242] "Multiple means of communication" refers to methods that allow data to be sent and received using different types of communication technologies and platforms, thereby enabling the aggregation of information.

[0243] "Means for receiving and integrating information" refers to the function of a server that takes in data from various communication sources and consolidates it into a format that can be centrally managed.

[0244] Natural language processing technology is a technique that analyzes text data and converts human language into a format that machines can understand. This makes it possible to accurately extract the meaning and intent of information.

[0245] "Means for extracting task-related information" refers to a function that identifies and extracts data related to specific actions or plans from the received information.

[0246] "Methods for automatically generating tasks" refer to functions that automatically initiate new tasks and activities as plans based on extracted information.

[0247] "Means of recording in a management database" refers to a function that records generated tasks and saves them in a database for later reference.

[0248] A "means of recall" refers to a function that issues notifications or alerts to draw the user's attention based on set time periods or conditions.

[0249] This invention is a system that aggregates information from communication means to efficiently manage tasks. For the system to function, the server, terminals, and users must each fulfill their respective roles.

[0250] The server uses APIs and webhooks to receive information from various communication methods. This allows data to be aggregated and centrally managed from various platforms used by users, such as chat services and email services. The received data is analyzed within the server using natural language processing technology. Possible software used includes Apache OpenNLP and Google's Natural Language API. Through analysis, important task-related information is extracted from the message, such as deadlines and assigned personnel.

[0251] Based on the extracted information, the server automatically generates tasks and records them in the management database. Examples of databases used include MySQL and PostgreSQL. This automatically assigns attributes such as deadlines, assignees, and priorities to each task, improving user work efficiency.

[0252] The terminal intuitively displays task information retrieved from the server to the user through its user interface. To achieve this, the terminal can use a web browser or a dedicated application. Furthermore, the terminal sends pop-up notifications and email alerts to the user based on configured reminder functions, prompting them to remember important task deadlines.

[0253] Users can view task information displayed by this system and edit it as needed. They can change task deadlines, adjust assignees, and perform other actions. These changes are reflected on the server in real time, and the management database is also updated accordingly.

[0254] For example, when a person in charge of a product development project is preparing for the next meeting, they can prompt the AI ​​model with "I want to check the preparation status for the next meeting," and the relevant tasks and schedule will be automatically generated and managed. This allows the user to further improve their work efficiency.

[0255] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0256] Step 1:

[0257] The server aggregates information from various communication platforms. Specifically, it receives messages from chat tools and email services via APIs and webhooks. The input is communication data from each tool, and the output is a unified, unanalyzed dataset. This process makes it possible to aggregate information from all the platforms used by the user in one place.

[0258] Step 2:

[0259] The server analyzes the received data using natural language processing techniques. Technologies used include Apache OpenNLP and Google's Natural Language API. The input is unanalyzed message data, and the output is a dataset containing task-related information. Through this information analysis, for example, information about the task content and deadline can be extracted from a message such as "Submit the Project X report by next Tuesday."

[0260] Step 3:

[0261] The server automatically generates tasks based on the analyzed information and saves them to the management database. The input is the analyzed task information, and the output is the recorded task database entry. At this stage, attributes such as the task assignee, deadline, and priority are set. Possible databases to use include MySQL and PostgreSQL.

[0262] Step 4:

[0263] The terminal displays task information to the user through a user interface. The input is task data received from the server, and the output is task information visually displayed to the user. This allows the user to intuitively check the content and status of each task.

[0264] Step 5:

[0265] The device notifies the user based on the set reminder time. Inputs include task deadlines and reminder settings, while outputs include pop-up notifications and email alerts. This process allows users to manage their schedules effectively without missing important deadlines.

[0266] Step 6:

[0267] Users review the displayed task information and edit it as needed. Input is the editing instructions received through the user interface, and output is the updated task information. For example, users can extend the task deadline or change the assignee, and these changes are reflected on the server in real time, updating the database.

[0268] (Application Example 1)

[0269] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0270] In modern industrial fields, particularly in automated factory environments, there is a need to efficiently manage information from diverse communication platforms while ensuring that autonomous devices such as robots reliably understand and execute instructions. However, conventional systems have shortcomings in analyzing received communication information and automating work instructions, resulting in decreased work efficiency. This invention aims to solve these information management and automation problems, enabling autonomous devices to reliably perform their tasks.

[0271] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0272] In this invention, the server includes means for receiving and analyzing communication information, means for automatically generating work instruction information based on the analysis results, and means for recording the generated work instruction information. This enables autonomous equipment in a factory to efficiently receive and execute work instructions, thereby improving work efficiency and ensuring accurate work execution.

[0273] "Communication information" is a general term for data and messages received from various information platforms and communication devices.

[0274] "Analysis" is the process of deciphering received communication information and extracting the necessary information.

[0275] "Work instruction information" refers to specific tasks and commands that autonomous devices should perform based on the analyzed information.

[0276] "Recording" refers to saving the generated work instruction information to storage such as a database.

[0277] "Execution means" refers to the hardware and software functions that carry out tasks based on received work instruction information.

[0278] "Notification" refers to the act of informing relevant parties about the progress or completion status of a task.

[0279] The system that realizes this invention consists of programs that enable autonomous factory equipment to efficiently perform its tasks. The server receives manufacturing instruction information from various communication platforms using APIs and webhooks. This information is analyzed using natural language processing technology to extract work instruction information. Natural language processing libraries such as Spacy are mainly used.

[0280] The server generates specific work instruction information based on the analyzed data and records this information in a database. At this time, attributes such as the content of the work and the deadline are set for the instruction information. Autonomous machines within the factory receive instructions from the server and execute the tasks. When a task is completed, the machine notifies the server of its completion status, and the information is communicated to the relevant parties.

[0281] As a concrete example, in a factory production line, an instruction to "assemble part A and part B" is sent to a server via email. The server analyzes the instruction and issues a command to the robot to begin assembly. The robot receives the instruction, starts the work, and notifies the server upon completion. The entire system is designed for smooth information flow and efficient work execution.

[0282] When using a generative AI model, it is possible to predict the next task instruction based on past instruction logs. Examples of prompt sentences include "Please predict the next necessary factory tasks: 1. Assembly of part A and part B, 2. To the testing process by 5 PM, 3. The number of required workers", etc. Using this prompt, it is possible to request the AI model to predict task instructions.

[0283] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0284] Step 1:

[0285] The server receives manufacturing instruction information from the communication platform through APIs or webhooks. As inputs, there is raw data sent from each platform, which is captured in stream format and saved in the communication log.

[0286] Step 2:

[0287] The server analyzes the received manufacturing instruction information using natural language processing technology. Specifically, libraries such as Spacy are used to extract keywords from the text and organize and list the information related to the business instruction. The input is the raw text message, and the output is the structured task information.

[0288] Step 3:

[0289] The server generates business instruction information based on the analyzed data and sets attributes such as the content and deadline of the task. The input is the task information included in the output of Step 2, and the output generates business instruction data including specific work instructions.

[0290] Step 4:

[0291] The server records the generated work instruction information in the database. This is done by an insertion process using SQL. The input is the work instruction data generated in step 3, and the output is a new record in the database.

[0292] Step 5:

[0293] The terminal receives work instruction information sent from the server and notifies autonomous devices of this information. The input is work instruction data from the server, and the output is commands sent to the devices. A communication protocol is used in this process.

[0294] Step 6:

[0295] Autonomous devices begin work based on received instructions and notify the server of their status upon completion. Input is the work command sent to the device, and output is the work completion notification to the server, which updates the corresponding record in the database.

[0296] Step 7:

[0297] The user inputs prompts using a generative AI model and predicts the next task instruction based on past instruction logs. In this process, the generated prompts and past data are given to the AI ​​as input, and a list of predicted tasks is received as output.

[0298] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0299] This invention is a system that recognizes emotions through communication with the user and applies this recognition to task management and reminder functions. This system integrates an emotion engine that analyzes the user's emotions and dynamically adjusts task information based on that information. The specific operation of this system is described below.

[0300] The server receives data that can measure emotions, just like regular communication data. This data includes voice tone, message text codes, and facial expressions extracted from the user's video. The analysis engine processes this data to determine the user's emotional state (e.g., stress, relaxation, concentration). This emotional data is treated as useful metadata for processing tasks.

[0301] The terminal provides a task management utility, integrating and displaying task information and sentiment analysis results sent from the server on the user's face. The sentiment engine allows the terminal to dynamically adjust task management information based on the user's current emotional state. For example, if the system determines that the user is under significant stress, it can postpone lower-priority tasks and shift high-load tasks to lower-load ones.

[0302] Furthermore, reminder messages are customized according to the user's mood. Based on the sentiment analysis results, the server adjusts the wording and tone of the reminders to create more user-friendly communication. For example, it sends friendly messages when the user is relaxed and concise notifications when the user is focused.

[0303] Users can receive emotion-based feedback through their devices and easily edit and manage task information to maximize their work efficiency. This provides task management and communication optimized for each individual user, thereby improving work performance. In this way, the present invention functions as a means to realize user-centered, flexible task management and provides a business support system with a more human-centered approach.

[0304] The following describes the processing flow.

[0305] Step 1:

[0306] The server receives voice, text, and video data transmitted from various communication tools of the user. These data include the tone of the user's voice, facial expressions, and emotional expressions within the text for analysis by the emotion engine.

[0307] Step 2:

[0308] The emotion engine of the server analyzes the received data to determine the user's current emotional state. For example, it performs processes such as detecting stress from voice data or extracting positive intentions from text information.

[0309] Step 3:

[0310] Based on the results of the emotion analysis, the server dynamically adjusts the existing task information. When the user is feeling stressed, changes such as lowering the priority of tasks or postponing the execution of tasks that can be put on hold are made.

[0311] Step 4:

[0312] The terminal receives the updated information from the server and displays the latest task information and emotional state on the user interface. The user can thereby immediately check the optimized task information according to their situation.

[0313] Step 5:

[0314] The terminal sends a customized reminder notification to the user according to the analysis results by the emotion engine. For example, when the user is not feeling anxious, the tone is adjusted, such as sending a notification with a relaxed text.

[0315] Step 6:

[0316] The user can further customize their task information using the editing function provided on the terminal. Thereby, task adjustments according to the emotional state are made at the discretion of the user themselves.

[0317] Step 7:

[0318] The server updates the task database based on user edits, ensuring the entire system is up-to-date. This provides the necessary information to be prepared for the next task cycle.

[0319] (Example 2)

[0320] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0321] In today's information society, individuals experience various stresses and emotional fluctuations, but traditional task management systems fail to take these emotional factors into account. As a result, people suffer from unnecessary stress and inefficient scheduling, leading to a decline in the quality of management.

[0322] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0323] In this invention, the server includes means for receiving and analyzing biosignals, means for determining emotional states, and means for dynamically adjusting task information. This enables flexible task management and customized reminders tailored to individual emotional states.

[0324] "Communication" is a general term for the technologies and processes used to send and receive data and information.

[0325] "Biosignals" refer to signals emitted from the human body, including data that indicates emotions and states such as voice and facial expressions.

[0326] "Analysis" is the process of breaking down data into smaller parts to verify and understand it, and extracting specific information.

[0327] "Emotional state" refers to a person's psychological and physiological state, including stress, relaxation, and concentration.

[0328] "Task information" refers to detailed data about the tasks and schedules that a user needs to complete.

[0329] "Dynamic adjustment" refers to the process of changing content and settings in real time in response to changes in circumstances and conditions.

[0330] A "reminder" is the act of sending a notification to a user to remind them of specific information or a task.

[0331] A "user interface" refers to the structure of screens and control systems that allow a user to interact with a system.

[0332] "Editable" means that existing information can be changed, deleted, or added.

[0333] This system provides task management and reminder functions based on the user's emotions. A specific implementation is shown below.

[0334] The server receives biometric signals such as voice, text, and video collected from the user. This is done using speech recognition software and image analysis tools (e.g., common speech recognition APIs, image processing libraries). The server preprocesses this data and uses an analysis engine (e.g., a natural language processing (NLP) engine) to determine the user's emotional state. Based on the analysis results, the server generates metadata indicating the emotional state.

[0335] The device dynamically adjusts task information according to the user's emotional state. Specifically, it utilizes task management software (e.g., a typical task management application) to change the schedule and tasks displayed to the user. For example, if the user is under high stress, it will postpone low-priority tasks and recommend lighter tasks that match their relaxed state.

[0336] Furthermore, the reminder function is also customized according to the user's emotional state. The server adjusts the tone and content of reminder messages and sends notifications through the engagement software. Friendly messages are sent when the user is relaxed, and concise messages when the user is focused.

[0337] Furthermore, users can freely edit task information and adjust settings through their devices. This allows users to enjoy task management optimized for their emotional state, enabling them to perform their daily tasks efficiently.

[0338] Examples of prompt messages include the following:

[0339] "Generate task management suggestions suitable for when the user's emotional state is stressed."

[0340] "Please design a reminder message for users who are in a relaxed state."

[0341] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0342] Step 1:

[0343] The server receives input data from the user, such as voice, text, and video. This data is then used to collect biometric signals. Specifically, a speech recognition API is used to convert voice to text, and an image processing library is used to extract facial expression data from video. This data is output as pre-processed input data for determining the user's emotions.

[0344] Step 2:

[0345] The server passes pre-processed data as input to the analysis engine, which then calculates the user's emotional state. The analysis engine determines the emotional label from voice tone, text sentiment analysis, and video facial recognition. As a result, metadata indicating the emotional state is output.

[0346] Step 3:

[0347] The device receives emotion metadata sent from the server and inputs it into the task management application. Based on this information, task information is dynamically adjusted. Specifically, high-priority tasks are displayed first, and if the user is stressed, for example, high-load tasks are postponed. The adjusted task information is output and displayed on the user interface.

[0348] Step 4:

[0349] The server generates reminder messages based on the user's emotional state. These messages are customized according to the emotional state and sent to the user through the engagement platform. For example, a relaxed user will receive a message in a friendly tone, while a focused user will receive concise instructions.

[0350] Step 5:

[0351] Users can view task information and reminder messages provided through their device and make modifications as needed. User changes are saved in real time on the device and referenced during subsequent feedback. This feedback loop improves user productivity.

[0352] (Application Example 2)

[0353] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0354] Traditional task management systems process tasks uniformly without considering the user's emotional state, making it difficult to maximize work efficiency while reducing the psychological burden on users. In particular, there was a need for dynamic prioritization based on emotional information such as stress levels and concentration, as well as optimization of reminder notifications.

[0355] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0356] In this invention, the server includes means for receiving and analyzing communication information, means for automatically generating business information based on the analysis results, means for recording the generated business information, means for dynamically adjusting the business content based on changes in circumstances, means for optimizing the business information based on the user's emotional state, and means for providing notifications based on the deadline for the business. This enables the dynamic optimization of business content in accordance with the user's emotional state, reducing psychological burden and allowing for efficient business execution.

[0357] "Communication information" refers to data and signals transmitted and received via a network, and is the subject of analysis.

[0358] "Analysis results" refer to judgments based on extracted information and data obtained by processing communication information.

[0359] "Business information" refers to information about the user's work and processes, generated based on the analysis results.

[0360] "Recording" means saving generated business information so that it can be searched and referenced later.

[0361] "Changes in circumstances" refers to temporal or conditional changes in the work environment or the user's condition.

[0362] "Dynamic adjustment" means changing the content and priorities of tasks in real time in response to changing circumstances.

[0363] "Emotional state" refers to information that indicates the user's psychological or physiological state, such as stress, concentration, or relaxation.

[0364] "Optimization" means adjusting business information under given conditions to make it efficient and effective.

[0365] "Providing notifications" refers to the act of conveying necessary information to users based on set conditions and timeframes.

[0366] This invention is a system for optimizing business information according to the user's emotional state. The server first receives communication information via the network. This includes voice tone, text messages, and facial expression data from images acquired through a camera. By analyzing this information, the server determines the user's emotional state.

[0367] The server dynamically generates business information based on analysis results and adjusts the business content according to changes in the situation. This business information is transmitted to the terminal and notified through the user interface. The terminal uses a digital display and speaker to notify the user. It can also accept user input and edit the business information.

[0368] The hardware used includes home robots and smartphones, while the software utilizes natural language processing engines and facial recognition algorithms. Specifically, Google Cloud Speech-to-Text is used as the speech analysis engine, and OpenCV is used for facial recognition.

[0369] As a concrete example, a home robot checks the user's voice commands and facial expressions, and if it detects a stressed state, it delays the execution of cleaning tasks and plays music instead to promote relaxation. When the robot is relaxed, it provides a gentle voice notification to encourage task completion.

[0370] Example prompt for a generative AI model: "Suggest the best notification method when the user is relaxed."

[0371] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0372] Step 1:

[0373] The server receives communication information from the user via the network. This information includes voice tone, text messages, and facial expression data from images captured via the camera. The input data is stored as an initial stage.

[0374] Step 2:

[0375] The server applies an emotion analysis engine to the received communication information. It uses Google Cloud Speech-to-Text for speech analysis and OpenCV for facial recognition to classify the user's emotional state into categories such as stress, relaxation, and concentration. Based on this, emotion state data is output as the analysis result.

[0376] Step 3:

[0377] The server dynamically generates business information based on the analysis results. The priority and deadlines of this business information are adjusted according to the user's emotional state. For example, tasks are changed to low-load tasks during a stressful state. After data processing, the adjusted task information is output.

[0378] Step 4:

[0379] The server records the generated business information. This information is stored in a database for later reference and editing. The input in this step is the adjusted business information, and the output is the recorded business information.

[0380] Step 5:

[0381] The terminal notifies the user of business information transmitted from the server through a user interface. Notifications are displayed on a digital screen, and voice guidance is also available. Input is business information from the server, and output is visual and auditory notification to the user.

[0382] Step 6:

[0383] Users view and edit business information via a terminal. Input consists of editing commands based on user instructions, and output is the updated business information. The information is then sent back to the server for further processing based on the user's actions.

[0384] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0385] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0386] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0387] [Third Embodiment]

[0388] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0389] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0390] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0391] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0392] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0393] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0394] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0395] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0396] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0397] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0398] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0399] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0400] This invention provides a system for efficiently managing communication data received from various communication devices and information platforms, and for automatically generating and reminding business tasks. This system integrates the functions of data reception, analysis, task generation, recording, reminder, and notification, and its specific operation is described below.

[0401] The server receives message data from various communication tools using APIs and webhooks. This makes it possible to centrally collect information from multiple chat services and email accounts used by the user. The received data is analyzed within the server using natural language processing technology to extract information relevant to the task. For example, from a message stating, "The Project X report needs to be submitted by next Tuesday," information such as "Report submission" and "Deadline: Next Tuesday" would be extracted.

[0402] Next, the server uses this extracted information to record it as a new task in the task management database. Each task is automatically assigned attributes such as assignee, deadline, and priority.

[0403] The device displays the latest task information from the server through its user interface. This allows the user to intuitively grasp the list of tasks and check the progress of each task. The device also sends pop-up notifications and email alerts to the user based on set reminder times. In this way, it prevents the user from missing important tasks.

[0404] Users can view and edit task information displayed on their devices. For example, they can freely change task deadlines or adjust assignees. These changes are immediately reflected on the server, and the database is updated accordingly.

[0405] The system of the present invention integrates the above functions to eliminate information inconsistencies and loss that occur between communication devices, thereby improving operational efficiency. Furthermore, it reduces the operational burden of task management for users, enabling more efficient work execution.

[0406] The following describes the processing flow.

[0407] Step 1:

[0408] The server receives message data from various communication tools using APIs and webhooks. This process centrally aggregates data from all chat services and email accounts used by the user.

[0409] Step 2:

[0410] The server sends the received message data to the natural language processing engine. The engine analyzes the message content, recognizes and extracts keywords and phrases relevant to the task.

[0411] Step 3:

[0412] The server formats the extracted task-related information and records it as a new task entry in the task management database. At this stage, basic properties such as the assignee, deadline, and priority are also automatically set.

[0413] Step 4:

[0414] The terminal retrieves task information updated by the server and displays the latest task list in the user interface. It is designed to allow users to visually and intuitively grasp important information.

[0415] Step 5:

[0416] The device sends reminders to the user via pop-up notifications and audio alerts based on the set reminder time. These notifications are sent at the appropriate time to ensure that task deadlines are not missed.

[0417] Step 6:

[0418] Users can view task information displayed on their device and edit tasks as needed. Changes made to task properties are immediately updated in the database via the server.

[0419] (Example 1)

[0420] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0421] There are problems such as information being dispersed from various communication methods and not being properly centrally managed, and productivity decreasing due to manual task management. In addition, deadlines and priorities are often missed, leading to unnecessary delays and errors. There is a need to resolve these situations and achieve efficient information management and task management.

[0422] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0423] In this invention, the server includes means for receiving and integrating information from multiple communication means, means for analyzing the received information and extracting task-related information using natural language processing technology, and means for automatically generating tasks based on the extracted information and recording them in a management database. This enables efficient centralization of information and automated task management and recording.

[0424] "Multiple means of communication" refers to methods that allow data to be sent and received using different types of communication technologies and platforms, thereby enabling the aggregation of information.

[0425] "Means for receiving and integrating information" refers to the function of a server that takes in data from various communication sources and consolidates it into a format that can be centrally managed.

[0426] Natural language processing technology is a technique that analyzes text data and converts human language into a format that machines can understand. This makes it possible to accurately extract the meaning and intent of information.

[0427] "Means for extracting task-related information" refers to a function that identifies and extracts data related to specific actions or plans from the received information.

[0428] "Methods for automatically generating tasks" refer to functions that automatically initiate new tasks and activities as plans based on extracted information.

[0429] "Means of recording in a management database" refers to the function of recording generated tasks and saving them in a database so that they can be referenced later.

[0430] A "means of recall" refers to a function that issues notifications or alerts to draw the user's attention based on set time periods or conditions.

[0431] This invention is a system that aggregates information from communication means to efficiently manage tasks. For the system to function, the server, terminals, and users must each fulfill their respective roles.

[0432] The server uses APIs and webhooks to receive information from various communication methods. This allows data to be aggregated and centrally managed from various platforms used by users, such as chat services and email services. The received data is analyzed within the server using natural language processing technology. Possible software used includes Apache OpenNLP and Google's Natural Language API. Through analysis, important task-related information is extracted from the message, such as deadlines and assigned personnel.

[0433] Based on the extracted information, the server automatically generates tasks and records them in the management database. Examples of databases used include MySQL and PostgreSQL. This automatically assigns attributes such as deadlines, assignees, and priorities to each task, improving user work efficiency.

[0434] The terminal intuitively displays task information retrieved from the server to the user through its user interface. To achieve this, the terminal can use a web browser or a dedicated application. Furthermore, the terminal sends pop-up notifications and email alerts to the user based on configured reminder functions, prompting them to remember important task deadlines.

[0435] Users can view task information displayed by this system and edit it as needed. They can change task deadlines, adjust assignees, and perform other actions. These changes are reflected on the server in real time, and the management database is also updated accordingly.

[0436] For example, when a person in charge of a product development project is preparing for the next meeting, they can prompt the AI ​​model with "I want to check the preparation status for the next meeting," and the relevant tasks and schedule will be automatically generated and managed. This allows the user to further improve their work efficiency.

[0437] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0438] Step 1:

[0439] The server aggregates information from various communication platforms. Specifically, it receives messages from chat tools and email services via APIs and webhooks. The input is communication data from each tool, and the output is a unified, unanalyzed dataset. This process makes it possible to aggregate information from all the platforms used by the user in one place.

[0440] Step 2:

[0441] The server analyzes the received data using natural language processing techniques. Technologies used include Apache OpenNLP and Google's Natural Language API. The input is unanalyzed message data, and the output is a dataset containing task-related information. Through this information analysis, for example, information about the task content and deadline can be extracted from a message such as "Submit the Project X report by next Tuesday."

[0442] Step 3:

[0443] The server automatically generates tasks based on the analyzed information and saves them to the management database. The input is the analyzed task information, and the output is the recorded task database entry. At this stage, attributes such as the task assignee, deadline, and priority are set. Possible databases to use include MySQL and PostgreSQL.

[0444] Step 4:

[0445] The terminal displays task information to the user through a user interface. The input is task data received from the server, and the output is task information visually displayed to the user. This allows the user to intuitively check the content and status of each task.

[0446] Step 5:

[0447] The device notifies the user based on the set reminder time. Inputs include task deadlines and reminder settings, while outputs include pop-up notifications and email alerts. This process allows users to manage their schedules effectively without missing important deadlines.

[0448] Step 6:

[0449] Users review the displayed task information and edit it as needed. Input is the editing instructions received through the user interface, and output is the updated task information. For example, users can extend the task deadline or change the assignee, and these changes are reflected on the server in real time, updating the database.

[0450] (Application Example 1)

[0451] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0452] In modern industrial fields, particularly in automated factory environments, there is a need to efficiently manage information from diverse communication platforms while ensuring that autonomous devices such as robots reliably understand and execute instructions. However, conventional systems have shortcomings in analyzing received communication information and automating work instructions, resulting in decreased work efficiency. This invention aims to solve these information management and automation problems, enabling autonomous devices to reliably perform their tasks.

[0453] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0454] In this invention, the server includes means for receiving and analyzing communication information, means for automatically generating work instruction information based on the analysis results, and means for recording the generated work instruction information. This enables autonomous equipment in a factory to efficiently receive and execute work instructions, thereby improving work efficiency and ensuring accurate work execution.

[0455] "Communication information" is a general term for data and messages received from various information platforms and communication devices.

[0456] "Analysis" is the process of deciphering received communication information and extracting the necessary information.

[0457] "Work instruction information" refers to specific tasks and commands that autonomous devices should perform based on the analyzed information.

[0458] "Recording" refers to saving the generated work instruction information to storage such as a database.

[0459] "Execution means" refers to the hardware and software functions that carry out tasks based on received work instruction information.

[0460] "Notification" refers to the act of informing relevant parties about the progress or completion status of a task.

[0461] The system that realizes this invention consists of programs that enable autonomous factory equipment to efficiently perform its tasks. The server receives manufacturing instruction information from various communication platforms using APIs and webhooks. This information is analyzed using natural language processing technology to extract work instruction information. Natural language processing libraries such as Spacy are mainly used.

[0462] The server generates specific work instruction information based on the analyzed data and records this information in a database. At this time, attributes such as the content of the work and the deadline are set for the instruction information. Autonomous machines within the factory receive instructions from the server and execute the tasks. When a task is completed, the machine notifies the server of its completion status, and the information is communicated to the relevant parties.

[0463] As a concrete example, in a factory production line, an instruction to "assemble part A and part B" is sent to a server via email. The server analyzes the instruction and issues a command to the robot to begin assembly. The robot receives the instruction, starts the work, and notifies the server upon completion. The entire system is designed for smooth information flow and efficient work execution.

[0464] When using a generative AI model, it is possible to predict the next task instruction based on past instruction logs. Examples of prompts include: "Predict the next factory tasks required: 1. Assemble parts A and B, 2. Proceed to the testing phase by 5 p.m., 3. Number of workers required." Using this prompt, you can request the AI ​​model to predict task instructions.

[0465] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0466] Step 1:

[0467] The server receives manufacturing instruction information from the communication platform via APIs and webhooks. The input consists of raw data sent from each platform, which is captured in a stream format and stored in the communication log.

[0468] Step 2:

[0469] The server analyzes the received manufacturing instruction information using natural language processing techniques. Specifically, it uses libraries such as Spacy to extract keywords from the text and organize and list the information related to the work instructions. The input is raw text messages, and the output is structured task information.

[0470] Step 3:

[0471] The server generates work instruction information based on the analyzed data and sets attributes such as task content and deadlines. The input is the task information included in the output of step 2, and the output generates work instruction data that includes specific work instructions.

[0472] Step 4:

[0473] The server records the generated work instruction information in the database. This is done by an insertion process using SQL. The input is the work instruction data generated in step 3, and the output is a new record in the database.

[0474] Step 5:

[0475] The terminal receives work instruction information sent from the server and notifies autonomous devices of this information. The input is work instruction data from the server, and the output is commands sent to the devices. A communication protocol is used in this process.

[0476] Step 6:

[0477] Autonomous devices begin work based on received instructions and notify the server of their status upon completion. Input is the work command sent to the device, and output is the completion notification to the server, which updates the corresponding record in the database.

[0478] Step 7:

[0479] The user inputs prompts using a generative AI model and predicts the next task instruction based on past instruction logs. In this process, the generated prompts and past data are given to the AI ​​as input, and a list of predicted tasks is received as output.

[0480] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0481] This invention is a system that recognizes emotions through communication with the user and applies this recognition to task management and reminder functions. This system integrates an emotion engine that analyzes the user's emotions and dynamically adjusts task information based on that information. The specific operation of this system is described below.

[0482] The server receives data that can measure emotions, just like regular communication data. This data includes voice tone, message text codes, and facial expressions extracted from the user's video. The analysis engine processes this data to determine the user's emotional state (e.g., stress, relaxation, concentration). This emotional data is treated as useful metadata for processing tasks.

[0483] The terminal provides a task management utility, integrating and displaying task information and sentiment analysis results sent from the server on the user's face. The sentiment engine allows the terminal to dynamically adjust task management information based on the user's current emotional state. For example, if the system determines that the user is under significant stress, it can postpone lower-priority tasks and shift high-load tasks to lower-load ones.

[0484] Furthermore, reminder messages are customized according to the user's mood. Based on the sentiment analysis results, the server adjusts the wording and tone of the reminders to create more user-friendly communication. For example, it sends friendly messages when the user is relaxed and concise notifications when the user is focused.

[0485] Users can receive emotion-based feedback through their devices and easily edit and manage task information to maximize their work efficiency. This provides task management and communication optimized for each individual user, thereby improving work performance. In this way, the present invention functions as a means to realize user-centered, flexible task management and provides a business support system with a more human-centered approach.

[0486] The following describes the processing flow.

[0487] Step 1:

[0488] The server receives audio, text, and video data sent from the user's various communication tools. This data includes the user's voice tone, facial expressions, and emotional expressions in the text for analysis by the emotion engine.

[0489] Step 2:

[0490] The server's emotion engine analyzes received data to determine the user's current emotional state. For example, it might detect stress from voice data or extract positive intentions from text information.

[0491] Step 3:

[0492] The server dynamically adjusts existing task information based on the results of sentiment analysis. If the user is experiencing stress, changes are made such as lowering the priority of tasks or postponing the execution of tasks that can wait.

[0493] Step 4:

[0494] The terminal receives updates from the server and displays the latest task information and emotional status on the user interface. This allows the user to immediately see optimized task information tailored to their situation.

[0495] Step 5:

[0496] The device sends customized reminder notifications to the user based on the analysis results from the emotion engine. For example, if the user is not feeling anxious, the tone of the notification will be adjusted to use a relaxed tone.

[0497] Step 6:

[0498] Users can further customize their task information using the editing functions provided on their device. This allows users to adjust tasks according to their emotional state at their own discretion.

[0499] Step 7:

[0500] The server updates the task database based on user edits, ensuring the entire system is up-to-date. This provides the necessary information to be prepared for the next task cycle.

[0501] (Example 2)

[0502] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0503] In today's information society, individuals experience various stresses and emotional fluctuations, but traditional task management systems fail to take these emotional factors into account. As a result, people suffer from unnecessary stress and inefficient scheduling, leading to a decline in the quality of management.

[0504] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0505] In this invention, the server includes means for receiving and analyzing biosignals, means for determining emotional states, and means for dynamically adjusting task information. This enables flexible task management and customized reminders tailored to individual emotional states.

[0506] "Communication" is a general term for the technologies and processes used to send and receive data and information.

[0507] "Biosignals" refer to signals emitted from the human body, including data that indicates emotions and states such as voice and facial expressions.

[0508] "Analysis" is the process of breaking down data into smaller parts to verify and understand it, and extracting specific information.

[0509] "Emotional state" refers to a person's psychological and physiological state, including stress, relaxation, and concentration.

[0510] "Task information" refers to detailed data about the tasks and schedules that a user needs to complete.

[0511] "Dynamic adjustment" refers to the process of changing content and settings in real time in response to changes in circumstances and conditions.

[0512] A "reminder" is the act of sending a notification to a user to remind them of specific information or a task.

[0513] A "user interface" refers to the structure of screens and control systems that allow a user to interact with a system.

[0514] "Editable" means that existing information can be changed, deleted, or added.

[0515] This system provides task management and reminder functions based on the user's emotions. A specific implementation is shown below.

[0516] The server receives biometric signals such as voice, text, and video collected from the user. This is done using speech recognition software and image analysis tools (e.g., common speech recognition APIs, image processing libraries). The server preprocesses this data and uses an analysis engine (e.g., a natural language processing (NLP) engine) to determine the user's emotional state. Based on the analysis results, the server generates metadata indicating the emotional state.

[0517] The device dynamically adjusts task information according to the user's emotional state. Specifically, it utilizes task management software (e.g., a typical task management application) to change the schedule and tasks displayed to the user. For example, if the user is under high stress, it will postpone low-priority tasks and recommend lighter tasks that match their relaxed state.

[0518] Furthermore, the reminder function is also customized according to the user's emotional state. The server adjusts the tone and content of reminder messages and sends notifications through the engagement software. Friendly messages are sent when the user is relaxed, and concise messages when the user is focused.

[0519] Furthermore, users can freely edit task information and adjust settings through their devices. This allows users to enjoy task management optimized for their emotional state, enabling them to perform their daily tasks efficiently.

[0520] Examples of prompt messages include the following:

[0521] "Generate task management suggestions suitable for when the user's emotional state is stressed."

[0522] "Please design a reminder message for users who are in a relaxed state."

[0523] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0524] Step 1:

[0525] The server receives input data from the user, such as voice, text, and video. This data is then used to collect biometric signals. Specifically, a speech recognition API is used to convert voice to text, and an image processing library is used to extract facial expression data from video. This data is output as pre-processed input data for determining the user's emotions.

[0526] Step 2:

[0527] The server passes pre-processed data as input to the analysis engine, which then calculates the user's emotional state. The analysis engine determines the emotional label from voice tone, text sentiment analysis, and video facial recognition. As a result, metadata indicating the emotional state is output.

[0528] Step 3:

[0529] The device receives emotion metadata sent from the server and inputs it into the task management application. Based on this information, task information is dynamically adjusted. Specifically, high-priority tasks are displayed first, and if the user is stressed, for example, high-load tasks are postponed. The adjusted task information is output and displayed on the user interface.

[0530] Step 4:

[0531] The server generates reminder messages based on the user's emotional state. These messages are customized according to the emotional state and sent to the user through the engagement platform. For example, a relaxed user will receive a message in a friendly tone, while a focused user will receive concise instructions.

[0532] Step 5:

[0533] Users can view task information and reminder messages provided through their device and make modifications as needed. User changes are saved in real time on the device and referenced during subsequent feedback. This feedback loop improves user productivity.

[0534] (Application Example 2)

[0535] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0536] Traditional task management systems process tasks uniformly without considering the user's emotional state, making it difficult to maximize work efficiency while reducing the psychological burden on users. In particular, there was a need for dynamic prioritization based on emotional information such as stress levels and concentration, as well as optimization of reminder notifications.

[0537] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0538] In this invention, the server includes means for receiving and analyzing communication information, means for automatically generating business information based on the analysis results, means for recording the generated business information, means for dynamically adjusting the business content based on changes in circumstances, means for optimizing the business information based on the user's emotional state, and means for providing notifications based on the deadline for the business. This enables the dynamic optimization of business content in accordance with the user's emotional state, reducing psychological burden and allowing for efficient business execution.

[0539] "Communication information" refers to data and signals transmitted and received via a network, and is the subject of analysis.

[0540] "Analysis results" refer to judgments based on extracted information and data obtained by processing communication information.

[0541] "Business information" refers to information about the user's work and processes, generated based on the analysis results.

[0542] "Recording" means saving generated business information so that it can be searched and referenced later.

[0543] "Changes in circumstances" refers to temporal or conditional changes in the work environment or the user's condition.

[0544] "Dynamic adjustment" means changing the content and priorities of tasks in real time in response to changing circumstances.

[0545] "Emotional state" refers to information that indicates the user's psychological or physiological state, such as stress, concentration, or relaxation.

[0546] "Optimization" means adjusting business information under given conditions to make it efficient and effective.

[0547] "Providing notifications" refers to the act of conveying necessary information to users based on set conditions and timeframes.

[0548] This invention is a system for optimizing business information according to the user's emotional state. The server first receives communication information via the network. This includes voice tone, text messages, and facial expression data from images acquired through a camera. By analyzing this information, the server determines the user's emotional state.

[0549] The server dynamically generates business information based on analysis results and adjusts the business content according to changes in the situation. This business information is transmitted to the terminal and notified through the user interface. The terminal uses a digital display and speaker to notify the user. It can also accept user input and edit the business information.

[0550] The hardware used includes home robots and smartphones, while the software utilizes natural language processing engines and facial recognition algorithms. Specifically, Google Cloud Speech-to-Text is used as the speech analysis engine, and OpenCV is used for facial recognition.

[0551] As a concrete example, a home robot checks the user's voice commands and facial expressions, and if it detects a stressed state, it delays the execution of cleaning tasks and plays music instead to promote relaxation. When the robot is relaxed, it provides a gentle voice notification to encourage task completion.

[0552] Example prompt for a generative AI model: "Suggest the best notification method when the user is relaxed."

[0553] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0554] Step 1:

[0555] The server receives communication information from the user via the network. This information includes voice tone, text messages, and facial expression data from images captured via the camera. The input data is stored as an initial stage.

[0556] Step 2:

[0557] The server applies an emotion analysis engine to the received communication information. It uses Google Cloud Speech-to-Text for speech analysis and OpenCV for facial recognition to classify the user's emotional state into categories such as stress, relaxation, and concentration. Based on this, emotion state data is output as the analysis result.

[0558] Step 3:

[0559] The server dynamically generates business information based on the analysis results. The priority and deadlines of this business information are adjusted according to the user's emotional state. For example, tasks are changed to low-load tasks during a stressful state. After data processing, the adjusted task information is output.

[0560] Step 4:

[0561] The server records the generated business information. This information is stored in a database for later reference and editing. The input in this step is the adjusted business information, and the output is the recorded business information.

[0562] Step 5:

[0563] The terminal notifies the user of business information transmitted from the server through a user interface. Notifications are displayed on a digital screen, and voice guidance is also available. Input is business information from the server, and output is visual and auditory notification to the user.

[0564] Step 6:

[0565] Users view and edit business information via a terminal. Input consists of editing commands based on user instructions, and output is the updated business information. The information is then sent back to the server for further processing based on the user's actions.

[0566] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0567] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0568] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0569] [Fourth Embodiment]

[0570] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0571] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0572] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0573] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0574] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0575] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0576] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0577] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0578] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0579] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0580] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0581] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0582] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0583] This invention provides a system for efficiently managing communication data received from various communication devices and information platforms, and for automatically generating and reminding business tasks. This system integrates the functions of data reception, analysis, task generation, recording, reminder, and notification, and its specific operation is described below.

[0584] The server receives message data from various communication tools using APIs and webhooks. This makes it possible to centrally collect information from multiple chat services and email accounts used by the user. The received data is analyzed within the server using natural language processing technology to extract information relevant to the task. For example, from a message stating, "The Project X report needs to be submitted by next Tuesday," information such as "Report submission" and "Deadline: Next Tuesday" would be extracted.

[0585] Next, the server uses this extracted information to record it as a new task in the task management database. Each task is automatically assigned attributes such as assignee, deadline, and priority.

[0586] The device displays the latest task information from the server through its user interface. This allows the user to intuitively grasp the list of tasks and check the progress of each task. The device also sends pop-up notifications and email alerts to the user based on set reminder times. In this way, it prevents the user from missing important tasks.

[0587] Users can view and edit task information displayed on their devices. For example, they can freely change task deadlines or adjust assignees. These changes are immediately reflected on the server, and the database is updated accordingly.

[0588] The system of the present invention integrates the above functions to eliminate information inconsistencies and loss that occur between communication devices, thereby improving operational efficiency. Furthermore, it reduces the operational burden of task management for users, enabling more efficient work execution.

[0589] The following describes the processing flow.

[0590] Step 1:

[0591] The server receives message data from various communication tools using APIs and webhooks. This process centrally aggregates data from all chat services and email accounts used by the user.

[0592] Step 2:

[0593] The server sends the received message data to the natural language processing engine. The engine analyzes the message content, recognizes and extracts keywords and phrases relevant to the task.

[0594] Step 3:

[0595] The server formats the extracted task-related information and records it as a new task entry in the task management database. At this stage, basic properties such as the assignee, deadline, and priority are also automatically set.

[0596] Step 4:

[0597] The terminal retrieves task information updated by the server and displays the latest task list in the user interface. It is designed to allow users to visually and intuitively grasp important information.

[0598] Step 5:

[0599] The device sends reminders to the user via pop-up notifications and audio alerts based on the set reminder time. These notifications are sent at the appropriate time to ensure that task deadlines are not missed.

[0600] Step 6:

[0601] Users can view task information displayed on their device and edit tasks as needed. Changes made to task properties are immediately updated in the database via the server.

[0602] (Example 1)

[0603] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0604] There are problems such as information being dispersed from various communication methods and not being properly centrally managed, and productivity decreasing due to manual task management. In addition, deadlines and priorities are often missed, leading to unnecessary delays and errors. There is a need to resolve these situations and achieve efficient information management and task management.

[0605] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0606] In this invention, the server includes means for receiving and integrating information from multiple communication means, means for analyzing the received information and extracting task-related information using natural language processing technology, and means for automatically generating tasks based on the extracted information and recording them in a management database. This enables efficient centralization of information and automated task management and recording.

[0607] "Multiple means of communication" refers to methods that allow data to be sent and received using different types of communication technologies and platforms, thereby enabling the aggregation of information.

[0608] "Means for receiving and integrating information" refers to the function of a server that takes in data from various communication sources and consolidates it into a format that can be centrally managed.

[0609] Natural language processing technology is a technique that analyzes text data and converts human language into a format that machines can understand. This makes it possible to accurately extract the meaning and intent of information.

[0610] "Means for extracting task-related information" refers to a function that identifies and extracts data related to specific actions or plans from the received information.

[0611] "Methods for automatically generating tasks" refer to functions that automatically initiate new tasks and activities as plans based on extracted information.

[0612] "Means of recording in a management database" refers to the function of recording generated tasks and saving them in a database so that they can be referenced later.

[0613] A "means of recall" refers to a function that issues notifications or alerts to draw the user's attention based on set time periods or conditions.

[0614] This invention is a system that aggregates information from communication means to efficiently manage tasks. For the system to function, the server, terminals, and users must each fulfill their respective roles.

[0615] The server uses APIs and webhooks to receive information from various communication methods. This allows data to be aggregated and centrally managed from various platforms used by users, such as chat services and email services. The received data is analyzed within the server using natural language processing technology. Possible software used includes Apache OpenNLP and Google's Natural Language API. Through analysis, important task-related information is extracted from the message, such as deadlines and assigned personnel.

[0616] Based on the extracted information, the server automatically generates tasks and records them in the management database. Examples of databases used include MySQL and PostgreSQL. This automatically assigns attributes such as deadlines, assignees, and priorities to each task, improving user work efficiency.

[0617] The terminal intuitively displays task information retrieved from the server to the user through its user interface. To achieve this, the terminal can use a web browser or a dedicated application. Furthermore, the terminal sends pop-up notifications and email alerts to the user based on configured reminder functions, prompting them to remember important task deadlines.

[0618] Users can view task information displayed by this system and edit it as needed. They can change task deadlines, adjust assignees, and perform other actions. These changes are reflected on the server in real time, and the management database is also updated accordingly.

[0619] For example, when a person in charge of a product development project is preparing for the next meeting, they can prompt the AI ​​model with "I want to check the preparation status for the next meeting," and the relevant tasks and schedule will be automatically generated and managed. This allows the user to further improve their work efficiency.

[0620] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0621] Step 1:

[0622] The server aggregates information from various communication platforms. Specifically, it receives messages from chat tools and email services via APIs and webhooks. The input is communication data from each tool, and the output is a unified, unanalyzed dataset. This process makes it possible to aggregate information from all the platforms used by the user in one place.

[0623] Step 2:

[0624] The server analyzes the received data using natural language processing techniques. Technologies used include Apache OpenNLP and Google's Natural Language API. The input is unanalyzed message data, and the output is a dataset containing task-related information. Through this information analysis, for example, information about the task content and deadline can be extracted from a message such as "Submit the Project X report by next Tuesday."

[0625] Step 3:

[0626] The server automatically generates tasks based on the analyzed information and saves them to the management database. The input is the analyzed task information, and the output is the recorded task database entry. At this stage, attributes such as the task assignee, deadline, and priority are set. Possible databases to use include MySQL and PostgreSQL.

[0627] Step 4:

[0628] The terminal displays task information to the user through a user interface. The input is task data received from the server, and the output is task information visually displayed to the user. This allows the user to intuitively check the content and status of each task.

[0629] Step 5:

[0630] The device notifies the user based on the set reminder time. Inputs include task deadlines and reminder settings, while outputs include pop-up notifications and email alerts. This process allows users to manage their schedules effectively without missing important deadlines.

[0631] Step 6:

[0632] Users review the displayed task information and edit it as needed. Input is the editing instructions received through the user interface, and output is the updated task information. For example, users can extend the task deadline or change the assignee, and these changes are reflected on the server in real time, updating the database.

[0633] (Application Example 1)

[0634] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] In modern industrial fields, particularly in automated factory environments, there is a need to efficiently manage information from diverse communication platforms while ensuring that autonomous devices such as robots reliably understand and execute instructions. However, conventional systems have shortcomings in analyzing received communication information and automating work instructions, resulting in decreased work efficiency. This invention aims to solve these information management and automation problems, enabling autonomous devices to reliably perform their tasks.

[0636] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0637] In this invention, the server includes means for receiving and analyzing communication information, means for automatically generating work instruction information based on the analysis results, and means for recording the generated work instruction information. This enables autonomous equipment in a factory to efficiently receive and execute work instructions, thereby improving work efficiency and ensuring accurate work execution.

[0638] "Communication information" is a general term for data and messages received from various information platforms and communication devices.

[0639] "Analysis" is the process of deciphering received communication information and extracting the necessary information.

[0640] "Work instruction information" refers to specific tasks and commands that autonomous devices should perform based on the analyzed information.

[0641] "Recording" refers to saving the generated work instruction information to storage such as a database.

[0642] "Execution means" refers to the hardware and software functions that carry out tasks based on received work instruction information.

[0643] "Notification" refers to the act of informing relevant parties about the progress or completion status of a task.

[0644] The system that realizes this invention consists of programs that enable autonomous factory equipment to efficiently perform its tasks. The server receives manufacturing instruction information from various communication platforms using APIs and webhooks. This information is analyzed using natural language processing technology to extract work instruction information. Natural language processing libraries such as Spacy are mainly used.

[0645] The server generates specific work instruction information based on the analyzed data and records this information in a database. At this time, attributes such as the content of the work and the deadline are set for the instruction information. Autonomous machines within the factory receive instructions from the server and execute the tasks. When a task is completed, the machine notifies the server of its completion status, and the information is communicated to the relevant parties.

[0646] As a concrete example, in a factory production line, an instruction to "assemble part A and part B" is sent to a server via email. The server analyzes the instruction and issues a command to the robot to begin assembly. The robot receives the instruction, starts the work, and notifies the server upon completion. The entire system is designed for smooth information flow and efficient work execution.

[0647] When using a generative AI model, it is possible to predict the next task instruction based on past instruction logs. Examples of prompts include: "Predict the next factory tasks required: 1. Assemble parts A and B, 2. Proceed to the testing phase by 5 p.m., 3. Number of workers required." Using this prompt, you can request the AI ​​model to predict task instructions.

[0648] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0649] Step 1:

[0650] The server receives manufacturing instruction information from the communication platform via APIs and webhooks. The input consists of raw data sent from each platform, which is captured in a stream format and stored in the communication log.

[0651] Step 2:

[0652] The server analyzes the received manufacturing instruction information using natural language processing techniques. Specifically, it uses libraries such as Spacy to extract keywords from the text and organize and list the information related to the work instructions. The input is raw text messages, and the output is structured task information.

[0653] Step 3:

[0654] The server generates work instruction information based on the analyzed data and sets attributes such as task content and deadlines. The input is the task information included in the output of step 2, and the output generates work instruction data that includes specific work instructions.

[0655] Step 4:

[0656] The server records the generated work instruction information in the database. This is done by an insertion process using SQL. The input is the work instruction data generated in step 3, and the output is a new record in the database.

[0657] Step 5:

[0658] The terminal receives work instruction information sent from the server and notifies autonomous devices of this information. The input is work instruction data from the server, and the output is commands sent to the devices. A communication protocol is used in this process.

[0659] Step 6:

[0660] Autonomous devices begin work based on received instructions and notify the server of their status upon completion. Input is the work command sent to the device, and output is the completion notification to the server, which updates the corresponding record in the database.

[0661] Step 7:

[0662] The user inputs prompts using a generative AI model and predicts the next task instruction based on past instruction logs. In this process, the generated prompts and past data are given to the AI ​​as input, and a list of predicted tasks is received as output.

[0663] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0664] This invention is a system that recognizes emotions through communication with the user and applies this recognition to task management and reminder functions. This system integrates an emotion engine that analyzes the user's emotions and dynamically adjusts task information based on that information. The specific operation of this system is described below.

[0665] The server receives data that can measure emotions, just like regular communication data. This data includes voice tone, message text codes, and facial expressions extracted from the user's video. The analysis engine processes this data to determine the user's emotional state (e.g., stress, relaxation, concentration). This emotional data is treated as useful metadata for processing tasks.

[0666] The terminal provides a task management utility, integrating and displaying task information and sentiment analysis results sent from the server on the user's face. The sentiment engine allows the terminal to dynamically adjust task management information based on the user's current emotional state. For example, if the system determines that the user is under significant stress, it can postpone lower-priority tasks and shift high-load tasks to lower-load ones.

[0667] Furthermore, reminder messages are customized according to the user's mood. Based on the sentiment analysis results, the server adjusts the wording and tone of the reminders to create more user-friendly communication. For example, it sends friendly messages when the user is relaxed and concise notifications when the user is focused.

[0668] Users can receive emotion-based feedback through their devices and easily edit and manage task information to maximize their work efficiency. This provides task management and communication optimized for each individual user, thereby improving work performance. In this way, the present invention functions as a means to realize user-centered, flexible task management and provides a business support system with a more human-centered approach.

[0669] The following describes the processing flow.

[0670] Step 1:

[0671] The server receives audio, text, and video data sent from the user's various communication tools. This data includes the user's voice tone, facial expressions, and emotional expressions in the text for analysis by the emotion engine.

[0672] Step 2:

[0673] The server's emotion engine analyzes received data to determine the user's current emotional state. For example, it might detect stress from voice data or extract positive intentions from text information.

[0674] Step 3:

[0675] The server dynamically adjusts existing task information based on the results of sentiment analysis. If the user is experiencing stress, changes are made such as lowering the priority of tasks or postponing the execution of tasks that can wait.

[0676] Step 4:

[0677] The terminal receives updates from the server and displays the latest task information and emotional status on the user interface. This allows the user to immediately see optimized task information tailored to their situation.

[0678] Step 5:

[0679] The device sends customized reminder notifications to the user based on the analysis results from the emotion engine. For example, if the user is not feeling anxious, the tone of the notification will be adjusted to use a relaxed tone.

[0680] Step 6:

[0681] Users can further customize their task information using the editing functions provided on their device. This allows users to adjust tasks according to their emotional state at their own discretion.

[0682] Step 7:

[0683] The server updates the task database based on user edits, ensuring the entire system is up-to-date. This provides the necessary information to be prepared for the next task cycle.

[0684] (Example 2)

[0685] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0686] In today's information society, individuals experience various stresses and emotional fluctuations, but traditional task management systems fail to take these emotional factors into account. As a result, people suffer from unnecessary stress and inefficient scheduling, leading to a decline in the quality of management.

[0687] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0688] In this invention, the server includes means for receiving and analyzing biosignals, means for determining emotional states, and means for dynamically adjusting task information. This enables flexible task management and customized reminders tailored to individual emotional states.

[0689] "Communication" is a general term for the technologies and processes used to send and receive data and information.

[0690] "Biosignals" refer to signals emitted from the human body, including data that indicates emotions and states such as voice and facial expressions.

[0691] "Analysis" is the process of breaking down data into smaller parts to verify and understand it, and extracting specific information.

[0692] "Emotional state" refers to a person's psychological and physiological state, including stress, relaxation, and concentration.

[0693] "Task information" refers to detailed data about the tasks and schedules that a user needs to complete.

[0694] "Dynamic adjustment" refers to the process of changing content and settings in real time in response to changes in circumstances and conditions.

[0695] A "reminder" is the act of sending a notification to a user to remind them of specific information or a task.

[0696] A "user interface" refers to the structure of screens and control systems that allow a user to interact with a system.

[0697] "Editable" means that existing information can be changed, deleted, or added.

[0698] This system provides task management and reminder functions based on the user's emotions. A specific implementation is shown below.

[0699] The server receives biometric signals such as voice, text, and video collected from the user. This is done using speech recognition software and image analysis tools (e.g., common speech recognition APIs, image processing libraries). The server preprocesses this data and uses an analysis engine (e.g., a natural language processing (NLP) engine) to determine the user's emotional state. Based on the analysis results, the server generates metadata indicating the emotional state.

[0700] The device dynamically adjusts task information according to the user's emotional state. Specifically, it utilizes task management software (e.g., a typical task management application) to change the schedule and tasks displayed to the user. For example, if the user is under high stress, it will postpone low-priority tasks and recommend lighter tasks that match their relaxed state.

[0701] Furthermore, the reminder function is also customized according to the user's emotional state. The server adjusts the tone and content of reminder messages and sends notifications through the engagement software. Friendly messages are sent when the user is relaxed, and concise messages when the user is focused.

[0702] Furthermore, users can freely edit task information and adjust settings through their devices. This allows users to enjoy task management optimized for their emotional state, enabling them to perform their daily tasks efficiently.

[0703] Examples of prompt messages include the following:

[0704] "Generate task management suggestions suitable for when the user's emotional state is stressed."

[0705] "Please design a reminder message for users who are in a relaxed state."

[0706] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0707] Step 1:

[0708] The server receives input data from the user, such as voice, text, and video. This data is then used to collect biometric signals. Specifically, a speech recognition API is used to convert voice to text, and an image processing library is used to extract facial expression data from video. This data is output as pre-processed input data for determining the user's emotions.

[0709] Step 2:

[0710] The server passes pre-processed data as input to the analysis engine, which then calculates the user's emotional state. The analysis engine determines the emotional label from voice tone, text sentiment analysis, and video facial recognition. As a result, metadata indicating the emotional state is output.

[0711] Step 3:

[0712] The device receives emotion metadata sent from the server and inputs it into the task management application. Based on this information, task information is dynamically adjusted. Specifically, high-priority tasks are displayed first, and if the user is stressed, for example, high-load tasks are postponed. The adjusted task information is output and displayed on the user interface.

[0713] Step 4:

[0714] The server generates reminder messages based on the user's emotional state. These messages are customized according to the emotional state and sent to the user through the engagement platform. For example, a relaxed user will receive a message in a friendly tone, while a focused user will receive concise instructions.

[0715] Step 5:

[0716] Users can view task information and reminder messages provided through their device and make modifications as needed. User changes are saved in real time on the device and referenced during subsequent feedback. This feedback loop improves user productivity.

[0717] (Application Example 2)

[0718] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0719] Traditional task management systems process tasks uniformly without considering the user's emotional state, making it difficult to maximize work efficiency while reducing the psychological burden on users. In particular, there was a need for dynamic prioritization based on emotional information such as stress levels and concentration, as well as optimization of reminder notifications.

[0720] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0721] In this invention, the server includes means for receiving and analyzing communication information, means for automatically generating business information based on the analysis results, means for recording the generated business information, means for dynamically adjusting the business content based on changes in circumstances, means for optimizing the business information based on the user's emotional state, and means for providing notifications based on the deadline for the business. This enables the dynamic optimization of business content in accordance with the user's emotional state, reducing psychological burden and allowing for efficient business execution.

[0722] "Communication information" refers to data and signals transmitted and received via a network, and is the subject of analysis.

[0723] "Analysis results" refer to judgments based on extracted information and data obtained by processing communication information.

[0724] "Business information" refers to information about the user's work and processes, generated based on the analysis results.

[0725] "Recording" means saving generated business information so that it can be searched and referenced later.

[0726] "Changes in circumstances" refers to temporal or conditional changes in the work environment or the user's condition.

[0727] "Dynamic adjustment" means changing the content and priorities of tasks in real time in response to changing circumstances.

[0728] "Emotional state" refers to information that indicates the user's psychological or physiological state, such as stress, concentration, or relaxation.

[0729] "Optimization" means adjusting business information under given conditions to make it efficient and effective.

[0730] "Providing notifications" refers to the act of conveying necessary information to users based on set conditions and timeframes.

[0731] This invention is a system for optimizing business information according to the user's emotional state. The server first receives communication information via the network. This includes voice tone, text messages, and facial expression data from images acquired through a camera. By analyzing this information, the server determines the user's emotional state.

[0732] The server dynamically generates business information based on the analysis results and adjusts the business content according to changes in the situation. This business information is sent to the terminal and notified through the user interface. The terminal notifies the user using a digital display and speaker. It can also accept user input and edit the business information.

[0733] The hardware used includes home robots and smartphones, while the software utilizes natural language processing engines and facial recognition algorithms. Specifically, Google Cloud Speech-to-Text is used as the speech analysis engine, and OpenCV is used for facial recognition.

[0734] As a concrete example, a home robot checks the user's voice commands and facial expressions, and if it detects a stressed state, it delays the execution of cleaning tasks and plays music instead to promote relaxation. When the robot is relaxed, it provides a gentle voice notification to encourage task completion.

[0735] Example prompt for a generative AI model: "Suggest the best notification method when the user is relaxed."

[0736] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0737] Step 1:

[0738] The server receives communication information from the user via the network. This information includes voice tone, text messages, and facial expression data from images captured via the camera. The input data is stored as an initial stage.

[0739] Step 2:

[0740] The server applies an emotion analysis engine to the received communication information. It uses Google Cloud Speech-to-Text for speech analysis and OpenCV for facial recognition to classify the user's emotional state into categories such as stress, relaxation, and concentration. Based on this, emotion state data is output as the analysis result.

[0741] Step 3:

[0742] The server dynamically generates business information based on the analysis results. The priority and deadlines of this business information are adjusted according to the user's emotional state. For example, tasks are changed to low-load tasks during a stressful state. After data processing, the adjusted task information is output.

[0743] Step 4:

[0744] The server records the generated business information. This information is stored in a database for later reference and editing. The input in this step is the adjusted business information, and the output is the recorded business information.

[0745] Step 5:

[0746] The terminal notifies the user of business information transmitted from the server through a user interface. Notifications are displayed on a digital screen, and voice guidance is also available. Input is business information from the server, and output is visual and auditory notification to the user.

[0747] Step 6:

[0748] Users view and edit business information via a terminal. Input consists of editing commands based on user instructions, and output is the updated business information. The information is then sent back to the server for further processing based on the user's actions.

[0749] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0750] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0751] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0752] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0753] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0754] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0755] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0756] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0757] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0758] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0759] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0760] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0761] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0762] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0763] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0764] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0765] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0766] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0767] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0768] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0769] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0770] The following is further disclosed regarding the embodiments described above.

[0771] (Claim 1)

[0772] A means of receiving and analyzing communication data,

[0773] A means for automatically generating task information based on the aforementioned analysis results,

[0774] A means for recording the generated task information,

[0775] A means of sending reminders based on task deadlines,

[0776] A system that includes this.

[0777] (Claim 2)

[0778] The system according to claim 1, further comprising means for notifying the user of the aforementioned reminder information through a user interface.

[0779] (Claim 3)

[0780] The system according to claim 1, comprising means for making task information editable based on user input.

[0781] "Example 1"

[0782] (Claim 1)

[0783] A means for receiving and integrating information from multiple communication methods,

[0784] A means for analyzing the received information and extracting task-related information using natural language processing technology,

[0785] A means for automatically generating tasks based on the extracted information and recording them in a management database,

[0786] A means to automatically set deadlines, assignees, and priorities as attributes of managed tasks,

[0787] A means of reminding users based on the task deadline,

[0788] A system that includes this.

[0789] (Claim 2)

[0790] The system according to claim 1, further comprising means for notifying the user of the recall information by display means and for issuing a warning by electronic message function.

[0791] (Claim 3)

[0792] The system according to claim 1, comprising means for enabling the modification of task information based on modification instructions from a user.

[0793] "Application Example 1"

[0794] (Claim 1)

[0795] A means for receiving and analyzing communication information,

[0796] A means for automatically generating work instruction information based on the aforementioned analysis results,

[0797] A means for recording the generated work instruction information,

[0798] Means for executing the aforementioned work instruction information,

[0799] A means of notifying the completion status of the work,

[0800] A system that includes this.

[0801] (Claim 2)

[0802] The system according to claim 1, further comprising means for notifying the user of notification information through an operation screen.

[0803] (Claim 3)

[0804] The system according to claim 1, comprising means for updating work instruction information based on input from an operating terminal.

[0805] "Example 2 of combining an emotion engine"

[0806] (Claim 1)

[0807] A means for receiving biological signals through communication and analyzing those signals,

[0808] A means for determining a person's emotional state based on the aforementioned analysis results,

[0809] A means for dynamically adjusting task information based on the identified emotional state,

[0810] A means of recording adjusted task information,

[0811] A means of providing customized reminders according to the user's emotional state,

[0812] A system that includes this.

[0813] (Claim 2)

[0814] The system according to claim 1, further comprising means for notifying the aforementioned reminder information via a user interface.

[0815] (Claim 3)

[0816] The system according to claim 1, comprising means for making task information editable based on human input.

[0817] "Application example 2 when combining with an emotional engine"

[0818] (Claim 1)

[0819] A means for receiving and analyzing communication information,

[0820] A means for automatically generating business information based on the aforementioned analysis results,

[0821] A means of recording the generated business information,

[0822] A means of dynamically adjusting work content based on changes in circumstances,

[0823] A means of optimizing business information based on the emotional state of users,

[0824] Means of providing notification based on the deadline for the work,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, comprising means for transmitting the notification information through a display device.

[0828] (Claim 3)

[0829] The system according to claim 1, comprising means for enabling editing of business information based on instructions from a user. [Explanation of symbols]

[0830] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving and analyzing communication information, A means for automatically generating work instruction information based on the aforementioned analysis results, A means for recording the generated work instruction information, Means for executing the aforementioned work instruction information, A means of notifying the completion status of the work, A system that includes this.

2. The system according to claim 1, further comprising means for notifying the user of notification information through an operation screen.

3. The system according to claim 1, comprising means for updating work instruction information based on input from an operating terminal.

Images