system

Abstract

Provide a system. 【Solution means】 Input means for inputting tasks with voice or character information, Analysis means for analyzing the input tasks and estimating classification, priority, and deadline, Planning means for calculating processing procedures and predicted required time based on the analyzed task information, Schedule cooperation means for proposing an optimal execution date and time from the calculated information and automatically adjusting the schedule in cooperation with a schedule management program, Communication analysis means for analyzing the content of communication information and generating appropriate response content, Learning means for collecting evaluations from users and improving the proposal accuracy of the system, Incorporated in household work equipment, and household environment application means for optimizing the work and communication response in the household environment for each of the above means, A system including.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, many people have problems in efficiently managing personal and professional tasks. Existing task management tools often require a lot of effort in task classification, priority setting, and schedule adjustment due to changes. Also, in email correspondence, while a prompt reply according to importance is required, the work of considering the optimal reply content for each individual email requires time and effort. The problem to be solved by this invention is to unify task management and email correspondence, enable efficient and accurate processing, and reduce the time and effort of users.

Means for Solving the Problems

[0005] This invention provides a user interface for inputting tasks via voice or text, and an analysis means for analyzing the input tasks to estimate their category, priority, and deadline. Furthermore, it provides a planning means for calculating processing procedures and estimated time required based on the analyzed task information, and a scheduling means for suggesting the optimal implementation date and time and automatically adjusting the schedule in conjunction with a calendar application. It also includes an email analysis means for analyzing the content of received emails and generating appropriate reply content, and a learning means for collecting user feedback to improve the accuracy of the system's suggestions. By integrating these means, task management and email response can be streamlined, and user time management can be optimized.

[0006] "User interface means" refers to an interface function within a system that allows the user to input tasks via voice or text.

[0007] The "analysis means" refers to a function that analyzes the input task and estimates its category, priority, and deadline.

[0008] A "planning tool" is a function that calculates processing procedures and estimated time required based on analyzed task information.

[0009] The "schedule integration method" is a function that suggests the optimal date and time for implementation based on calculated information and automatically adjusts the schedule in conjunction with a calendar application.

[0010] "Email analysis tools" are functions that analyze the content of received emails and generate appropriate reply content.

[0011] A "learning tool" is a function that collects user feedback and improves the accuracy of the system's suggestions. [Brief explanation of the drawing]

[0012] [Figure 1]This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a tagged processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a tagged RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a tagged storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, a tagged communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention is a system that aims to improve the efficiency of task management and email response using AI technology. This system functions in cooperation with a server, terminals, and users, and each function exchanges data with others to manage tasks.

[0034] First, the user inputs the task via voice or text through the device. The device uses voice recognition to convert the input into text data and sends the task information to the server. The server analyzes the received task using natural language processing technology to estimate the task's category, priority, and due date.

[0035] Based on the analysis results, the server calculates the task processing steps and estimated time required. Using this information, the server proposes the optimal date and time for execution and notifies the terminal. The terminal then receives this proposal and automatically adjusts its schedule in conjunction with its calendar application.

[0036] Furthermore, when a user opens an email they have received on their device, the device sends the email content to the server. The server analyzes the email content and generates an appropriate reply. This generated reply is sent to the device and presented to the user. The user reviews the suggested reply, makes any necessary modifications, and then sends it.

[0037] Furthermore, the terminal records user feedback and sends it to the server. The server uses this feedback to train the system and improve the accuracy of its suggestions. In this way, the efficiency of task management and email response is increased.

[0038] As a concrete example, consider a scenario where a user enters the task "Prepare presentation materials for next week" into their terminal. The server analyzes this task, sets its importance level, and suggests the optimal time to complete it. Simultaneously, when the server receives an inquiry email from a client, it generates a reply such as "Thank you for your inquiry. We will respond with details shortly," and displays it to the user. This significantly reduces the time the user spends on tasks and email correspondence.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The user enters the task into the device via voice or text. The device uses speech recognition to convert the voice to text. This text data is then sent to the server.

[0042] Step 2:

[0043] The server analyzes the received task data using natural language processing techniques. This analysis estimates the task category, priority, and deadline.

[0044] Step 3:

[0045] The server calculates the task processing steps and estimated time required based on the analysis results. It then proposes the optimal execution date and time based on the calculation results and sends that information to the terminal.

[0046] Step 4:

[0047] The device receives a suggestion from the server, integrates with the calendar application, and automatically adds the task to the user's schedule. After adding the task, it notifies the user of the schedule change.

[0048] Step 5:

[0049] The user opens the received email on their device. The device sends the email content to the server. The server analyzes the email content and generates an appropriate reply.

[0050] Step 6:

[0051] The server sends the generated reply to the terminal and presents it to the user. The user reviews the suggested reply, makes any necessary corrections, and replies to the email.

[0052] Step 7:

[0053] The terminal sends the results of the user's response to the server. The server uses this feedback data in its learning process to improve the accuracy of its suggestions.

[0054] (Example 1)

[0055] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0056] In today's work environment, efficient task management and rapid email response are crucial for improving work performance. However, traditional systems often require manual task input, analysis, and scheduling, which is time-consuming and labor-intensive. Furthermore, users must each individually consider how to respond to incoming emails, which reduces productivity. To address these challenges, more efficient and automated task management and email response systems are needed.

[0057] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0058] In this invention, the server includes user interaction means, analysis means, and communication analysis means. This enables the automation of efficient task input and analysis, and accurate responses to received emails.

[0059] A "user interaction means" is an interface means that allows the user to input tasks via voice or text.

[0060] "Analysis means" refers to means equipped with functions for analyzing input tasks and estimating their classification, priority, and deadline.

[0061] A "planning tool" is a means for calculating the processing order and estimated time required based on the analyzed task information.

[0062] A "schedule coordination method" is a means of suggesting the optimal date and time for implementation based on calculated information, and automatically adjusting the schedule in conjunction with a time management application program.

[0063] A "communication analysis tool" is a means for analyzing the content of a received communication and generating an appropriate reply.

[0064] "Learning methods" refer to means of collecting feedback from users and improving the accuracy of the system's suggestions.

[0065] A "conversion method" is a means of converting input into text data using speech recognition functionality.

[0066] "Generation means" refers to means for generating proposals and reply texts using a generative AI model.

[0067] This invention is a system that utilizes AI technology to streamline task management and email handling. This system works through the interaction of a server, terminals, and users, achieving its functions as follows:

[0068] First, the user inputs the task via voice or text through the device. The device has a voice recognition function that converts the voice input into text data. The user can input a task by voice, such as "Prepare the presentation materials for next week." This converted text data is sent to the server. For example, a common voice recognition API can be used for voice recognition.

[0069] The server analyzes the received text data using natural language processing techniques. This analysis allows it to estimate the category of the task, its priority, and its deadline. Generative AI models are used to perform the analysis efficiently and accurately. Based on the analysis results, the server calculates the task processing order and estimated time required. Based on these calculations, the server proposes the optimal execution date and time to the user.

[0070] The proposed schedule will be notified to the user via the device. The device will work with the calendar application to automatically adjust the schedule. Therefore, integration with time management applications such as the Google Calendar API is possible.

[0071] Furthermore, when a user opens an email on their device, the email content is sent to the server. The server analyzes the communication and generates an appropriate reply. Using a generation AI model, a quick and appropriate reply can be constructed. A reply such as "Thank you for your inquiry. We will contact you with details shortly." is generated and presented to the user.

[0072] The generated reply can be reviewed by the user, modified if necessary, and then sent. Finally, the device sends the user's feedback to the server. The server uses this feedback to further improve the accuracy of the suggestions and replies generated by the AI ​​model.

[0073] An example of a prompt message might be, "Please suggest a method to optimize task management by converting voice-input tasks into text." In this way, each element of the invention works together to streamline the user's task management and email response.

[0074] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0075] Step 1:

[0076] The user inputs the task via voice or text. The input data here is the user's spoken or written information. Upon receiving this input, the terminal uses speech recognition to convert the voice input into text. A common speech recognition API is often used for speech recognition. The output is the text data passed to the next stage of processing.

[0077] Step 2:

[0078] The terminal sends the converted text data to the server. The server uses natural language processing techniques to analyze this text data as input. A generative AI model is used to estimate the data's category, priority, and deadline. This analysis helps understand the task details and provides categorized task information as output.

[0079] Step 3:

[0080] The server calculates the task processing order and estimated time required based on the task information obtained through analysis. This generates a proposed feasible schedule. The input here is the analyzed task information, and the output is information on the proposed execution date and time.

[0081] Step 4:

[0082] Upon receiving schedule suggestions from the server, the terminal interacts with its calendar application. This automatically adjusts the user's schedule. Specifically, it updates appointments through a time management application program. The input is the schedule suggestion information. The output is the adjusted calendar information.

[0083] Step 5:

[0084] When a user opens an email on their device, the email content is sent to the server. The server analyzes the communication content as input and uses a generative AI model to generate an appropriate reply. This streamlines the email reply process. The output is the generated reply.

[0085] Step 6:

[0086] The generated reply is sent to the terminal and presented to the user. The user can review the presented reply and make any necessary modifications. The input is the generated reply, and the output after the user's review and modifications is the final reply.

[0087] Step 7:

[0088] The terminal sends user feedback to the server. This feedback becomes input data, which the server uses to train the system. The generative AI model improves the accuracy of its suggestions and replies based on this feedback. The output is the improved suggestion accuracy.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] The decline in efficiency in household tasks and electronic communication is a factor that increases the burden on individuals in their daily lives. In particular, communication tasks such as task management and replying to emails are time-consuming and require quick and accurate responses, so many consumers desire increased efficiency. This invention aims to provide a technology that enables individuals to make effective use of their time by improving the efficiency of such household tasks and electronic communication.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes input means for inputting tasks as voice or text information, analysis means for analyzing the input tasks and estimating their classification, priority, and deadline, and communication analysis means for analyzing the content of communication information and generating appropriate response content. This makes it possible to efficiently optimize tasks and communication responses within the home.

[0094] "Input method" refers to a function for registering tasks in the system using voice or text information.

[0095] "Analysis tools" refer to functions that analyze, classify, prioritize, and estimate deadlines for input tasks.

[0096] A "planning tool" is a function that calculates the processing steps and estimated time required based on the information of the analyzed task.

[0097] The "schedule coordination method" is a function that proposes the optimal execution date and time based on calculated information and automatically adjusts the schedule in conjunction with the schedule management program.

[0098] A "communication analysis means" is a function that analyzes the content of received communication information and automatically generates an appropriate response.

[0099] A "learning tool" is a function that collects evaluations from users and uses that information to improve the accuracy of the system's suggestions.

[0100] "Home environment application means" refers to functions in home work equipment that adapt the aforementioned functions to the home environment and optimize work and communication capabilities.

[0101] The system implementing this invention is installed in a work device used in the home. The user inputs tasks using voice or text information. In the case of voice input, the terminal uses speech recognition software (e.g., Google Cloud Speech-to-Text API) to convert the voice into text information. The terminal then sends the input information to the server.

[0102] The server analyzes the received information using natural language processing technology (e.g., OpenAI®, GPT-3®), classifies tasks, and estimates priorities and deadlines. Based on the analysis results, the server uses a planning tool to calculate processing steps and estimated time required. It also proposes the optimal date and time for implementation and automatically adjusts the schedule by synchronizing with a schedule management program using a scheduling linkage tool.

[0103] Furthermore, the server analyzes the received communication information using communication analysis tools and automatically generates an appropriate reply. This generated reply is sent to the terminal and presented to the user. The user can review this suggestion and modify it as needed.

[0104] Through user feedback, the server uses learning mechanisms to improve the accuracy of system suggestions. This feedback is then used to optimize in-home work and communication capabilities through home environment application mechanisms.

[0105] For example, if a user enters a task such as "I will do some gardening next weekend," the server will suggest the best date and reflect it in the schedule management program. When a message arrives via email requesting a reply to "check the stock of gardening tools," the system can automatically generate a reply such as "Stock is available. We will proceed with preparing for shipment."

[0106] Examples of prompts for the generating AI model include: "Task: I'm planning to do some gardening next weekend. Please suggest the best plan," and "Message: Please reply asking for confirmation of the availability of gardening tools."

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] The user inputs the task via voice or text. If the input is voice, the terminal uses speech recognition software to convert the voice into text. This input data forms the basis for the next analysis step. The input is task information, and the output is text information.

[0110] Step 2:

[0111] The terminal sends task data, converted into text information, to the server. The server analyzes this data using natural language processing techniques to classify, prioritize, and estimate the deadlines for tasks. This analysis involves understanding the meaning and context of vocabulary and classifying information using a generative AI model. The input is the converted text information, and the output is the data structure of the analysis results.

[0112] Step 3:

[0113] The server uses the analysis results to calculate the processing steps and estimated time required through a planning mechanism. The calculated information is used to propose the optimal implementation date and time. This step is a process of algorithmically calculating the required time and optimizing the schedule based on previous analysis results. The input is the analysis results, and the output is the required time and proposed implementation date and time.

[0114] Step 4:

[0115] The server synchronizes the calculation results with the schedule management program via a scheduling linkage mechanism. This process automatically reflects the proposed implementation date and time in the user's schedule. The input is the proposed implementation date and time, and the output is the updated schedule.

[0116] Step 5:

[0117] When a user receives a new message, the terminal sends the communication information to the server. The server analyzes the content using communication analysis tools and automatically generates the optimal reply. In this process, the intent of the received message is analyzed, and an AI model is used to create an appropriate response. The input is the received communication content, and the output is the generated reply.

[0118] Step 6:

[0119] The terminal presents the generated response to the user, who can then modify it as needed. If feedback is provided, it is sent back to the server, and the suggestion accuracy improves through learning mechanisms. The input is user feedback, and the output is the improved response accuracy.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] This invention relates to a task management system incorporating an emotion engine for recognizing user emotions, in which the server, terminal, and user work in conjunction with each other. The system aims to improve and streamline user time management and communication by integrating speech recognition, natural language processing, and emotion recognition technologies.

[0122] First, the user inputs the task via voice or text. The device uses speech recognition to convert the voice to text and an emotion engine to analyze the user's emotions from the input voice and text. The text data and emotion data are then sent to the server.

[0123] The server analyzes the received task data using natural language processing techniques to estimate the task category, priority, and deadline. Simultaneously, it adjusts task priorities based on sentiment data and changes the tone of communication as needed.

[0124] Based on the analyzed information, the server calculates the task's processing steps and estimated time required, and proposes the optimal date and time for execution. This information is sent to the terminal, which then automatically adjusts the schedule in conjunction with its calendar application. The user is then notified of the revised schedule.

[0125] Regarding incoming emails, when a user opens an email on their device, the device sends the email content to the server. The server analyzes the email content using natural language processing and generates a reply that matches the user's emotions using an emotion engine. This reply is sent to the device, and after the user reviews it, it is modified as needed before being sent again.

[0126] The terminal also records user actions and feedback and sends them to the server. The server uses this data for learning, improving the overall accuracy of suggestions and the quality of user support within the system.

[0127] As a concrete example, consider a case where a user enters the task "Prepare next month's project report." If the emotion engine detects that the user is expressing anxiety, the server will take this emotion into consideration and provide more detailed procedural suggestions. Similarly, when receiving an email from a client, if the user expresses joy, the reply can reflect this and suggest a more friendly tone. This enables a more empathetic response to the user's emotions, improving efficiency and satisfaction.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] The user enters a new task into the device via voice or text. In the case of voice input, the device uses speech recognition to convert the voice into text data.

[0131] Step 2:

[0132] The device passes the entered text data to the emotion engine, which analyzes the user's emotions from their voice tone and text. The emotion data is then sent to the server along with the text data.

[0133] Step 3:

[0134] The server analyzes the received task data using natural language processing via an analysis tool to estimate the task category, priority, and deadline.

[0135] Step 4:

[0136] The server uses sentiment data to adjust the priority of analysis results. For example, if a user is experiencing stress, it may increase the priority of tasks or provide more detailed guidance.

[0137] Step 5:

[0138] The server calculates the processing steps and estimated time required based on the analyzed task information and sentiment data, and proposes the optimal date and time for implementation. It then sends this proposal to the terminal.

[0139] Step 6:

[0140] The device uses information received from the server to interact with a calendar application and automatically adjusts the schedule based on the suggested date and time. It then notifies the user of the changed schedule.

[0141] Step 7:

[0142] When a user opens an email they received on their device, the device sends the email content to the server. The server analyzes the email content and generates a reply that matches the user's sentiment data.

[0143] Step 8:

[0144] The server sends the generated reply to the terminal. The user reviews the suggested reply, makes any necessary corrections, and sends the email.

[0145] Step 9:

[0146] The device records user actions and feedback and sends them to the server. The server learns from this data to improve the accuracy of future suggestions.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] In today's information society, users face a massive amount of tasks and information daily. In particular, there is a need for efficient task management methods that take user emotions into consideration, especially when it comes to prioritizing and scheduling tasks. However, conventional task management systems often lack sufficient emotional analysis and automated task adjustments based on this analysis, leading to high levels of user stress. To improve this situation, it is necessary to provide a method for efficiently managing tasks while considering user emotions.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes an interface means for inputting tasks by voice or text and generating sentiment data for sentiment analysis; an analysis means for analyzing the input tasks and sentiment data, estimating categories, priorities, and deadlines, and adjusting priorities based on the sentiment data; and a planning means for calculating processing procedures and estimated required time based on the analyzed task information and adjusted information, and proposing the optimal date and time for implementation. This enables task management that takes user sentiment into account and efficient time allocation.

[0152] An "interface means" is a device that allows the user to input tasks via voice or text and generates emotional data for sentiment analysis.

[0153] The "analysis means" is a device that analyzes input task and sentiment data, estimates the task category, priority, and deadline, and adjusts the priority based on these.

[0154] The "planning tool" is a device that calculates processing procedures and estimated required time based on analyzed task information and adjusted information, and proposes the optimal date and time for implementation.

[0155] A "schedule linking mechanism" is a system that automatically adjusts the schedule by linking calculated information with a calendar function and notifies the user of the changes.

[0156] A "communication analysis means" is a device that analyzes the content of received communications and has the function of generating appropriate reply content based on the user's emotions.

[0157] "Learning tools" are used to record user behavior and feedback, and to improve the overall accuracy of the system's suggestions.

[0158] "Emotional data" refers to information that represents a user's emotional state and is generated from voice and text.

[0159] A "generative AI model" is a model that utilizes artificial intelligence technology to generate content and analysis results based on input data.

[0160] A "prompt" is input text used to instruct a generative AI model to generate specific content.

[0161] This invention is a task management system incorporating an emotion engine for recognizing user emotions, in which the server, terminal, and user work in coordination with each other. This system aims to improve the efficiency of user time management and communication by integrating speech recognition technology, natural language processing technology, and emotion recognition technology.

[0162] The user begins by inputting tasks via voice or text using a smartphone or computer terminal. The terminal uses speech recognition software (a common example being a speech recognition API) to convert the speech to text. Here, cloud-based speech recognition services via an internet connection are typically used as the speech recognition technology. Next, the terminal uses an emotion engine to analyze the user's emotions from the input text or speech. Sentiment analysis utilizes natural language processing techniques, such as using an emotion analysis API.

[0163] The analyzed text and sentiment data are sent to the server. The server receives the relational data and utilizes natural language processing techniques to estimate the task category, priority, and deadline. Based on the input sentiment data, the server dynamically adjusts the task priority and appropriately modifies the tone of communication to match the user's emotional state. Based on this information, the server calculates the task processing steps and estimated time required, and suggests the optimal date and time for completion to the terminal.

[0164] The device automatically adjusts the user's schedule in conjunction with its calendar function based on the received implementation date and time, and notifies the user of any changes.

[0165] As a concrete example, suppose a user inputs the task "Prepare next month's project report" via voice. If the emotion engine detects the user's anxiety, the server will explicitly provide detailed step-by-step suggestions. This allows the user to proceed with the task in a way that is sensitive to their emotions. As an additional example, a prompt might read, "Generate detailed task step-by-step suggestions to provide if the user is feeling anxious."

[0166] This allows the system to provide better task management while taking user emotions into consideration, thereby improving user work efficiency and satisfaction.

[0167] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0168] Step 1:

[0169] Users input tasks using their smartphones or computers, either by voice or text. This input includes specific tasks such as "Submit the report by next week." When using voice input, the device's microphone is used to capture the data as audio.

[0170] Step 2:

[0171] The device uses a speech recognition engine to convert speech data into text data. This engine typically utilizes a cloud-based speech recognition service. Input: speech data, Output: text data.

[0172] Step 3:

[0173] The device uses an emotion engine to analyze text data and identify the user's emotions. Specifically, it uses natural language processing techniques to extract emotions (joy, anxiety, anger, etc.) from word choice and context. Input: Text data, Output: Emotion data.

[0174] Step 4:

[0175] The device sends the analyzed text data and sentiment data together to the server. During this process, the data is encrypted according to a security protocol. Input: Text data and sentiment data; Output: Data transmission to the server.

[0176] Step 5:

[0177] The server processes the received task data and sentiment data using an analysis tool. The analysis tool utilizes natural language processing techniques to estimate the task category, priority, and deadline. Task priority is dynamically adjusted based on the sentiment data. Input: Task data and sentiment data; Output: Analyzed task information.

[0178] Step 6:

[0179] The server uses planning tools to calculate the task processing steps and estimated time required based on the analysis results, and then proposes the optimal execution date and time. In this process, it generates an efficient schedule using relevant algorithms. Input: Analyzed task information; Output: Proposal of the optimal execution date and time.

[0180] Step 7:

[0181] The terminal receives implementation date and time information from the server and automatically adjusts the user's schedule by synchronizing with the calendar function. This adjustment is then notified to the user, prompting confirmation. Input: Optimal implementation date and time; Output: Adjusted schedule and notification.

[0182] Step 8:

[0183] The server analyzes the content of received emails and uses communication analysis tools to generate response content based on the user's emotions. A generation AI model supports this process, and prompts are used to generate appropriate response text. Input: Email data, Output: Response text.

[0184] Step 9:

[0185] The terminal receives the generated reply from the server and presents it to the user. The user can review the content, make any necessary modifications, and then send it. Input: Generated reply; Output: Reply ready to send.

[0186] Step 10:

[0187] The terminal records user actions and feedback and sends them to the server. The server uses this data as a learning tool to improve the accuracy of the system's suggestions and the quality of user responses. Input: Feedback data; Output: Improvement of system accuracy and quality.

[0188] (Application Example 2)

[0189] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0190] In task management for the elderly and those requiring care, conventional systems have a challenge in flexibly responding to the user's emotional state. This can lead to a lack of support that is sensitive to the user's feelings, potentially resulting in decreased satisfaction and a decline in the quality of care.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0192] In this invention, the server includes information input means, data analysis means, and emotion analysis means. This makes it possible to adjust the priority of care tasks according to the user's emotional state and to provide reminders and encouraging messages.

[0193] "Information input means" refers to devices or interfaces used to input tasks using voice or text.

[0194] "Data analysis methods" refer to the techniques and algorithms used to analyze input tasks and estimate their categories, importance, and deadlines.

[0195] A "planning tool" is a function that calculates processing procedures and estimated time required based on analyzed task information.

[0196] "Time management integration means" refers to a technology that proposes the optimal date and time for implementation based on calculated information and automatically adjusts schedules in conjunction with time management applications.

[0197] A "message analysis means" is a function that analyzes the content of received information and generates an appropriate response.

[0198] A "data learning method" is a learning function that collects user feedback and improves the accuracy of the system's suggestions.

[0199] "Emotional analysis tools" are technologies that analyze a user's emotions and adjust the priority of tasks according to their emotional state.

[0200] This invention provides a task management system that incorporates an emotion engine to recognize user emotions. This system functions in cooperation with a server and a smart device (e.g., a smartphone or smart glasses). Specific embodiments are described below.

[0201] Users input tasks using voice or text via a smart device. The device is equipped with the Google Cloud Speech-to-Text API, which converts the voice data into text. Then, IBM Watson® sentiment analysis engine is used to analyze the user's emotions from the text. This process is handled by both the information input device and the sentiment analysis device.

[0202] Next, the device sends the analyzed text and sentiment data to an Amazon Web Services (AWS®) server. The server analyzes this data using data analysis tools to estimate the task category, importance, and deadline. Furthermore, a planning tool calculates the processing steps and estimated time required.

[0203] The server takes user sentiment data into consideration and proposes the optimal date and time for implementation through a time management integration system, reflecting this in the calendar service on AWS. This information is then sent back to the device and integrated into the user's schedule.

[0204] Furthermore, when a user opens received information on their device, the device sends its contents to the server. The server analyzes this information using message parsing tools and generates an appropriate response that matches the user's emotions. This response is then provided to the user, facilitating smoother communication.

[0205] The device also collects user feedback and sends it to a server. The server uses data learning methods to utilize this feedback to improve the system. For example, if an elderly person says, "I want to relax today," the system will analyze their emotion as a relaxed state and suggest leisurely activities, enabling task suggestions tailored to the user's emotions.

[0206] An example of a prompt message is: "Analyze the user's emotions and suggest the most appropriate caregiving tasks for their current mood. Also, let me know if there are any tasks that should be prioritized."

[0207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0208] Step 1:

[0209] Users input tasks using voice or text via a smart device. For voice input, the Google Cloud Speech-to-Text API is used to convert the voice data to text. The converted text and voice data are the inputs. The output is text data ready for analysis.

[0210] Step 2:

[0211] The terminal uses IBM Watson's sentiment analysis engine to analyze the input text data. Specifically, it extracts the user's emotions (e.g., joy, anxiety, sadness) from the text data. The input is text data, and the output is sentiment data. This data is stored for use in later processes.

[0212] Step 3:

[0213] The terminal sends text data and sentiment data to the server. The server uses data analysis tools to analyze the received text data and estimate the task category, importance, and deadline. The input is text data and sentiment data, and the output is analyzed task attribute data.

[0214] Step 4:

[0215] The server uses a planning mechanism to calculate processing steps and estimated time required from the analyzed task attribute data. The input is task attribute data, and the output is detailed task planning data.

[0216] Step 5:

[0217] The server uses a time management integration mechanism to suggest the optimal date and time for implementation based on the planning data. This information is used in conjunction with the AWS Calendar service to automatically adjust schedules. The input is planning data, and the output is updated schedule information.

[0218] Step 6:

[0219] When a user opens received information on their device, the device sends its contents to the server. The server uses message parsing tools to analyze this information and generate an appropriate response that matches the user's emotions. The input is the data of the received information, and the output is the response message.

[0220] Step 7:

[0221] The terminal collects user feedback and sends it to the server. The server uses data learning tools to analyze this feedback data and improve the accuracy of the system's proposals. The input is the feedback data, and the output is the improved system proposal model.

[0222] Step 8:

[0223] The device uses a generative AI model to optimize task suggestions based on the user's emotions. The output is a list of tasks optimized for each individual user. Through this process, a system that flexibly responds to the user's emotions is realized.

[0224] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0225] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0226] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0227] [Second Embodiment]

[0228] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0229] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0230] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0231] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0232] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0233] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0234] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0235] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0236] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0237] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0238] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0239] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0240] This invention is a system that aims to improve the efficiency of task management and email response using AI technology. This system functions in cooperation with a server, terminals, and users, and each function exchanges data with others to manage tasks.

[0241] First, the user inputs the task via voice or text through the device. The device uses voice recognition to convert the input into text data and sends the task information to the server. The server analyzes the received task using natural language processing technology to estimate the task's category, priority, and due date.

[0242] Based on the analysis results, the server calculates the task processing steps and estimated time required. Using this information, the server proposes the optimal date and time for execution and notifies the terminal. The terminal then receives this proposal and automatically adjusts its schedule in conjunction with its calendar application.

[0243] Furthermore, when a user opens an email they have received on their device, the device sends the email content to the server. The server analyzes the email content and generates an appropriate reply. This generated reply is sent to the device and presented to the user. The user reviews the suggested reply, makes any necessary modifications, and then sends it.

[0244] Furthermore, the terminal records user feedback and sends it to the server. The server uses this feedback to train the system and improve the accuracy of its suggestions. In this way, the efficiency of task management and email response is increased.

[0245] As a concrete example, consider a scenario where a user enters the task "Prepare presentation materials for next week" into their terminal. The server analyzes this task, sets its importance level, and suggests the optimal time to complete it. Simultaneously, when the server receives an inquiry email from a client, it generates a reply such as "Thank you for your inquiry. We will respond with details shortly," and displays it to the user. This significantly reduces the time the user spends on tasks and email correspondence.

[0246] The following describes the processing flow.

[0247] Step 1:

[0248] The user enters the task into the device via voice or text. The device uses speech recognition to convert the voice to text. This text data is then sent to the server.

[0249] Step 2:

[0250] The server analyzes the received task data using natural language processing techniques. This analysis estimates the task category, priority, and deadline.

[0251] Step 3:

[0252] The server calculates the task processing steps and estimated time required based on the analysis results. It then proposes the optimal execution date and time based on the calculation results and sends that information to the terminal.

[0253] Step 4:

[0254] The device receives a suggestion from the server, integrates with the calendar application, and automatically adds the task to the user's schedule. After adding the task, it notifies the user of the schedule change.

[0255] Step 5:

[0256] The user opens the received email on their device. The device sends the email content to the server. The server analyzes the email content and generates an appropriate reply.

[0257] Step 6:

[0258] The server sends the generated reply to the terminal and presents it to the user. The user reviews the suggested reply, makes any necessary corrections, and replies to the email.

[0259] Step 7:

[0260] The terminal sends the results of the user's response to the server. The server uses this feedback data in its learning process to improve the accuracy of its suggestions.

[0261] (Example 1)

[0262] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0263] In today's work environment, efficient task management and rapid email response are crucial for improving work performance. However, traditional systems often require manual task input, analysis, and scheduling, which is time-consuming and labor-intensive. Furthermore, users must each individually consider how to respond to incoming emails, which reduces productivity. To address these challenges, more efficient and automated task management and email response systems are needed.

[0264] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0265] In this invention, the server includes user interaction means, analysis means, and communication analysis means. This enables the automation of efficient task input and analysis, and accurate responses to received emails.

[0266] A "user interaction means" is an interface means that allows the user to input tasks via voice or text.

[0267] "Analysis means" refers to means equipped with functions for analyzing input tasks and estimating their classification, priority, and deadline.

[0268] A "planning tool" is a means for calculating the processing order and estimated time required based on the analyzed task information.

[0269] A "schedule coordination method" is a means of suggesting the optimal date and time for implementation based on calculated information, and automatically adjusting the schedule in conjunction with a time management application program.

[0270] A "communication analysis tool" is a means for analyzing the content of a received communication and generating an appropriate reply.

[0271] "Learning methods" refer to means of collecting feedback from users and improving the accuracy of the system's suggestions.

[0272] A "conversion method" is a means of converting input into text data using speech recognition functionality.

[0273] "Generation means" refers to means for generating proposals and reply texts using a generative AI model.

[0274] This invention is a system that utilizes AI technology to streamline task management and email handling. This system works through the interaction of a server, terminals, and users, achieving its functions as follows:

[0275] First, the user inputs the task via voice or text through the device. The device has a voice recognition function that converts the voice input into text data. The user can input a task by voice, such as "Prepare the presentation materials for next week." This converted text data is sent to the server. For example, a common voice recognition API can be used for voice recognition.

[0276] The server analyzes the received text data using natural language processing techniques. This analysis allows it to estimate the category of the task, its priority, and its deadline. Generative AI models are used to perform the analysis efficiently and accurately. Based on the analysis results, the server calculates the task processing order and estimated time required. Based on these calculations, the server proposes the optimal execution date and time to the user.

[0277] The proposed schedule will be notified to the user via the device. The device will work with the calendar application to automatically adjust the schedule. Therefore, integration with time management applications such as the Google Calendar API is possible.

[0278] Furthermore, when the user opens the received email on the terminal, the email content is sent to the server. The server analyzes the communication content and generates an appropriate reply. Using a generation AI model, a quick and appropriate reply can be constructed. A reply such as "Thank you for your inquiry. We will reply in detail soon." is generated and presented to the user.

[0279] The generated reply can be confirmed by the user, corrected if necessary, and then sent. Finally, the terminal sends the feedback from the user to the server. The server utilizes this feedback to further improve the accuracy of the proposals and replies by the generation AI model.

[0280] As an example of the prompt text, content such as "Please convert the voice-input task into text and propose a method to optimize task management" can be considered. In this way, each element of the invention can cooperate to improve the efficiency of the user's task management and email response.

[0281] The flow of the specific process in Example 1 will be described using FIG. 11.

[0282] Step 1:

[0283] The user inputs a task either by voice or text. The input data here is the user's speech or character information. Upon receiving this input, the terminal uses a speech recognition function to convert the voice input into text if it is a voice input. Generally, a common speech recognition API is used for speech recognition. The output is text data that is passed to the next stage of the process.

[0284] Step 2:

[0285] The terminal sends the converted text data to the server. The server analyzes this text data as input using natural language processing technology. It estimates the category, priority, and deadline of the data by utilizing a generative AI model. Through this analysis, the details of the task are understood, and task information classified as output is obtained.

[0286] Step 3:

[0287] Based on the task information obtained from the analysis, the server calculates the processing order and estimated required time of the task. As a result, a proposal for an executable schedule is generated. The input here is the analyzed task information, and the output is the information on the proposed execution date and time.

[0288] Step 4:

[0289] Receiving the schedule proposal sent from the server, the terminal cooperates with the calendar application. Thereby, the user's schedule is automatically adjusted. Specifically, the schedule is updated through a time management application program. The input is the schedule proposal information, and the output is the adjusted calendar information.

[0290] Step 5:

[0291] When the user opens the received email on the terminal, the content of the email is sent to the server. The server analyzes the content of the communication as input and generates an appropriate reply text using a generative AI model. Thereby, the email reply work is made more efficient. The output is the generated reply text.

[0292] Step 6:

[0293] The generated reply text is sent to the terminal and presented to the user. The user can check the presented reply text and make necessary corrections to it. The input is the generated reply text, and the output after the user's confirmation and correction is the final reply text.

[0294] Step 7:

[0295] The terminal sends user feedback to the server. This feedback becomes input data, which the server uses to train the system. The generative AI model improves the accuracy of its suggestions and replies based on this feedback. The output is the improved suggestion accuracy.

[0296] (Application Example 1)

[0297] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0298] The decline in efficiency in household tasks and electronic communication is a factor that increases the burden on individuals in their daily lives. In particular, communication tasks such as task management and replying to emails are time-consuming and require quick and accurate responses, so many consumers desire increased efficiency. This invention aims to provide a technology that enables individuals to make effective use of their time by improving the efficiency of such household tasks and electronic communication.

[0299] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0300] In this invention, the server includes input means for inputting tasks as voice or text information, analysis means for analyzing the input tasks and estimating their classification, priority, and deadline, and communication analysis means for analyzing the content of communication information and generating appropriate response content. This makes it possible to efficiently optimize tasks and communication responses within the home.

[0301] "Input method" refers to a function for registering tasks in the system using voice or text information.

[0302] "Analysis tools" refer to functions that analyze, classify, prioritize, and estimate deadlines for input tasks.

[0303] The "planning means" is a function that calculates the processing procedure and the predicted required time based on the information of the analyzed task.

[0304] The "schedule coordination means" is a function that proposes the optimal execution date and time based on the calculated information and automatically adjusts the schedule in conjunction with the schedule management program.

[0305] The "communication analysis means" is a function that analyzes the content of the received communication information and automatically generates an appropriate response.

[0306] The "learning means" is a function that collects evaluations from the user and uses the information to improve the proposal accuracy of the system.

[0307] The "home environment application means" is a function that adapts each of the above functions to the home environment in home working devices and optimizes work and communication responses.

[0308] The system for implementing this invention is installed in a working device used within a home. The user inputs a task using voice or text information. In the case of voice input, the terminal uses voice recognition software (e.g., Google Cloud Speech-to-Text API) to convert the voice into text information. Then, the terminal transmits the input information to the server.

[0309] The server analyzes the received information using natural language processing technology (e.g., OpenAI GPT-3), classifies the tasks, and estimates the priority and deadline. Based on the analysis results, the server uses the planning means to calculate the processing procedure and the predicted required time. Also, it proposes the optimal implementation date and time and automatically adjusts the schedule by synchronizing with the schedule management program using the schedule coordination means.

[0310] Furthermore, the server analyzes the received communication information using the communication analysis means and automatically generates an appropriate reply. This generated reply is transmitted to the terminal and presented to the user. The user can confirm this proposal and modify it as necessary.

[0311] Through user feedback, the server uses learning mechanisms to improve the accuracy of system suggestions. This feedback is then used to optimize in-home work and communication capabilities through home environment application mechanisms.

[0312] For example, if a user enters a task such as "I will do some gardening next weekend," the server will suggest the best date and reflect it in the schedule management program. When a message arrives via email requesting a reply to "check the stock of gardening tools," the system can automatically generate a reply such as "Stock is available. We will proceed with preparing for shipment."

[0313] Examples of prompts for the generating AI model include: "Task: I'm planning to do some gardening next weekend. Please suggest the best plan," and "Message: Please reply asking for confirmation of the availability of gardening tools."

[0314] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0315] Step 1:

[0316] The user inputs the task via voice or text. If the input is voice, the terminal uses speech recognition software to convert the voice into text. This input data forms the basis for the next analysis step. The input is task information, and the output is text information.

[0317] Step 2:

[0318] The terminal sends task data, converted into text information, to the server. The server analyzes this data using natural language processing techniques to classify, prioritize, and estimate the deadlines for tasks. This analysis involves understanding the meaning and context of vocabulary and classifying information using a generative AI model. The input is the converted text information, and the output is the data structure of the analysis results.

[0319] Step 3:

[0320] The server uses the analysis results to calculate the processing steps and estimated time required through a planning mechanism. The calculated information is used to propose the optimal implementation date and time. This step is a process of algorithmically calculating the required time and optimizing the schedule based on previous analysis results. The input is the analysis results, and the output is the required time and proposed implementation date and time.

[0321] Step 4:

[0322] The server synchronizes the calculation results with the schedule management program via a scheduling linkage mechanism. This process automatically reflects the proposed implementation date and time in the user's schedule. The input is the proposed implementation date and time, and the output is the updated schedule.

[0323] Step 5:

[0324] When a user receives a new message, the terminal sends the communication information to the server. The server analyzes the content using communication analysis tools and automatically generates the optimal reply. In this process, the intent of the received message is analyzed, and an AI model is used to create an appropriate response. The input is the received communication content, and the output is the generated reply.

[0325] Step 6:

[0326] The terminal presents the generated response to the user, who can then modify it as needed. If feedback is provided, it is sent back to the server, and the suggestion accuracy improves through learning mechanisms. The input is user feedback, and the output is the improved response accuracy.

[0327] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0328] This invention relates to a task management system incorporating an emotion engine for recognizing user emotions, in which the server, terminal, and user work in conjunction with each other. The system aims to improve and streamline user time management and communication by integrating speech recognition, natural language processing, and emotion recognition technologies.

[0329] First, the user inputs the task via voice or text. The device uses speech recognition to convert the voice to text and an emotion engine to analyze the user's emotions from the input voice and text. The text data and emotion data are then sent to the server.

[0330] The server analyzes the received task data using natural language processing techniques to estimate the task category, priority, and deadline. Simultaneously, it adjusts task priorities based on sentiment data and changes the tone of communication as needed.

[0331] Based on the analyzed information, the server calculates the task's processing steps and estimated time required, and proposes the optimal date and time for execution. This information is sent to the terminal, which then automatically adjusts the schedule in conjunction with its calendar application. The user is then notified of the revised schedule.

[0332] Regarding incoming emails, when a user opens an email on their device, the device sends the email content to the server. The server analyzes the email content using natural language processing and generates a reply that matches the user's emotions using an emotion engine. This reply is sent to the device, and after the user reviews it, it is modified as needed before being sent again.

[0333] The terminal also records user actions and feedback and sends them to the server. The server uses this data for learning, improving the overall accuracy of suggestions and the quality of user support within the system.

[0334] As a concrete example, consider a case where a user enters the task "Prepare next month's project report." If the emotion engine detects that the user is expressing anxiety, the server will take this emotion into consideration and provide more detailed procedural suggestions. Similarly, when receiving an email from a client, if the user expresses joy, the reply can reflect this and suggest a more friendly tone. This enables a more empathetic response to the user's emotions, improving efficiency and satisfaction.

[0335] The following describes the processing flow.

[0336] Step 1:

[0337] The user enters a new task into the device via voice or text. In the case of voice input, the device uses speech recognition to convert the voice into text data.

[0338] Step 2:

[0339] The device passes the entered text data to the emotion engine, which analyzes the user's emotions from their voice tone and text. The emotion data is then sent to the server along with the text data.

[0340] Step 3:

[0341] The server analyzes the received task data using natural language processing via an analysis tool to estimate the task category, priority, and deadline.

[0342] Step 4:

[0343] The server uses sentiment data to adjust the priority of analysis results. For example, if a user is experiencing stress, it may increase the priority of tasks or provide more detailed guidance.

[0344] Step 5:

[0345] The server calculates the processing steps and estimated time required based on the analyzed task information and sentiment data, and proposes the optimal date and time for implementation. It then sends this proposal to the terminal.

[0346] Step 6:

[0347] The device uses information received from the server to interact with a calendar application and automatically adjusts the schedule based on the suggested date and time. It then notifies the user of the changed schedule.

[0348] Step 7:

[0349] When a user opens an email they received on their device, the device sends the email content to the server. The server analyzes the email content and generates a reply that matches the user's sentiment data.

[0350] Step 8:

[0351] The server sends the generated reply to the terminal. The user reviews the suggested reply, makes any necessary corrections, and sends the email.

[0352] Step 9:

[0353] The device records user actions and feedback and sends them to the server. The server learns from this data to improve the accuracy of future suggestions.

[0354] (Example 2)

[0355] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0356] In today's information society, users face a massive amount of tasks and information daily. In particular, there is a need for efficient task management methods that take user emotions into consideration, especially when it comes to prioritizing and scheduling tasks. However, conventional task management systems often lack sufficient emotional analysis and automated task adjustments based on this analysis, leading to high levels of user stress. To improve this situation, it is necessary to provide a method for efficiently managing tasks while considering user emotions.

[0357] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0358] In this invention, the server includes an interface means for inputting tasks by voice or text and generating sentiment data for sentiment analysis; an analysis means for analyzing the input tasks and sentiment data, estimating categories, priorities, and deadlines, and adjusting priorities based on the sentiment data; and a planning means for calculating processing procedures and estimated required time based on the analyzed task information and adjusted information, and proposing the optimal date and time for implementation. This enables task management that takes user sentiment into account and efficient time allocation.

[0359] An "interface means" is a device that allows the user to input tasks via voice or text and generates emotional data for sentiment analysis.

[0360] The "analysis means" is a device that analyzes input task and sentiment data, estimates the task category, priority, and deadline, and adjusts the priority based on these.

[0361] The "planning tool" is a device that calculates processing procedures and estimated required time based on analyzed task information and adjusted information, and proposes the optimal date and time for implementation.

[0362] A "schedule linking mechanism" is a system that automatically adjusts the schedule by linking calculated information with a calendar function and notifies the user of the changes.

[0363] A "communication analysis means" is a device that analyzes the content of received communications and has the function of generating appropriate reply content based on the user's emotions.

[0364] "Learning tools" are used to record user behavior and feedback, and to improve the overall accuracy of the system's suggestions.

[0365] "Emotional data" refers to information that represents a user's emotional state and is generated from voice and text.

[0366] A "generative AI model" is a model that utilizes artificial intelligence technology to generate content and analysis results based on input data.

[0367] A "prompt" is input text used to instruct a generative AI model to generate specific content.

[0368] This invention is a task management system incorporating an emotion engine for recognizing user emotions, in which the server, terminal, and user work in coordination with each other. This system aims to improve the efficiency of user time management and communication by integrating speech recognition technology, natural language processing technology, and emotion recognition technology.

[0369] The user begins by inputting tasks via voice or text using a smartphone or computer terminal. The terminal uses speech recognition software (a common example being a speech recognition API) to convert the speech to text. Here, cloud-based speech recognition services via an internet connection are typically used as the speech recognition technology. Next, the terminal uses an emotion engine to analyze the user's emotions from the input text or speech. Sentiment analysis utilizes natural language processing techniques, such as using an emotion analysis API.

[0370] The analyzed text and sentiment data are sent to the server. The server receives the relational data and utilizes natural language processing techniques to estimate the task category, priority, and deadline. Based on the input sentiment data, the server dynamically adjusts the task priority and appropriately modifies the tone of communication to match the user's emotional state. Based on this information, the server calculates the task processing steps and estimated time required, and suggests the optimal date and time for completion to the terminal.

[0371] The device automatically adjusts the user's schedule in conjunction with its calendar function based on the received implementation date and time, and notifies the user of any changes.

[0372] As a concrete example, suppose a user inputs the task "Prepare next month's project report" via voice. If the emotion engine detects the user's anxiety, the server will explicitly provide detailed step-by-step suggestions. This allows the user to proceed with the task in a way that is sensitive to their emotions. As an additional example, a prompt might read, "Generate detailed task step-by-step suggestions to provide if the user is feeling anxious."

[0373] This allows the system to provide better task management while taking user emotions into consideration, thereby improving user work efficiency and satisfaction.

[0374] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0375] Step 1:

[0376] Users input tasks using their smartphones or computers, either by voice or text. This input includes specific tasks such as "Submit the report by next week." When using voice input, the device's microphone is used to capture the data as audio.

[0377] Step 2:

[0378] The device uses a speech recognition engine to convert speech data into text data. This engine typically utilizes a cloud-based speech recognition service. Input: speech data, Output: text data.

[0379] Step 3:

[0380] The device uses an emotion engine to analyze text data and identify the user's emotions. Specifically, it uses natural language processing techniques to extract emotions (joy, anxiety, anger, etc.) from word choice and context. Input: Text data, Output: Emotion data.

[0381] Step 4:

[0382] The device sends the analyzed text data and sentiment data together to the server. During this process, the data is encrypted according to a security protocol. Input: Text data and sentiment data; Output: Data transmission to the server.

[0383] Step 5:

[0384] The server processes the received task data and sentiment data using an analysis tool. The analysis tool utilizes natural language processing techniques to estimate the task category, priority, and deadline. Task priority is dynamically adjusted based on the sentiment data. Input: Task data and sentiment data; Output: Analyzed task information.

[0385] Step 6:

[0386] The server uses planning tools to calculate the task processing steps and estimated time required based on the analysis results, and then proposes the optimal execution date and time. In this process, it generates an efficient schedule using relevant algorithms. Input: Analyzed task information; Output: Proposal of the optimal execution date and time.

[0387] Step 7:

[0388] The terminal receives implementation date and time information from the server and automatically adjusts the user's schedule by synchronizing with the calendar function. This adjustment is then notified to the user, prompting confirmation. Input: Optimal implementation date and time; Output: Adjusted schedule and notification.

[0389] Step 8:

[0390] The server analyzes the content of received emails and uses communication analysis tools to generate response content based on the user's emotions. A generation AI model supports this process, and prompts are used to generate appropriate response text. Input: Email data, Output: Response text.

[0391] Step 9:

[0392] The terminal receives the generated reply from the server and presents it to the user. The user can review the content, make any necessary modifications, and then send it. Input: Generated reply; Output: Reply ready to send.

[0393] Step 10:

[0394] The terminal records user actions and feedback and sends them to the server. The server uses this data as a learning tool to improve the accuracy of the system's suggestions and the quality of user responses. Input: Feedback data; Output: Improvement of system accuracy and quality.

[0395] (Application Example 2)

[0396] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0397] In task management for the elderly and those requiring care, conventional systems have a challenge in flexibly responding to the user's emotional state. This can lead to a lack of support that is sensitive to the user's feelings, potentially resulting in decreased satisfaction and a decline in the quality of care.

[0398] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0399] In this invention, the server includes information input means, data analysis means, and emotion analysis means. This makes it possible to adjust the priority of care tasks according to the user's emotional state and to provide reminders and encouraging messages.

[0400] "Information input means" refers to devices or interfaces used to input tasks using voice or text.

[0401] "Data analysis methods" refer to the techniques and algorithms used to analyze input tasks and estimate their categories, importance, and deadlines.

[0402] A "planning tool" is a function that calculates processing procedures and estimated time required based on analyzed task information.

[0403] "Time management integration means" refers to a technology that proposes the optimal date and time for implementation based on calculated information and automatically adjusts schedules in conjunction with time management applications.

[0404] A "message analysis means" is a function that analyzes the content of received information and generates an appropriate response.

[0405] A "data learning method" is a learning function that collects user feedback and improves the accuracy of the system's suggestions.

[0406] "Emotional analysis tools" are technologies that analyze a user's emotions and adjust the priority of tasks according to their emotional state.

[0407] This invention provides a task management system that incorporates an emotion engine to recognize user emotions. This system functions in cooperation with a server and a smart device (e.g., a smartphone or smart glasses). Specific embodiments are described below.

[0408] Users input tasks using voice or text via a smart device. The device is equipped with the Google Cloud Speech-to-Text API, which converts the voice data into text. Then, IBM Watson's sentiment analysis engine is used to analyze the user's emotions from the text. This process is handled by both the information input device and the sentiment analysis device.

[0409] Next, the device sends the analyzed text and sentiment data to an Amazon Web Services (AWS) server. The server analyzes this data using data analysis tools to estimate the task category, importance, and deadline. Furthermore, a planning tool calculates the processing steps and estimated time required.

[0410] The server takes user sentiment data into consideration and proposes the optimal date and time for implementation through a time management integration system, reflecting this in the calendar service on AWS. This information is then sent back to the device and integrated into the user's schedule.

[0411] Furthermore, when a user opens received information on their device, the device sends its contents to the server. The server analyzes this information using message parsing tools and generates an appropriate response that matches the user's emotions. This response is then provided to the user, facilitating smoother communication.

[0412] The device also collects user feedback and sends it to a server. The server uses data learning methods to utilize this feedback to improve the system. For example, if an elderly person says, "I want to relax today," the system will analyze their emotion as a relaxed state and suggest leisurely activities, enabling task suggestions tailored to the user's emotions.

[0413] An example of a prompt message is: "Analyze the user's emotions and suggest the most appropriate caregiving tasks for their current mood. Also, let me know if there are any tasks that should be prioritized."

[0414] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0415] Step 1:

[0416] Users input tasks using voice or text via a smart device. For voice input, the Google Cloud Speech-to-Text API is used to convert the voice data to text. The converted text and voice data are the inputs. The output is text data ready for analysis.

[0417] Step 2:

[0418] The terminal uses IBM Watson's sentiment analysis engine to analyze the input text data. Specifically, it extracts the user's emotions (e.g., joy, anxiety, sadness) from the text data. The input is text data, and the output is sentiment data. This data is stored for use in later processes.

[0419] Step 3:

[0420] The terminal sends text data and sentiment data to the server. The server uses data analysis tools to analyze the received text data and estimate the task category, importance, and deadline. The input is text data and sentiment data, and the output is analyzed task attribute data.

[0421] Step 4:

[0422] The server uses a planning mechanism to calculate processing steps and estimated time required from the analyzed task attribute data. The input is task attribute data, and the output is detailed task planning data.

[0423] Step 5:

[0424] The server uses a time management integration mechanism to suggest the optimal date and time for implementation based on the planning data. This information is used in conjunction with the AWS Calendar service to automatically adjust schedules. The input is planning data, and the output is updated schedule information.

[0425] Step 6:

[0426] When a user opens received information on their device, the device sends its contents to the server. The server uses message parsing tools to analyze this information and generate an appropriate response that matches the user's emotions. The input is the data of the received information, and the output is the response message.

[0427] Step 7:

[0428] The terminal collects user feedback and sends it to the server. The server uses data learning tools to analyze this feedback data and improve the accuracy of the system's proposals. The input is the feedback data, and the output is the improved system proposal model.

[0429] Step 8:

[0430] The device uses a generative AI model to optimize task suggestions based on the user's emotions. The output is a list of tasks optimized for each individual user. Through this process, a system that flexibly responds to the user's emotions is realized.

[0431] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0432] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0433] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0434] [Third Embodiment]

[0435] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0436] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0437] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0438] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0439] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0440] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0441] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0442] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0443] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0444] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0445] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0446] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0447] This invention is a system that aims to improve the efficiency of task management and email response using AI technology. This system functions in cooperation with a server, terminals, and users, and each function exchanges data with others to manage tasks.

[0448] First, the user inputs the task via voice or text through the device. The device uses voice recognition to convert the input into text data and sends the task information to the server. The server analyzes the received task using natural language processing technology to estimate the task's category, priority, and due date.

[0449] Based on the analysis results, the server calculates the task processing steps and estimated time required. Using this information, the server proposes the optimal date and time for execution and notifies the terminal. The terminal then receives this proposal and automatically adjusts its schedule in conjunction with its calendar application.

[0450] Furthermore, when a user opens an email they have received on their device, the device sends the email content to the server. The server analyzes the email content and generates an appropriate reply. This generated reply is sent to the device and presented to the user. The user reviews the suggested reply, makes any necessary modifications, and then sends it.

[0451] Furthermore, the terminal records user feedback and sends it to the server. The server uses this feedback to train the system and improve the accuracy of its suggestions. In this way, the efficiency of task management and email response is increased.

[0452] As a concrete example, consider a scenario where a user enters the task "Prepare presentation materials for next week" into their terminal. The server analyzes this task, sets its importance level, and suggests the optimal time to complete it. Simultaneously, when the server receives an inquiry email from a client, it generates a reply such as "Thank you for your inquiry. We will respond with details shortly," and displays it to the user. This significantly reduces the time the user spends on tasks and email correspondence.

[0453] The following describes the processing flow.

[0454] Step 1:

[0455] The user enters the task into the device via voice or text. The device uses speech recognition to convert the voice to text. This text data is then sent to the server.

[0456] Step 2:

[0457] The server analyzes the received task data using natural language processing techniques. This analysis estimates the task category, priority, and deadline.

[0458] Step 3:

[0459] The server calculates the task processing steps and estimated time required based on the analysis results. It then proposes the optimal execution date and time based on the calculation results and sends that information to the terminal.

[0460] Step 4:

[0461] The device receives a suggestion from the server, integrates with the calendar application, and automatically adds the task to the user's schedule. After adding the task, it notifies the user of the schedule change.

[0462] Step 5:

[0463] The user opens the received email on their device. The device sends the email content to the server. The server analyzes the email content and generates an appropriate reply.

[0464] Step 6:

[0465] The server sends the generated reply to the terminal and presents it to the user. The user reviews the suggested reply, makes any necessary corrections, and replies to the email.

[0466] Step 7:

[0467] The terminal sends the results of the user's response to the server. The server uses this feedback data in its learning process to improve the accuracy of its suggestions.

[0468] (Example 1)

[0469] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0470] In today's work environment, efficient task management and rapid email response are crucial for improving work performance. However, traditional systems often require manual task input, analysis, and scheduling, which is time-consuming and labor-intensive. Furthermore, users must each individually consider how to respond to incoming emails, which reduces productivity. To address these challenges, more efficient and automated task management and email response systems are needed.

[0471] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0472] In this invention, the server includes user interaction means, analysis means, and communication analysis means. This enables the automation of efficient task input and analysis, and accurate responses to received emails.

[0473] A "user interaction means" is an interface means that allows the user to input tasks via voice or text.

[0474] "Analysis means" refers to means equipped with functions for analyzing input tasks and estimating their classification, priority, and deadline.

[0475] A "planning tool" is a means for calculating the processing order and estimated time required based on the analyzed task information.

[0476] A "schedule coordination method" is a means of suggesting the optimal date and time for implementation based on calculated information, and automatically adjusting the schedule in conjunction with a time management application program.

[0477] A "communication analysis tool" is a means for analyzing the content of a received communication and generating an appropriate reply.

[0478] "Learning methods" refer to means of collecting feedback from users and improving the accuracy of the system's suggestions.

[0479] A "conversion method" is a means of converting input into text data using speech recognition functionality.

[0480] "Generation means" refers to means for generating proposals and reply texts using a generative AI model.

[0481] This invention is a system that utilizes AI technology to streamline task management and email handling. This system works through the interaction of a server, terminals, and users, achieving its functions as follows:

[0482] First, the user inputs the task via voice or text through the device. The device has a voice recognition function that converts the voice input into text data. The user can input a task by voice, such as "Prepare the presentation materials for next week." This converted text data is sent to the server. For example, a common voice recognition API can be used for voice recognition.

[0483] The server analyzes the received text data using natural language processing techniques. This analysis allows it to estimate the category of the task, its priority, and its deadline. Generative AI models are used to perform the analysis efficiently and accurately. Based on the analysis results, the server calculates the task processing order and estimated time required. Based on these calculations, the server proposes the optimal execution date and time to the user.

[0484] The proposed schedule will be notified to the user via the device. The device will work with the calendar application to automatically adjust the schedule. Therefore, integration with time management applications such as the Google Calendar API is possible.

[0485] Furthermore, when a user opens an email on their device, the email content is sent to the server. The server analyzes the communication and generates an appropriate reply. Using a generation AI model, a quick and appropriate reply can be constructed. A reply such as "Thank you for your inquiry. We will contact you with details shortly." is generated and presented to the user.

[0486] The generated reply can be reviewed by the user, modified if necessary, and then sent. Finally, the device sends the user's feedback to the server. The server uses this feedback to further improve the accuracy of the suggestions and replies generated by the AI ​​model.

[0487] An example of a prompt message might be, "Please suggest a method to optimize task management by converting voice-input tasks into text." In this way, each element of the invention works together to streamline the user's task management and email response.

[0488] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0489] Step 1:

[0490] The user inputs the task via voice or text. The input data here is the user's spoken or written information. Upon receiving this input, the terminal uses speech recognition to convert the voice input into text. A common speech recognition API is often used for speech recognition. The output is the text data passed to the next stage of processing.

[0491] Step 2:

[0492] The terminal sends the converted text data to the server. The server uses natural language processing techniques to analyze this text data as input. A generative AI model is used to estimate the data's category, priority, and deadline. This analysis helps understand the task details and provides categorized task information as output.

[0493] Step 3:

[0494] The server calculates the task processing order and estimated time required based on the task information obtained through analysis. This generates a proposed feasible schedule. The input here is the analyzed task information, and the output is information on the proposed execution date and time.

[0495] Step 4:

[0496] Upon receiving schedule suggestions from the server, the terminal interacts with its calendar application. This automatically adjusts the user's schedule. Specifically, it updates appointments through a time management application program. The input is the schedule suggestion information. The output is the adjusted calendar information.

[0497] Step 5:

[0498] When a user opens an email on their device, the email content is sent to the server. The server analyzes the communication content as input and uses a generative AI model to generate an appropriate reply. This streamlines the email reply process. The output is the generated reply.

[0499] Step 6:

[0500] The generated reply is sent to the terminal and presented to the user. The user can review the presented reply and make any necessary modifications. The input is the generated reply, and the output after the user's review and modifications is the final reply.

[0501] Step 7:

[0502] The terminal sends user feedback to the server. This feedback becomes input data, which the server uses to train the system. The generative AI model improves the accuracy of its suggestions and replies based on this feedback. The output is the improved suggestion accuracy.

[0503] (Application Example 1)

[0504] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0505] The decline in efficiency in household tasks and electronic communication is a factor that increases the burden on individuals in their daily lives. In particular, communication tasks such as task management and replying to emails are time-consuming and require quick and accurate responses, so many consumers desire increased efficiency. This invention aims to provide a technology that enables individuals to make effective use of their time by improving the efficiency of such household tasks and electronic communication.

[0506] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0507] In this invention, the server includes input means for inputting tasks as voice or text information, analysis means for analyzing the input tasks and estimating their classification, priority, and deadline, and communication analysis means for analyzing the content of communication information and generating appropriate response content. This makes it possible to efficiently optimize tasks and communication responses within the home.

[0508] "Input method" refers to a function for registering tasks in the system using voice or text information.

[0509] "Analysis tools" refer to functions that analyze, classify, prioritize, and estimate deadlines for input tasks.

[0510] A "planning tool" is a function that calculates the processing steps and estimated time required based on the information of the analyzed task.

[0511] The "schedule coordination method" is a function that proposes the optimal execution date and time based on calculated information and automatically adjusts the schedule in conjunction with the schedule management program.

[0512] A "communication analysis means" is a function that analyzes the content of received communication information and automatically generates an appropriate response.

[0513] A "learning tool" is a function that collects evaluations from users and uses that information to improve the accuracy of the system's suggestions.

[0514] "Home environment application means" refers to functions in home work equipment that adapt the aforementioned functions to the home environment and optimize work and communication capabilities.

[0515] The system implementing this invention is installed in a work device used in the home. The user inputs tasks using voice or text information. In the case of voice input, the terminal uses speech recognition software (e.g., Google Cloud Speech-to-Text API) to convert the voice into text information. The terminal then sends the input information to the server.

[0516] The server analyzes the received information using natural language processing techniques (e.g., OpenAI GPT-3) to classify tasks and estimate their priorities and deadlines. Based on the analysis results, the server uses a planning tool to calculate the processing steps and estimated time required. It also proposes the optimal date and time for implementation and automatically adjusts the schedule by synchronizing it with a schedule management program using a scheduling linkage tool.

[0517] Furthermore, the server analyzes the received communication information using communication analysis tools and automatically generates an appropriate reply. This generated reply is sent to the terminal and presented to the user. The user can review this suggestion and modify it as needed.

[0518] Through user feedback, the server uses learning mechanisms to improve the accuracy of system suggestions. This feedback is then used to optimize in-home work and communication capabilities through home environment application mechanisms.

[0519] For example, if a user enters a task such as "I will do some gardening next weekend," the server will suggest the best date and reflect it in the schedule management program. When a message arrives via email requesting a reply to "check the stock of gardening tools," the system can automatically generate a reply such as "Stock is available. We will proceed with preparing for shipment."

[0520] Examples of prompts for the generating AI model include: "Task: I'm planning to do some gardening next weekend. Please suggest the best plan," and "Message: Please reply asking for confirmation of the availability of gardening tools."

[0521] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0522] Step 1:

[0523] The user inputs the task via voice or text. If the input is voice, the terminal uses speech recognition software to convert the voice into text. This input data forms the basis for the next analysis step. The input is task information, and the output is text information.

[0524] Step 2:

[0525] The terminal sends task data, converted into text information, to the server. The server analyzes this data using natural language processing techniques to classify, prioritize, and estimate the deadlines for tasks. This analysis involves understanding the meaning and context of vocabulary and classifying information using a generative AI model. The input is the converted text information, and the output is the data structure of the analysis results.

[0526] Step 3:

[0527] The server uses the analysis results to calculate the processing steps and estimated time required through a planning mechanism. The calculated information is used to propose the optimal implementation date and time. This step is a process of algorithmically calculating the required time and optimizing the schedule based on previous analysis results. The input is the analysis results, and the output is the required time and proposed implementation date and time.

[0528] Step 4:

[0529] The server synchronizes the calculation results with the schedule management program via a scheduling linkage mechanism. This process automatically reflects the proposed implementation date and time in the user's schedule. The input is the proposed implementation date and time, and the output is the updated schedule.

[0530] Step 5:

[0531] When a user receives a new message, the terminal sends the communication information to the server. The server analyzes the content using communication analysis tools and automatically generates the optimal reply. In this process, the intent of the received message is analyzed, and an AI model is used to create an appropriate response. The input is the received communication content, and the output is the generated reply.

[0532] Step 6:

[0533] The terminal presents the generated response to the user, who can then modify it as needed. If feedback is provided, it is sent back to the server, and the suggestion accuracy improves through learning mechanisms. The input is user feedback, and the output is the improved response accuracy.

[0534] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0535] This invention relates to a task management system incorporating an emotion engine for recognizing user emotions, in which the server, terminal, and user work in conjunction with each other. The system aims to improve and streamline user time management and communication by integrating speech recognition, natural language processing, and emotion recognition technologies.

[0536] First, the user inputs the task via voice or text. The device uses speech recognition to convert the voice to text and an emotion engine to analyze the user's emotions from the input voice and text. The text data and emotion data are then sent to the server.

[0537] The server analyzes the received task data using natural language processing techniques to estimate the task category, priority, and deadline. Simultaneously, it adjusts task priorities based on sentiment data and changes the tone of communication as needed.

[0538] Based on the analyzed information, the server calculates the task's processing steps and estimated time required, and proposes the optimal date and time for execution. This information is sent to the terminal, which then automatically adjusts the schedule in conjunction with its calendar application. The user is then notified of the revised schedule.

[0539] Regarding incoming emails, when a user opens an email on their device, the device sends the email content to the server. The server analyzes the email content using natural language processing and generates a reply that matches the user's emotions using an emotion engine. This reply is sent to the device, and after the user reviews it, it is modified as needed before being sent again.

[0540] The terminal also records user actions and feedback and sends them to the server. The server uses this data for learning, improving the overall accuracy of suggestions and the quality of user support within the system.

[0541] As a concrete example, consider a case where a user enters the task "Prepare next month's project report." If the emotion engine detects that the user is expressing anxiety, the server will take this emotion into consideration and provide more detailed procedural suggestions. Similarly, when receiving an email from a client, if the user expresses joy, the reply can reflect this and suggest a more friendly tone. This enables a more empathetic response to the user's emotions, improving efficiency and satisfaction.

[0542] The following describes the processing flow.

[0543] Step 1:

[0544] The user enters a new task into the device via voice or text. In the case of voice input, the device uses speech recognition to convert the voice into text data.

[0545] Step 2:

[0546] The device passes the entered text data to the emotion engine, which analyzes the user's emotions from their voice tone and text. The emotion data is then sent to the server along with the text data.

[0547] Step 3:

[0548] The server analyzes the received task data using natural language processing via an analysis tool to estimate the task category, priority, and deadline.

[0549] Step 4:

[0550] The server uses sentiment data to adjust the priority of analysis results. For example, if a user is experiencing stress, it may increase the priority of tasks or provide more detailed guidance.

[0551] Step 5:

[0552] The server calculates the processing steps and estimated time required based on the analyzed task information and sentiment data, and proposes the optimal date and time for implementation. It then sends this proposal to the terminal.

[0553] Step 6:

[0554] The device uses information received from the server to interact with a calendar application and automatically adjusts the schedule based on the suggested date and time. It then notifies the user of the changed schedule.

[0555] Step 7:

[0556] When a user opens an email they received on their device, the device sends the email content to the server. The server analyzes the email content and generates a reply that matches the user's sentiment data.

[0557] Step 8:

[0558] The server sends the generated reply to the terminal. The user reviews the suggested reply, makes any necessary corrections, and sends the email.

[0559] Step 9:

[0560] The device records user actions and feedback and sends them to the server. The server learns from this data to improve the accuracy of future suggestions.

[0561] (Example 2)

[0562] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0563] In today's information society, users face a massive amount of tasks and information daily. In particular, there is a need for efficient task management methods that take user emotions into consideration, especially when it comes to prioritizing and scheduling tasks. However, conventional task management systems often lack sufficient emotional analysis and automated task adjustments based on this analysis, leading to high levels of user stress. To improve this situation, it is necessary to provide a method for efficiently managing tasks while considering user emotions.

[0564] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0565] In this invention, the server includes an interface means for inputting tasks by voice or text and generating sentiment data for sentiment analysis; an analysis means for analyzing the input tasks and sentiment data, estimating categories, priorities, and deadlines, and adjusting priorities based on the sentiment data; and a planning means for calculating processing procedures and estimated required time based on the analyzed task information and adjusted information, and proposing the optimal date and time for implementation. This enables task management that takes user sentiment into account and efficient time allocation.

[0566] An "interface means" is a device that allows the user to input tasks via voice or text and generates emotional data for sentiment analysis.

[0567] The "analysis means" is a device that analyzes input task and sentiment data, estimates the task category, priority, and deadline, and adjusts the priority based on these.

[0568] The "planning tool" is a device that calculates processing procedures and estimated required time based on analyzed task information and adjusted information, and proposes the optimal date and time for implementation.

[0569] A "schedule linking mechanism" is a system that automatically adjusts the schedule by linking calculated information with a calendar function and notifies the user of the changes.

[0570] A "communication analysis means" is a device that analyzes the content of received communications and has the function of generating appropriate reply content based on the user's emotions.

[0571] "Learning tools" are used to record user behavior and feedback, and to improve the overall accuracy of the system's suggestions.

[0572] "Emotional data" refers to information that represents a user's emotional state and is generated from voice and text.

[0573] A "generative AI model" is a model that utilizes artificial intelligence technology to generate content and analysis results based on input data.

[0574] A "prompt" is input text used to instruct a generative AI model to generate specific content.

[0575] This invention is a task management system incorporating an emotion engine for recognizing user emotions, in which the server, terminal, and user work in coordination with each other. This system aims to improve the efficiency of user time management and communication by integrating speech recognition technology, natural language processing technology, and emotion recognition technology.

[0576] The user begins by inputting tasks via voice or text using a smartphone or computer terminal. The terminal uses speech recognition software (a common example being a speech recognition API) to convert the speech to text. Here, cloud-based speech recognition services via an internet connection are typically used as the speech recognition technology. Next, the terminal uses an emotion engine to analyze the user's emotions from the input text or speech. Sentiment analysis utilizes natural language processing techniques, such as using an emotion analysis API.

[0577] The analyzed text and sentiment data are sent to the server. The server receives the relational data and utilizes natural language processing techniques to estimate the task category, priority, and deadline. Based on the input sentiment data, the server dynamically adjusts the task priority and appropriately modifies the tone of communication to match the user's emotional state. Based on this information, the server calculates the task processing steps and estimated time required, and suggests the optimal date and time for completion to the terminal.

[0578] The device automatically adjusts the user's schedule in conjunction with its calendar function based on the received implementation date and time, and notifies the user of any changes.

[0579] As a concrete example, suppose a user inputs the task "Prepare next month's project report" via voice. If the emotion engine detects the user's anxiety, the server will explicitly provide detailed step-by-step suggestions. This allows the user to proceed with the task in a way that is sensitive to their emotions. As an additional example, a prompt might read, "Generate detailed task step-by-step suggestions to provide if the user is feeling anxious."

[0580] This allows the system to provide better task management while taking user emotions into consideration, thereby improving user work efficiency and satisfaction.

[0581] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0582] Step 1:

[0583] Users input tasks using their smartphones or computers, either by voice or text. This input includes specific tasks such as "Submit the report by next week." When using voice input, the device's microphone is used to capture the data as audio.

[0584] Step 2:

[0585] The device uses a speech recognition engine to convert speech data into text data. This engine typically utilizes a cloud-based speech recognition service. Input: speech data, Output: text data.

[0586] Step 3:

[0587] The device uses an emotion engine to analyze text data and identify the user's emotions. Specifically, it uses natural language processing techniques to extract emotions (joy, anxiety, anger, etc.) from word choice and context. Input: Text data, Output: Emotion data.

[0588] Step 4:

[0589] The device sends the analyzed text data and sentiment data together to the server. During this process, the data is encrypted according to a security protocol. Input: Text data and sentiment data; Output: Data transmission to the server.

[0590] Step 5:

[0591] The server processes the received task data and sentiment data using an analysis tool. The analysis tool utilizes natural language processing techniques to estimate the task category, priority, and deadline. Task priority is dynamically adjusted based on the sentiment data. Input: Task data and sentiment data; Output: Analyzed task information.

[0592] Step 6:

[0593] The server uses planning tools to calculate the task processing steps and estimated time required based on the analysis results, and then proposes the optimal execution date and time. In this process, it generates an efficient schedule using relevant algorithms. Input: Analyzed task information; Output: Proposal of the optimal execution date and time.

[0594] Step 7:

[0595] The terminal receives implementation date and time information from the server and automatically adjusts the user's schedule by synchronizing with the calendar function. This adjustment is then notified to the user, prompting confirmation. Input: Optimal implementation date and time; Output: Adjusted schedule and notification.

[0596] Step 8:

[0597] The server analyzes the content of received emails and uses communication analysis tools to generate response content based on the user's emotions. A generation AI model supports this process, and prompts are used to generate appropriate response text. Input: Email data, Output: Response text.

[0598] Step 9:

[0599] The terminal receives the generated reply from the server and presents it to the user. The user can review the content, make any necessary modifications, and then send it. Input: Generated reply; Output: Reply ready to send.

[0600] Step 10:

[0601] The terminal records user actions and feedback and sends them to the server. The server uses this data as a learning tool to improve the accuracy of the system's suggestions and the quality of user responses. Input: Feedback data; Output: Improvement of system accuracy and quality.

[0602] (Application Example 2)

[0603] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0604] In task management for the elderly and those requiring care, conventional systems have a challenge in flexibly responding to the user's emotional state. This can lead to a lack of support that is sensitive to the user's feelings, potentially resulting in decreased satisfaction and a decline in the quality of care.

[0605] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0606] In this invention, the server includes information input means, data analysis means, and emotion analysis means. This makes it possible to adjust the priority of care tasks according to the user's emotional state and to provide reminders and encouraging messages.

[0607] "Information input means" refers to devices or interfaces used to input tasks using voice or text.

[0608] "Data analysis methods" refer to the techniques and algorithms used to analyze input tasks and estimate their categories, importance, and deadlines.

[0609] A "planning tool" is a function that calculates processing procedures and estimated time required based on analyzed task information.

[0610] "Time management integration means" refers to a technology that proposes the optimal date and time for implementation based on calculated information and automatically adjusts schedules in conjunction with time management applications.

[0611] A "message analysis means" is a function that analyzes the content of received information and generates an appropriate response.

[0612] A "data learning method" is a learning function that collects user feedback and improves the accuracy of the system's suggestions.

[0613] "Emotional analysis tools" are technologies that analyze a user's emotions and adjust the priority of tasks according to their emotional state.

[0614] This invention provides a task management system that incorporates an emotion engine to recognize user emotions. This system functions in cooperation with a server and a smart device (e.g., a smartphone or smart glasses). Specific embodiments are described below.

[0615] Users input tasks using voice or text via a smart device. The device is equipped with the Google Cloud Speech-to-Text API, which converts the voice data into text. Then, IBM Watson's sentiment analysis engine is used to analyze the user's emotions from the text. This process is handled by both the information input device and the sentiment analysis device.

[0616] Next, the device sends the analyzed text and sentiment data to an Amazon Web Services (AWS) server. The server analyzes this data using data analysis tools to estimate the task category, importance, and deadline. Furthermore, a planning tool calculates the processing steps and estimated time required.

[0617] The server takes user sentiment data into consideration and proposes the optimal date and time for implementation through a time management integration system, reflecting this in the calendar service on AWS. This information is then sent back to the device and integrated into the user's schedule.

[0618] Furthermore, when a user opens received information on their device, the device sends its contents to the server. The server analyzes this information using message parsing tools and generates an appropriate response that matches the user's emotions. This response is then provided to the user, facilitating smoother communication.

[0619] The device also collects user feedback and sends it to a server. The server uses data learning methods to utilize this feedback to improve the system. For example, if an elderly person says, "I want to relax today," the system will analyze their emotion as a relaxed state and suggest leisurely activities, enabling task suggestions tailored to the user's emotions.

[0620] An example of a prompt message is: "Analyze the user's emotions and suggest the most appropriate caregiving tasks for their current mood. Also, let me know if there are any tasks that should be prioritized."

[0621] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0622] Step 1:

[0623] Users input tasks using voice or text via a smart device. For voice input, the Google Cloud Speech-to-Text API is used to convert the voice data to text. The converted text and voice data are the inputs. The output is text data ready for analysis.

[0624] Step 2:

[0625] The terminal uses IBM Watson's sentiment analysis engine to analyze the input text data. Specifically, it extracts the user's emotions (e.g., joy, anxiety, sadness) from the text data. The input is text data, and the output is sentiment data. This data is stored for use in later processes.

[0626] Step 3:

[0627] The terminal sends text data and sentiment data to the server. The server uses data analysis tools to analyze the received text data and estimate the task category, importance, and deadline. The input is text data and sentiment data, and the output is analyzed task attribute data.

[0628] Step 4:

[0629] The server uses a planning mechanism to calculate processing steps and estimated time required from the analyzed task attribute data. The input is task attribute data, and the output is detailed task planning data.

[0630] Step 5:

[0631] The server uses a time management integration mechanism to suggest the optimal date and time for implementation based on the planning data. This information is used in conjunction with the AWS Calendar service to automatically adjust schedules. The input is planning data, and the output is updated schedule information.

[0632] Step 6:

[0633] When a user opens received information on their device, the device sends its contents to the server. The server uses message parsing tools to analyze this information and generate an appropriate response that matches the user's emotions. The input is the data of the received information, and the output is the response message.

[0634] Step 7:

[0635] The terminal collects user feedback and sends it to the server. The server uses data learning tools to analyze this feedback data and improve the accuracy of the system's proposals. The input is the feedback data, and the output is the improved system proposal model.

[0636] Step 8:

[0637] The device uses a generative AI model to optimize task suggestions based on the user's emotions. The output is a list of tasks optimized for each individual user. Through this process, a system that flexibly responds to the user's emotions is realized.

[0638] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0639] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0640] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0641] [Fourth Embodiment]

[0642] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0643] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0644] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0645] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0646] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0647] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0648] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0649] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0650] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0651] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0652] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0653] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0654] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0655] This invention is a system that aims to improve the efficiency of task management and email response using AI technology. This system functions in cooperation with a server, terminals, and users, and each function exchanges data with others to manage tasks.

[0656] First, the user inputs the task via voice or text through the device. The device uses voice recognition to convert the input into text data and sends the task information to the server. The server analyzes the received task using natural language processing technology to estimate the task's category, priority, and due date.

[0657] Based on the analysis results, the server calculates the task processing steps and estimated time required. Using this information, the server proposes the optimal date and time for execution and notifies the terminal. The terminal then receives this proposal and automatically adjusts its schedule in conjunction with its calendar application.

[0658] Furthermore, when a user opens an email they have received on their device, the device sends the email content to the server. The server analyzes the email content and generates an appropriate reply. This generated reply is sent to the device and presented to the user. The user reviews the suggested reply, makes any necessary modifications, and then sends it.

[0659] Furthermore, the terminal records user feedback and sends it to the server. The server uses this feedback to train the system and improve the accuracy of its suggestions. In this way, the efficiency of task management and email response is increased.

[0660] As a concrete example, consider a scenario where a user enters the task "Prepare presentation materials for next week" into their terminal. The server analyzes this task, sets its importance level, and suggests the optimal time to complete it. Simultaneously, when the server receives an inquiry email from a client, it generates a reply such as "Thank you for your inquiry. We will respond with details shortly," and displays it to the user. This significantly reduces the time the user spends on tasks and email correspondence.

[0661] The following describes the processing flow.

[0662] Step 1:

[0663] The user enters the task into the device via voice or text. The device uses speech recognition to convert the voice to text. This text data is then sent to the server.

[0664] Step 2:

[0665] The server analyzes the received task data using natural language processing techniques. This analysis estimates the task category, priority, and deadline.

[0666] Step 3:

[0667] The server calculates the task processing steps and estimated time required based on the analysis results. It then proposes the optimal execution date and time based on the calculation results and sends that information to the terminal.

[0668] Step 4:

[0669] The device receives a suggestion from the server, integrates with the calendar application, and automatically adds the task to the user's schedule. After adding the task, it notifies the user of the schedule change.

[0670] Step 5:

[0671] The user opens the received email on their device. The device sends the email content to the server. The server analyzes the email content and generates an appropriate reply.

[0672] Step 6:

[0673] The server sends the generated reply to the terminal and presents it to the user. The user reviews the suggested reply, makes any necessary corrections, and replies to the email.

[0674] Step 7:

[0675] The terminal sends the results of the user's response to the server. The server uses this feedback data in its learning process to improve the accuracy of its suggestions.

[0676] (Example 1)

[0677] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0678] In today's work environment, efficient task management and rapid email response are crucial for improving work performance. However, traditional systems often require manual task input, analysis, and scheduling, which is time-consuming and labor-intensive. Furthermore, users must each individually consider how to respond to incoming emails, which reduces productivity. To address these challenges, more efficient and automated task management and email response systems are needed.

[0679] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0680] In this invention, the server includes user interaction means, analysis means, and communication analysis means. This enables the automation of efficient task input and analysis, and accurate responses to received emails.

[0681] A "user interaction means" is an interface means that allows the user to input tasks via voice or text.

[0682] "Analysis means" refers to means equipped with functions for analyzing input tasks and estimating their classification, priority, and deadline.

[0683] A "planning tool" is a means for calculating the processing order and estimated time required based on the analyzed task information.

[0684] A "schedule coordination method" is a means of suggesting the optimal date and time for implementation based on calculated information, and automatically adjusting the schedule in conjunction with a time management application program.

[0685] A "communication analysis tool" is a means for analyzing the content of a received communication and generating an appropriate reply.

[0686] "Learning methods" refer to means of collecting feedback from users and improving the accuracy of the system's suggestions.

[0687] A "conversion method" is a means of converting input into text data using speech recognition functionality.

[0688] "Generation means" refers to means for generating proposals and reply texts using a generative AI model.

[0689] This invention is a system that utilizes AI technology to streamline task management and email handling. This system works through the interaction of a server, terminals, and users, achieving its functions as follows:

[0690] First, the user inputs the task via voice or text through the device. The device has a voice recognition function that converts the voice input into text data. The user can input a task by voice, such as "Prepare the presentation materials for next week." This converted text data is sent to the server. For example, a common voice recognition API can be used for voice recognition.

[0691] The server analyzes the received text data using natural language processing techniques. This analysis allows it to estimate the category of the task, its priority, and its deadline. Generative AI models are used to perform the analysis efficiently and accurately. Based on the analysis results, the server calculates the task processing order and estimated time required. Based on these calculations, the server proposes the optimal execution date and time to the user.

[0692] The proposed schedule will be notified to the user via the device. The device will work with the calendar application to automatically adjust the schedule. Therefore, integration with time management applications such as the Google Calendar API is possible.

[0693] Furthermore, when a user opens an email on their device, the email content is sent to the server. The server analyzes the communication and generates an appropriate reply. Using a generation AI model, a quick and appropriate reply can be constructed. A reply such as "Thank you for your inquiry. We will contact you with details shortly." is generated and presented to the user.

[0694] The generated reply can be reviewed by the user, modified if necessary, and then sent. Finally, the device sends the user's feedback to the server. The server uses this feedback to further improve the accuracy of the suggestions and replies generated by the AI ​​model.

[0695] An example of a prompt message might be, "Please suggest a method to optimize task management by converting voice-input tasks into text." In this way, each element of the invention works together to streamline the user's task management and email response.

[0696] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0697] Step 1:

[0698] The user inputs the task via voice or text. The input data here is the user's spoken or written information. Upon receiving this input, the terminal uses speech recognition to convert the voice input into text. A common speech recognition API is often used for speech recognition. The output is the text data passed to the next stage of processing.

[0699] Step 2:

[0700] The terminal sends the converted text data to the server. The server uses natural language processing techniques to analyze this text data as input. A generative AI model is used to estimate the data's category, priority, and deadline. This analysis helps understand the task details and provides categorized task information as output.

[0701] Step 3:

[0702] The server calculates the task processing order and estimated time required based on the task information obtained through analysis. This generates a proposed feasible schedule. The input here is the analyzed task information, and the output is information on the proposed execution date and time.

[0703] Step 4:

[0704] Upon receiving schedule suggestions from the server, the terminal interacts with its calendar application. This automatically adjusts the user's schedule. Specifically, it updates appointments through a time management application program. The input is the schedule suggestion information. The output is the adjusted calendar information.

[0705] Step 5:

[0706] When a user opens an email on their device, the email content is sent to the server. The server analyzes the communication content as input and uses a generative AI model to generate an appropriate reply. This streamlines the email reply process. The output is the generated reply.

[0707] Step 6:

[0708] The generated reply is sent to the terminal and presented to the user. The user can review the presented reply and make any necessary modifications. The input is the generated reply, and the output after the user's review and modifications is the final reply.

[0709] Step 7:

[0710] The terminal sends user feedback to the server. This feedback becomes input data, which the server uses to train the system. The generative AI model improves the accuracy of its suggestions and replies based on this feedback. The output is the improved suggestion accuracy.

[0711] (Application Example 1)

[0712] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0713] The decline in efficiency in household tasks and electronic communication is a factor that increases the burden on individuals in their daily lives. In particular, communication tasks such as task management and replying to emails are time-consuming and require quick and accurate responses, so many consumers desire increased efficiency. This invention aims to provide a technology that enables individuals to make effective use of their time by improving the efficiency of such household tasks and electronic communication.

[0714] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0715] In this invention, the server includes input means for inputting tasks as voice or text information, analysis means for analyzing the input tasks and estimating their classification, priority, and deadline, and communication analysis means for analyzing the content of communication information and generating appropriate response content. This makes it possible to efficiently optimize tasks and communication responses within the home.

[0716] "Input method" refers to a function for registering tasks in the system using voice or text information.

[0717] "Analysis tools" refer to functions that analyze, classify, prioritize, and estimate deadlines for input tasks.

[0718] A "planning tool" is a function that calculates the processing steps and estimated time required based on the information of the analyzed task.

[0719] The "schedule coordination method" is a function that proposes the optimal execution date and time based on calculated information and automatically adjusts the schedule in conjunction with the schedule management program.

[0720] A "communication analysis means" is a function that analyzes the content of received communication information and automatically generates an appropriate response.

[0721] A "learning tool" is a function that collects evaluations from users and uses that information to improve the accuracy of the system's suggestions.

[0722] "Home environment application means" refers to functions in home work equipment that adapt the aforementioned functions to the home environment and optimize work and communication capabilities.

[0723] The system implementing this invention is installed in a work device used in the home. The user inputs tasks using voice or text information. In the case of voice input, the terminal uses speech recognition software (e.g., Google Cloud Speech-to-Text API) to convert the voice into text information. The terminal then sends the input information to the server.

[0724] The server analyzes the received information using natural language processing techniques (e.g., OpenAI GPT-3) to classify tasks and estimate their priorities and deadlines. Based on the analysis results, the server uses a planning tool to calculate the processing steps and estimated time required. It also proposes the optimal date and time for implementation and automatically adjusts the schedule by synchronizing it with a schedule management program using a scheduling linkage tool.

[0725] Furthermore, the server analyzes the received communication information using communication analysis tools and automatically generates an appropriate reply. This generated reply is sent to the terminal and presented to the user. The user can review this suggestion and modify it as needed.

[0726] Through user feedback, the server uses learning mechanisms to improve the accuracy of system suggestions. This feedback is then used to optimize in-home work and communication capabilities through home environment application mechanisms.

[0727] For example, if a user enters a task such as "I will do some gardening next weekend," the server will suggest the best date and reflect it in the schedule management program. When a message arrives via email requesting a reply to "check the stock of gardening tools," the system can automatically generate a reply such as "Stock is available. We will proceed with preparing for shipment."

[0728] Examples of prompts for the generating AI model include: "Task: I'm planning to do some gardening next weekend. Please suggest the best plan," and "Message: Please reply asking for confirmation of the availability of gardening tools."

[0729] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0730] Step 1:

[0731] The user inputs the task via voice or text. If the input is voice, the terminal uses speech recognition software to convert the voice into text. This input data forms the basis for the next analysis step. The input is task information, and the output is text information.

[0732] Step 2:

[0733] The terminal sends task data, converted into text information, to the server. The server analyzes this data using natural language processing techniques to classify, prioritize, and estimate the deadlines for tasks. This analysis involves understanding the meaning and context of vocabulary and classifying information using a generative AI model. The input is the converted text information, and the output is the data structure of the analysis results.

[0734] Step 3:

[0735] The server uses the analysis results to calculate the processing steps and estimated time required through a planning mechanism. The calculated information is used to propose the optimal implementation date and time. This step is a process of algorithmically calculating the required time and optimizing the schedule based on previous analysis results. The input is the analysis results, and the output is the required time and proposed implementation date and time.

[0736] Step 4:

[0737] The server synchronizes the calculation results with the schedule management program via a scheduling linkage mechanism. This process automatically reflects the proposed implementation date and time in the user's schedule. The input is the proposed implementation date and time, and the output is the updated schedule.

[0738] Step 5:

[0739] When a user receives a new message, the terminal sends the communication information to the server. The server analyzes the content using communication analysis tools and automatically generates the optimal reply. In this process, the intent of the received message is analyzed, and an AI model is used to create an appropriate response. The input is the received communication content, and the output is the generated reply.

[0740] Step 6:

[0741] The terminal presents the generated response to the user, who can then modify it as needed. If feedback is provided, it is sent back to the server, and the suggestion accuracy improves through learning mechanisms. The input is user feedback, and the output is the improved response accuracy.

[0742] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0743] This invention relates to a task management system incorporating an emotion engine for recognizing user emotions, in which the server, terminal, and user work in conjunction with each other. The system aims to improve and streamline user time management and communication by integrating speech recognition, natural language processing, and emotion recognition technologies.

[0744] First, the user inputs the task via voice or text. The device uses speech recognition to convert the voice to text and an emotion engine to analyze the user's emotions from the input voice and text. The text data and emotion data are then sent to the server.

[0745] The server analyzes the received task data using natural language processing techniques to estimate the task category, priority, and deadline. Simultaneously, it adjusts task priorities based on sentiment data and changes the tone of communication as needed.

[0746] Based on the analyzed information, the server calculates the task's processing steps and estimated time required, and proposes the optimal date and time for execution. This information is sent to the terminal, which then automatically adjusts the schedule in conjunction with its calendar application. The user is then notified of the revised schedule.

[0747] Regarding incoming emails, when a user opens an email on their device, the device sends the email content to the server. The server analyzes the email content using natural language processing and generates a reply that matches the user's emotions using an emotion engine. This reply is sent to the device, and after the user reviews it, it is modified as needed before being sent again.

[0748] The terminal also records user actions and feedback and sends them to the server. The server uses this data for learning, improving the overall accuracy of suggestions and the quality of user support within the system.

[0749] As a concrete example, consider a case where a user enters the task "Prepare next month's project report." If the emotion engine detects that the user is expressing anxiety, the server will take this emotion into consideration and provide more detailed procedural suggestions. Similarly, when receiving an email from a client, if the user expresses joy, the reply can reflect this and suggest a more friendly tone. This enables a more empathetic response to the user's emotions, improving efficiency and satisfaction.

[0750] The following describes the processing flow.

[0751] Step 1:

[0752] The user enters a new task into the device via voice or text. In the case of voice input, the device uses speech recognition to convert the voice into text data.

[0753] Step 2:

[0754] The device passes the entered text data to the emotion engine, which analyzes the user's emotions from their voice tone and text. The emotion data is then sent to the server along with the text data.

[0755] Step 3:

[0756] The server analyzes the received task data using natural language processing via an analysis tool to estimate the task category, priority, and deadline.

[0757] Step 4:

[0758] The server uses sentiment data to adjust the priority of analysis results. For example, if a user is experiencing stress, it may increase the priority of tasks or provide more detailed guidance.

[0759] Step 5:

[0760] The server calculates the processing steps and estimated time required based on the analyzed task information and sentiment data, and proposes the optimal date and time for implementation. It then sends this proposal to the terminal.

[0761] Step 6:

[0762] The device uses information received from the server to interact with a calendar application and automatically adjusts the schedule based on the suggested date and time. It then notifies the user of the changed schedule.

[0763] Step 7:

[0764] When a user opens an email they received on their device, the device sends the email content to the server. The server analyzes the email content and generates a reply that matches the user's sentiment data.

[0765] Step 8:

[0766] The server sends the generated reply to the terminal. The user reviews the suggested reply, makes any necessary corrections, and sends the email.

[0767] Step 9:

[0768] The device records user actions and feedback and sends them to the server. The server learns from this data to improve the accuracy of future suggestions.

[0769] (Example 2)

[0770] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0771] In today's information society, users face a massive amount of tasks and information daily. In particular, there is a need for efficient task management methods that take user emotions into consideration, especially when it comes to prioritizing and scheduling tasks. However, conventional task management systems often lack sufficient emotional analysis and automated task adjustments based on this analysis, leading to high levels of user stress. To improve this situation, it is necessary to provide a method for efficiently managing tasks while considering user emotions.

[0772] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0773] In this invention, the server includes an interface means for inputting tasks by voice or text and generating sentiment data for sentiment analysis; an analysis means for analyzing the input tasks and sentiment data, estimating categories, priorities, and deadlines, and adjusting priorities based on the sentiment data; and a planning means for calculating processing procedures and estimated required time based on the analyzed task information and adjusted information, and proposing the optimal date and time for implementation. This enables task management that takes user sentiment into account and efficient time allocation.

[0774] An "interface means" is a device that allows the user to input tasks via voice or text and generates emotional data for sentiment analysis.

[0775] The "analysis means" is a device that analyzes input task and sentiment data, estimates the task category, priority, and deadline, and adjusts the priority based on these.

[0776] The "planning tool" is a device that calculates processing procedures and estimated required time based on analyzed task information and adjusted information, and proposes the optimal date and time for implementation.

[0777] A "schedule linking mechanism" is a system that automatically adjusts the schedule by linking calculated information with a calendar function and notifies the user of the changes.

[0778] A "communication analysis means" is a device that analyzes the content of received communications and has the function of generating appropriate reply content based on the user's emotions.

[0779] "Learning tools" are used to record user behavior and feedback, and to improve the overall accuracy of the system's suggestions.

[0780] "Emotional data" refers to information that represents a user's emotional state and is generated from voice and text.

[0781] A "generative AI model" is a model that utilizes artificial intelligence technology to generate content and analysis results based on input data.

[0782] A "prompt" is input text used to instruct a generative AI model to generate specific content.

[0783] This invention is a task management system incorporating an emotion engine for recognizing user emotions, in which the server, terminal, and user work in coordination with each other. This system aims to improve the efficiency of user time management and communication by integrating speech recognition technology, natural language processing technology, and emotion recognition technology.

[0784] The user begins by inputting tasks via voice or text using a smartphone or computer terminal. The terminal uses speech recognition software (a common example being a speech recognition API) to convert the speech to text. Here, cloud-based speech recognition services via an internet connection are typically used as the speech recognition technology. Next, the terminal uses an emotion engine to analyze the user's emotions from the input text or speech. Sentiment analysis utilizes natural language processing techniques, such as using an emotion analysis API.

[0785] The analyzed text and sentiment data are sent to the server. The server receives the relational data and utilizes natural language processing techniques to estimate the task category, priority, and deadline. Based on the input sentiment data, the server dynamically adjusts the task priority and appropriately modifies the tone of communication to match the user's emotional state. Based on this information, the server calculates the task processing steps and estimated time required, and suggests the optimal date and time for completion to the terminal.

[0786] The device automatically adjusts the user's schedule in conjunction with its calendar function based on the received implementation date and time, and notifies the user of any changes.

[0787] As a concrete example, suppose a user inputs the task "Prepare next month's project report" via voice. If the emotion engine detects the user's anxiety, the server will explicitly provide detailed step-by-step suggestions. This allows the user to proceed with the task in a way that is sensitive to their emotions. As an additional example, a prompt might read, "Generate detailed task step-by-step suggestions to provide if the user is feeling anxious."

[0788] This allows the system to provide better task management while taking user emotions into consideration, thereby improving user work efficiency and satisfaction.

[0789] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0790] Step 1:

[0791] Users input tasks using their smartphones or computers, either by voice or text. This input includes specific tasks such as "Submit the report by next week." When using voice input, the device's microphone is used to capture the data as audio.

[0792] Step 2:

[0793] The device uses a speech recognition engine to convert speech data into text data. This engine typically utilizes a cloud-based speech recognition service. Input: speech data, Output: text data.

[0794] Step 3:

[0795] The device uses an emotion engine to analyze text data and identify the user's emotions. Specifically, it uses natural language processing techniques to extract emotions (joy, anxiety, anger, etc.) from word choice and context. Input: Text data, Output: Emotion data.

[0796] Step 4:

[0797] The device sends the analyzed text data and sentiment data together to the server. During this process, the data is encrypted according to a security protocol. Input: Text data and sentiment data; Output: Data transmission to the server.

[0798] Step 5:

[0799] The server processes the received task data and sentiment data using an analysis tool. The analysis tool utilizes natural language processing techniques to estimate the task category, priority, and deadline. Task priority is dynamically adjusted based on the sentiment data. Input: Task data and sentiment data; Output: Analyzed task information.

[0800] Step 6:

[0801] The server uses planning tools to calculate the task processing steps and estimated time required based on the analysis results, and then proposes the optimal execution date and time. In this process, it generates an efficient schedule using relevant algorithms. Input: Analyzed task information; Output: Proposal of the optimal execution date and time.

[0802] Step 7:

[0803] The terminal receives implementation date and time information from the server and automatically adjusts the user's schedule by synchronizing with the calendar function. This adjustment is then notified to the user, prompting confirmation. Input: Optimal implementation date and time; Output: Adjusted schedule and notification.

[0804] Step 8:

[0805] The server analyzes the content of received emails and uses communication analysis tools to generate response content based on the user's emotions. A generation AI model supports this process, and prompts are used to generate appropriate response text. Input: Email data, Output: Response text.

[0806] Step 9:

[0807] The terminal receives the generated reply from the server and presents it to the user. The user can review the content, make any necessary modifications, and then send it. Input: Generated reply; Output: Reply ready to send.

[0808] Step 10:

[0809] The terminal records user actions and feedback and sends them to the server. The server uses this data as a learning tool to improve the accuracy of the system's suggestions and the quality of user responses. Input: Feedback data; Output: Improvement of system accuracy and quality.

[0810] (Application Example 2)

[0811] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0812] In task management for the elderly and those requiring care, conventional systems have a challenge in flexibly responding to the user's emotional state. This can lead to a lack of support that is sensitive to the user's feelings, potentially resulting in decreased satisfaction and a decline in the quality of care.

[0813] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0814] In this invention, the server includes information input means, data analysis means, and emotion analysis means. This makes it possible to adjust the priority of care tasks according to the user's emotional state and to provide reminders and encouraging messages.

[0815] "Information input means" refers to devices or interfaces used to input tasks using voice or text.

[0816] "Data analysis methods" refer to the techniques and algorithms used to analyze input tasks and estimate their categories, importance, and deadlines.

[0817] A "planning tool" is a function that calculates processing procedures and estimated time required based on analyzed task information.

[0818] "Time management integration means" refers to a technology that proposes the optimal date and time for implementation based on calculated information and automatically adjusts schedules in conjunction with time management applications.

[0819] A "message analysis means" is a function that analyzes the content of received information and generates an appropriate response.

[0820] A "data learning method" is a learning function that collects user feedback and improves the accuracy of the system's suggestions.

[0821] "Emotional analysis tools" are technologies that analyze a user's emotions and adjust the priority of tasks according to their emotional state.

[0822] This invention provides a task management system that incorporates an emotion engine to recognize user emotions. This system functions in cooperation with a server and a smart device (e.g., a smartphone or smart glasses). Specific embodiments are described below.

[0823] Users input tasks using voice or text via a smart device. The device is equipped with the Google Cloud Speech-to-Text API, which converts the voice data into text. Then, IBM Watson's sentiment analysis engine is used to analyze the user's emotions from the text. This process is handled by both the information input device and the sentiment analysis device.

[0824] Next, the device sends the analyzed text and sentiment data to an Amazon Web Services (AWS) server. The server analyzes this data using data analysis tools to estimate the task category, importance, and deadline. Furthermore, a planning tool calculates the processing steps and estimated time required.

[0825] The server takes user sentiment data into consideration and proposes the optimal date and time for implementation through a time management integration system, reflecting this in the calendar service on AWS. This information is then sent back to the device and integrated into the user's schedule.

[0826] Furthermore, when a user opens received information on their device, the device sends its contents to the server. The server analyzes this information using message parsing tools and generates an appropriate response that matches the user's emotions. This response is then provided to the user, facilitating smoother communication.

[0827] The device also collects user feedback and sends it to a server. The server uses data learning methods to utilize this feedback to improve the system. For example, if an elderly person says, "I want to relax today," the system will analyze their emotion as a relaxed state and suggest leisurely activities, enabling task suggestions tailored to the user's emotions.

[0828] An example of a prompt message is: "Analyze the user's emotions and suggest the most appropriate caregiving tasks for their current mood. Also, let me know if there are any tasks that should be prioritized."

[0829] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0830] Step 1:

[0831] Users input tasks using voice or text via a smart device. For voice input, the Google Cloud Speech-to-Text API is used to convert the voice data to text. The converted text and voice data are the inputs. The output is text data ready for analysis.

[0832] Step 2:

[0833] The terminal uses IBM Watson's sentiment analysis engine to analyze the input text data. Specifically, it extracts the user's emotions (e.g., joy, anxiety, sadness) from the text data. The input is text data, and the output is sentiment data. This data is stored for use in later processes.

[0834] Step 3:

[0835] The terminal sends text data and sentiment data to the server. The server uses data analysis tools to analyze the received text data and estimate the task category, importance, and deadline. The input is text data and sentiment data, and the output is analyzed task attribute data.

[0836] Step 4:

[0837] The server uses a planning mechanism to calculate processing steps and estimated time required from the analyzed task attribute data. The input is task attribute data, and the output is detailed task planning data.

[0838] Step 5:

[0839] The server uses a time management integration mechanism to suggest the optimal date and time for implementation based on the planning data. This information is used in conjunction with the AWS Calendar service to automatically adjust schedules. The input is planning data, and the output is updated schedule information.

[0840] Step 6:

[0841] When a user opens received information on their device, the device sends its contents to the server. The server uses message parsing tools to analyze this information and generate an appropriate response that matches the user's emotions. The input is the data of the received information, and the output is the response message.

[0842] Step 7:

[0843] The terminal collects user feedback and sends it to the server. The server uses data learning tools to analyze this feedback data and improve the accuracy of the system's proposals. The input is the feedback data, and the output is the improved system proposal model.

[0844] Step 8:

[0845] The device uses a generative AI model to optimize task suggestions based on the user's emotions. The output is a list of tasks optimized for each individual user. Through this process, a system that flexibly responds to the user's emotions is realized.

[0846] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0847] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0848] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0849] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0850] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0851] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0852] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0853] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0854] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0855] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0856] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0857] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0858] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0859] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0860] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0861] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0862] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0863] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0864] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0865] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0866] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0867] The following is further disclosed regarding the embodiments described above.

[0868] (Claim 1)

[0869] A user interface means for inputting tasks by voice or text,

[0870] An analysis means for analyzing the input task and estimating its category, priority, and deadline,

[0871] A planning means for calculating the processing procedure and estimated time required based on the analyzed task information,

[0872] A scheduling integration method that proposes the optimal date and time for implementation based on calculated information and automatically adjusts the schedule in conjunction with a calendar application,

[0873] A means of analyzing the content of received emails and generating appropriate reply content,

[0874] A task management system that includes a learning mechanism to collect user feedback and improve the accuracy of the system's suggestions.

[0875] (Claim 2)

[0876] The system according to claim 1, wherein the user interface means is equipped with a voice recognition function.

[0877] (Claim 3)

[0878] The system according to claim 1, wherein the analysis means uses natural language processing technology.

[0879] "Example 1"

[0880] (Claim 1)

[0881] A user interaction means for inputting tasks by voice or text,

[0882] An analysis means for analyzing the input tasks and estimating their classification, priority, and deadline,

[0883] A planning means for calculating the processing order and estimated time required based on the analyzed task information,

[0884] A scheduling integration method that proposes the optimal date and time for implementation based on calculated information and automatically adjusts the schedule in conjunction with a time management application program,

[0885] A communication analysis means for analyzing the content of received communications and generating appropriate reply content,

[0886] A learning method to collect feedback from users and improve the accuracy of the system's suggestions,

[0887] A conversion method for converting input into text data using speech recognition functionality,

[0888] A system including a generation means for generating proposals and reply texts using a generative AI model.

[0889] (Claim 2)

[0890] The system according to claim 1, wherein the user interaction means is equipped with a voice recognition function.

[0891] (Claim 3)

[0892] The system according to claim 1, wherein the analysis means uses natural language processing technology.

[0893] "Application Example 1"

[0894] (Claim 1)

[0895] An input method for entering tasks using voice or text information,

[0896] An analytical means for analyzing the input tasks and estimating their classification, priority, and deadline,

[0897] A planning means for calculating processing procedures and estimated required time based on analyzed task information,

[0898] A scheduling mechanism that proposes the optimal date and time based on calculated information and automatically adjusts the schedule in conjunction with a schedule management program,

[0899] A communication analysis means for analyzing the content of communication information and generating appropriate response content,

[0900] A learning method to collect user feedback and improve the accuracy of the system's suggestions,

[0901] Incorporated into home work equipment, the above means are home environment application means for optimizing work and communication capabilities in a home environment,

[0902] A system that includes this.

[0903] (Claim 2)

[0904] The system according to claim 1, wherein the input means is equipped with a voice conversion function.

[0905] (Claim 3)

[0906] The system according to claim 1, wherein the analysis means uses natural language processing technology.

[0907] "Example 2 of combining an emotion engine"

[0908] (Claim 1)

[0909] An interface means for inputting tasks via voice or text and generating sentiment data for sentiment analysis,

[0910] The aforementioned input task and sentiment data are analyzed, the category, priority and deadline are estimated, and the priority is adjusted based on the sentiment data.

[0911] A planning means for calculating processing procedures and estimated required time based on analyzed task information and adjusted information, and for proposing the optimal implementation date and time,

[0912] A scheduling integration method that automatically adjusts the schedule by linking the calculated information with the calendar function and notifies the user of the changed schedule,

[0913] A communication analysis means for analyzing the content of received communications and generating appropriate reply content based on the user's emotions,

[0914] A system that records user behavior and feedback, and includes learning mechanisms to improve the accuracy of the system's suggestions.

[0915] (Claim 2)

[0916] The system according to claim 1, wherein the interface means is equipped with a voice recognition function and integrates an emotion engine.

[0917] (Claim 3)

[0918] The system according to claim 1, wherein the analysis means uses natural language processing technology and a generative AI model, and applies prompt sentences based on sentiment data.

[0919] "Application example 2 when combining with an emotional engine"

[0920] (Claim 1)

[0921] Information input means for entering tasks by voice or text,

[0922] A data analysis means for analyzing the input tasks and estimating their categories, importance, and deadlines,

[0923] A planning means for calculating processing procedures and estimated required time based on analyzed task information,

[0924] A time management integration method that proposes the optimal date and time for implementation based on calculated information and automatically adjusts schedules in conjunction with a time management application,

[0925] A message analysis means for analyzing the content of received information and generating appropriate response content,

[0926] A data learning method for collecting user feedback and improving the accuracy of system suggestions,

[0927] A means of emotional analysis to analyze the user's emotions, adjust the priority of caregiving tasks according to their emotional state, and provide reminders and encouraging messages,

[0928] A system that includes this.

[0929] (Claim 2)

[0930] The system according to claim 1, wherein the information input means is equipped with a voice recognition function.

[0931] (Claim 3)

[0932] The system according to claim 1, wherein the data analysis means uses natural language processing technology. [Explanation of Symbols]

[0933] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input method for entering tasks using voice or text information, An analytical means for analyzing the input tasks and estimating their classification, priority, and deadline, A planning means for calculating processing procedures and estimated required time based on analyzed task information, A scheduling mechanism that proposes the optimal date and time based on calculated information and automatically adjusts the schedule in conjunction with a schedule management program, A communication analysis means for analyzing the content of communication information and generating appropriate response content, A learning method to collect user feedback and improve the accuracy of the system's suggestions, Incorporated into home work equipment, the above means are home environment application means for optimizing work and communication capabilities in a home environment, A system that includes this.

2. The system according to claim 1, wherein the input means is equipped with a voice conversion function.

3. The system according to claim 1, wherein the analysis means uses natural language processing technology.

Images