system

Abstract

We provide the system. [Solution] A means of receiving email data and analyzing its content using natural language processing, A means of automatically classifying emails into different categories based on the analysis results, A method for automatically summarizing the content of each email, A means of prioritizing emails based on their importance and deadline, A means of generating a reply template for a message that requires a response, A means of notifying reminders about important emails that have not yet been processed, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a modern business environment, especially in positions such as executives and management positions, a large amount of email management is required every day. As a result, important emails are overlooked or responses are delayed, leading to problems such as a decrease in work efficiency, an increase in stress, and a breakdown in work-life balance. Since these problems cannot be sufficiently solved by conventional manual email management, there is a need for a system that can perform email management efficiently and accurately.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides a system that automatically categorizes emails by receiving email data and analyzing its content using natural language processing. This system summarizes the content of each email and automatically prioritizes them based on importance and deadline, thereby achieving efficient email management. Furthermore, it also has a function to automatically generate reply templates for emails that require a reply and to notify users of reminders for important emails that have not yet been processed, thereby reducing the burden on users and improving work efficiency and productivity.

[0006] "Email data" refers to a collection of information sent and received as email, and is typically digital data including sender, recipient, subject, body, and attachments.

[0007] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, and it involves analyzing text data using machine learning and rule-based algorithms.

[0008] "Analysis" is the process of analyzing given data and extracting meaning and characteristics, and in this invention in particular, it is performed to determine the category and importance from the content of emails.

[0009] "Automatic classification" is the process of organizing and assigning data to specific categories or groups based on predefined criteria.

[0010] "Summary" refers to expressing the main points of a document or piece of information in a shortened form, and in this system, it is used to allow users to quickly grasp the content of an email.

[0011] "Priority setting" refers to determining the order in which multiple tasks or items should be performed based on their importance and urgency.

[0012] A "reply template" is a standardized template for reply documents, designed to facilitate efficient communication.

[0013] A "reminder" is a function that notifies or alerts the user at a specific time or in a specific situation, according to pre-set conditions. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The email management system of this invention is realized through the cooperation of a server and a terminal, and provides a function to efficiently manage the user's received emails. When the server receives a new email from the mail server, it first analyzes the content of the email using natural language processing technology. Based on the analysis results, the server classifies the email into categories such as "Notification," "Action Required," and "Reply Required."

[0036] Next, the server summarizes the content of each email and prioritizes them based on importance and deadlines. This allows the user interface to quickly identify the most important emails. The terminal then sorts and displays the email list by priority based on the information received from the server. Through the terminal, the user can review the email summary and access the detailed content as needed.

[0037] Furthermore, for emails that the server determines require a reply, it automatically generates a reply template based on pre-registered reply history and template information. The terminal presents this to the user, who can edit it as needed and easily send a reply. The server also has a function to periodically issue reminders for important unprocessed emails. This function allows the terminal to notify the user through a designated communication tool, preventing delays in response.

[0038] As a concrete example, among the numerous emails a user receives via the server, "project progress report requests" from their supervisor are classified with high priority, and timely notifications are sent via a reminder function as time passes. In addition, a reply template generation function automatically generates a reply template for the "progress report" and presents it to the user. This process prevents important emails from being overlooked and enables more efficient work.

[0039] The features described above allow users to significantly reduce the time spent on email management, enabling them to focus on other important tasks. This system is particularly useful for users who need to handle a large volume of emails daily, especially in a business environment.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server accesses the user's mail server and periodically retrieves new mail data. In this process, it analyzes the mail metadata (sender, received date and time, subject) and stores it in a database.

[0043] Step 2:

[0044] The server passes the body of the retrieved email data to a natural language processing engine for text analysis. Based on the analysis results, the emails are categorized as "Notification," "Action Required," or "Reply Required."

[0045] Step 3:

[0046] The server uses the analysis results to generate a summary from the email content. In this process, it extracts key keywords and context to create a shortened summary.

[0047] Step 4:

[0048] The server determines the importance of an email by scoring the sender's reputation and the urgency of the email. Prioritization is set based on deadline information and the presence of important keywords.

[0049] Step 5:

[0050] The terminal uses the summary, classification, and priority information of emails received from the server to display a list of emails in the user interface. At this time, the emails are sorted and displayed to the user in descending order of priority.

[0051] Step 6:

[0052] Users can view emails displayed with high priority on the device's interface and open the detailed contents of emails as needed. This enables them to quickly check and respond to important emails.

[0053] Step 7:

[0054] The server automatically generates reply templates for emails that require a response. It creates the most suitable template based on past email reply history and pre-configured phrases.

[0055] Step 8:

[0056] The device presents the user with a generated reply template, allowing the user to quickly respond by editing it.

[0057] Step 9:

[0058] The server monitors important pending emails in real time and issues reminders if replies are delayed. These reminders are sent to the user via a designated communication tool.

[0059] Step 10:

[0060] Users can receive reminder notifications through their devices, ensuring they don't forget to respond to important emails. This improves their work efficiency.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] In today's business environment, the volume of electronic messages users receive daily is increasing, requiring the rapid and efficient management of important messages. In particular, there are challenges such as overlooking important messages, the invalidation of preventative measures, and the difficulty of responding in a timely manner. Unless these challenges are addressed, user productivity will decline, and the risk of missing important business opportunities will increase.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes information processing means for receiving and analyzing the meaning of electronic messages, information processing means for classifying electronic messages into multiple categories based on the analysis results, and information processing means for condensing the information of each electronic message. This enables efficient management of electronic messages, prevention of overlooking important information, and rapid response.

[0066] "Electronic messages" refer to digital communications, including text and attachments, that are sent and received via the internet or other networks.

[0067] "Information processing means" refers to programs and algorithms designed to achieve a specific purpose on computers or servers.

[0068] "Meaning analysis" refers to the process of understanding the content and intent of text using natural language processing techniques, and then classifying and judging it.

[0069] "Classifying" means grouping information and dividing it into categories based on predetermined criteria.

[0070] "Information condensation" means extracting the most important content from the original information, shortening it, and expressing it in a summarized form.

[0071] "Setting priorities" means determining the order of tasks or items based on their importance and urgency according to some criteria.

[0072] A "reply template" is a pre-prepared set of sentences that can be used as a response to an electronic message without having to rewrite them.

[0073] "Notifying a user of an alert" means sending an alert or notification to inform them of the importance or urgency of a certain matter.

[0074] A "communication information management device" is a system device or application for efficiently organizing, classifying, and managing received communications.

[0075] The communication information management device according to this invention is a system in which a server and a terminal work together to efficiently manage received electronic messages. Specifically, the server first receives electronic messages via the internet. The received messages are analyzed using software for natural language processing, such as a generative AI model. This model utilizes open-source natural language processing tools or general generative AI models to understand the received data and analyze the intent and content of the message.

[0076] Based on the analysis results, the server categorizes electronic messages into categories such as "important," "normal," and "low priority." Furthermore, it evaluates the importance and urgency of each message and sets priorities according to specific rules. This process takes into account past communication history and pre-configured rules, and the AI ​​model supports classification and evaluation using appropriate prompt statements.

[0077] The server sends the returned information and priority information to the terminal. The terminal receives this information and displays it on the interface in a format accessible to the user. The UI is customized as needed so that high-priority messages stand out.

[0078] Users can view a summary of the message through their terminal and review the details as needed. If a reply is required, the server generates a reply template based on predefined templates and suggests it to the user. The user can review the template and use it as is, or edit it to send a reply.

[0079] Furthermore, the server has a function to periodically alert users to unprocessed messages. This function includes a reminder feature linked to the device, which, for example, issues a notification if a message has not been processed within a specified time.

[0080] As a concrete example, consider a scenario where a user receives a large volume of electronic messages daily. This system ensures that important requests from superiors are handled quickly. The server automatically generates reply templates, allowing users to efficiently submit progress reports.

[0081] Examples of prompts for generative AI models:

[0082] "Categorize the new messages and evaluate their priority."

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The server receives new electronic messages from the mail server. The input consists of the electronic message and its metadata (sender, recipient, date and time, etc.). The server receives this and stores it in a temporary database in preparation for the next parsing process.

[0086] Step 2:

[0087] The server uses a generative AI model to analyze the content of received messages. The input is the body of the electronic message. The AI ​​model uses prompts to analyze the content and determine the message's theme, importance, and necessary actions. The output is the analysis result, including recommended categories and action information.

[0088] Step 3:

[0089] The server categorizes electronic messages based on the analysis results. The input is the analysis results from step 2. The server uses pre-configured rules to classify messages into categories such as "Important," "Normal," and "Low Priority." The output is the classified category information.

[0090] Step 4:

[0091] The server generates summaries and prioritizes each message. The inputs are the content of the electronic messages and the classification results from step 3. A generative AI model is used to summarize the content and assess importance based on known information. The output is the summary and prioritization information.

[0092] Step 5:

[0093] The server sends organized message data to the terminal. The input is summarized and prioritized message information. By sending this to the terminal, the terminal can display the information to the user. The output is the terminal's display interface.

[0094] Step 6:

[0095] The terminal displays messages based on the information it receives, highlighting those with higher priority. The input is data sent from the server. The terminal sorts this data according to priority and presents it visually to the user. The output is an organized message list in the user interface.

[0096] Step 7:

[0097] Users can instantly access important messages through their terminal and view details if necessary. The input is a summary message displayed on the terminal's interface. Based on the displayed information, the user selects an action and opens details as needed. The output is the detailed information viewed by the user and the action selected.

[0098] Step 8:

[0099] The server generates a reply template for messages that require a response. The input consists of the message information requiring a response and past communication history data. The AI ​​model generates an appropriate reply template based on this information. The output is the reply template.

[0100] Step 9:

[0101] The terminal presents the user with a generated reply template and supports editing and sending as needed. The input is template information sent from the server. The user can review and edit the reply based on this. The output is the reply finalized by the user.

[0102] Step 10:

[0103] The server periodically issues reminders for important messages that have not yet been processed. The input is information about the unprocessed messages and their priority. The server creates a reminder notification at the specified time and sends it to the device. The output is a notification alert serving as a reminder.

[0104] (Application Example 1)

[0105] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0106] In the business environment, companies receive a large volume of emails and messages daily. This leads to challenges such as delays in reviewing and responding to important messages, resulting in decreased operational efficiency. Furthermore, there is a risk of missed opportunities due to overlooking crucial information. In this context, there is a need for technology that enables the rapid and accurate management and processing of email information.

[0107] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0108] In this invention, the server includes means for receiving email information and analyzing its content using natural language processing techniques; means for automatically classifying emails into different categories based on the analysis results; means for automatically summarizing the content of each email; means for determining priority based on the importance and deadline of the emails; means for generating templates for creating replies to information that requires a response; means for providing a function to notify users of important unprocessed emails; and means for efficiently processing payment confirmations and customer inquiries via smart devices. This enables the rapid and efficient management of email information.

[0109] "Email information" refers to all electronic documents, including text and data, that are transmitted via electronic communication networks and presented to recipients.

[0110] "Natural language processing techniques" refer to technologies that enable computers to understand and analyze the language that humans use on a daily basis.

[0111] "Analysis results" refer to structured data and insights obtained after analyzing email information using natural language processing techniques.

[0112] "Classification" refers to the process of grouping data based on specific criteria or attributes.

[0113] "Summarization" refers to the act of extracting the essential parts from the original information and expressing them in a concise manner.

[0114] "Priority" refers to the order in which tasks or actions are assigned to indicate their importance or urgency.

[0115] A "reply template" refers to a pre-prepared text format designed to allow for quick and efficient responses to specific inquiries.

[0116] A "notification function" refers to a mechanism that presents warnings or information to prompt users to take action when certain conditions are met.

[0117] A "smart device" refers to a portable device that has internet connectivity and advanced features that allow it to run various applications.

[0118] "Payment confirmation" refers to the actions or processes taken to verify that a transaction has been successfully completed and that the money has been transferred correctly.

[0119] "Customer inquiry processing" refers to a series of activities aimed at providing appropriate information and resolving problems in response to questions and requests from customers.

[0120] The system implementing this invention consists of a server and a smart device, efficiently managing email information and supporting the user's work. The server retrieves email information from a mail server and analyzes the email content using natural language processing techniques. This is done using Python and natural language processing libraries (e.g., spaCy, NLTK). Next, based on the analysis results, the server automatically divides the emails into multiple categories. Machine learning algorithms are employed for classification, analyzing the characteristics of the email information and assigning appropriate tags.

[0121] The server further summarizes the content of each email, simplifying the essence of the information. This uses summarization technology to extract only the important points. Based on the importance and deadline of the email information, the server prioritizes the emails and sends the results to the smart device. The smart device then displays the received data in a user interface in a format that is easy for the user to understand.

[0122] For information requiring a response, the server utilizes pre-prepared templates to generate a template for creating a reply. These templates are generated from past contact history and registered formats, enabling quick and efficient responses. Furthermore, the server uses a notification function to alert users if important emails remain unprocessed. Smart devices provide users with relevant information in real time to facilitate smoother processing of important payment confirmations and customer inquiries.

[0123] As an example of its use, an accounting staff member can check an "overdue payment notification" email on their smart device and quickly complete a reply using a suggested reply template from the server. A specific example of a prompt message would be: "Analyze the following email and select the appropriate category based on its content. Furthermore, if a reply is required, generate an automated template." In this way, the entire business process can be streamlined.

[0124] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0125] Step 1:

[0126] The server retrieves new email information from the mail server. The input requires access information to the mail server. The server retrieves the email information and outputs it as text data.

[0127] Step 2:

[0128] The server analyzes the content of acquired email information using natural language processing techniques. It receives text data as input and performs analysis via a generative AI model. Through data analysis, it extracts important keywords and phrases from emails and outputs structured data. Specifically, it utilizes natural language processing libraries (e.g., spaCy, NLTK) to understand the context and extract necessary information.

[0129] Step 3:

[0130] The server automatically sorts emails into different categories based on the analysis results. It uses structured data as input. The server outputs email data that has been analyzed using a machine learning algorithm and assigned categories (e.g., "payment confirmation," "customer inquiry," etc.).

[0131] Step 4:

[0132] The server summarizes each email, extracting only the essential information to create a short summary. It consumes categorized email data as input. Through the summarization process, it outputs a concise summary.

[0133] Step 5:

[0134] The server prioritizes emails based on their importance and deadline. It uses summaries and email attribute information as input. The server performs calculations based on this information and outputs prioritized email data.

[0135] Step 6:

[0136] The terminal displays prioritized email data received from the server on the user interface. It takes prioritized email data as input, formats it for easy user understanding, and outputs it.

[0137] Step 7:

[0138] The server generates a reply template for emails it determines require a response. It uses the email content and past communication history as input. A template generation algorithm is used to automatically output the template.

[0139] Step 8:

[0140] The server periodically sends notifications about important unprocessed emails. It uses email status information and importance level as input to generate and output alert data. Specifically, it sends reminders to smart devices.

[0141] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0142] This invention combines an emotion engine with an email management system to achieve efficient and personalized email management that takes user emotions into account. When the server receives user email data, it analyzes the content using natural language processing. This analysis automatically classifies emails into categories such as "Notification," "Action Required," and "Reply Required."

[0143] Furthermore, the server uses an emotion engine to recognize the user's emotions from the email body. Emotion recognition is based on specific vocabulary, writing style, and the sender's past emotional tendencies contained in the email. The recognized emotion information is used to prioritize emails. For example, an email that is urgent and contains an emotionally unstable tone may be given a higher priority.

[0144] The server also utilizes the sentiment engine when generating reply templates. By analyzing the sentiment history of past email replies and reflecting the appropriate tone in the reply template based on that analysis, it provides reply content that matches the user's communication style.

[0145] The device displays received email information on the user interface based on priority. Users can view a list of emails that reflect sentiment information, enabling them to respond quickly and appropriately to important emails. In addition, the reminder notification function uses sentiment information to determine the most appropriate notification timing and send notifications to the device.

[0146] For example, if a user receives a "product trouble report" email from a customer that indicates heightened emotions, this email is immediately set to high priority. Furthermore, the server provides an appropriate reply template based on the emotions expressed, allowing the user to quickly respond appropriately to the situation. This system is expected to improve the efficiency of email management and enhance business performance.

[0147] The following describes the processing flow.

[0148] Step 1:

[0149] The server periodically retrieves new emails from the user's mail server and stores the email metadata and body in a database. During retrieval, it analyzes basic information, including the email sender and subject.

[0150] Step 2:

[0151] The server uses a natural language processing engine to analyze the email body. Based on the analysis, the email is automatically categorized into one of the following categories: "Notification," "Action Required," or "Reply Required."

[0152] Step 3:

[0153] The server utilizes an emotion engine to recognize emotions from the email body. Specifically, it identifies emotional expressions within the text and calculates an emotion score based on them.

[0154] Step 4:

[0155] The server determines the priority of emails by considering recognized sentiment information and other importance indicators (e.g., the importance of the sender and the urgency of the subject). This priority is used to determine the order in which emails are processed.

[0156] Step 5:

[0157] The terminal sorts the email list by priority based on classification information, sentiment information, and priority received from the server, and displays it on the user interface. This allows the user to quickly review their emails.

[0158] Step 6:

[0159] Users can view summaries and sentiment information of displayed emails on their devices, and open the emails if they wish to view more detailed information. This allows them to efficiently grasp important content.

[0160] Step 7:

[0161] For emails classified as requiring a reply, the server automatically generates a reply template with an appropriate tone using an emotion engine. The template is then customized based on past reply history and emotion information.

[0162] Step 8:

[0163] The device presents the user with a generated reply template, which the user can then edit and quickly submit the necessary reply.

[0164] Step 9:

[0165] The server monitors the database and issues reminders for important unprocessed emails at the optimal time based on sentiment.

[0166] Step 10:

[0167] Users receive emotion-based reminder notifications through their devices, prompting them to take action. This ensures timely responses to important emails.

[0168] (Example 2)

[0169] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0170] In email management, users are required to efficiently categorize large volumes of emails and respond appropriately. However, current systems lack the ability to consider the emotional state of emails or prioritize them based on importance, leading to challenges such as decreased user efficiency. Furthermore, there is a lack of mechanisms to generate personalized replies using past communication history.

[0171] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0172] In this invention, the server includes means for receiving email information and analyzing its content using natural language processing; means for automatically classifying the information into different types based on the analysis results; means for identifying the emotional state of the information using an emotion analysis function; means for setting priorities based on the identified emotional state and urgency; means for analyzing the previous communication history and generating an appropriate reply format; and means for notifying important unprocessed information. This enables effective prioritization and the generation of personalized replies according to the content and emotional state of emails.

[0173] "Email information" refers to digital documents, including text data and attachments, that are sent or received via electronic communication.

[0174] "Natural language processing" is a technology that enables computers to understand, analyze, and automatically process human language.

[0175] "Sentiment analysis function" refers to algorithms and technologies used to identify emotions and emotional tendencies from text data.

[0176] "Prioritizing" refers to the act of evaluating the necessity of processing or responding to tasks based on specific criteria and determining the order in which they are performed.

[0177] "Communication history" refers to records of past email transmissions and receptions, as well as their content, and is data used for future processing and decision-making.

[0178] "Reply format generation" refers to the process of automatically creating the structure and content of a reply message that should be sent.

[0179] "Notifications" refer to alerts or messages that inform users of relevant information.

[0180] In this embodiment of the invention, the server, terminal, and user functions are integrated to manage email. The respective roles and technologies used in their processing are described below.

[0181] The server first receives email information sent by the user. This information is transmitted as digital data over the internet. On the server, natural language processing (NLP) techniques built with Python or other programming languages ​​are used to analyze the content of the email. Specifically, libraries such as NLTK and spaCy are utilized to extract and classify information from the text. Based on the analysis results, the emails are automatically classified into categories such as "Notification," "Action Required," and "Reply Required," and further processing is performed based on this classification information.

[0182] Next, the server uses sentiment analysis capabilities to analyze the emotional state of received emails. This analysis identifies emotions based on data obtained from the vocabulary and writing style of the email body, as well as past communication history. For sentiment analysis, publicly available sentiment analysis platforms (e.g., IBM Watson® Natural Language Understanding or Microsoft® Azure® Text Analytics) are applied. Using these results, email priorities are determined, with emotionally urgent information being given a higher priority.

[0183] Furthermore, the server utilizes a generative AI model to generate reply templates. Leveraging generative models such as OpenAI®, it executes prompts to automatically create appropriate reply formats. For example, a prompt such as "Generate an appropriate reply to this email" can be used. This process refers to past communication history and provides templates tailored to the user's communication style.

[0184] The terminal receives information from the server, then sorts it based on priority and displays it on the user interface. Users can view the email list on the displayed interface and easily identify emails that require priority. This enables quick responses, and the reminder function prevents important emails from being overlooked.

[0185] For example, when a user receives a "product trouble report email" from a client that indicates heightened emotions, the server classifies this email as high priority and sends an appropriate reply template to the user's device. The user can then quickly respond using the provided reply as a reference.

[0186] In this way, by linking the server and terminal, users can improve the efficiency of email management and enhance their work performance.

[0187] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0188] Step 1:

[0189] The server receives email information from the user. This email information is stored as electronic data on the server's storage via the internet. The input is a new email message received in the user's email account, and the output is the raw email data.

[0190] Step 2:

[0191] The server performs natural language processing on the received email data. Specifically, it analyzes the text data of the email body and subject line to extract keywords and grammatical structures. This is done using libraries such as Python's NLTK and spaCy. The input is the raw email data saved in step 1, and the output is the analyzed text data and its features.

[0192] Step 3:

[0193] The server automatically classifies emails into different categories (e.g., "Notification," "Action Required," "Reply Required") based on the analyzed text data. Machine learning algorithms are commonly used for this classification. The input is the feature data obtained in step 2, and the output is the classified email category.

[0194] Step 4:

[0195] The server uses an emotion analysis engine to identify emotional states from email data. This is done based on specific vocabulary and stylistic patterns. The input is the analysis data from step 2, and the output is an emotional index (e.g., positive, negative, neutral).

[0196] Step 5:

[0197] The server prioritizes emails based on identified emotional state and category information. High priority is given to emails deemed urgent or emotionally important. The input is the output data from steps 3 and 4, and the output is email prioritization information.

[0198] Step 6:

[0199] The server uses a generative AI model to generate appropriate reply templates. The prompt text used as input is something like, "Generate an appropriate reply for this email." The input consists of the sentiment data from step 4 and the required prompt text, and the output is the generated reply template.

[0200] Step 7:

[0201] The terminal receives information from the server and displays emails organized according to priority in the user interface. Through this interface, users can review email content and respond quickly to important emails. Inputs include prioritized email information and reply templates from the server, while output is a list of emails displayed on the screen.

[0202] (Application Example 2)

[0203] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0204] In today's communication society, users have to deal with a large volume of emails daily, some of which contain emotional content. However, conventional email management systems are unable to properly prioritize emails based on emotion recognition, potentially causing important emails to be overlooked. Furthermore, automatically generating reply content tailored to individual user emotions has been difficult. A system is needed to solve these problems.

[0205] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0206] In this invention, the server includes means for receiving email data and analyzing its content using natural language processing; means for automatically classifying emails into different categories based on the analysis results; means for analyzing the emotional state of emails and prioritizing them based on emotional information; means for generating reply templates for messages requiring a reply, taking into account the emotional history; and means for a robot to notify important unprocessed emails of reminders. This enables efficient management that takes into account the emotional information of emails.

[0207] "Email data" refers to a collection of information sent and received between users in electronic communication.

[0208] "Natural language processing" is a technology that uses computers to analyze and understand human language.

[0209] "Analysis results" refer to data that shows the classification and characteristics of information obtained after data processing.

[0210] A "category" refers to a group of emails that are analyzed and classified according to their characteristics and content.

[0211] "Emotional state" refers to the sender's emotions and psychological condition, as identified based on the content and expression of the email.

[0212] "Emotional information" specifically refers to data associated with emotions after they have been extracted.

[0213] "Priority" refers to a relative order set to determine the importance and urgency of processing or responding to tasks.

[0214] "Emotional history" refers to the accumulation of past emotion-related data, which is used to analyze a user's emotional tendencies.

[0215] A "reply template" refers to a document format that has a standard reply content for an email pre-configured.

[0216] A "robot" refers to a mechanical device that has a specific function and operates autonomously or according to a program.

[0217] A "reminder" is a notification or alarm that reminds a user of a specific matter or task.

[0218] The system for implementing this invention can incorporate an emotion engine to enhance efficiency and personalization in email management. First, upon receiving email data, the server analyzes the content using natural language processing (NLP) techniques. This process includes contextual analysis using Spacy and emotion analysis using TextBlob. The analyzed email data is categorized into categories such as "Reply Required," "Action Needed," and "Notification," and emotion information is extracted to identify the emotional state of the email.

[0219] Next, the server prioritizes emails based on the extracted sentiment information. For example, emails containing emotionally unstable tones or those of high urgency are given priority. For emails requiring a reply, a reply template is automatically generated, taking into account the user's past emotional history, ensuring a response that matches the user's communication style.

[0220] By running this system, consumer robots streamline email processing on behalf of users. The robots list important pending emails on the user interface based on priority and inform users of their importance through reminder notifications. This allows users to efficiently review a list of emails that take sentiment into account, enabling them to respond quickly and appropriately.

[0221] For example, if a user receives a "product trouble report" email, the system will set this email as a high priority and suggest a reply template that reflects the user's emotions. In this system, which uses a generative AI model, the prompt "Design a program that analyzes the content of emails received by users based on their emotions and categories and provides advice on how to reply" can be used.

[0222] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0223] Step 1:

[0224] The server receives the user's email data. The received email data includes information such as the sender, subject, and body. Based on this data, the server proceeds to the next analysis step.

[0225] Step 2:

[0226] The server analyzes the content of received email data using natural language processing. Specifically, it uses Spacy to analyze the context of the email body and identify parts of speech and sentence structure. The input is the email body text, and the output is the result of the contextual analysis.

[0227] Step 3:

[0228] The server classifies emails into different categories based on the analysis results. Here, emails are categorized into "Reply Required," "Action Needed," "Notification," etc., based on keywords and context. The input is the result of contextual analysis, and the output is email data categorized accordingly.

[0229] Step 4:

[0230] The server uses TextBlob to perform sentiment analysis on emails and identify their emotional state. In this step, the email body is used again as input, and positive or negative emotions are calculated from the vocabulary and tone, and output as a sentiment score.

[0231] Step 5:

[0232] The server uses sentiment scores and email category information to prioritize each email. Emails deemed to be emotionally unstable and urgent are given a higher priority. The input is sentiment score and category information, and the output is a list of emails with their priorities set.

[0233] Step 6:

[0234] The server considers past sentiment history and generates reply templates for emails that require a response. This step uses past reply history data and the current sentiment score as input, and outputs an appropriate tone and template.

[0235] Step 7:

[0236] The terminal displays a priority-based email list on the user interface. The user can visually review this list and determine which emails should be prioritized. The input is a priority-based email list from the server, and the output is the displayed email list.

[0237] Step 8:

[0238] A robot will notify users of important emails that have not yet been processed. This step uses a list of high-priority emails that have not been dealt with as input, and the output is a reminder notification that appears on the user's device.

[0239] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0240] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0241] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0242] [Second Embodiment]

[0243] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0244] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0245] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0246] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0247] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0248] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0249] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0250] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0251] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0252] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0253] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0254] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0255] The email management system of this invention is realized through the cooperation of a server and a terminal, and provides a function to efficiently manage the user's received emails. When the server receives a new email from the mail server, it first analyzes the content of the email using natural language processing technology. Based on the analysis results, the server classifies the email into categories such as "Notification," "Action Required," and "Reply Required."

[0256] Next, the server summarizes the content of each email and prioritizes them based on importance and deadlines. This allows the user interface to quickly identify the most important emails. The terminal then sorts and displays the email list by priority based on the information received from the server. Through the terminal, the user can review the email summary and access the detailed content as needed.

[0257] Furthermore, for emails that the server determines require a reply, it automatically generates a reply template based on pre-registered reply history and template information. The terminal presents this to the user, who can edit it as needed and easily send a reply. The server also has a function to periodically issue reminders for important unprocessed emails. This function allows the terminal to notify the user through a designated communication tool, preventing delays in response.

[0258] As a concrete example, among the numerous emails a user receives via the server, "project progress report requests" from their supervisor are classified with high priority, and timely notifications are sent via a reminder function as time passes. In addition, a reply template generation function automatically generates a reply template for the "progress report" and presents it to the user. This process prevents important emails from being overlooked and enables more efficient work.

[0259] The features described above allow users to significantly reduce the time spent on email management, enabling them to focus on other important tasks. This system is particularly useful for users who need to handle a large volume of emails daily, especially in a business environment.

[0260] The following describes the processing flow.

[0261] Step 1:

[0262] The server accesses the user's mail server and periodically retrieves new mail data. In this process, it analyzes the mail metadata (sender, received date and time, subject) and stores it in a database.

[0263] Step 2:

[0264] The server passes the body of the retrieved email data to a natural language processing engine for text analysis. Based on the analysis results, the emails are categorized as "Notification," "Action Required," or "Reply Required."

[0265] Step 3:

[0266] The server uses the analysis results to generate a summary from the email content. In this process, it extracts key keywords and context to create a shortened summary.

[0267] Step 4:

[0268] The server determines the importance of an email by scoring the sender's reputation and the urgency of the email. Prioritization is set based on deadline information and the presence of important keywords.

[0269] Step 5:

[0270] The terminal uses the summary, classification, and priority information of emails received from the server to display a list of emails in the user interface. At this time, the emails are sorted and displayed to the user in descending order of priority.

[0271] Step 6:

[0272] Users can view emails displayed with high priority on the device's interface and open the detailed contents of emails as needed. This enables them to quickly check and respond to important emails.

[0273] Step 7:

[0274] The server automatically generates a reply template for emails that require a response. It creates an optimal template based on past email reply histories and pre-set phrases.

[0275] Step 8:

[0276] The terminal presents the generated reply template to the user, and the user can quickly reply by editing it.

[0277] Step 9:

[0278] The server monitors unprocessed important emails in real-time and issues a reminder if the reply is delayed. This reminder is notified to the user via a specified communication tool.

[0279] Step 10:

[0280] By receiving the reminder notification through the terminal, the user can ensure that they do not forget to respond to important emails, thereby improving the user's work efficiency.

[0281] (Example 1)

[0282] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 2l4 are referred to as the "terminal".

[0283] In a modern business environment, the amount of electronic messages received by users daily is increasing, and it is required to manage important messages quickly and efficiently. In particular, overlooking important messages, invalidation of prevention measures, and difficulty in timely replies are cited. Unless these issues are resolved, the productivity of users will decline, and the risk of missing important business opportunities will increase.

[0284] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0285] In this invention, the server includes information processing means for receiving an electronic message and analyzing its meaning, information processing means for classifying the electronic message into a plurality of categories based on the analysis result, and information processing means for condensing the information of each electronic message. As a result, efficient management of electronic messages, prevention of overlooking important information, and prompt response become possible.

[0286] An "electronic message" refers to digital communication including text and attached files transmitted and received through the Internet or other networks.

[0287] "Information processing means" refers to programs or algorithms designed to achieve a specific purpose on a computer or server.

[0288] "Analyzing the meaning" refers to a process of understanding the content and intention of text using natural language processing technology and performing classification and judgment.

[0289] "Classifying" means grouping information based on predetermined criteria and dividing it into categories.

[0290] "Condensing information" means extracting important content from the original information, shortening it, and expressing it in a summary form.

[0291] "Setting priorities" means determining an order according to the importance and urgency of tasks or items based on certain criteria.

[0292] A "reply template" refers to a template that can be used as it is without rewriting a series of prepared sentences as a reply to an electronic message.

[0293] "Notifying to arouse attention" means sending alerts or notifications to inform users of the importance and urgency of certain matters.

[0294] A "communication information management device" is a system device or application for efficiently organizing, classifying, and managing received communications.

[0295] The communication information management device according to this invention is a system in which a server and a terminal work together to efficiently manage received electronic messages. Specifically, the server first receives electronic messages via the internet. The received messages are analyzed using software for natural language processing, such as a generative AI model. This model utilizes open-source natural language processing tools or general generative AI models to understand the received data and analyze the intent and content of the message.

[0296] Based on the analysis results, the server categorizes electronic messages into categories such as "important," "normal," and "low priority." Furthermore, it evaluates the importance and urgency of each message and sets priorities according to specific rules. This process takes into account past communication history and pre-configured rules, and the AI ​​model supports classification and evaluation using appropriate prompt statements.

[0297] The server sends the returned information and priority information to the terminal. The terminal receives this information and displays it on the interface in a format accessible to the user. The UI is customized as needed so that high-priority messages stand out.

[0298] Users can view a summary of the message through their terminal and review the details as needed. If a reply is required, the server generates a reply template based on predefined templates and suggests it to the user. The user can review the template and use it as is, or edit it to send a reply.

[0299] Furthermore, the server has a function to periodically alert users to unprocessed messages. This function includes a reminder feature linked to the device, which, for example, issues a notification if a message has not been processed within a specified time.

[0300] As a specific example, assume a scenario where a user receives a large number of electronic messages every day. With this system, important requests from superiors can be quickly addressed. Using the reply template automatically generated by the server, the user can efficiently provide progress reports.

[0301] Example of a prompt sentence for the generation AI model:

[0302] "Please classify the category of the new message and evaluate its priority."

[0303] The flow of the specific process in Example 1 will be described using FIG. 11.

[0304] Step 1:

[0305] The server receives new electronic messages from the mail server. The input is the electronic message and its metadata (sender, recipient, date and time, etc.). The server receives this and stores it in a temporary database to prepare for the following analysis process.

[0306] Step 2:

[0307] The server analyzes the content of the received message using the generation AI model. The input is the body of the electronic message. The AI model analyzes the content using the prompt sentence and determines the theme, importance, and necessary actions of the message. The output is the recommended category and action information as the analysis result.

[0308] Step 3:

[0309] The server classifies the electronic message into categories based on the analysis result. The input is the analysis result of Step 2. The server uses pre-set rules to classify the message into categories such as "important", "normal", "low priority", etc. The output is the classified category information.

[0310] Step 4:

[0311] The server generates summaries and prioritizes each message. The inputs are the content of the electronic messages and the classification results from step 3. A generative AI model is used to summarize the content and assess importance based on known information. The output is the summary and prioritization information.

[0312] Step 5:

[0313] The server sends organized message data to the terminal. The input is summarized and prioritized message information. By sending this to the terminal, the terminal can display the information to the user. The output is the terminal's display interface.

[0314] Step 6:

[0315] The terminal displays messages based on the information it receives, highlighting those with higher priority. The input is data sent from the server. The terminal sorts this data according to priority and presents it visually to the user. The output is an organized message list in the user interface.

[0316] Step 7:

[0317] Users can instantly access important messages through their terminal and view details if necessary. The input is a summary message displayed on the terminal's interface. Based on the displayed information, the user selects an action and opens details as needed. The output is the detailed information viewed by the user and the action selected.

[0318] Step 8:

[0319] The server generates a reply template for messages that require a response. The input consists of the message information requiring a response and past communication history data. The AI ​​model generates an appropriate reply template based on this information. The output is the reply template.

[0320] Step 9:

[0321] The terminal presents the user with a generated reply template and supports editing and sending as needed. The input is template information sent from the server. The user can review and edit the reply based on this. The output is the reply finalized by the user.

[0322] Step 10:

[0323] The server periodically issues reminders for important messages that have not yet been processed. The input is information about the unprocessed messages and their priority. The server creates a reminder notification at the specified time and sends it to the device. The output is a notification alert serving as a reminder.

[0324] (Application Example 1)

[0325] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0326] In the business environment, companies receive a large volume of emails and messages daily. This leads to challenges such as delays in reviewing and responding to important messages, resulting in decreased operational efficiency. Furthermore, there is a risk of missed opportunities due to overlooking crucial information. In this context, there is a need for technology that enables the rapid and accurate management and processing of email information.

[0327] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0328] In this invention, the server includes means for receiving email information and analyzing its content using natural language processing techniques; means for automatically classifying emails into different categories based on the analysis results; means for automatically summarizing the content of each email; means for determining priority based on the importance and deadline of the emails; means for generating templates for creating replies to information that requires a response; means for providing a function to notify users of important unprocessed emails; and means for efficiently processing payment confirmations and customer inquiries via smart devices. This enables the rapid and efficient management of email information.

[0329] "Email information" refers to all electronic documents, including text and data, that are transmitted via electronic communication networks and presented to recipients.

[0330] "Natural language processing techniques" refer to technologies that enable computers to understand and analyze the language that humans use on a daily basis.

[0331] "Analysis results" refer to structured data and insights obtained after analyzing email information using natural language processing techniques.

[0332] "Classification" refers to the process of grouping data based on specific criteria or attributes.

[0333] "Summarization" refers to the act of extracting the essential parts from the original information and expressing them in a concise manner.

[0334] "Priority" refers to the order in which tasks or actions are assigned to indicate their importance or urgency.

[0335] A "reply template" refers to a pre-prepared text format designed to allow for quick and efficient responses to specific inquiries.

[0336] A "notification function" refers to a mechanism that presents warnings or information to prompt users to take action when certain conditions are met.

[0337] A "smart device" refers to a portable device that has internet connectivity and advanced features that allow it to run various applications.

[0338] "Payment confirmation" refers to the actions or processes taken to verify that a transaction has been successfully completed and that the money has been transferred correctly.

[0339] "Customer inquiry processing" refers to a series of activities aimed at providing appropriate information and resolving problems in response to questions and requests from customers.

[0340] The system implementing this invention consists of a server and a smart device, efficiently managing email information and supporting the user's work. The server retrieves email information from a mail server and analyzes the email content using natural language processing techniques. This is done using Python and natural language processing libraries (e.g., spaCy, NLTK). Next, based on the analysis results, the server automatically divides the emails into multiple categories. Machine learning algorithms are employed for classification, analyzing the characteristics of the email information and assigning appropriate tags.

[0341] The server further summarizes the content of each email, simplifying the essence of the information. This uses summarization technology to extract only the important points. Based on the importance and deadline of the email information, the server prioritizes the emails and sends the results to the smart device. The smart device then displays the received data in a user interface in a format that is easy for the user to understand.

[0342] For information requiring a response, the server utilizes pre-prepared templates to generate a template for creating a reply. These templates are generated from past contact history and registered formats, enabling quick and efficient responses. Furthermore, the server uses a notification function to alert users if important emails remain unprocessed. Smart devices provide users with relevant information in real time to facilitate smoother processing of important payment confirmations and customer inquiries.

[0343] As an example of its use, an accounting staff member can check an "overdue payment notification" email on their smart device and quickly complete a reply using a suggested reply template from the server. A specific example of a prompt message would be: "Analyze the following email and select the appropriate category based on its content. Furthermore, if a reply is required, generate an automated template." In this way, the entire business process can be streamlined.

[0344] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0345] Step 1:

[0346] The server retrieves new email information from the mail server. The input requires access information to the mail server. The server retrieves the email information and outputs it as text data.

[0347] Step 2:

[0348] The server analyzes the content of acquired email information using natural language processing techniques. It receives text data as input and performs analysis via a generative AI model. Through data analysis, it extracts important keywords and phrases from emails and outputs structured data. Specifically, it utilizes natural language processing libraries (e.g., spaCy, NLTK) to understand the context and extract necessary information.

[0349] Step 3:

[0350] The server automatically sorts emails into different categories based on the analysis results. It uses structured data as input. The server outputs email data that has been analyzed using a machine learning algorithm and assigned categories (e.g., "payment confirmation," "customer inquiry," etc.).

[0351] Step 4:

[0352] The server summarizes each email, extracting only the essential information to create a short summary. It consumes categorized email data as input. Through the summarization process, it outputs a concise summary.

[0353] Step 5:

[0354] The server prioritizes emails based on their importance and deadline. It uses summaries and email attribute information as input. The server performs calculations based on this information and outputs prioritized email data.

[0355] Step 6:

[0356] The terminal displays prioritized email data received from the server on the user interface. It takes prioritized email data as input, formats it for easy user understanding, and outputs it.

[0357] Step 7:

[0358] The server generates a reply template for emails it determines require a response. It uses the email content and past communication history as input. A template generation algorithm is used to automatically output the template.

[0359] Step 8:

[0360] The server periodically sends notifications about important unprocessed emails. It uses email status information and importance level as input to generate and output alert data. Specifically, it sends reminders to smart devices.

[0361] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0362] This invention combines an emotion engine with an email management system to achieve efficient and personalized email management that takes user emotions into account. When the server receives user email data, it analyzes the content using natural language processing. This analysis automatically classifies emails into categories such as "Notification," "Action Required," and "Reply Required."

[0363] Furthermore, the server uses an emotion engine to recognize the user's emotions from the email body. Emotion recognition is based on specific vocabulary, writing style, and the sender's past emotional tendencies contained in the email. The recognized emotion information is used to prioritize emails. For example, an email that is urgent and contains an emotionally unstable tone may be given a higher priority.

[0364] The server also utilizes the sentiment engine when generating reply templates. By analyzing the sentiment history of past email replies and reflecting the appropriate tone in the reply template based on that analysis, it provides reply content that matches the user's communication style.

[0365] The device displays received email information on the user interface based on priority. Users can view a list of emails that reflect sentiment information, enabling them to respond quickly and appropriately to important emails. In addition, the reminder notification function uses sentiment information to determine the most appropriate notification timing and send notifications to the device.

[0366] For example, if a user receives a "product trouble report" email from a customer that indicates heightened emotions, this email is immediately set to high priority. Furthermore, the server provides an appropriate reply template based on the emotions expressed, allowing the user to quickly respond appropriately to the situation. This system is expected to improve the efficiency of email management and enhance business performance.

[0367] The following describes the processing flow.

[0368] Step 1:

[0369] The server periodically retrieves new emails from the user's mail server and stores the email metadata and body in a database. During retrieval, it analyzes basic information, including the email sender and subject.

[0370] Step 2:

[0371] The server uses a natural language processing engine to analyze the email body. Based on the analysis, the email is automatically categorized into one of the following categories: "Notification," "Action Required," or "Reply Required."

[0372] Step 3:

[0373] The server utilizes an emotion engine to recognize emotions from the email body. Specifically, it identifies emotional expressions within the text and calculates an emotion score based on them.

[0374] Step 4:

[0375] The server determines the priority of emails by considering recognized sentiment information and other importance indicators (e.g., the importance of the sender and the urgency of the subject). This priority is used to determine the order in which emails are processed.

[0376] Step 5:

[0377] The terminal sorts the email list by priority based on classification information, sentiment information, and priority received from the server, and displays it on the user interface. This allows the user to quickly review their emails.

[0378] Step 6:

[0379] Users can view summaries and sentiment information of displayed emails on their devices, and open the emails if they wish to view more detailed information. This allows them to efficiently grasp important content.

[0380] Step 7:

[0381] For emails classified as requiring a reply, the server automatically generates a reply template with an appropriate tone using an emotion engine. The template is then customized based on past reply history and emotion information.

[0382] Step 8:

[0383] The device presents the user with a generated reply template, which the user can then edit and quickly submit the necessary reply.

[0384] Step 9:

[0385] The server monitors the database and issues reminders for important unprocessed emails at the optimal time based on sentiment.

[0386] Step 10:

[0387] Users receive emotion-based reminder notifications through their devices, prompting them to take action. This ensures timely responses to important emails.

[0388] (Example 2)

[0389] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0390] In email management, users are required to efficiently categorize large volumes of emails and respond appropriately. However, current systems lack the ability to consider the emotional state of emails or prioritize them based on importance, leading to challenges such as decreased user efficiency. Furthermore, there is a lack of mechanisms to generate personalized replies using past communication history.

[0391] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0392] In this invention, the server includes means for receiving email information and analyzing its content using natural language processing; means for automatically classifying the information into different types based on the analysis results; means for identifying the emotional state of the information using an emotion analysis function; means for setting priorities based on the identified emotional state and urgency; means for analyzing the previous communication history and generating an appropriate reply format; and means for notifying important unprocessed information. This enables effective prioritization and the generation of personalized replies according to the content and emotional state of emails.

[0393] "Email information" refers to digital documents, including text data and attachments, that are sent or received via electronic communication.

[0394] "Natural language processing" is a technology that enables computers to understand, analyze, and automatically process human language.

[0395] "Sentiment analysis function" refers to algorithms and technologies used to identify emotions and emotional tendencies from text data.

[0396] "Prioritizing" refers to the act of evaluating the necessity of processing or responding to tasks based on specific criteria and determining the order in which they are performed.

[0397] "Communication history" refers to records of past email transmissions and receptions, as well as their content, and is data used for future processing and decision-making.

[0398] "Reply format generation" refers to the process of automatically creating the structure and content of a reply message that should be sent.

[0399] "Notifications" refer to alerts or messages that inform users of relevant information.

[0400] In this embodiment of the invention, the server, terminal, and user functions are integrated to manage email. The respective roles and technologies used in their processing are described below.

[0401] The server first receives email information sent by the user. This information is transmitted as digital data over the internet. On the server, natural language processing (NLP) techniques built with Python or other programming languages ​​are used to analyze the content of the email. Specifically, libraries such as NLTK and spaCy are utilized to extract and classify information from the text. Based on the analysis results, the emails are automatically classified into categories such as "Notification," "Action Required," and "Reply Required," and further processing is performed based on this classification information.

[0402] Next, the server uses sentiment analysis capabilities to analyze the emotional state of received emails. This analysis identifies emotions based on data obtained from the vocabulary and writing style of the email body, as well as past communication history. For sentiment analysis, publicly available sentiment analysis platforms (e.g., IBM Watson Natural Language Understanding or Microsoft Azure Text Analytics) are applied. Using these results, email priorities are determined, with emotionally urgent information being given a higher priority.

[0403] Furthermore, the server utilizes a generative AI model to generate reply templates. Leveraging OpenAI's generative model and other tools, it executes prompts to automatically create appropriate reply formats. For example, a prompt such as "Generate an appropriate reply to this email" can be used. This process refers to past communication history and provides templates tailored to the user's communication style.

[0404] The terminal receives information from the server, then sorts it based on priority and displays it on the user interface. Users can view the email list on the displayed interface and easily identify emails that require priority. This enables quick responses, and the reminder function prevents important emails from being overlooked.

[0405] For example, when a user receives a "product trouble report email" from a client that indicates heightened emotions, the server classifies this email as high priority and sends an appropriate reply template to the user's device. The user can then quickly respond using the provided reply as a reference.

[0406] In this way, by linking the server and terminal, users can improve the efficiency of email management and enhance their work performance.

[0407] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0408] Step 1:

[0409] The server receives email information from the user. This email information is stored as electronic data on the server's storage via the internet. The input is a new email message received in the user's email account, and the output is the raw email data.

[0410] Step 2:

[0411] The server performs natural language processing on the received email data. Specifically, it analyzes the text data of the email body and subject line to extract keywords and grammatical structures. This is done using libraries such as Python's NLTK and spaCy. The input is the raw email data saved in step 1, and the output is the analyzed text data and its features.

[0412] Step 3:

[0413] The server automatically classifies emails into different categories (e.g., "Notification," "Action Required," "Reply Required") based on the analyzed text data. Machine learning algorithms are commonly used for this classification. The input is the feature data obtained in step 2, and the output is the classified email category.

[0414] Step 4:

[0415] The server uses an emotion analysis engine to identify emotional states from email data. This is done based on specific vocabulary and stylistic patterns. The input is the analysis data from step 2, and the output is an emotional index (e.g., positive, negative, neutral).

[0416] Step 5:

[0417] The server prioritizes emails based on identified emotional state and category information. High priority is given to emails deemed urgent or emotionally important. The input is the output data from steps 3 and 4, and the output is email prioritization information.

[0418] Step 6:

[0419] The server uses a generative AI model to generate appropriate reply templates. The prompt text used as input is something like, "Generate an appropriate reply for this email." The input consists of the sentiment data from step 4 and the required prompt text, and the output is the generated reply template.

[0420] Step 7:

[0421] The terminal receives information from the server and displays emails organized according to priority in the user interface. Through this interface, users can review email content and respond quickly to important emails. Inputs include prioritized email information and reply templates from the server, while output is a list of emails displayed on the screen.

[0422] (Application Example 2)

[0423] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0424] In today's communication society, users have to deal with a large volume of emails daily, some of which contain emotional content. However, conventional email management systems are unable to properly prioritize emails based on emotion recognition, potentially causing important emails to be overlooked. Furthermore, automatically generating reply content tailored to individual user emotions has been difficult. A system is needed to solve these problems.

[0425] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0426] In this invention, the server includes means for receiving email data and analyzing its content using natural language processing; means for automatically classifying emails into different categories based on the analysis results; means for analyzing the emotional state of emails and prioritizing them based on emotional information; means for generating reply templates for messages requiring a reply, taking into account the emotional history; and means for a robot to notify important unprocessed emails of reminders. This enables efficient management that takes into account the emotional information of emails.

[0427] "Email data" refers to a collection of information sent and received between users in electronic communication.

[0428] "Natural language processing" is a technology that uses computers to analyze and understand human language.

[0429] "Analysis results" refer to data that shows the classification and characteristics of information obtained after data processing.

[0430] A "category" refers to a group of emails that are analyzed and classified according to their characteristics and content.

[0431] "Emotional state" refers to the sender's emotions and psychological condition, as identified based on the content and expression of the email.

[0432] "Emotional information" specifically refers to data associated with emotions after they have been extracted.

[0433] "Priority" refers to a relative order set to determine the importance and urgency of processing or responding to tasks.

[0434] "Emotional history" refers to the accumulation of past emotion-related data, which is used to analyze a user's emotional tendencies.

[0435] A "reply template" refers to a document format that has a standard reply content for an email pre-configured.

[0436] A "robot" refers to a mechanical device that has a specific function and operates autonomously or according to a program.

[0437] A "reminder" is a notification or alarm that reminds a user of a specific matter or task.

[0438] The system for implementing this invention can incorporate an emotion engine to enhance efficiency and personalization in email management. First, upon receiving email data, the server analyzes the content using natural language processing (NLP) techniques. This process includes contextual analysis using Spacy and emotion analysis using TextBlob. The analyzed email data is categorized into categories such as "Reply Required," "Action Needed," and "Notification," and emotion information is extracted to identify the emotional state of the email.

[0439] Next, the server prioritizes emails based on the extracted sentiment information. For example, emails containing emotionally unstable tones or those of high urgency are given priority. For emails requiring a reply, a reply template is automatically generated, taking into account the user's past emotional history, ensuring a response that matches the user's communication style.

[0440] By running this system, consumer robots streamline email processing on behalf of users. The robots list important pending emails on the user interface based on priority and inform users of their importance through reminder notifications. This allows users to efficiently review a list of emails that take sentiment into account, enabling them to respond quickly and appropriately.

[0441] For example, if a user receives a "product trouble report" email, the system will set this email as a high priority and suggest a reply template that reflects the user's emotions. In this system, which uses a generative AI model, the prompt "Design a program that analyzes the content of emails received by users based on their emotions and categories and provides advice on how to reply" can be used.

[0442] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0443] Step 1:

[0444] The server receives the user's email data. The received email data includes information such as the sender, subject, and body. Based on this data, the server proceeds to the next analysis step.

[0445] Step 2:

[0446] The server analyzes the content of received email data using natural language processing. Specifically, it uses Spacy to analyze the context of the email body and identify parts of speech and sentence structure. The input is the email body text, and the output is the result of the contextual analysis.

[0447] Step 3:

[0448] The server classifies emails into different categories based on the analysis results. Here, emails are categorized into "Reply Required," "Action Needed," "Notification," etc., based on keywords and context. The input is the result of contextual analysis, and the output is email data categorized accordingly.

[0449] Step 4:

[0450] The server uses TextBlob to perform sentiment analysis on emails and identify their emotional state. In this step, the email body is used again as input, and positive or negative emotions are calculated from the vocabulary and tone, and output as a sentiment score.

[0451] Step 5:

[0452] The server uses sentiment scores and email category information to prioritize each email. Emails deemed to be emotionally unstable and urgent are given a higher priority. The input is sentiment score and category information, and the output is a list of emails with their priorities set.

[0453] Step 6:

[0454] The server considers past sentiment history and generates reply templates for emails that require a response. This step uses past reply history data and the current sentiment score as input, and outputs an appropriate tone and template.

[0455] Step 7:

[0456] The terminal displays a priority-based email list on the user interface. The user can visually review this list and determine which emails should be prioritized. The input is a priority-based email list from the server, and the output is the displayed email list.

[0457] Step 8:

[0458] A robot will notify users of important emails that have not yet been processed. This step uses a list of high-priority emails that have not been dealt with as input, and the output is a reminder notification that appears on the user's device.

[0459] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0460] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0461] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0462] [Third Embodiment]

[0463] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0464] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0465] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0466] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0467] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0468] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0469] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0470] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0471] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0472] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0473] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0474] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0475] The email management system of this invention is realized through the cooperation of a server and a terminal, and provides a function to efficiently manage the user's received emails. When the server receives a new email from the mail server, it first analyzes the content of the email using natural language processing technology. Based on the analysis results, the server classifies the email into categories such as "Notification," "Action Required," and "Reply Required."

[0476] Next, the server summarizes the content of each email and prioritizes them based on importance and deadlines. This allows the user interface to quickly identify the most important emails. The terminal then sorts and displays the email list by priority based on the information received from the server. Through the terminal, the user can review the email summary and access the detailed content as needed.

[0477] Furthermore, for emails that the server determines require a reply, it automatically generates a reply template based on pre-registered reply history and template information. The terminal presents this to the user, who can edit it as needed and easily send a reply. The server also has a function to periodically issue reminders for important unprocessed emails. This function allows the terminal to notify the user through a designated communication tool, preventing delays in response.

[0478] As a concrete example, among the numerous emails a user receives via the server, "project progress report requests" from their supervisor are classified with high priority, and timely notifications are sent via a reminder function as time passes. In addition, a reply template generation function automatically generates a reply template for the "progress report" and presents it to the user. This process prevents important emails from being overlooked and enables more efficient work.

[0479] The features described above allow users to significantly reduce the time spent on email management, enabling them to focus on other important tasks. This system is particularly useful for users who need to handle a large volume of emails daily, especially in a business environment.

[0480] The following describes the processing flow.

[0481] Step 1:

[0482] The server accesses the user's mail server and periodically retrieves new mail data. In this process, it analyzes the mail metadata (sender, received date and time, subject) and stores it in a database.

[0483] Step 2:

[0484] The server passes the body of the retrieved email data to a natural language processing engine for text analysis. Based on the analysis results, the emails are categorized as "Notification," "Action Required," or "Reply Required."

[0485] Step 3:

[0486] The server uses the analysis results to generate a summary from the email content. In this process, it extracts key keywords and context to create a shortened summary.

[0487] Step 4:

[0488] The server determines the importance of an email by scoring the sender's reputation and the urgency of the email. Prioritization is set based on deadline information and the presence of important keywords.

[0489] Step 5:

[0490] The terminal uses the summary, classification, and priority information of emails received from the server to display a list of emails in the user interface. At this time, the emails are sorted and displayed to the user in descending order of priority.

[0491] Step 6:

[0492] Users can view emails displayed with high priority on the device's interface and open the detailed contents of emails as needed. This enables them to quickly check and respond to important emails.

[0493] Step 7:

[0494] The server automatically generates reply templates for emails that require a response. It creates the most suitable template based on past email reply history and pre-configured phrases.

[0495] Step 8:

[0496] The device presents the user with a generated reply template, allowing the user to quickly respond by editing it.

[0497] Step 9:

[0498] The server monitors important pending emails in real time and issues reminders if replies are delayed. These reminders are sent to the user via a designated communication tool.

[0499] Step 10:

[0500] Users can receive reminder notifications through their devices, ensuring they don't forget to respond to important emails. This improves their work efficiency.

[0501] (Example 1)

[0502] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0503] In today's business environment, the volume of electronic messages users receive daily is increasing, requiring the rapid and efficient management of important messages. In particular, there are challenges such as overlooking important messages, the invalidation of preventative measures, and the difficulty of responding in a timely manner. Unless these challenges are addressed, user productivity will decline, and the risk of missing important business opportunities will increase.

[0504] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0505] In this invention, the server includes information processing means for receiving and analyzing the meaning of electronic messages, information processing means for classifying electronic messages into multiple categories based on the analysis results, and information processing means for condensing the information of each electronic message. This enables efficient management of electronic messages, prevention of overlooking important information, and rapid response.

[0506] "Electronic messages" refer to digital communications, including text and attachments, that are sent and received via the internet or other networks.

[0507] "Information processing means" refers to programs and algorithms designed to achieve a specific purpose on computers or servers.

[0508] "Meaning analysis" refers to the process of understanding the content and intent of text using natural language processing techniques, and then classifying and judging it.

[0509] "Classifying" means grouping information and dividing it into categories based on predetermined criteria.

[0510] "Information condensation" means extracting the most important content from the original information, shortening it, and expressing it in a summarized form.

[0511] "Setting priorities" means determining the order of tasks or items based on their importance and urgency according to some criteria.

[0512] A "reply template" is a pre-prepared set of sentences that can be used as a response to an electronic message without having to rewrite them.

[0513] "Notifying a user of an alert" means sending an alert or notification to inform them of the importance or urgency of a certain matter.

[0514] A "communication information management device" is a system device or application for efficiently organizing, classifying, and managing received communications.

[0515] The communication information management device according to this invention is a system in which a server and a terminal work together to efficiently manage received electronic messages. Specifically, the server first receives electronic messages via the internet. The received messages are analyzed using software for natural language processing, such as a generative AI model. This model utilizes open-source natural language processing tools or general generative AI models to understand the received data and analyze the intent and content of the message.

[0516] Based on the analysis results, the server categorizes electronic messages into categories such as "important," "normal," and "low priority." Furthermore, it evaluates the importance and urgency of each message and sets priorities according to specific rules. This process takes into account past communication history and pre-configured rules, and the AI ​​model supports classification and evaluation using appropriate prompt statements.

[0517] The server sends the returned information and priority information to the terminal. The terminal receives this information and displays it on the interface in a format accessible to the user. The UI is customized as needed so that high-priority messages stand out.

[0518] Users can view a summary of the message through their terminal and review the details as needed. If a reply is required, the server generates a reply template based on predefined templates and suggests it to the user. The user can review the template and use it as is, or edit it to send a reply.

[0519] Furthermore, the server has a function to periodically alert users to unprocessed messages. This function includes a reminder feature linked to the device, which, for example, issues a notification if a message has not been processed within a specified time.

[0520] As a concrete example, consider a scenario where a user receives a large volume of electronic messages daily. This system ensures that important requests from superiors are handled quickly. The server automatically generates reply templates, allowing users to efficiently submit progress reports.

[0521] Examples of prompts for generative AI models:

[0522] "Categorize the new messages and evaluate their priority."

[0523] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0524] Step 1:

[0525] The server receives new electronic messages from the mail server. The input consists of the electronic message and its metadata (sender, recipient, date and time, etc.). The server receives this and stores it in a temporary database in preparation for the next parsing process.

[0526] Step 2:

[0527] The server uses a generative AI model to analyze the content of received messages. The input is the body of the electronic message. The AI ​​model uses prompts to analyze the content and determine the message's theme, importance, and necessary actions. The output is the analysis result, including recommended categories and action information.

[0528] Step 3:

[0529] The server categorizes electronic messages based on the analysis results. The input is the analysis results from step 2. The server uses pre-configured rules to classify messages into categories such as "Important," "Normal," and "Low Priority." The output is the classified category information.

[0530] Step 4:

[0531] The server generates summaries and prioritizes each message. The inputs are the content of the electronic messages and the classification results from step 3. A generative AI model is used to summarize the content and assess importance based on known information. The output is the summary and prioritization information.

[0532] Step 5:

[0533] The server sends organized message data to the terminal. The input is summarized and prioritized message information. By sending this to the terminal, the terminal can display the information to the user. The output is the terminal's display interface.

[0534] Step 6:

[0535] The terminal displays messages based on the information it receives, highlighting those with higher priority. The input is data sent from the server. The terminal sorts this data according to priority and presents it visually to the user. The output is an organized message list in the user interface.

[0536] Step 7:

[0537] Users can instantly access important messages through their terminal and view details if necessary. The input is a summary message displayed on the terminal's interface. Based on the displayed information, the user selects an action and opens details as needed. The output is the detailed information viewed by the user and the action selected.

[0538] Step 8:

[0539] The server generates a reply template for messages that require a response. The input consists of the message information requiring a response and past communication history data. The AI ​​model generates an appropriate reply template based on this information. The output is the reply template.

[0540] Step 9:

[0541] The terminal presents the user with a generated reply template and supports editing and sending as needed. The input is template information sent from the server. The user can review and edit the reply based on this. The output is the reply finalized by the user.

[0542] Step 10:

[0543] The server periodically issues reminders for important messages that have not yet been processed. The input is information about the unprocessed messages and their priority. The server creates a reminder notification at the specified time and sends it to the device. The output is a notification alert serving as a reminder.

[0544] (Application Example 1)

[0545] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0546] In the business environment, companies receive a large volume of emails and messages daily. This leads to challenges such as delays in reviewing and responding to important messages, resulting in decreased operational efficiency. Furthermore, there is a risk of missed opportunities due to overlooking crucial information. In this context, there is a need for technology that enables the rapid and accurate management and processing of email information.

[0547] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0548] In this invention, the server includes means for receiving email information and analyzing its content using natural language processing techniques; means for automatically classifying emails into different categories based on the analysis results; means for automatically summarizing the content of each email; means for determining priority based on the importance and deadline of the emails; means for generating templates for creating replies to information that requires a response; means for providing a function to notify users of important unprocessed emails; and means for efficiently processing payment confirmations and customer inquiries via smart devices. This enables the rapid and efficient management of email information.

[0549] "Email information" refers to all electronic documents, including text and data, that are transmitted via electronic communication networks and presented to recipients.

[0550] "Natural language processing techniques" refer to technologies that enable computers to understand and analyze the language that humans use on a daily basis.

[0551] "Analysis results" refer to structured data and insights obtained after analyzing email information using natural language processing techniques.

[0552] "Classification" refers to the process of grouping data based on specific criteria or attributes.

[0553] "Summarization" refers to the act of extracting the essential parts from the original information and expressing them in a concise manner.

[0554] "Priority" refers to the order in which tasks or actions are assigned to indicate their importance or urgency.

[0555] A "reply template" refers to a pre-prepared text format designed to allow for quick and efficient responses to specific inquiries.

[0556] A "notification function" refers to a mechanism that presents warnings or information to prompt users to take action when certain conditions are met.

[0557] A "smart device" refers to a portable device that has internet connectivity and advanced features that allow it to run various applications.

[0558] "Payment confirmation" refers to the actions or processes taken to verify that a transaction has been successfully completed and that the money has been transferred correctly.

[0559] "Customer inquiry processing" refers to a series of activities aimed at providing appropriate information and resolving problems in response to questions and requests from customers.

[0560] The system implementing this invention consists of a server and a smart device, efficiently managing email information and supporting the user's work. The server retrieves email information from a mail server and analyzes the email content using natural language processing techniques. This is done using Python and natural language processing libraries (e.g., spaCy, NLTK). Next, based on the analysis results, the server automatically divides the emails into multiple categories. Machine learning algorithms are employed for classification, analyzing the characteristics of the email information and assigning appropriate tags.

[0561] The server further summarizes the content of each email, simplifying the essence of the information. This uses summarization technology to extract only the important points. Based on the importance and deadline of the email information, the server prioritizes the emails and sends the results to the smart device. The smart device then displays the received data in a user interface in a format that is easy for the user to understand.

[0562] For information requiring a response, the server utilizes pre-prepared templates to generate a template for creating a reply. These templates are generated from past contact history and registered formats, enabling quick and efficient responses. Furthermore, the server uses a notification function to alert users if important emails remain unprocessed. Smart devices provide users with relevant information in real time to facilitate smoother processing of important payment confirmations and customer inquiries.

[0563] As an example of its use, an accounting staff member can check an "overdue payment notification" email on their smart device and quickly complete a reply using a suggested reply template from the server. A specific example of a prompt message would be: "Analyze the following email and select the appropriate category based on its content. Furthermore, if a reply is required, generate an automated template." In this way, the entire business process can be streamlined.

[0564] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0565] Step 1:

[0566] The server retrieves new email information from the mail server. The input requires access information to the mail server. The server retrieves the email information and outputs it as text data.

[0567] Step 2:

[0568] The server analyzes the content of acquired email information using natural language processing techniques. It receives text data as input and performs analysis via a generative AI model. Through data analysis, it extracts important keywords and phrases from emails and outputs structured data. Specifically, it utilizes natural language processing libraries (e.g., spaCy, NLTK) to understand the context and extract necessary information.

[0569] Step 3:

[0570] The server automatically sorts emails into different categories based on the analysis results. It uses structured data as input. The server outputs email data that has been analyzed using a machine learning algorithm and assigned categories (e.g., "payment confirmation," "customer inquiry," etc.).

[0571] Step 4:

[0572] The server summarizes each email, extracting only the essential information to create a short summary. It consumes categorized email data as input. Through the summarization process, it outputs a concise summary.

[0573] Step 5:

[0574] The server prioritizes emails based on their importance and deadline. It uses summaries and email attribute information as input. The server performs calculations based on this information and outputs prioritized email data.

[0575] Step 6:

[0576] The terminal displays prioritized email data received from the server on the user interface. It takes prioritized email data as input, formats it for easy user understanding, and outputs it.

[0577] Step 7:

[0578] The server generates a reply template for emails it determines require a response. It uses the email content and past communication history as input. A template generation algorithm is used to automatically output the template.

[0579] Step 8:

[0580] The server periodically sends notifications about important unprocessed emails. It uses email status information and importance level as input to generate and output alert data. Specifically, it sends reminders to smart devices.

[0581] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0582] This invention combines an emotion engine with an email management system to achieve efficient and personalized email management that takes user emotions into account. When the server receives user email data, it analyzes the content using natural language processing. This analysis automatically classifies emails into categories such as "Notification," "Action Required," and "Reply Required."

[0583] Furthermore, the server uses an emotion engine to recognize the user's emotions from the email body. Emotion recognition is based on specific vocabulary, writing style, and the sender's past emotional tendencies contained in the email. The recognized emotion information is used to prioritize emails. For example, an email that is urgent and contains an emotionally unstable tone may be given a higher priority.

[0584] The server also utilizes the sentiment engine when generating reply templates. By analyzing the sentiment history of past email replies and reflecting the appropriate tone in the reply template based on that analysis, it provides reply content that matches the user's communication style.

[0585] The device displays received email information on the user interface based on priority. Users can view a list of emails that reflect sentiment information, enabling them to respond quickly and appropriately to important emails. In addition, the reminder notification function uses sentiment information to determine the most appropriate notification timing and send notifications to the device.

[0586] For example, if a user receives a "product trouble report" email from a customer that indicates heightened emotions, this email is immediately set to high priority. Furthermore, the server provides an appropriate reply template based on the emotions expressed, allowing the user to quickly respond appropriately to the situation. This system is expected to improve the efficiency of email management and enhance business performance.

[0587] The following describes the processing flow.

[0588] Step 1:

[0589] The server periodically retrieves new emails from the user's mail server and stores the email metadata and body in a database. During retrieval, it analyzes basic information, including the email sender and subject.

[0590] Step 2:

[0591] The server uses a natural language processing engine to analyze the email body. Based on the analysis, the email is automatically categorized into one of the following categories: "Notification," "Action Required," or "Reply Required."

[0592] Step 3:

[0593] The server utilizes an emotion engine to recognize emotions from the email body. Specifically, it identifies emotional expressions within the text and calculates an emotion score based on them.

[0594] Step 4:

[0595] The server determines the priority of emails by considering recognized sentiment information and other importance indicators (e.g., the importance of the sender and the urgency of the subject). This priority is used to determine the order in which emails are processed.

[0596] Step 5:

[0597] The terminal sorts the email list by priority based on classification information, sentiment information, and priority received from the server, and displays it on the user interface. This allows the user to quickly review their emails.

[0598] Step 6:

[0599] Users can view summaries and sentiment information of displayed emails on their devices, and open the emails if they wish to view more detailed information. This allows them to efficiently grasp important content.

[0600] Step 7:

[0601] For emails classified as requiring a reply, the server automatically generates a reply template with an appropriate tone using an emotion engine. The template is then customized based on past reply history and emotion information.

[0602] Step 8:

[0603] The device presents the user with a generated reply template, which the user can then edit and quickly submit the necessary reply.

[0604] Step 9:

[0605] The server monitors the database and issues reminders for important unprocessed emails at the optimal time based on sentiment.

[0606] Step 10:

[0607] Users receive emotion-based reminder notifications through their devices, prompting them to take action. This ensures timely responses to important emails.

[0608] (Example 2)

[0609] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0610] In email management, users are required to efficiently categorize large volumes of emails and respond appropriately. However, current systems lack the ability to consider the emotional state of emails or prioritize them based on importance, leading to challenges such as decreased user efficiency. Furthermore, there is a lack of mechanisms to generate personalized replies using past communication history.

[0611] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0612] In this invention, the server includes means for receiving email information and analyzing its content using natural language processing; means for automatically classifying the information into different types based on the analysis results; means for identifying the emotional state of the information using an emotion analysis function; means for setting priorities based on the identified emotional state and urgency; means for analyzing the previous communication history and generating an appropriate reply format; and means for notifying important unprocessed information. This enables effective prioritization and the generation of personalized replies according to the content and emotional state of emails.

[0613] "Email information" refers to digital documents, including text data and attachments, that are sent or received via electronic communication.

[0614] "Natural language processing" is a technology that enables computers to understand, analyze, and automatically process human language.

[0615] "Sentiment analysis function" refers to algorithms and technologies used to identify emotions and emotional tendencies from text data.

[0616] "Prioritizing" refers to the act of evaluating the necessity of processing or responding to tasks based on specific criteria and determining the order in which they are performed.

[0617] "Communication history" refers to records of past email transmissions and receptions, as well as their content, and is data used for future processing and decision-making.

[0618] "Reply format generation" refers to the process of automatically creating the structure and content of a reply message that should be sent.

[0619] "Notifications" refer to alerts or messages that inform users of relevant information.

[0620] In this embodiment of the invention, the server, terminal, and user functions are integrated to manage email. The respective roles and technologies used in their processing are described below.

[0621] The server first receives email information sent by the user. This information is transmitted as digital data over the internet. On the server, natural language processing (NLP) techniques built with Python or other programming languages ​​are used to analyze the content of the email. Specifically, libraries such as NLTK and spaCy are utilized to extract and classify information from the text. Based on the analysis results, the emails are automatically classified into categories such as "Notification," "Action Required," and "Reply Required," and further processing is performed based on this classification information.

[0622] Next, the server uses sentiment analysis capabilities to analyze the emotional state of received emails. This analysis identifies emotions based on data obtained from the vocabulary and writing style of the email body, as well as past communication history. For sentiment analysis, publicly available sentiment analysis platforms (e.g., IBM Watson Natural Language Understanding or Microsoft Azure Text Analytics) are applied. Using these results, email priorities are determined, with emotionally urgent information being given a higher priority.

[0623] Furthermore, the server utilizes a generative AI model to generate reply templates. Leveraging OpenAI's generative model and other tools, it executes prompts to automatically create appropriate reply formats. For example, a prompt such as "Generate an appropriate reply to this email" can be used. This process refers to past communication history and provides templates tailored to the user's communication style.

[0624] The terminal receives information from the server, then sorts it based on priority and displays it on the user interface. Users can view the email list on the displayed interface and easily identify emails that require priority. This enables quick responses, and the reminder function prevents important emails from being overlooked.

[0625] For example, when a user receives a "product trouble report email" from a client that indicates heightened emotions, the server classifies this email as high priority and sends an appropriate reply template to the user's device. The user can then quickly respond using the provided reply as a reference.

[0626] In this way, by linking the server and terminal, users can improve the efficiency of email management and enhance their work performance.

[0627] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0628] Step 1:

[0629] The server receives email information from the user. This email information is stored as electronic data on the server's storage via the internet. The input is a new email message received in the user's email account, and the output is the raw email data.

[0630] Step 2:

[0631] The server performs natural language processing on the received email data. Specifically, it analyzes the text data of the email body and subject line to extract keywords and grammatical structures. This is done using libraries such as Python's NLTK and spaCy. The input is the raw email data saved in step 1, and the output is the analyzed text data and its features.

[0632] Step 3:

[0633] The server automatically classifies emails into different categories (e.g., "Notification," "Action Required," "Reply Required") based on the analyzed text data. Machine learning algorithms are commonly used for this classification. The input is the feature data obtained in step 2, and the output is the classified email category.

[0634] Step 4:

[0635] The server uses an emotion analysis engine to identify emotional states from email data. This is done based on specific vocabulary and stylistic patterns. The input is the analysis data from step 2, and the output is an emotional index (e.g., positive, negative, neutral).

[0636] Step 5:

[0637] The server prioritizes emails based on identified emotional state and category information. High priority is given to emails deemed urgent or emotionally important. The input is the output data from steps 3 and 4, and the output is email prioritization information.

[0638] Step 6:

[0639] The server uses a generative AI model to generate appropriate reply templates. The prompt text used as input is something like, "Generate an appropriate reply for this email." The input consists of the sentiment data from step 4 and the required prompt text, and the output is the generated reply template.

[0640] Step 7:

[0641] The terminal receives information from the server and displays emails organized according to priority in the user interface. Through this interface, users can review email content and respond quickly to important emails. Inputs include prioritized email information and reply templates from the server, while output is a list of emails displayed on the screen.

[0642] (Application Example 2)

[0643] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0644] In today's communication society, users have to deal with a large volume of emails daily, some of which contain emotional content. However, conventional email management systems are unable to properly prioritize emails based on emotion recognition, potentially causing important emails to be overlooked. Furthermore, automatically generating reply content tailored to individual user emotions has been difficult. A system is needed to solve these problems.

[0645] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0646] In this invention, the server includes means for receiving email data and analyzing its content using natural language processing; means for automatically classifying emails into different categories based on the analysis results; means for analyzing the emotional state of emails and prioritizing them based on emotional information; means for generating reply templates for messages requiring a reply, taking into account the emotional history; and means for a robot to notify important unprocessed emails of reminders. This enables efficient management that takes into account the emotional information of emails.

[0647] "Email data" refers to a collection of information sent and received between users in electronic communication.

[0648] "Natural language processing" is a technology that uses computers to analyze and understand human language.

[0649] "Analysis results" refer to data that shows the classification and characteristics of information obtained after data processing.

[0650] A "category" refers to a group of emails that are analyzed and classified according to their characteristics and content.

[0651] "Emotional state" refers to the sender's emotions and psychological condition, as identified based on the content and expression of the email.

[0652] "Emotional information" specifically refers to data associated with emotions after they have been extracted.

[0653] "Priority" refers to a relative order set to determine the importance and urgency of processing or responding to tasks.

[0654] "Emotional history" refers to the accumulation of past emotion-related data, which is used to analyze a user's emotional tendencies.

[0655] A "reply template" refers to a document format that has a standard reply content for an email pre-configured.

[0656] A "robot" refers to a mechanical device that has a specific function and operates autonomously or according to a program.

[0657] A "reminder" is a notification or alarm that reminds a user of a specific matter or task.

[0658] The system for implementing this invention can incorporate an emotion engine to enhance efficiency and personalization in email management. First, upon receiving email data, the server analyzes the content using natural language processing (NLP) techniques. This process includes contextual analysis using Spacy and emotion analysis using TextBlob. The analyzed email data is categorized into categories such as "Reply Required," "Action Needed," and "Notification," and emotion information is extracted to identify the emotional state of the email.

[0659] Next, the server prioritizes emails based on the extracted sentiment information. For example, emails containing emotionally unstable tones or those of high urgency are given priority. For emails requiring a reply, a reply template is automatically generated, taking into account the user's past emotional history, ensuring a response that matches the user's communication style.

[0660] By running this system, consumer robots streamline email processing on behalf of users. The robots list important pending emails on the user interface based on priority and inform users of their importance through reminder notifications. This allows users to efficiently review a list of emails that take sentiment into account, enabling them to respond quickly and appropriately.

[0661] For example, if a user receives a "product trouble report" email, the system will set this email as a high priority and suggest a reply template that reflects the user's emotions. In this system, which uses a generative AI model, the prompt "Design a program that analyzes the content of emails received by users based on their emotions and categories and provides advice on how to reply" can be used.

[0662] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0663] Step 1:

[0664] The server receives the user's email data. The received email data includes information such as the sender, subject, and body. Based on this data, the server proceeds to the next analysis step.

[0665] Step 2:

[0666] The server analyzes the content of received email data using natural language processing. Specifically, it uses Spacy to analyze the context of the email body and identify parts of speech and sentence structure. The input is the email body text, and the output is the result of the contextual analysis.

[0667] Step 3:

[0668] The server classifies emails into different categories based on the analysis results. Here, emails are categorized into "Reply Required," "Action Needed," "Notification," etc., based on keywords and context. The input is the result of contextual analysis, and the output is email data categorized accordingly.

[0669] Step 4:

[0670] The server uses TextBlob to perform sentiment analysis on emails and identify their emotional state. In this step, the email body is used again as input, and positive or negative emotions are calculated from the vocabulary and tone, and output as a sentiment score.

[0671] Step 5:

[0672] The server uses sentiment scores and email category information to prioritize each email. Emails deemed to be emotionally unstable and urgent are given a higher priority. The input is sentiment score and category information, and the output is a list of emails with their priorities set.

[0673] Step 6:

[0674] The server considers past sentiment history and generates reply templates for emails that require a response. This step uses past reply history data and the current sentiment score as input, and outputs an appropriate tone and template.

[0675] Step 7:

[0676] The terminal displays a priority-based email list on the user interface. The user can visually review this list and determine which emails should be prioritized. The input is a priority-based email list from the server, and the output is the displayed email list.

[0677] Step 8:

[0678] A robot will notify users of important emails that have not yet been processed. This step uses a list of high-priority emails that have not been dealt with as input, and the output is a reminder notification that appears on the user's device.

[0679] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0680] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0681] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0682] [Fourth Embodiment]

[0683] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0684] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0685] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0686] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0687] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0688] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0689] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0690] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0691] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0692] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0693] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0694] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0695] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0696] The email management system of this invention is realized through the cooperation of a server and a terminal, and provides a function to efficiently manage the user's received emails. When the server receives a new email from the mail server, it first analyzes the content of the email using natural language processing technology. Based on the analysis results, the server classifies the email into categories such as "Notification," "Action Required," and "Reply Required."

[0697] Next, the server summarizes the content of each email and prioritizes them based on importance and deadlines. This allows the user interface to quickly identify the most important emails. The terminal then sorts and displays the email list by priority based on the information received from the server. Through the terminal, the user can review the email summary and access the detailed content as needed.

[0698] Furthermore, for emails that the server determines require a reply, it automatically generates a reply template based on pre-registered reply history and template information. The terminal presents this to the user, who can edit it as needed and easily send a reply. The server also has a function to periodically issue reminders for important unprocessed emails. This function allows the terminal to notify the user through a designated communication tool, preventing delays in response.

[0699] As a concrete example, among the numerous emails a user receives via the server, "project progress report requests" from their supervisor are classified with high priority, and timely notifications are sent via a reminder function as time passes. In addition, a reply template generation function automatically generates a reply template for the "progress report" and presents it to the user. This process prevents important emails from being overlooked and enables more efficient work.

[0700] The features described above allow users to significantly reduce the time spent on email management, enabling them to focus on other important tasks. This system is particularly useful for users who need to handle a large volume of emails daily, especially in a business environment.

[0701] The following describes the processing flow.

[0702] Step 1:

[0703] The server accesses the user's mail server and periodically retrieves new mail data. In this process, it analyzes the mail metadata (sender, received date and time, subject) and stores it in a database.

[0704] Step 2:

[0705] The server passes the body of the retrieved email data to a natural language processing engine for text analysis. Based on the analysis results, the emails are categorized as "Notification," "Action Required," or "Reply Required."

[0706] Step 3:

[0707] The server uses the analysis results to generate a summary from the email content. In this process, it extracts key keywords and context to create a shortened summary.

[0708] Step 4:

[0709] The server determines the importance of an email by scoring the sender's reputation and the urgency of the email. Prioritization is set based on deadline information and the presence of important keywords.

[0710] Step 5:

[0711] The terminal uses the summary, classification, and priority information of emails received from the server to display a list of emails in the user interface. At this time, the emails are sorted and displayed to the user in descending order of priority.

[0712] Step 6:

[0713] Users can view emails displayed with high priority on the device's interface and open the detailed contents of emails as needed. This enables them to quickly check and respond to important emails.

[0714] Step 7:

[0715] The server automatically generates reply templates for emails that require a response. It creates the most suitable template based on past email reply history and pre-configured phrases.

[0716] Step 8:

[0717] The device presents the user with a generated reply template, allowing the user to quickly respond by editing it.

[0718] Step 9:

[0719] The server monitors important pending emails in real time and issues reminders if replies are delayed. These reminders are sent to the user via a designated communication tool.

[0720] Step 10:

[0721] Users can receive reminder notifications through their devices, ensuring they don't forget to respond to important emails. This improves their work efficiency.

[0722] (Example 1)

[0723] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0724] In today's business environment, the volume of electronic messages users receive daily is increasing, requiring the rapid and efficient management of important messages. In particular, there are challenges such as overlooking important messages, the invalidation of preventative measures, and the difficulty of responding in a timely manner. Unless these challenges are addressed, user productivity will decline, and the risk of missing important business opportunities will increase.

[0725] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0726] In this invention, the server includes information processing means for receiving and analyzing the meaning of electronic messages, information processing means for classifying electronic messages into multiple categories based on the analysis results, and information processing means for condensing the information of each electronic message. This enables efficient management of electronic messages, prevention of overlooking important information, and rapid response.

[0727] "Electronic messages" refer to digital communications, including text and attachments, that are sent and received via the internet or other networks.

[0728] "Information processing means" refers to programs and algorithms designed to achieve a specific purpose on computers or servers.

[0729] "Meaning analysis" refers to the process of understanding the content and intent of text using natural language processing techniques, and then classifying and judging it.

[0730] "Classifying" means grouping information and dividing it into categories based on predetermined criteria.

[0731] "Information condensation" means extracting the most important content from the original information, shortening it, and expressing it in a summarized form.

[0732] "Setting priorities" means determining the order of tasks or items based on their importance and urgency according to some criteria.

[0733] A "reply template" is a pre-prepared set of sentences that can be used as a response to an electronic message without having to rewrite them.

[0734] "Notifying a user of an alert" means sending an alert or notification to inform them of the importance or urgency of a certain matter.

[0735] A "communication information management device" is a system device or application for efficiently organizing, classifying, and managing received communications.

[0736] The communication information management device according to this invention is a system in which a server and a terminal work together to efficiently manage received electronic messages. Specifically, the server first receives electronic messages via the internet. The received messages are analyzed using software for natural language processing, such as a generative AI model. This model utilizes open-source natural language processing tools or general generative AI models to understand the received data and analyze the intent and content of the message.

[0737] Based on the analysis results, the server categorizes electronic messages into categories such as "important," "normal," and "low priority." Furthermore, it evaluates the importance and urgency of each message and sets priorities according to specific rules. This process takes into account past communication history and pre-configured rules, and the AI ​​model supports classification and evaluation using appropriate prompt statements.

[0738] The server sends the returned information and priority information to the terminal. The terminal receives this information and displays it on the interface in a format accessible to the user. The UI is customized as needed so that high-priority messages stand out.

[0739] Users can view a summary of the message through their terminal and review the details as needed. If a reply is required, the server generates a reply template based on predefined templates and suggests it to the user. The user can review the template and use it as is, or edit it to send a reply.

[0740] Furthermore, the server has a function to periodically alert users to unprocessed messages. This function includes a reminder feature linked to the device, which, for example, issues a notification if a message has not been processed within a specified time.

[0741] As a concrete example, consider a scenario where a user receives a large volume of electronic messages daily. This system ensures that important requests from superiors are handled quickly. The server automatically generates reply templates, allowing users to efficiently submit progress reports.

[0742] Examples of prompts for generative AI models:

[0743] "Categorize the new messages and evaluate their priority."

[0744] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0745] Step 1:

[0746] The server receives new electronic messages from the mail server. The input consists of the electronic message and its metadata (sender, recipient, date and time, etc.). The server receives this and stores it in a temporary database in preparation for the next parsing process.

[0747] Step 2:

[0748] The server uses a generative AI model to analyze the content of received messages. The input is the body of the electronic message. The AI ​​model uses prompts to analyze the content and determine the message's theme, importance, and necessary actions. The output is the analysis result, including recommended categories and action information.

[0749] Step 3:

[0750] The server categorizes electronic messages based on the analysis results. The input is the analysis results from step 2. The server uses pre-configured rules to classify messages into categories such as "Important," "Normal," and "Low Priority." The output is the classified category information.

[0751] Step 4:

[0752] The server generates summaries and prioritizes each message. The inputs are the content of the electronic messages and the classification results from step 3. A generative AI model is used to summarize the content and assess importance based on known information. The output is the summary and prioritization information.

[0753] Step 5:

[0754] The server sends organized message data to the terminal. The input is summarized and prioritized message information. By sending this to the terminal, the terminal can display the information to the user. The output is the terminal's display interface.

[0755] Step 6:

[0756] The terminal displays messages based on the information it receives, highlighting those with higher priority. The input is data sent from the server. The terminal sorts this data according to priority and presents it visually to the user. The output is an organized message list in the user interface.

[0757] Step 7:

[0758] Users can instantly access important messages through their terminal and view details if necessary. The input is a summary message displayed on the terminal's interface. Based on the displayed information, the user selects an action and opens details as needed. The output is the detailed information viewed by the user and the action selected.

[0759] Step 8:

[0760] The server generates a reply template for messages that require a response. The input consists of the message information requiring a response and past communication history data. The AI ​​model generates an appropriate reply template based on this information. The output is the reply template.

[0761] Step 9:

[0762] The terminal presents the user with a generated reply template and supports editing and sending as needed. The input is template information sent from the server. The user can review and edit the reply based on this. The output is the reply finalized by the user.

[0763] Step 10:

[0764] The server periodically issues reminders for important messages that have not yet been processed. The input is information about the unprocessed messages and their priority. The server creates a reminder notification at the specified time and sends it to the device. The output is a notification alert serving as a reminder.

[0765] (Application Example 1)

[0766] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0767] In the business environment, companies receive a large volume of emails and messages daily. This leads to challenges such as delays in reviewing and responding to important messages, resulting in decreased operational efficiency. Furthermore, there is a risk of missed opportunities due to overlooking crucial information. In this context, there is a need for technology that enables the rapid and accurate management and processing of email information.

[0768] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0769] In this invention, the server includes means for receiving email information and analyzing its content using natural language processing techniques; means for automatically classifying emails into different categories based on the analysis results; means for automatically summarizing the content of each email; means for determining priority based on the importance and deadline of the emails; means for generating templates for creating replies to information that requires a response; means for providing a function to notify users of important unprocessed emails; and means for efficiently processing payment confirmations and customer inquiries via smart devices. This enables the rapid and efficient management of email information.

[0770] "Email information" refers to all electronic documents, including text and data, that are transmitted via electronic communication networks and presented to recipients.

[0771] "Natural language processing techniques" refer to technologies that enable computers to understand and analyze the language that humans use on a daily basis.

[0772] "Analysis results" refer to structured data and insights obtained after analyzing email information using natural language processing techniques.

[0773] "Classification" refers to the process of grouping data based on specific criteria or attributes.

[0774] "Summarization" refers to the act of extracting the essential parts from the original information and expressing them in a concise manner.

[0775] "Priority" refers to the order in which tasks or actions are assigned to indicate their importance or urgency.

[0776] A "reply template" refers to a pre-prepared text format designed to allow for quick and efficient responses to specific inquiries.

[0777] A "notification function" refers to a mechanism that presents warnings or information to prompt users to take action when certain conditions are met.

[0778] A "smart device" refers to a portable device that has internet connectivity and advanced features that allow it to run various applications.

[0779] "Payment confirmation" refers to the actions or processes taken to verify that a transaction has been successfully completed and that the money has been transferred correctly.

[0780] "Customer inquiry processing" refers to a series of activities aimed at providing appropriate information and resolving problems in response to questions and requests from customers.

[0781] The system implementing this invention consists of a server and a smart device, efficiently managing email information and supporting the user's work. The server retrieves email information from a mail server and analyzes the email content using natural language processing techniques. This is done using Python and natural language processing libraries (e.g., spaCy, NLTK). Next, based on the analysis results, the server automatically divides the emails into multiple categories. Machine learning algorithms are employed for classification, analyzing the characteristics of the email information and assigning appropriate tags.

[0782] The server further summarizes the content of each email, simplifying the essence of the information. This uses summarization technology to extract only the important points. Based on the importance and deadline of the email information, the server prioritizes the emails and sends the results to the smart device. The smart device then displays the received data in a user interface in a format that is easy for the user to understand.

[0783] For information requiring a response, the server utilizes pre-prepared templates to generate a template for creating a reply. These templates are generated from past contact history and registered formats, enabling quick and efficient responses. Furthermore, the server uses a notification function to alert users if important emails remain unprocessed. Smart devices provide users with relevant information in real time to facilitate smoother processing of important payment confirmations and customer inquiries.

[0784] As an example of its use, an accounting staff member can check an "overdue payment notification" email on their smart device and quickly complete a reply using a suggested reply template from the server. A specific example of a prompt message would be: "Analyze the following email and select the appropriate category based on its content. Furthermore, if a reply is required, generate an automated template." In this way, the entire business process can be streamlined.

[0785] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0786] Step 1:

[0787] The server retrieves new email information from the mail server. The input requires access information to the mail server. The server retrieves the email information and outputs it as text data.

[0788] Step 2:

[0789] The server analyzes the content of acquired email information using natural language processing techniques. It receives text data as input and performs analysis via a generative AI model. Through data analysis, it extracts important keywords and phrases from emails and outputs structured data. Specifically, it utilizes natural language processing libraries (e.g., spaCy, NLTK) to understand the context and extract necessary information.

[0790] Step 3:

[0791] The server automatically sorts emails into different categories based on the analysis results. It uses structured data as input. The server outputs email data that has been analyzed using a machine learning algorithm and assigned categories (e.g., "payment confirmation," "customer inquiry," etc.).

[0792] Step 4:

[0793] The server summarizes each email, extracting only the essential information to create a short summary. It consumes categorized email data as input. Through the summarization process, it outputs a concise summary.

[0794] Step 5:

[0795] The server prioritizes emails based on their importance and deadline. It uses summaries and email attribute information as input. The server performs calculations based on this information and outputs prioritized email data.

[0796] Step 6:

[0797] The terminal displays prioritized email data received from the server on the user interface. It takes prioritized email data as input, formats it for easy user understanding, and outputs it.

[0798] Step 7:

[0799] The server generates a reply template for emails it determines require a response. It uses the email content and past communication history as input. A template generation algorithm is used to automatically output the template.

[0800] Step 8:

[0801] The server periodically sends notifications about important unprocessed emails. It uses email status information and importance level as input to generate and output alert data. Specifically, it sends reminders to smart devices.

[0802] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0803] This invention combines an emotion engine with an email management system to achieve efficient and personalized email management that takes user emotions into account. When the server receives user email data, it analyzes the content using natural language processing. This analysis automatically classifies emails into categories such as "Notification," "Action Required," and "Reply Required."

[0804] Furthermore, the server uses an emotion engine to recognize the user's emotions from the email body. Emotion recognition is based on specific vocabulary, writing style, and the sender's past emotional tendencies contained in the email. The recognized emotion information is used to prioritize emails. For example, an email that is urgent and contains an emotionally unstable tone may be given a higher priority.

[0805] The server also utilizes the sentiment engine when generating reply templates. By analyzing the sentiment history of past email replies and reflecting the appropriate tone in the reply template based on that analysis, it provides reply content that matches the user's communication style.

[0806] The device displays received email information on the user interface based on priority. Users can view a list of emails that reflect sentiment information, enabling them to respond quickly and appropriately to important emails. In addition, the reminder notification function uses sentiment information to determine the most appropriate notification timing and send notifications to the device.

[0807] For example, if a user receives a "product trouble report" email from a customer that indicates heightened emotions, this email is immediately set to high priority. Furthermore, the server provides an appropriate reply template based on the emotions expressed, allowing the user to quickly respond appropriately to the situation. This system is expected to improve the efficiency of email management and enhance business performance.

[0808] The following describes the processing flow.

[0809] Step 1:

[0810] The server periodically retrieves new emails from the user's mail server and stores the email metadata and body in a database. During retrieval, it analyzes basic information, including the email sender and subject.

[0811] Step 2:

[0812] The server uses a natural language processing engine to analyze the email body. Based on the analysis, the email is automatically categorized into one of the following categories: "Notification," "Action Required," or "Reply Required."

[0813] Step 3:

[0814] The server utilizes an emotion engine to recognize emotions from the email body. Specifically, it identifies emotional expressions within the text and calculates an emotion score based on them.

[0815] Step 4:

[0816] The server determines the priority of emails by considering recognized sentiment information and other importance indicators (e.g., the importance of the sender and the urgency of the subject). This priority is used to determine the order in which emails are processed.

[0817] Step 5:

[0818] The terminal sorts the email list by priority based on classification information, sentiment information, and priority received from the server, and displays it on the user interface. This allows the user to quickly review their emails.

[0819] Step 6:

[0820] Users can view summaries and sentiment information of displayed emails on their devices, and open the emails if they wish to view more detailed information. This allows them to efficiently grasp important content.

[0821] Step 7:

[0822] For emails classified as requiring a reply, the server automatically generates a reply template with an appropriate tone using an emotion engine. The template is then customized based on past reply history and emotion information.

[0823] Step 8:

[0824] The device presents the user with a generated reply template, which the user can then edit and quickly submit the necessary reply.

[0825] Step 9:

[0826] The server monitors the database and issues reminders for important unprocessed emails at the optimal time based on sentiment.

[0827] Step 10:

[0828] Users receive emotion-based reminder notifications through their devices, prompting them to take action. This ensures timely responses to important emails.

[0829] (Example 2)

[0830] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0831] In email management, users are required to efficiently categorize large volumes of emails and respond appropriately. However, current systems lack the ability to consider the emotional state of emails or prioritize them based on importance, leading to challenges such as decreased user efficiency. Furthermore, there is a lack of mechanisms to generate personalized replies using past communication history.

[0832] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0833] In this invention, the server includes means for receiving email information and analyzing its content using natural language processing; means for automatically classifying the information into different types based on the analysis results; means for identifying the emotional state of the information using an emotion analysis function; means for setting priorities based on the identified emotional state and urgency; means for analyzing the previous communication history and generating an appropriate reply format; and means for notifying important unprocessed information. This enables effective prioritization and the generation of personalized replies according to the content and emotional state of emails.

[0834] "Email information" refers to digital documents, including text data and attachments, that are sent or received via electronic communication.

[0835] "Natural language processing" is a technology that enables computers to understand, analyze, and automatically process human language.

[0836] "Sentiment analysis function" refers to algorithms and technologies used to identify emotions and emotional tendencies from text data.

[0837] "Prioritizing" refers to the act of evaluating the necessity of processing or responding to tasks based on specific criteria and determining the order in which they are performed.

[0838] "Communication history" refers to records of past email transmissions and receptions, as well as their content, and is data used for future processing and decision-making.

[0839] "Reply format generation" refers to the process of automatically creating the structure and content of a reply message that should be sent.

[0840] "Notifications" refer to alerts or messages that inform users of relevant information.

[0841] In this embodiment of the invention, the server, terminal, and user functions are integrated to manage email. The respective roles and technologies used in their processing are described below.

[0842] The server first receives email information sent by the user. This information is transmitted as digital data over the internet. On the server, natural language processing (NLP) techniques built with Python or other programming languages ​​are used to analyze the content of the email. Specifically, libraries such as NLTK and spaCy are utilized to extract and classify information from the text. Based on the analysis results, the emails are automatically classified into categories such as "Notification," "Action Required," and "Reply Required," and further processing is performed based on this classification information.

[0843] Next, the server uses sentiment analysis capabilities to analyze the emotional state of received emails. This analysis identifies emotions based on data obtained from the vocabulary and writing style of the email body, as well as past communication history. For sentiment analysis, publicly available sentiment analysis platforms (e.g., IBM Watson Natural Language Understanding or Microsoft Azure Text Analytics) are applied. Using these results, email priorities are determined, with emotionally urgent information being given a higher priority.

[0844] Furthermore, the server utilizes a generative AI model to generate reply templates. Leveraging OpenAI's generative model and other tools, it executes prompts to automatically create appropriate reply formats. For example, a prompt such as "Generate an appropriate reply to this email" can be used. This process refers to past communication history and provides templates tailored to the user's communication style.

[0845] The terminal receives information from the server, then sorts it based on priority and displays it on the user interface. Users can view the email list on the displayed interface and easily identify emails that require priority. This enables quick responses, and the reminder function prevents important emails from being overlooked.

[0846] For example, when a user receives a "product trouble report email" from a client that indicates heightened emotions, the server classifies this email as high priority and sends an appropriate reply template to the user's device. The user can then quickly respond using the provided reply as a reference.

[0847] In this way, by linking the server and terminal, users can improve the efficiency of email management and enhance their work performance.

[0848] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0849] Step 1:

[0850] The server receives email information from the user. This email information is stored as electronic data on the server's storage via the internet. The input is a new email message received in the user's email account, and the output is the raw email data.

[0851] Step 2:

[0852] The server performs natural language processing on the received email data. Specifically, it analyzes the text data of the email body and subject line to extract keywords and grammatical structures. This is done using libraries such as Python's NLTK and spaCy. The input is the raw email data saved in step 1, and the output is the analyzed text data and its features.

[0853] Step 3:

[0854] The server automatically classifies emails into different categories (e.g., "Notification," "Action Required," "Reply Required") based on the analyzed text data. Machine learning algorithms are commonly used for this classification. The input is the feature data obtained in step 2, and the output is the classified email category.

[0855] Step 4:

[0856] The server uses an emotion analysis engine to identify emotional states from email data. This is done based on specific vocabulary and stylistic patterns. The input is the analysis data from step 2, and the output is an emotional index (e.g., positive, negative, neutral).

[0857] Step 5:

[0858] The server prioritizes emails based on identified emotional state and category information. High priority is given to emails deemed urgent or emotionally important. The input is the output data from steps 3 and 4, and the output is email prioritization information.

[0859] Step 6:

[0860] The server uses a generative AI model to generate appropriate reply templates. The prompt text used as input is something like, "Generate an appropriate reply for this email." The input consists of the sentiment data from step 4 and the required prompt text, and the output is the generated reply template.

[0861] Step 7:

[0862] The terminal receives information from the server and displays emails organized according to priority in the user interface. Through this interface, users can review email content and respond quickly to important emails. Inputs include prioritized email information and reply templates from the server, while output is a list of emails displayed on the screen.

[0863] (Application Example 2)

[0864] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0865] In today's communication society, users have to deal with a large volume of emails daily, some of which contain emotional content. However, conventional email management systems are unable to properly prioritize emails based on emotion recognition, potentially causing important emails to be overlooked. Furthermore, automatically generating reply content tailored to individual user emotions has been difficult. A system is needed to solve these problems.

[0866] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0867] In this invention, the server includes means for receiving email data and analyzing its content using natural language processing; means for automatically classifying emails into different categories based on the analysis results; means for analyzing the emotional state of emails and prioritizing them based on emotional information; means for generating reply templates for messages requiring a reply, taking into account the emotional history; and means for a robot to notify important unprocessed emails of reminders. This enables efficient management that takes into account the emotional information of emails.

[0868] "Email data" refers to a collection of information sent and received between users in electronic communication.

[0869] "Natural language processing" is a technology that uses computers to analyze and understand human language.

[0870] "Analysis results" refer to data that shows the classification and characteristics of information obtained after data processing.

[0871] A "category" refers to a group of emails that are analyzed and classified according to their characteristics and content.

[0872] "Emotional state" refers to the sender's emotions and psychological condition, as identified based on the content and expression of the email.

[0873] "Emotional information" specifically refers to data associated with emotions after they have been extracted.

[0874] "Priority" refers to a relative order set to determine the importance and urgency of processing or responding to tasks.

[0875] "Emotional history" refers to the accumulation of past emotion-related data, which is used to analyze a user's emotional tendencies.

[0876] A "reply template" refers to a document format that has a standard reply content for an email pre-configured.

[0877] A "robot" refers to a mechanical device that has a specific function and operates autonomously or according to a program.

[0878] A "reminder" is a notification or alarm that reminds a user of a specific matter or task.

[0879] The system for implementing this invention can incorporate an emotion engine to enhance efficiency and personalization in email management. First, upon receiving email data, the server analyzes the content using natural language processing (NLP) techniques. This process includes contextual analysis using Spacy and emotion analysis using TextBlob. The analyzed email data is categorized into categories such as "Reply Required," "Action Needed," and "Notification," and emotion information is extracted to identify the emotional state of the email.

[0880] Next, the server prioritizes emails based on the extracted sentiment information. For example, emails containing emotionally unstable tones or those of high urgency are given priority. For emails requiring a reply, a reply template is automatically generated, taking into account the user's past emotional history, ensuring a response that matches the user's communication style.

[0881] By running this system, consumer robots streamline email processing on behalf of users. The robots list important pending emails on the user interface based on priority and inform users of their importance through reminder notifications. This allows users to efficiently review a list of emails that take sentiment into account, enabling them to respond quickly and appropriately.

[0882] For example, if a user receives a "product trouble report" email, the system will set this email as a high priority and suggest a reply template that reflects the user's emotions. In this system, which uses a generative AI model, the prompt "Design a program that analyzes the content of emails received by users based on their emotions and categories and provides advice on how to reply" can be used.

[0883] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0884] Step 1:

[0885] The server receives the user's email data. The received email data includes information such as the sender, subject, and body. Based on this data, the server proceeds to the next analysis step.

[0886] Step 2:

[0887] The server analyzes the content of received email data using natural language processing. Specifically, it uses Spacy to analyze the context of the email body and identify parts of speech and sentence structure. The input is the email body text, and the output is the result of the contextual analysis.

[0888] Step 3:

[0889] The server classifies emails into different categories based on the analysis results. Here, emails are categorized into "Reply Required," "Action Needed," "Notification," etc., based on keywords and context. The input is the result of contextual analysis, and the output is email data categorized accordingly.

[0890] Step 4:

[0891] The server uses TextBlob to perform sentiment analysis on emails and identify their emotional state. In this step, the email body is used again as input, and positive or negative emotions are calculated from the vocabulary and tone, and output as a sentiment score.

[0892] Step 5:

[0893] The server uses sentiment scores and email category information to prioritize each email. Emails deemed to be emotionally unstable and urgent are given a higher priority. The input is sentiment score and category information, and the output is a list of emails with their priorities set.

[0894] Step 6:

[0895] The server considers past sentiment history and generates reply templates for emails that require a response. This step uses past reply history data and the current sentiment score as input, and outputs an appropriate tone and template.

[0896] Step 7:

[0897] The terminal displays a priority-based email list on the user interface. The user can visually review this list and determine which emails should be prioritized. The input is a priority-based email list from the server, and the output is the displayed email list.

[0898] Step 8:

[0899] A robot will notify users of important emails that have not yet been processed. This step uses a list of high-priority emails that have not been dealt with as input, and the output is a reminder notification that appears on the user's device.

[0900] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0901] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0902] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0903] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0904] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0905] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0906] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0907] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0908] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0909] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0910] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0911] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0912] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0913] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0914] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0915] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0916] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0917] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0918] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0919] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0920] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0921] The following is further disclosed regarding the embodiments described above.

[0922] (Claim 1)

[0923] A means of receiving email data and analyzing its content using natural language processing,

[0924] A means of automatically classifying emails into different categories based on the analysis results,

[0925] A method for automatically summarizing the content of each email,

[0926] A means of prioritizing emails based on their importance and deadline,

[0927] A means of generating a reply template for a message that requires a response,

[0928] A means of notifying reminders about important emails that have not yet been processed,

[0929] A system that includes this.

[0930] (Claim 2)

[0931] The system according to claim 1, further comprising means for sorting email data based on priority and displaying it in a list on a user interface.

[0932] (Claim 3)

[0933] The system according to claim 1, comprising means for using past communication history to generate a reply template.

[0934] "Example 1"

[0935] (Claim 1)

[0936] Information processing means for receiving electronic messages and analyzing their meaning,

[0937] Information processing means for classifying electronic messages into multiple categories based on analysis results,

[0938] An information processing means for condensing the information of each electronic message,

[0939] Information processing means for setting priorities based on the importance and deadline of electronic messages,

[0940] Information processing means for creating reply templates for electronic messages that require a response,

[0941] Information processing means for notifying attention to unprocessed important electronic messages,

[0942] A communication information management device that includes this.

[0943] (Claim 2)

[0944] The communication information management device according to claim 1, further comprising information processing means for arranging electronic messages according to priority and displaying them in a list on a user display device.

[0945] (Claim 3)

[0946] The communication information management device according to claim 1, comprising information processing means for using past communication records to create reply templates.

[0947] "Application Example 1"

[0948] (Claim 1)

[0949] A means of receiving email information and analyzing its content using natural language processing techniques,

[0950] A method for automatically separating emails into different categories based on analysis results,

[0951] A method for automatically summarizing the content of each email,

[0952] A means of prioritizing emails based on their importance and deadlines,

[0953] A means of generating a template for creating a reply to information that requires a response,

[0954] A means of providing a function to notify important emails that have not yet been processed,

[0955] A means of efficiently processing payment confirmations and customer inquiries via smart devices,

[0956] A system that includes this.

[0957] (Claim 2)

[0958] The system according to claim 1, further comprising means for sorting email information based on priority and displaying it in a list on the user's operation screen.

[0959] (Claim 3)

[0960] The system according to claim 1, comprising means for utilizing past contact history to generate a template for creating a reply.

[0961] "Example 2 of combining an emotion engine"

[0962] (Claim 1)

[0963] A means of receiving email information and analyzing its content using natural language processing,

[0964] A means for automatically classifying information into different types based on the analysis results,

[0965] A means of identifying the emotional state of information using an emotion analysis function,

[0966] A means of setting priorities based on identified emotional states and urgency,

[0967] A means of analyzing the previous communication history and generating an appropriate reply format,

[0968] Means of notifying about important unprocessed information,

[0969] A system that includes this.

[0970] (Claim 2)

[0971] The system according to claim 1, comprising means for organizing information based on priority and displaying it in a list on a user interface.

[0972] (Claim 3)

[0973] The system according to claim 1, comprising means for considering past communication history when generating a reply.

[0974] "Application example 2 when combining with an emotional engine"

[0975] (Claim 1)

[0976] A means of receiving email data and analyzing its content using natural language processing,

[0977] A means of automatically classifying emails into different categories based on the analysis results,

[0978] A method for analyzing the emotional state of emails and prioritizing them based on emotional information,

[0979] A means of generating a reply template for a message that requires a response, taking into account the emotional history,

[0980] A method for using a robot to send reminders about important emails that have not yet been processed,

[0981] A system that includes this.

[0982] (Claim 2)

[0983] The system according to claim 1, further comprising means for sorting email data based on priority and displaying it in a list on a user interface.

[0984] (Claim 3)

[0985] The system according to claim 1, comprising means for utilizing past communication history and emotional tendencies in generating reply templates. [Explanation of Symbols]

[0986] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving email data and analyzing its content using natural language processing, A means of automatically classifying emails into different categories based on the analysis results, A method for automatically summarizing the content of each email, A means of prioritizing emails based on their importance and deadline, A means of generating a reply template for a message that requires a response, A means of notifying reminders about important emails that have not yet been processed, A system that includes this.

2. The system according to claim 1, further comprising means for sorting email data based on priority and displaying it in a list on a user interface.

3. The system according to claim 1, comprising means for using past communication history to generate a reply template.

Images