system

Abstract

Provide a system. [Solution] a means for collecting emails; A means of classifying collected emails into multiple categories; a means for rating the importance of each email; A means of summarizing the content of emails for each category; means for generating a list of actions to be performed by the user; means for delivering said summary and action list to a user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the volume of emails received by companies and organizations has increased. Accordingly, a wide variety of emails are received daily, including important business instructions, customer communications, information sharing, and company-wide announcements. In such situations, it is difficult to check each and every email and process it without missing any important information, which can lead to a decline in business efficiency and the speed of customer responses. The present invention aims to solve this problem by providing a system that automatically generates summaries of the contents of all received emails and provides a list of actions to be taken, thereby ensuring thorough responses. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including means for collecting emails, means for classifying the collected emails into multiple categories, means for evaluating the importance of each email, means for summarizing the content of the emails for each category, means for generating a list of actions to be taken by the user, and means for delivering the summaries and action lists to the user. The email importance evaluation means calculates an importance score and determines the priority of the action list based on the score. The summary generation means uses natural language generation technology to extract key points from the emails and generate concise summaries. This allows the user to quickly understand the content of the emails they receive and take all important actions without missing anything.

[0006] "Email" refers to a digital message sent over a communications network, with content including text, images, audio, and video.

[0007] A "collection method" is a mechanism for obtaining email from a particular mail server or account and storing it within the system.

[0008] A "classification method" is an algorithm or function that sorts collected email into predefined categories based on content.

[0009] "Means for assessing importance" refers to a process for analyzing the content, sender, keywords, etc. of an email and assessing the importance of the email as a number or rank.

[0010] A "summarization method" is a technique or algorithm used to extract the main points from the full text of an email and summarize them in a concise form.

[0011] The "means for generating an action list" is a mechanism that presents a list of specific actions that the user should take based on the importance assessment and summary.

[0012] The "distribution means" is a communication means for transmitting the generated summary and action list to the user's terminal and displaying them to the user.

[0013] "Natural language generation technology" is a technology for automatically creating sentences in natural language that are easy for humans to understand from text data. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention relates to a system that collects e-mails, automatically classifies them, evaluates their importance, generates summaries, creates action lists, and distributes the results of these processes. This system is composed of the following elements: a server, a terminal, and a user.

[0036] System Overview

[0037] The system helps users manage their email more efficiently by providing a range of features, including:

[0038] 1. Email Collection:

[0039] The server accesses the mail server and collects received emails for a specific period (e.g., one day).

[0040] 2. Email Classification:

[0041] The server analyzes the content of the emails collected and classifies them into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0042] 3. Importance rating:

[0043] The server calculates an importance score based on the content of each email and prioritizes the emails.

[0044] 4. Summary generation:

[0045] The server uses machine learning and natural language generation techniques to generate a summary of each email.

[0046] 5. Create an action list:

[0047] The server creates a list of actions for the user to take based on the importance score and summary.

[0048] 6. Delivery of results:

[0049] The server sends the generated summary and action list to the terminal and displays them to the user.

[0050] Program processing overview

[0051] Email collection

[0052] 1. Server:

[0053] Every night, all received emails for the day are retrieved from the mail server using the IMAP protocol.

[0054] Store the retrieved emails in a database and save the metadata of each email (sender, recipient, subject, body, date and time).

[0055] Email Classification

[0056] 2. Server:

[0057] Each email is extracted from the database and its content is analyzed using natural language processing technology.

[0058] Based on the analysis results, the information is classified into the following categories: work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0059] The classification results are stored in a database.

[0060] Importance rating

[0061] 3. Server:

[0062] Scoring the importance of each email based on a list of keywords and senders.

[0063] Emails containing important keywords (e.g., "urgent," "important") or from specific senders are given a higher score.

[0064] Save the importance score in a database.

[0065] Summary Generation

[0066] 4. Server:

[0067] Natural language generation technology is used to generate a summary of each email.

[0068] Divide each email into a concise, few-line summary (e.g., "New project materials needed to be created").

[0069] The generated summary is stored in a database.

[0070] Action List Creation

[0071] 5. Server:

[0072] Based on the importance score and summary, a list of actions for the user to perform is generated.

[0073] Action lists are prioritized according to importance (e.g., 1. Create documentation for new projects, 2. Respond to customer inquiries).

[0074] Store the action list in a database.

[0075] Results distribution

[0076] 6. Server:

[0077] The generated summary and action list are compiled into a single report.

[0078] Send the report to the user's device.

[0079] 7. Terminal:

[0080] Receive the report and display it to the user (e.g., display it on the notification screen of a PC or smartphone).

[0081] Specific examples

[0082] For example, if the incoming email on October 1, 2023 is:

[0083] First message: "Work instructions: Create materials for a new project"

[0084] 2nd message: "Contact from customer: Product inquiry"

[0085] Third email: "Information sharing: Market trend report"

[0086] 4th message: "Company-wide notice: System maintenance notice"

[0087] 5th message: "Advertisement: Announcement of new product release"

[0088] Once the server has collected the above emails, classified them, ranked them by importance, generated summaries, and created action lists, the output will look something like this:

[0089] Summary example

[0090] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0091] Summary of customer email: "I need more information about product features"

[0092] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0093] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0094] Promotional email summary: "New product coming next month."

[0095] Action list to be taken

[0096] 1. Creating materials for a new project

[0097] 2. Detailed explanation of product features to customers

[0098] 3. Prepare for system maintenance this weekend

[0099] In this way, the system allows users to quickly grasp important information and efficiently take necessary actions.

[0100] The processing flow will be explained below.

[0101] Step 1:

[0102] The server connects to the mail server and retrieves all incoming emails for a specific period (e.g., one day) using the IMAP protocol. The server stores the retrieved emails in temporary storage.

[0103] Step 2:

[0104] The server reads the emails retrieved from temporary storage and stores them in a database, along with the metadata for each email (sender, recipient, subject, body, date and time).

[0105] Step 3:

[0106] The server retrieves each email stored in the database and analyzes its content using natural language processing (NLP) technology. Based on the analysis results, the email is automatically classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[0107] Step 4:

[0108] For each email that is categorized, the server calculates an importance score using keyword matching and sender lists. For example, emails containing keywords like "urgent" or "urgent" or from specific senders will receive a higher score.

[0109] Step 5:

[0110] The server uses natural language generation (NLG) technology to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[0111] Step 6:

[0112] The server generates a list of actions for the user to perform based on the importance scores and the generated summary, with the list of actions prioritized in order of importance.

[0113] Step 7:

[0114] The server compiles the summary and action list into a single report, which contains a summary of the messages in each category and the corresponding actions.

[0115] Step 8:

[0116] The server sends the compiled report to the user's device, and the report is provided to the user in the form of email, app notification, or dashboard display.

[0117] Step 9:

[0118] The terminal displays the received report to the user, who can then check the report and take the action indicated.

[0119] Example 1

[0120] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0121] In today's business environment, email is an important means of communication, but it can be difficult to efficiently manage the large amount of information and quickly take necessary action. Conventional email management systems lack the functionality to properly evaluate the importance and content of emails and clearly present the actions that users should take. As a result, users often miss important information or respond late. There is a need to solve this problem and improve user work efficiency.

[0122] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0123] In this invention, the server includes means for analyzing the metadata of emails and storing it in a database, means for analyzing the content of emails using natural language processing technology, and means for generating summaries of emails using natural language generation technology, which enable accurate evaluation of the importance of emails and specific suggestions for actions to be taken by the user.

[0124] "Email" means a digital message sent or received over the Internet or other communications network.

[0125] "Metadata" refers to information attached to an email, such as the sender, recipient, subject, body, date and time.

[0126] A "database" is a collection of digital data that systematically stores and manages collected emails and metadata.

[0127] "Natural language processing technology" refers to the technology of analyzing and understanding human language using a computer, and includes techniques such as tokenizing words and analyzing sentence structure.

[0128] "Natural language generation technology" refers to a technology that allows a computer to generate text in natural human language, and is used to summarize emails and extract information.

[0129] An "abstract" is a short sentence that succinctly summarizes the main points of the email.

[0130] The "importance score" is a number that indicates the importance of an email, calculated based on the email content and sender list.

[0131] An "action list" is a list of specific tasks or action items that a user must perform.

[0132] "Priority" refers to the ordering of multiple tasks or action items based on their importance.

[0133] "Report" refers to information provided to a user in the form of a document summarizing emails and / or a list of actions.

[0134] This invention relates to a system that collects, automatically classifies, evaluates the importance of, summarizes, creates an action list, and distributes e-mails, and is composed of elements of a server, a terminal, and a user. A specific embodiment of this system is shown below.

[0135] System Configuration

[0136] The system mainly consists of the following components:

[0137] Server: Use a Linux (registered trademark) server or cloud server (e.g., AWS (registered trademark) EC2).

[0138] Database: A relational database such as MySQL (registered trademark) or PostgreSQL is used.

[0139] Natural Language Processing (NLP) techniques: Use Python's NLTK and SpaCy libraries.

[0140] Natural Language Generation (NLG) technology: Utilizes OpenAI's GPT-3 or the latest GPT-4 models.

[0141] Terminal: A user device such as a PC or smartphone.

[0142] System Features

[0143] The system helps users manage their email more efficiently by providing the following features:

[0144] 1. Email Collection:

[0145] The server accesses the mail server and uses the IMAP protocol to collect incoming emails for a specific period (e.g., one day).

[0146] The collected email metadata (sender, recipient, subject, body, date and time) is stored in a database.

[0147] 2. Email Classification:

[0148] The server retrieves each stored email and analyzes its content using natural language processing technology.

[0149] Based on the analysis results, the data is classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising, and stored in a database.

[0150] 3. Importance rating:

[0151] The server calculates an importance score for each email based on its content, keywords, and sender list.

[0152] Save the importance score to the database and update the importance field of the email.

[0153] 4. Summary generation:

[0154] The server uses natural language generation techniques to generate a summary of each email.

[0155] The prompt sentence "Generate a summary of the following email: Email content" is input into the model.

[0156] The generated summary is stored in a database and the summary field corresponding to each email is updated.

[0157] 5. Create an action list:

[0158] The server lists action items for the user to perform based on the importance scores and the generated summary.

[0159] The action list is prioritized based on importance and stored in a database.

[0160] 6. Delivery of results:

[0161] The server compiles the generated summary and action list into a single report and sends it to the user's terminal.

[0162] The device displays the report received from the server to the user on a PC or smartphone.

[0163] Specific examples

[0164] For example, if the incoming email on October 1, 2023 is:

[0165] First message: "Work instructions: Create materials for a new project"

[0166] 2nd message: "Contact from customer: Product inquiry"

[0167] Third email: "Information sharing: Market trend report"

[0168] 4th message: "Company-wide notice: System maintenance notice"

[0169] 5th message: "Advertisement: Announcement of new product release"

[0170] Once the server has collected these emails, sorted them, ranked them by importance, generated summaries, and created action lists, the result will look something like this:

[0171] Summary example

[0172] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0173] Summary of customer email: "I need more information about product features"

[0174] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0175] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0176] Promotional email summary: "New product coming next month."

[0177] Action list to be taken

[0178] 1. Creating materials for a new project

[0179] 2. Detailed explanation of product features to customers

[0180] 3. Prepare for system maintenance this weekend

[0181] Prompt Sentence Examples

[0182] An example of a prompt sentence to input to the generative AI model is "Generate a summary of the following email: Work instructions: Create materials for a new project."

[0183] In this way, by using the system of the present invention, the user can quickly grasp important information and efficiently take the necessary actions.

[0184] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0185] Specific flow of program processing

[0186] Each processing step is described in detail below.

[0187] Step 1: Email Collection

[0188] server:

[0189] Input: Connect to a mail server (for example, using the IMAP protocol) at a fixed time every night.

[0190] Specific operation: The server retrieves unread emails and newly received emails for a specified period (e.g., the past 24 hours).

[0191] Data processing: Analyze the metadata of the acquired emails (sender, recipient, subject, body, date and time).

[0192] Output: Save the email metadata and body to a database.

[0193] Step 2: Email Classification

[0194] server:

[0195] Input: Collected emails stored in a database.

[0196] What it does: The server analyzes the content of each email using natural language processing (NLP) techniques, such as Python's NLTK or SpaCy.

[0197] Data processing: Based on the analysis results, data is classified into the following categories: work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0198] Output: Store the email classification results in a database and assign each email a corresponding category tag.

[0199] Step 3: Severity Assessment

[0200] server:

[0201] Input: Classified email.

[0202] What it does: The server calculates an importance score for each email based on its content, keywords, and sender list.

[0203] Data processing: For example, emails containing keywords such as "urgent" or "important" or emails from specific senders (such as your boss or important customers) are given a higher score.

[0204] Output: Save the importance score to the database and update the importance field of the email.

[0205] Step 4: Summary generation

[0206] server:

[0207] Input: Emails with importance scores.

[0208] What it does: It uses natural language generation technology (NLG technology, e.g., OpenAI's GPT-3 or GPT-4 models) to convert the content of each email into a concise summary.

[0209] Data processing: The prompt sentence "Generate a summary of the following email: Email content" is input into the model.

[0210] Output: Store the generated summaries in a database and update the summary field for each email.

[0211] Step 5: Create an Action List

[0212] server:

[0213] Input: Email with summary and importance score.

[0214] Specific Action: Based on the importance score and the generated summary, a list of action items for the user to take is generated.

[0215] Data manipulation: Prioritize action items based on importance.

[0216] Output: Stores the action list in the database and updates the user's action list field.

[0217] Step 6: Delivering results

[0218] server:

[0219] Input: The data from which the action list and summary are generated.

[0220] Specific behavior: Compile the generated summary and action list into a single report, for example, generate a report in PDF or HTML format.

[0221] Output: Sends the report to the user's terminal.

[0222] Device:

[0223] Input: The report received from the server.

[0224] Specific operation: The device displays the report to the user on a PC or smartphone, and notifies the user in real time using the notification function.

[0225] Output: User reviews the report and decides what action to take next.

[0226] By implementing this system's series of processes, users can quickly grasp important information and efficiently take the necessary actions.

[0227] (Application example 1)

[0228] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0229] In modern factories, a great deal of information is communicated via email, but it is difficult to efficiently manage the vast volume of emails and quickly take necessary actions. It is also difficult to distinguish between important and unimportant emails, which increases the risk of overlooking important information. This reduces factory operational efficiency and has a negative impact on productivity. To solve these problems, proper email management, automatic classification, importance assessment, rapid summary generation, and the creation and distribution of action lists are required.

[0230] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0231] In this invention, the server includes means for collecting emails, means for classifying the collected emails into multiple categories, means for rating the importance of each email, means for summarizing the content of the emails for each category, means for generating a list of actions to be taken by the user, means for distributing the summaries and the action list to the user, and means for automatically processing emails related to factory work and notifying the results to the manager's terminal, thereby enabling the factory manager to efficiently manage a huge amount of emails, quickly grasp important information, and immediately take necessary actions.

[0232] "Means for collecting email" refers to a system or software that connects to a mail server via the Internet and obtains emails sent to a specific account.

[0233] A "multiple categorization means" is an algorithm or software that automatically sorts retrieved emails into predefined categories based on their content.

[0234] The "means for assessing importance" is a system or software that analyzes the content of each email and calculates and scores the importance of the email.

[0235] The "summarization means" refers to a system or software that extracts key points from the body of an email and generates a concise summary using a machine learning algorithm or similar.

[0236] The "means for generating an action list" is a system or software that lists and prioritizes tasks that a user should perform based on the summary and importance score of each email.

[0237] The "distribution means" is a system or software for sending the generated summary and action list to the user's terminal and notifying them.

[0238] "Means for automatically processing emails related to factory operations and notifying the results to the manager's terminal" refers to a system or software that collects and analyzes emails containing business communications, instructions, progress reports, etc. in the factory, and delivers the results to the manager in real time.

[0239] This invention relates to a system that collects emails, automatically classifies them, evaluates their importance, creates summaries and action lists, and distributes the results. This system aims to efficiently manage business emails, particularly in factories, reducing the burden on managers and enabling them to quickly grasp important information.

[0240] The system is mainly composed of the server, terminal, and user elements. The role of each element and the specific processing method are explained below.

[0241] Server Roles and Operations

[0242] The server plays a central role in collecting, classifying, assessing the importance of emails, generating summaries, and creating and distributing action lists.

[0243] Email collection

[0244] Every night, the server retrieves incoming emails from the mail server using the IMAP protocol, and stores all email metadata (sender, recipient, subject, body, date and time) in a database.

[0245] Email Classification

[0246] The server retrieves emails from the database and analyzes their contents using natural language processing (NLP) technology, categorizing them into categories such as work instructions, production progress, emergency contacts, maintenance information, and advertisements. A keyword-based classification algorithm is used for the specific analysis.

[0247] Importance rating

[0248] The importance of each email is scored based on keywords and senders. Emails containing important keywords (e.g., "urgent" or "important") or from specific senders are given a higher score. This importance score is stored in a database.

[0249] Summary Generation

[0250] We use natural language generation techniques to generate summaries of each email, specifically by extracting a concise, few-line summary from the body of the email using the gensim library.

[0251] Action List Creation

[0252] Based on the importance score and summary, a list of actions for the user to perform is generated, prioritized according to importance.

[0253] Results distribution

[0254] The generated summary and action list are compiled and delivered to the user's terminal. The delivered notification is displayed on the administrator's terminal.

[0255] Device Role

[0256] The user's device receives the summary and action list sent from the server and notifies the administrator. The application on the device displays the notification content in an easy-to-understand manner, allowing the administrator to take action immediately.

[0257] User Roles

[0258] The administrator (user) can quickly take necessary action based on the provided summary and action list. For example, they can prioritize tasks of high importance, such as creating a production plan for a new project or responding to machine failures.

[0259] Specific examples

[0260] For example, if the following email is received on October 1, 2023:

[0261] Work instruction: "Create a production plan for a newly ordered project."

[0262] Manufacturing Progress: "Report on the current progress of the production line"

[0263] Emergency call: "There is a machine breakdown and we need to fix it quickly."

[0264] Maintenance information: "Check the scheduled maintenance schedule"

[0265] Advertisement: "Sales announcement for new materials from suppliers"

[0266] The server then categorizes and rates each email, generating a summary and action list like this:

[0267] Summary example

[0268] Summary of work instruction email: "Create a production plan for the new project this week"

[0269] Summary of manufacturing progress email: "Production line progress is on schedule"

[0270] Emergency message summary: "Machine breakdown, immediate repair required."

[0271] Maintenance information summary: "Scheduled maintenance scheduled for next week"

[0272] Summary of the advertising email: "Information on new materials"

[0273] Action list to be taken

[0274] 1. Creating production plans for new projects

[0275] 2. Response to machine breakdowns

[0276] 3. Preparation for scheduled maintenance

[0277] 4. Checking production progress

[0278] Example prompts for generative AI models

[0279] "Analyze the following email and generate a summary:

[0280] Subject: Create a production plan for a new project

[0281] Body: We need to create a production plan for a newly ordered project this week. Please confirm the detailed specifications and required resources and create an appropriate schedule.

[0282] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0283] Step 1:

[0284] The server establishes a connection to the mail server using the IMAP protocol and retrieves all emails received within a specific period of time. The server stores the metadata of the retrieved emails (sender, recipient, subject, body, date and time) in a database, making all incoming emails available as target input data.

[0285] Step 2:

[0286] The server retrieves each email from the database and analyzes its content using natural language processing technology. Based on the content of the email, the server categorizes it into categories such as work instructions, production progress, emergency contacts, maintenance information, and advertisements. The analyzed data is then stored in the database as output data categorized by category.

[0287] Step 3:

[0288] The server calculates an importance score based on the classified emails. Emails containing important keywords (e.g., "urgent" or "important") or from specific senders are given a higher score. This weights the importance of each email. The score is calculated based on the input data and added to the output data as the importance of each email.

[0289] Step 4:

[0290] The server uses natural language generation techniques, such as the gensim library, to generate a summary of each email's content. The body of each email is summarized into a concise, few-line summary, which is then stored in a database as the output of the analysis.

[0291] Step 5:

[0292] Based on the importance scores and summaries, the server generates a list of actions for the user to perform. The action list is prioritized by importance and identifies tasks to be performed. This action list is stored in a database as part of the output data.

[0293] Step 6:

[0294] The server compiles the generated summary and action list into a single report and sends it to the user's device, allowing the administrator to receive notifications on their smartphone or other device.

[0295] Step 7:

[0296] The terminal displays the received report and notifies the user. Specifically, the terminal's notification function is used to display a summary and action list, allowing the user to quickly take necessary actions based on this.

[0297] Prompt Sentence Examples

[0298] For example, the prompt to verify the generated summary is:

[0299] "Analyze the following email and generate a summary:

[0300] Subject: Create a production plan for a new project

[0301] Body: We need to create a production plan for a newly ordered project this week. Please confirm the detailed specifications and required resources and create an appropriate schedule.

[0302] Through the flow of steps, a system is realized that allows servers, terminals, and users to work together to significantly improve factory operational efficiency.

[0303] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0304] This invention relates to a system that collects emails, automatically classifies them, assesses their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. This system is composed of a server, a terminal, and users, and by combining it with an emotion engine, it provides information and generates action lists that take into account the user's emotional state.

[0305] System Overview

[0306] The system will help users streamline email management and emotion recognition by providing a series of features, including:

[0307] 1. Email Collection:

[0308] The server accesses the mail server and collects all incoming emails for a specific period (e.g., one day).

[0309] 2. Email Classification:

[0310] The server analyzes the content of the emails collected and classifies them into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0311] 3. Importance rating:

[0312] The server calculates an importance score based on the content of each email and prioritizes it.

[0313] 4. Summary generation:

[0314] The server uses machine learning and natural language generation techniques to generate a summary of each email.

[0315] 5. Create an action list:

[0316] The server generates a list of actions for the user to perform based on the importance score and summary text.

[0317] 6. Emotion recognition:

[0318] The server uses an emotion engine that recognizes the user's emotions, and reflects the emotion information when generating the email contents and action list.

[0319] 7. Delivery of results:

[0320] The server generates a report including the summary, action list, and emotion-based adjustments and sends it to the terminal for display to the user.

[0321] Program processing overview

[0322] Email collection

[0323] 1. Server:

[0324] Every night, all received emails for the day are retrieved from the mail server using the IMAP protocol.

[0325] Store the retrieved emails in a database and save the metadata of each email (sender, recipient, subject, body, date and time).

[0326] Email Classification

[0327] 2. Server:

[0328] The email is retrieved from the database and its content is analyzed using natural language processing techniques.

[0329] Based on the analysis results, the messages are sorted into the following categories: work instructions, customer contact, information sharing, company-wide announcements, and advertising.

[0330] The classification results are stored in a database.

[0331] Importance rating

[0332] 3. Server:

[0333] The importance of each email is scored based on a list of keywords and senders. For example, emails containing keywords such as "urgent" or "urgent" or from specific senders are given a higher score.

[0334] Save the importance score in a database.

[0335] Summary Generation

[0336] 4. Server:

[0337] Natural language generation technology is used to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[0338] The generated summaries are stored in a database.

[0339] Action List Creation

[0340] 5. Server:

[0341] Based on the importance score and summary, a list of actions to be performed by the user is generated. The action list is prioritized in order of importance (e.g., 1. Create documents for a new project, 2. Respond to customer inquiries).

[0342] Store the action list in a database.

[0343] emotion recognition

[0344] 6. Server:

[0345] Recognize the user's emotional state using an emotion engine, which analyzes emotional information detected from the user's voice, facial expressions, text input, etc.

[0346] The system adjusts email content and action list generation based on the user's recognized emotional state. For example, if the user is feeling stressed, it suggests prioritizing high-priority emails.

[0347] Results distribution

[0348] 7. Server:

[0349] The generated summary, action list, and report reflecting the sentiment information are compiled into a single document.

[0350] Send the report to the user's device.

[0351] 8. Terminal:

[0352] The received report is displayed to the user, who then reviews the report and takes the action indicated.

[0353] Specific examples

[0354] For example, if the incoming email on October 1, 2023 is:

[0355] First message: "Work instructions: Create materials for a new project"

[0356] 2nd message: "Contact from customer: Product inquiry"

[0357] Third email: "Information sharing: Market trend report"

[0358] 4th message: "Company-wide notice: System maintenance notice"

[0359] 5th message: "Advertisement: Announcement of new product release"

[0360] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[0361] Summary example

[0362] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0363] Summary of customer email: "I need more information about product features"

[0364] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0365] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0366] Promotional email summary: "New product coming next month."

[0367] Action list to be taken

[0368] 1. Creating materials for a new project

[0369] 2. Detailed explanation of product features to customers

[0370] 3. Prepare for system maintenance this weekend

[0371] emotion recognition

[0372] If the user is stressed, adjust the action list to be shorter and focus on the most important tasks to reduce stress.

[0373] In this way, the system allows users to quickly grasp important information and take necessary actions efficiently.By combining emotion engines, it provides adaptive support according to the user's emotional state.

[0374] The processing flow will be explained below.

[0375] Step 1:

[0376] The server connects to the mail server and retrieves all incoming emails for a specific period (e.g., one day) using the IMAP protocol. The server stores the retrieved emails in temporary storage.

[0377] Step 2:

[0378] The server reads the emails retrieved from temporary storage and stores them in a database, along with the metadata for each email (sender, recipient, subject, body, date and time).

[0379] Step 3:

[0380] The server retrieves each email stored in the database and analyzes its content using natural language processing (NLP) technology. Based on the analysis results, the email is automatically classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[0381] Step 4:

[0382] For each email that is categorized, the server calculates an importance score using keyword matching and sender lists. For example, emails containing keywords like "urgent" or "urgent" or from specific senders will be given a higher score.

[0383] Step 5:

[0384] The server uses natural language generation (NLG) technology to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[0385] Step 6:

[0386] The server generates a list of actions for the user to perform based on the importance scores and the generated summary, with the list of actions prioritized in order of importance.

[0387] Step 7:

[0388] The server uses an emotion engine to recognize the user's emotional state, which analyzes emotional information from the user's voice, facial expressions, text input, etc.

[0389] Step 8:

[0390] The server adjusts the action list or summary based on the perceived emotional state, for example, if the user is feeling stressed, prioritizing important tasks to reduce stress.

[0391] Step 9:

[0392] The server compiles the summary and action list into a single report, which contains a summary of the messages in each category and the corresponding actions.

[0393] Step 10:

[0394] The server sends the compiled report to the user's device, and the report is provided to the user in the form of email, app notification, or dashboard display.

[0395] Step 11:

[0396] The terminal displays the received report to the user, who then checks the report and takes the action indicated.

[0397] Step 12:

[0398] The user performs the actions indicated and inputs the results as feedback to the system, which allows the system to provide a more accurate action list and summary.

[0399] For example, if the email you received on October 1, 2023 is:

[0400] First message: "Work instructions: Create materials for a new project"

[0401] 2nd message: "Contact from customer: Product inquiry"

[0402] Third email: "Information sharing: Market trend report"

[0403] 4th message: "Company-wide notice: System maintenance notice"

[0404] 5th message: "Advertisement: Announcement of new product release"

[0405] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[0406] Summary example

[0407] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0408] Summary of customer email: "I need more information about product features"

[0409] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0410] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0411] Promotional email summary: "New product coming next month."

[0412] Action list to be taken

[0413] 1. Creating materials for a new project

[0414] 2. Detailed explanation of product features to customers

[0415] 3. Prepare for system maintenance this weekend

[0416] emotion recognition

[0417] If users are stressed, shorten the action list and adjust it to focus on the most important tasks.

[0418] In this way, the system allows users to quickly grasp important information and take necessary actions efficiently.By combining emotion engines, it provides adaptive support according to the user's emotional state.

[0419] Example 2

[0420] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0421] In conventional email management systems, received emails are classified, their importance is assessed, summaries are generated, and action lists are created individually, without taking into account the user's emotional state. As a result, users can become overwhelmed by the volume of emails and may miss important emails or tasks. Furthermore, when users are under stress, it is difficult for them to process emails appropriately.

[0422] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting emails, means for classifying the collected emails into a plurality of categories, means for evaluating the importance of each email, means for summarizing the contents of the emails for each category, means for generating a list of actions to be performed by the user, means for delivering the summaries and action list to the user, means for recognizing the user's emotional state, and means for adjusting the action list and summaries according to the emotional state. This enables the user to quickly grasp important information and receive adaptive support according to their emotional state.

[0423] "Email" means a digital message sent or received over the Internet or other digital network.

[0424] "Category" refers to a group of emails classified based on their contents, and examples include work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[0425] An "importance score" is a number used to rate the importance of an email and is calculated based on specific keywords and sender lists.

[0426] "Natural language generation technology" refers to technology that generates content in natural language that humans can understand, and includes machine learning and AI technologies.

[0427] A "summary" is a short sentence that extracts the main points from a longer text and summarizes them concisely.

[0428] An "action list" is a list of specific tasks or actions that a user must perform, prioritized based on importance.

[0429] "Emotional state" refers to the user's psychological state, and includes emotions such as stress, joy, and impatience.

[0430] An "emotion engine" is hardware or software used to recognize a user's emotions, and can obtain emotional information by analyzing voice, facial expressions, text input, etc.

[0431] "User" means an individual or organization that uses this system.

[0432] MODE FOR CARRYING OUT THE INVENTION

[0433] This invention relates to a system that collects emails, automatically classifies them, assesses their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. This system is composed of a server, a terminal, and users, and by combining it with an emotion engine, it provides information and generates action lists that take into account the user's emotional state.

[0434] The system configuration is as follows: first, the server accesses the mail server and collects all received emails for a specific period (e.g., one day). At this stage, the IMAP protocol is used to retrieve the latest emails from the mail server (e.g., Gmail server) and store them in a database (e.g., MySQL). Email metadata (sender, recipient, subject, body, date and time) is also saved at the same time.

[0435] The server then retrieves the email from the database and analyzes its content using natural language processing technology (e.g., Google® NLP API). Based on this analysis, the email is classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements. The classification results are also stored in the database.

[0436] The server also assigns a score to each email based on its importance, based on keywords (e.g., "urgent" or "urgent") and specific sender lists. This importance score is stored in a database for each email.

[0437] The server then uses machine learning and natural language generation techniques (e.g., OpenAI GPT) to generate a summary of each email. The summary extracts key information from the email body and presents it in a concise format of a few lines. The generated summary is also stored in a database.

[0438] Based on the generated summary and the importance score, the server generates a list of actions for the user to perform. The list of actions is prioritized in order of importance and stored in a database.

[0439] Furthermore, the server uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotional state. This analyzes emotional information detected from the user's voice, facial expressions, text input, etc. Based on the recognized emotional state, the email content and action list generation are adjusted. For example, if the user is feeling stressed, the server suggests prioritizing high-priority emails.

[0440] Finally, the server compiles the generated summary, action list, and emotional information into a report and sends it to the user's device. The device displays the received report to the user, who then checks the report and performs the requested actions.

[0441] Specific examples

[0442] Email received on October 1, 2023:

[0443] "Work instructions: Create materials for a new project"

[0444] "Customer Contact: Product Inquiry"

[0445] "Information Sharing: Market Trends Report"

[0446] "Company-wide Notice: System Maintenance Notice"

[0447] "Advertisement: Announcement of new product release"

[0448] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[0449] Summary example:

[0450] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0451] Summary of customer email: "I need more information about product features"

[0452] Summary of the information sharing email: "Certain sectors are experiencing rapid growth as a market trend."

[0453] Summary of company-wide email: "System maintenance is scheduled for this weekend."

[0454] Promotional email summary: "New product launching next month"

[0455] List of actions to be taken:

[0456] 1. Creating materials for a new project

[0457] 2. Detailed explanation of product features to customers

[0458] 3. Prepare for system maintenance this weekend

[0459] Emotion recognition:

[0460] If the user is stressed, shorten the action list to reduce stress and focus on the most important tasks.

[0461] Example of an input prompt for a generative AI model:

[0462] Analyze the content of received emails and categorize them into the following categories: work instructions, customer contact, information sharing, company-wide announcements, and promotional advertisements. Also, assign an importance score to each email and generate a summary. Then, create an action list and provide tailored results based on sentiment recognition.

[0463] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0464] Step 1:

[0465] Email collection

[0466] server:

[0467] Every night, the server connects to a mail server using the IMAP protocol, for example, to retrieve incoming emails from the Gmail server.

[0468] The retrieved emails are stored in a database (e.g., MySQL), and metadata for each email (sender, recipient, subject, body, date and time) is stored.

[0469] Input: Received email data from the mail server.

[0470] Output: Email data and metadata stored in a database.

[0471] Step 2:

[0472] Email Classification

[0473] server:

[0474] The server retrieves unclassified emails from the database and analyzes their content using natural language processing techniques (e.g., Google NLP API).

[0475] Based on the analysis results, the information is sorted into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0476] The classification results are stored in the database again.

[0477] Input: Unclassified email data from the database.

[0478] Output: Email data categorized by category.

[0479] Step 3:

[0480] Importance rating

[0481] server:

[0482] The server calculates an importance score for each email based on specific keywords (e.g., "urgent" or "urgent") and a list of specific senders.

[0483] An importance score is stored in the database for each email.

[0484] Input: Classified email data.

[0485] Output: Email data with importance scores assigned.

[0486] Step 4:

[0487] Summary Generation

[0488] server:

[0489] The server uses machine learning and natural language generation techniques (e.g., OpenAI GPT) to generate a summary of each email.

[0490] The summary extracts the key information from the email body and presents it in a concise format of a few lines.

[0491] The generated summaries are stored in a database.

[0492] Input: Email data with an importance score.

[0493] Output: The email data with the summary generated.

[0494] Step 5:

[0495] Action List Creation

[0496] server:

[0497] The server generates a list of actions for the user to perform based on the generated summary and importance score.

[0498] Action lists are prioritized in order of importance and stored in a database.

[0499] Input: The email data from which the summary was generated and the importance score.

[0500] Output: A prioritized list of actions.

[0501] Step 6:

[0502] emotion recognition

[0503] server:

[0504] The server uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotional state.

[0505] Emotional information is analyzed from the user's voice, facial expressions, text input, etc.

[0506] Based on the recognized emotional state, adjustments are made when generating email content and action lists.

[0507] Input: User voice, facial expressions, and text input.

[0508] Output: A tailored action list and summary.

[0509] Step 7:

[0510] Results distribution

[0511] server:

[0512] The server compiles the generated summary, action list, and report reflecting the emotional information into a single document.

[0513] Send the document to the user's device.

[0514] Input: Tailored action list and summary.

[0515] Output: Report sent to user terminal.

[0516] Device:

[0517] The terminal presents the received report to the user.

[0518] The user reviews the report and takes the indicated action.

[0519] Input: The report sent by the server.

[0520] Output: The report as displayed to the user.

[0521] (Application example 2)

[0522] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0523] With the spread of autonomous vehicles, drivers are required to efficiently and safely obtain important information while driving. Conventional systems require drivers to manually check emails, determine their importance, and decide on an action, which can distract them from driving. Furthermore, the lack of appropriate information provided takes into account the driver's emotional state, which can increase stress and fatigue. The present invention solves these problems, enabling drivers to safely and efficiently obtain email information and take appropriate action.

[0524] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0525] In this invention, the server includes means for collecting emails, means for classifying the collected emails into a plurality of categories, means for evaluating the importance of each email, means for summarizing the contents of the emails for each category, means for generating a list of actions to be performed by the user, means for delivering the summaries and the action list to the user, means for detecting the driver's state, and means for adjusting the action list based on emotional information. This allows the driver to quickly identify important emails while driving, and to sense stress and fatigue and respond efficiently based on the action list with adjusted priorities.

[0526] "Email" is a digital message sent and received over a computer network.

[0527] "Means of collection" refers to the communication protocols and software functions for obtaining email from a specific mail server.

[0528] A "classification means" is an algorithm or software function that sorts collected emails into categories based on their content.

[0529] A "means for assessing importance" is an algorithm or system that scores importance based on factors such as the content of an email and the sender.

[0530] "Summarization means" refers to natural language processing techniques and software functions that extract key information from lengthy emails and present it in a short form.

[0531] The "means for generating an action list" is a function that creates a list of specific tasks that the user should perform based on the summarized content and importance rating of the email.

[0532] "Means for distribution" refers to the communication protocol and software functions for transferring the generated summary and action list to the user's terminal and displaying or notifying them.

[0533] The "means for adjusting the action list based on emotional information" is a function that analyzes the driver's emotional state and automatically adjusts the priority and content of the action list according to that state.

[0534] "Means for sensing driver state" refers to sensors and software functions that analyze the driver's voice, facial expressions, and behavior to detect their emotional state.

[0535] This system allows drivers to safely and efficiently manage their emails. It collects emails, automatically classifies them, evaluates their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. The system is primarily composed of a server, a terminal, and users.

[0536] System Configuration

[0537] The server uses the IMAP protocol to collect emails from the mail server. The collected emails are stored in a database, and metadata (sender, recipient, subject, body, date and time) is saved. The server then applies natural language processing (NLP) technology to classify each email into multiple categories, such as work instructions, customer contact, information sharing, company-wide notices, and promotional advertisements. The classified emails are then scored using an importance assessment algorithm.

[0538] Based on the importance rating, the server extracts key information from the body of each email and uses natural language generation technology to generate a concise summary, helping drivers quickly grasp the information they need.

[0539] The server then generates an action list containing specific tasks the user needs to perform, prioritizing them in order of importance.The emotion engine also recognizes emotions from the user's voice, facial expressions, and text input, and reflects them in the action list.

[0540] Finally, the server compiles this information into a single report and sends it to the device (such as a smartphone), which notifies the user by voice or on-screen display.

[0541] Hardware and software used

[0542] Hardware

[0543] Server: A high-performance server for collecting emails, classifying them, assessing their importance, generating summaries, recognizing emotions, and creating action lists.

[0544] User device: smartphone or tablet

[0545] software

[0546] IMAP protocol: for collecting emails

[0547] Natural Language Processing (NLP) algorithms: for email classification and summary generation

[0548] Natural language generation technology: for content summarization

[0549] Emotion Engine: Recognizes the driver's emotional state

[0550] Database: To store collected emails

[0551] Specific examples

[0552] For example, a driver receives the following email on October 1, 2023:

[0553] First message: "Work instructions: Create materials for a new project"

[0554] 2nd message: "Contact from customer: Product inquiry"

[0555] Third email: "Information sharing: Market trend report"

[0556] 4th message: "Company-wide notice: System maintenance notice"

[0557] 5th message: "Advertisement: Announcement of new product release"

[0558] The server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, resulting in the following output:

[0559] Summary example

[0560] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0561] Summary of customer email: "I need more information about product features"

[0562] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0563] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0564] Promotional email summary: "New product coming next month."

[0565] Action list to be taken

[0566] 1. Creating materials for a new project

[0567] 2. Detailed explanation of product features to customers

[0568] 3. Prepare for system maintenance this weekend

[0569] Example prompts for generative AI models

[0570] Summarize the following email: "This is a customer inquiry. They would like more information on how to use the product. It's urgent."

[0571] The system is designed to help drivers respond to important emails while continuing to drive safely. It provides a concise summary of the email's key content and actions, reads them out loud, and uses emotion recognition to suggest the most appropriate response.

[0572] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0573] Step 1:

[0574] The server uses the IMAP protocol to collect all incoming emails from the mail server for a specified period (e.g., one day) and stores them in a database, including their metadata (sender, recipient, subject, body, date and time).

[0575] Input: Mail server account information, period specification

[0576] Output: Email list (including metadata)

[0577] Step 2:

[0578] The server retrieves the collected emails from the database and uses natural language processing (NLP) technology to analyze the email content and classify it into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[0579] Input: Email data collected in Step 1

[0580] Output: A list of emails sorted by category

[0581] Step 3:

[0582] The server calculates the importance score for each email based on the classified email content. For example, emails containing keywords such as "urgent" or "urgent" or emails from specific senders are given a higher score.

[0583] Input: Email data classified in Step 2

[0584] Output: Email data with importance scores assigned

[0585] Step 4:

[0586] The server uses natural language generation technology to generate a summary of each email, extracting key information from the email body and creating a concise summary of a few lines.

[0587] Input: Email data scored in Step 3

[0588] Output: Email data with summary generated

[0589] Step 5:

[0590] The server generates a list of actions for the user to perform based on the importance score and the summary sentence, and prioritizes the list of actions in order of importance.

[0591] Input: Email data summarized in step 4

[0592] Output: A list of actions ordered by importance

[0593] Step 6:

[0594] The server uses an emotion engine to recognize the user's (driver's) emotional state by analyzing data such as voice, facial expressions, and text input. Based on the recognized emotion, it adjusts the action list and notification content.

[0595] Input: Driver's voice, facial expressions, and text data

[0596] Output: Recognized emotional state and adjusted action list

[0597] Step 7:

[0598] The server compiles the generated summary, action list, and a report including adjustments based on the emotion information into a single document and sends it to the user's terminal.

[0599] Input: Summary sentence generated in step 4, Action list created in step 5, Emotion information recognized in step 6

[0600] Output: Report displayed on the user's terminal

[0601] Step 8:

[0602] The user's device will notify the user of the received report via voice or screen display, allowing the driver to quickly grasp important information while driving and efficiently take necessary actions.

[0603] Input: Report sent from the server

[0604] Output: Notification and display to the driver

[0605] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0606] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0607] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0608] [Second embodiment]

[0609] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0610] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0611] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0612] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0613] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0614] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0615] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0616] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0617] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0618] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0619] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0620] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0621] The present invention relates to a system that collects e-mails, automatically classifies them, evaluates their importance, generates summaries, creates action lists, and distributes the results of these processes. This system is composed of the following elements: a server, a terminal, and a user.

[0622] System Overview

[0623] The system helps users manage their email more efficiently by providing a range of features, including:

[0624] 1. Email Collection:

[0625] The server accesses the mail server and collects received emails for a specific period (e.g., one day).

[0626] 2. Email Classification:

[0627] The server analyzes the content of the emails collected and classifies them into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0628] 3. Importance rating:

[0629] The server calculates an importance score based on the content of each email and prioritizes the emails.

[0630] 4. Summary generation:

[0631] The server uses machine learning and natural language generation techniques to generate a summary of each email.

[0632] 5. Create an action list:

[0633] The server creates a list of actions for the user to take based on the importance score and summary.

[0634] 6. Delivery of results:

[0635] The server sends the generated summary and action list to the terminal and displays them to the user.

[0636] Program processing overview

[0637] Email collection

[0638] 1. Server:

[0639] Every night, all received emails for the day are retrieved from the mail server using the IMAP protocol.

[0640] Store the retrieved emails in a database and save the metadata of each email (sender, recipient, subject, body, date and time).

[0641] Email Classification

[0642] 2. Server:

[0643] Each email is extracted from the database and its content is analyzed using natural language processing technology.

[0644] Based on the analysis results, the information is classified into the following categories: work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0645] The classification results are stored in a database.

[0646] Importance rating

[0647] 3. Server:

[0648] Scoring the importance of each email based on a list of keywords and senders.

[0649] Emails containing important keywords (e.g., "urgent," "important") or from specific senders are given a higher score.

[0650] Save the importance score in a database.

[0651] Summary Generation

[0652] 4. Server:

[0653] Natural language generation technology is used to generate a summary of each email.

[0654] Divide each email into a concise, few-line summary (e.g., "New project materials needed to be created").

[0655] The generated summary is stored in a database.

[0656] Action List Creation

[0657] 5. Server:

[0658] Based on the importance score and summary, a list of actions for the user to perform is generated.

[0659] Action lists are prioritized according to importance (e.g., 1. Create documentation for new projects, 2. Respond to customer inquiries).

[0660] Store the action list in a database.

[0661] Results distribution

[0662] 6. Server:

[0663] The generated summary and action list are compiled into a single report.

[0664] Send the report to the user's device.

[0665] 7. Terminal:

[0666] Receive the report and display it to the user (e.g., display it on the notification screen of a PC or smartphone).

[0667] Specific examples

[0668] For example, if the incoming email on October 1, 2023 is:

[0669] First message: "Work instructions: Create materials for a new project"

[0670] 2nd message: "Contact from customer: Product inquiry"

[0671] Third email: "Information sharing: Market trend report"

[0672] 4th message: "Company-wide notice: System maintenance notice"

[0673] 5th message: "Advertisement: Announcement of new product release"

[0674] Once the server has collected the above emails, classified them, ranked them by importance, generated summaries, and created action lists, the output will look something like this:

[0675] Summary example

[0676] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0677] Summary of customer email: "I need more information about product features"

[0678] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0679] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0680] Promotional email summary: "New product coming next month."

[0681] Action list to be taken

[0682] 1. Creating materials for a new project

[0683] 2. Detailed explanation of product features to customers

[0684] 3. Prepare for system maintenance this weekend

[0685] In this way, the system allows users to quickly grasp important information and efficiently take necessary actions.

[0686] The processing flow will be explained below.

[0687] Step 1:

[0688] The server connects to the mail server and retrieves all incoming emails for a specific period (e.g., one day) using the IMAP protocol. The server stores the retrieved emails in temporary storage.

[0689] Step 2:

[0690] The server reads the emails retrieved from temporary storage and stores them in a database, along with the metadata for each email (sender, recipient, subject, body, date and time).

[0691] Step 3:

[0692] The server retrieves each email stored in the database and analyzes its content using natural language processing (NLP) technology. Based on the analysis results, the email is automatically classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[0693] Step 4:

[0694] For each email that is categorized, the server calculates an importance score using keyword matching and sender lists. For example, emails containing keywords like "urgent" or "urgent" or from specific senders will receive a higher score.

[0695] Step 5:

[0696] The server uses natural language generation (NLG) technology to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[0697] Step 6:

[0698] The server generates a list of actions for the user to perform based on the importance scores and the generated summary, with the list of actions prioritized in order of importance.

[0699] Step 7:

[0700] The server compiles the summary and action list into a single report, which contains a summary of the messages in each category and the corresponding actions.

[0701] Step 8:

[0702] The server sends the compiled report to the user's device, and the report is provided to the user in the form of email, app notification, or dashboard display.

[0703] Step 9:

[0704] The terminal displays the received report to the user, who can then check the report and take the action indicated.

[0705] Example 1

[0706] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0707] In today's business environment, email is an important means of communication, but it can be difficult to efficiently manage the large amount of information and quickly take necessary action. Conventional email management systems lack the functionality to properly evaluate the importance and content of emails and clearly present the actions that users should take. As a result, users often miss important information or respond late. There is a need to solve this problem and improve user work efficiency.

[0708] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0709] In this invention, the server includes means for analyzing the metadata of emails and storing it in a database, means for analyzing the content of emails using natural language processing technology, and means for generating summaries of emails using natural language generation technology, which enable accurate evaluation of the importance of emails and specific suggestions for actions to be taken by the user.

[0710] "Email" means a digital message sent or received over the Internet or other communications network.

[0711] "Metadata" refers to information attached to an email, such as the sender, recipient, subject, body, date and time.

[0712] A "database" is a collection of digital data that systematically stores and manages collected emails and metadata.

[0713] "Natural language processing technology" refers to the technology of analyzing and understanding human language using a computer, and includes techniques such as tokenizing words and analyzing sentence structure.

[0714] "Natural language generation technology" refers to a technology that allows a computer to generate text in natural human language, and is used to summarize emails and extract information.

[0715] An "abstract" is a short sentence that succinctly summarizes the main points of the email.

[0716] The "importance score" is a number that indicates the importance of an email, calculated based on the email content and sender list.

[0717] An "action list" is a list of specific tasks or action items that a user must perform.

[0718] "Priority" refers to the ordering of multiple tasks or action items based on their importance.

[0719] "Report" refers to information provided to a user in the form of a document summarizing emails and / or a list of actions.

[0720] This invention relates to a system that collects, automatically classifies, evaluates the importance of, summarizes, creates an action list, and distributes e-mails, and is composed of elements of a server, a terminal, and a user. A specific embodiment of this system is shown below.

[0721] System Configuration

[0722] The system mainly consists of the following components:

[0723] Server: Use a Linux server or cloud server (e.g. AWS EC2).

[0724] Database: Use a relational database such as MySQL or PostgreSQL.

[0725] Natural Language Processing (NLP) techniques: Use Python's NLTK and SpaCy libraries.

[0726] Natural Language Generation (NLG) technology: Uses OpenAI's GPT-3 or the latest GPT-4 model.

[0727] Terminal: A user device such as a PC or smartphone.

[0728] System Features

[0729] The system helps users manage their email more efficiently by providing the following features:

[0730] 1. Email Collection:

[0731] The server accesses the mail server and uses the IMAP protocol to collect incoming emails for a specific period (e.g., one day).

[0732] The collected email metadata (sender, recipient, subject, body, date and time) is stored in a database.

[0733] 2. Email Classification:

[0734] The server retrieves each stored email and analyzes its content using natural language processing technology.

[0735] Based on the analysis results, the data is classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising, and stored in a database.

[0736] 3. Importance rating:

[0737] The server calculates an importance score for each email based on its content, keywords, and sender list.

[0738] Save the importance score to the database and update the importance field of the email.

[0739] 4. Summary generation:

[0740] The server uses natural language generation techniques to generate a summary of each email.

[0741] The prompt sentence "Generate a summary of the following email: Email content" is input into the model.

[0742] The generated summary is stored in a database and the summary field corresponding to each email is updated.

[0743] 5. Create an action list:

[0744] The server lists action items for the user to perform based on the importance scores and the generated summary.

[0745] The action list is prioritized based on importance and stored in a database.

[0746] 6. Delivery of results:

[0747] The server compiles the generated summary and action list into a single report and sends it to the user's terminal.

[0748] The device displays the report received from the server to the user on a PC or smartphone.

[0749] Specific examples

[0750] For example, if the incoming email on October 1, 2023 is:

[0751] First message: "Work instructions: Create materials for a new project"

[0752] 2nd message: "Contact from customer: Product inquiry"

[0753] Third email: "Information sharing: Market trend report"

[0754] 4th message: "Company-wide notice: System maintenance notice"

[0755] 5th message: "Advertisement: Announcement of new product release"

[0756] Once the server has collected these emails, sorted them, ranked them by importance, generated summaries, and created action lists, the result will look something like this:

[0757] Summary example

[0758] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0759] Summary of customer email: "I need more information about product features"

[0760] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0761] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0762] Promotional email summary: "New product coming next month."

[0763] Action list to be taken

[0764] 1. Creating materials for a new project

[0765] 2. Detailed explanation of product features to customers

[0766] 3. Prepare for system maintenance this weekend

[0767] Prompt Sentence Examples

[0768] An example of a prompt sentence to input to the generative AI model is "Generate a summary of the following email: Work instructions: Create materials for a new project."

[0769] In this way, by using the system of the present invention, the user can quickly grasp important information and efficiently take the necessary actions.

[0770] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0771] Specific flow of program processing

[0772] Each processing step is described in detail below.

[0773] Step 1: Email Collection

[0774] server:

[0775] Input: Connect to a mail server (for example, using the IMAP protocol) at a fixed time every night.

[0776] Specific operation: The server retrieves unread emails and newly received emails for a specified period (e.g., the past 24 hours).

[0777] Data processing: Analyze the metadata of the acquired emails (sender, recipient, subject, body, date and time).

[0778] Output: Save the email metadata and body to a database.

[0779] Step 2: Email Classification

[0780] server:

[0781] Input: Collected emails stored in a database.

[0782] What it does: The server analyzes the content of each email using natural language processing (NLP) techniques, such as Python's NLTK or SpaCy.

[0783] Data processing: Based on the analysis results, data is classified into the following categories: work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0784] Output: Store the email classification results in a database and assign each email a corresponding category tag.

[0785] Step 3: Severity Assessment

[0786] server:

[0787] Input: Classified email.

[0788] What it does: The server calculates an importance score for each email based on its content, keywords, and sender list.

[0789] Data processing: For example, emails containing keywords such as "urgent" or "important" or emails from specific senders (such as your boss or important customers) are given a higher score.

[0790] Output: Save the importance score to the database and update the importance field of the email.

[0791] Step 4: Summary generation

[0792] server:

[0793] Input: Emails with importance scores.

[0794] What it does: It uses natural language generation technology (NLG technology, e.g., OpenAI's GPT-3 or GPT-4 models) to convert the content of each email into a concise summary.

[0795] Data processing: The prompt sentence "Generate a summary of the following email: Email content" is input into the model.

[0796] Output: Store the generated summaries in a database and update the summary field for each email.

[0797] Step 5: Create an Action List

[0798] server:

[0799] Input: Email with summary and importance score.

[0800] Specific Action: Based on the importance score and the generated summary, a list of action items for the user to take is generated.

[0801] Data manipulation: Prioritize action items based on importance.

[0802] Output: Stores the action list in the database and updates the user's action list field.

[0803] Step 6: Delivering results

[0804] server:

[0805] Input: The data from which the action list and summary are generated.

[0806] Specific behavior: Compile the generated summary and action list into a single report, for example, generate a report in PDF or HTML format.

[0807] Output: Sends the report to the user's terminal.

[0808] Device:

[0809] Input: The report received from the server.

[0810] Specific operation: The device displays the report to the user on a PC or smartphone, and notifies the user in real time using the notification function.

[0811] Output: User reviews the report and decides what action to take next.

[0812] By implementing this system's series of processes, users can quickly grasp important information and efficiently take the necessary actions.

[0813] (Application example 1)

[0814] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0815] In modern factories, a great deal of information is communicated via email, but it is difficult to efficiently manage the vast volume of emails and quickly take necessary actions. It is also difficult to distinguish between important and unimportant emails, which increases the risk of overlooking important information. This reduces factory operational efficiency and has a negative impact on productivity. To solve these problems, proper email management, automatic classification, importance assessment, rapid summary generation, and the creation and distribution of action lists are required.

[0816] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0817] In this invention, the server includes means for collecting emails, means for classifying the collected emails into multiple categories, means for rating the importance of each email, means for summarizing the content of the emails for each category, means for generating a list of actions to be taken by the user, means for distributing the summaries and the action list to the user, and means for automatically processing emails related to factory work and notifying the results to the manager's terminal, thereby enabling the factory manager to efficiently manage a huge amount of emails, quickly grasp important information, and immediately take necessary actions.

[0818] "Means for collecting email" refers to a system or software that connects to a mail server via the Internet and obtains emails sent to a specific account.

[0819] A "multiple categorization means" is an algorithm or software that automatically sorts retrieved emails into predefined categories based on their content.

[0820] The "means for assessing importance" is a system or software that analyzes the content of each email and calculates and scores the importance of the email.

[0821] The "summarization means" refers to a system or software that extracts key points from the body of an email and generates a concise summary using a machine learning algorithm or similar.

[0822] The "means for generating an action list" is a system or software that lists and prioritizes tasks that a user should perform based on the summary and importance score of each email.

[0823] The "distribution means" is a system or software for sending the generated summary and action list to the user's terminal and notifying them.

[0824] "Means for automatically processing emails related to factory operations and notifying the results to the manager's terminal" refers to a system or software that collects and analyzes emails containing business communications, instructions, progress reports, etc. in the factory, and delivers the results to the manager in real time.

[0825] This invention relates to a system that collects emails, automatically classifies them, evaluates their importance, creates summaries and action lists, and distributes the results. This system aims to efficiently manage business emails, particularly in factories, reducing the burden on managers and enabling them to quickly grasp important information.

[0826] The system is mainly composed of the server, terminal, and user elements. The role of each element and the specific processing method are explained below.

[0827] Server Roles and Operations

[0828] The server plays a central role in collecting, classifying, assessing the importance of emails, generating summaries, and creating and distributing action lists.

[0829] Email collection

[0830] Every night, the server retrieves incoming emails from the mail server using the IMAP protocol, and stores all email metadata (sender, recipient, subject, body, date and time) in a database.

[0831] Email Classification

[0832] The server retrieves emails from the database and analyzes their contents using natural language processing (NLP) technology, categorizing them into categories such as work instructions, production progress, emergency contacts, maintenance information, and advertisements. A keyword-based classification algorithm is used for the specific analysis.

[0833] Importance rating

[0834] The importance of each email is scored based on keywords and senders. Emails containing important keywords (e.g., "urgent" or "important") or from specific senders are given a higher score. This importance score is stored in a database.

[0835] Summary Generation

[0836] We use natural language generation techniques to generate summaries of each email, specifically by extracting a concise, few-line summary from the body of the email using the gensim library.

[0837] Action List Creation

[0838] Based on the importance score and summary, a list of actions for the user to perform is generated, prioritized according to importance.

[0839] Results distribution

[0840] The generated summary and action list are compiled and delivered to the user's terminal. The delivered notification is displayed on the administrator's terminal.

[0841] Device Role

[0842] The user's device receives the summary and action list sent from the server and notifies the administrator. The application on the device displays the notification content in an easy-to-understand manner, allowing the administrator to take action immediately.

[0843] User Roles

[0844] The administrator (user) can quickly take necessary action based on the provided summary and action list. For example, they can prioritize tasks of high importance, such as creating a production plan for a new project or responding to machine failures.

[0845] Specific examples

[0846] For example, if the following email is received on October 1, 2023:

[0847] Work instruction: "Create a production plan for a newly ordered project."

[0848] Manufacturing Progress: "Report on the current progress of the production line"

[0849] Emergency call: "There is a machine breakdown and we need to fix it quickly."

[0850] Maintenance information: "Check the scheduled maintenance schedule"

[0851] Advertisement: "Sales announcement for new materials from suppliers"

[0852] The server then categorizes and rates each email, generating a summary and action list like this:

[0853] Summary example

[0854] Summary of work instruction email: "Create a production plan for the new project this week"

[0855] Summary of manufacturing progress email: "Production line progress is on schedule"

[0856] Emergency message summary: "Machine breakdown, immediate repair required."

[0857] Maintenance information summary: "Scheduled maintenance scheduled for next week"

[0858] Summary of the advertising email: "Information on new materials"

[0859] Action list to be taken

[0860] 1. Creating production plans for new projects

[0861] 2. Response to machine breakdowns

[0862] 3. Preparation for scheduled maintenance

[0863] 4. Checking production progress

[0864] Example prompts for generative AI models

[0865] "Analyze the following email and generate a summary:

[0866] Subject: Create a production plan for a new project

[0867] Body: We need to create a production plan for a newly ordered project this week. Please confirm the detailed specifications and required resources and create an appropriate schedule.

[0868] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0869] Step 1:

[0870] The server establishes a connection to the mail server using the IMAP protocol and retrieves all emails received within a specific period of time. The server stores the metadata of the retrieved emails (sender, recipient, subject, body, date and time) in a database, making all incoming emails available as target input data.

[0871] Step 2:

[0872] The server retrieves each email from the database and analyzes its content using natural language processing technology. Based on the content of the email, the server categorizes it into categories such as work instructions, production progress, emergency contacts, maintenance information, and advertisements. The analyzed data is then stored in the database as output data categorized by category.

[0873] Step 3:

[0874] The server calculates an importance score based on the classified emails. Emails containing important keywords (e.g., "urgent" or "important") or from specific senders are given a higher score. This weights the importance of each email. The score is calculated based on the input data and added to the output data as the importance of each email.

[0875] Step 4:

[0876] The server uses natural language generation techniques, such as the gensim library, to generate a summary of each email's content. The body of each email is summarized into a concise, few-line summary, which is then stored in a database as the output of the analysis.

[0877] Step 5:

[0878] Based on the importance scores and summaries, the server generates a list of actions for the user to perform. The action list is prioritized by importance and identifies tasks to be performed. This action list is stored in a database as part of the output data.

[0879] Step 6:

[0880] The server compiles the generated summary and action list into a single report and sends it to the user's device, allowing the administrator to receive notifications on their smartphone or other device.

[0881] Step 7:

[0882] The terminal displays the received report and notifies the user. Specifically, the terminal's notification function is used to display a summary and action list, allowing the user to quickly take necessary actions based on this.

[0883] Prompt Sentence Examples

[0884] For example, the prompt to verify the generated summary is:

[0885] "Analyze the following email and generate a summary:

[0886] Subject: Create a production plan for a new project

[0887] Body: We need to create a production plan for a newly ordered project this week. Please confirm the detailed specifications and required resources and create an appropriate schedule.

[0888] Through the flow of steps, a system is realized that allows servers, terminals, and users to work together to significantly improve factory operational efficiency.

[0889] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0890] This invention relates to a system that collects emails, automatically classifies them, assesses their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. This system is composed of a server, a terminal, and users, and by combining it with an emotion engine, it provides information and generates action lists that take into account the user's emotional state.

[0891] System Overview

[0892] The system will help users streamline email management and emotion recognition by providing a series of features, including:

[0893] 1. Email Collection:

[0894] The server accesses the mail server and collects all incoming emails for a specific period (e.g., one day).

[0895] 2. Email Classification:

[0896] The server analyzes the content of the emails collected and classifies them into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[0897] 3. Importance rating:

[0898] The server calculates an importance score based on the content of each email and prioritizes it.

[0899] 4. Summary generation:

[0900] The server uses machine learning and natural language generation techniques to generate a summary of each email.

[0901] 5. Create an action list:

[0902] The server generates a list of actions for the user to perform based on the importance score and summary text.

[0903] 6. Emotion recognition:

[0904] The server uses an emotion engine that recognizes the user's emotions, and reflects the emotion information when generating the email contents and action list.

[0905] 7. Delivery of results:

[0906] The server generates a report including the summary, action list, and emotion-based adjustments and sends it to the terminal for display to the user.

[0907] Program processing overview

[0908] Email collection

[0909] 1. Server:

[0910] Every night, all received emails for the day are retrieved from the mail server using the IMAP protocol.

[0911] Store the retrieved emails in a database and save the metadata of each email (sender, recipient, subject, body, date and time).

[0912] Email Classification

[0913] 2. Server:

[0914] The email is retrieved from the database and its content is analyzed using natural language processing techniques.

[0915] Based on the analysis results, the messages are sorted into the following categories: work instructions, customer contact, information sharing, company-wide announcements, and advertising.

[0916] The classification results are stored in a database.

[0917] Importance rating

[0918] 3. Server:

[0919] The importance of each email is scored based on a list of keywords and senders. For example, emails containing keywords such as "urgent" or "urgent" or from specific senders are given a higher score.

[0920] Save the importance score in a database.

[0921] Summary Generation

[0922] 4. Server:

[0923] Natural language generation technology is used to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[0924] The generated summaries are stored in a database.

[0925] Action List Creation

[0926] 5. Server:

[0927] Based on the importance score and summary, a list of actions to be performed by the user is generated. The action list is prioritized in order of importance (e.g., 1. Create documents for a new project, 2. Respond to customer inquiries).

[0928] Store the action list in a database.

[0929] emotion recognition

[0930] 6. Server:

[0931] Recognize the user's emotional state using an emotion engine, which analyzes emotional information detected from the user's voice, facial expressions, text input, etc.

[0932] The system adjusts email content and action list generation based on the user's recognized emotional state. For example, if the user is feeling stressed, it suggests prioritizing high-priority emails.

[0933] Results distribution

[0934] 7. Server:

[0935] The generated summary, action list, and report reflecting the sentiment information are compiled into a single document.

[0936] Send the report to the user's device.

[0937] 8. Terminal:

[0938] The received report is displayed to the user, who then reviews the report and takes the action indicated.

[0939] Specific examples

[0940] For example, if the incoming email on October 1, 2023 is:

[0941] First message: "Work instructions: Create materials for a new project"

[0942] 2nd message: "Contact from customer: Product inquiry"

[0943] Third email: "Information sharing: Market trend report"

[0944] 4th message: "Company-wide notice: System maintenance notice"

[0945] 5th message: "Advertisement: Announcement of new product release"

[0946] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[0947] Summary example

[0948] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0949] Summary of customer email: "I need more information about product features"

[0950] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0951] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0952] Promotional email summary: "New product coming next month."

[0953] Action list to be taken

[0954] 1. Creating materials for a new project

[0955] 2. Detailed explanation of product features to customers

[0956] 3. Prepare for system maintenance this weekend

[0957] emotion recognition

[0958] If the user is stressed, adjust the action list to be shorter and focus on the most important tasks to reduce stress.

[0959] In this way, the system allows users to quickly grasp important information and take necessary actions efficiently.By combining emotion engines, it provides adaptive support according to the user's emotional state.

[0960] The processing flow will be explained below.

[0961] Step 1:

[0962] The server connects to the mail server and retrieves all incoming emails for a specific period (e.g., one day) using the IMAP protocol. The server stores the retrieved emails in temporary storage.

[0963] Step 2:

[0964] The server reads the emails retrieved from temporary storage and stores them in a database, along with the metadata for each email (sender, recipient, subject, body, date and time).

[0965] Step 3:

[0966] The server retrieves each email stored in the database and analyzes its content using natural language processing (NLP) technology. Based on the analysis results, the email is automatically classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[0967] Step 4:

[0968] For each email that is categorized, the server calculates an importance score using keyword matching and sender lists. For example, emails containing keywords like "urgent" or "urgent" or from specific senders will be given a higher score.

[0969] Step 5:

[0970] The server uses natural language generation (NLG) technology to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[0971] Step 6:

[0972] The server generates a list of actions for the user to perform based on the importance scores and the generated summary, with the list of actions prioritized in order of importance.

[0973] Step 7:

[0974] The server uses an emotion engine to recognize the user's emotional state, which analyzes emotional information from the user's voice, facial expressions, text input, etc.

[0975] Step 8:

[0976] The server adjusts the action list or summary based on the perceived emotional state, for example, if the user is feeling stressed, prioritizing important tasks to reduce stress.

[0977] Step 9:

[0978] The server compiles the summary and action list into a single report, which contains a summary of the messages in each category and the corresponding actions.

[0979] Step 10:

[0980] The server sends the compiled report to the user's device, and the report is provided to the user in the form of email, app notification, or dashboard display.

[0981] Step 11:

[0982] The terminal displays the received report to the user, who then checks the report and takes the action indicated.

[0983] Step 12:

[0984] The user performs the actions indicated and inputs the results as feedback to the system, which allows the system to provide a more accurate action list and summary.

[0985] For example, if the email you received on October 1, 2023 is:

[0986] First message: "Work instructions: Create materials for a new project"

[0987] 2nd message: "Contact from customer: Product inquiry"

[0988] Third email: "Information sharing: Market trend report"

[0989] 4th message: "Company-wide notice: System maintenance notice"

[0990] 5th message: "Advertisement: Announcement of new product release"

[0991] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[0992] Summary example

[0993] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[0994] Summary of customer email: "I need more information about product features"

[0995] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[0996] Summary of company-wide email: "System maintenance scheduled for this weekend."

[0997] Promotional email summary: "New product coming next month."

[0998] Action list to be taken

[0999] 1. Creating materials for a new project

[1000] 2. Detailed explanation of product features to customers

[1001] 3. Prepare for system maintenance this weekend

[1002] emotion recognition

[1003] If users are stressed, shorten the action list and adjust it to focus on the most important tasks.

[1004] In this way, the system allows users to quickly grasp important information and take necessary actions efficiently.By combining emotion engines, it provides adaptive support according to the user's emotional state.

[1005] Example 2

[1006] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1007] In conventional email management systems, received emails are classified, their importance is assessed, summaries are generated, and action lists are created individually, without taking into account the user's emotional state. As a result, users can become overwhelmed by the volume of emails and may miss important emails or tasks. Furthermore, when users are under stress, it is difficult for them to process emails appropriately.

[1008] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting emails, means for classifying the collected emails into a plurality of categories, means for evaluating the importance of each email, means for summarizing the contents of the emails for each category, means for generating a list of actions to be performed by the user, means for delivering the summaries and action list to the user, means for recognizing the user's emotional state, and means for adjusting the action list and summaries according to the emotional state. This enables the user to quickly grasp important information and receive adaptive support according to their emotional state.

[1009] "Email" means a digital message sent or received over the Internet or other digital network.

[1010] "Category" refers to a group of emails classified based on their contents, and examples include work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[1011] An "importance score" is a number used to rate the importance of an email and is calculated based on specific keywords and sender lists.

[1012] "Natural language generation technology" refers to technology that generates content in natural language that humans can understand, and includes machine learning and AI technologies.

[1013] A "summary" is a short sentence that extracts the main points from a longer text and summarizes them concisely.

[1014] An "action list" is a list of specific tasks or actions that a user must perform, prioritized based on importance.

[1015] "Emotional state" refers to the user's psychological state, and includes emotions such as stress, joy, and impatience.

[1016] An "emotion engine" is hardware or software used to recognize a user's emotions, and can obtain emotional information by analyzing voice, facial expressions, text input, etc.

[1017] "User" means an individual or organization that uses this system.

[1018] MODE FOR CARRYING OUT THE INVENTION

[1019] This invention relates to a system that collects emails, automatically classifies them, assesses their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. This system is composed of a server, a terminal, and users, and by combining it with an emotion engine, it provides information and generates action lists that take into account the user's emotional state.

[1020] The system configuration is as follows: first, the server accesses the mail server and collects all received emails for a specific period (e.g., one day). At this stage, the IMAP protocol is used to retrieve the latest emails from the mail server (e.g., Gmail server) and store them in a database (e.g., MySQL). Email metadata (sender, recipient, subject, body, date and time) is also saved at the same time.

[1021] The server then retrieves the email from the database and analyzes its content using natural language processing technology (e.g., Google NLP API). Based on this analysis, the email is classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements. The classification results are also stored in the database.

[1022] The server also assigns a score to each email based on its importance, based on keywords (e.g., "urgent" or "urgent") and specific sender lists. This importance score is stored in a database for each email.

[1023] The server then uses machine learning and natural language generation techniques (e.g., OpenAI GPT) to generate a summary of each email. The summary extracts key information from the email body and presents it in a concise format of a few lines. The generated summary is also stored in a database.

[1024] Based on the generated summary and the importance score, the server generates a list of actions for the user to perform. The list of actions is prioritized in order of importance and stored in a database.

[1025] Furthermore, the server uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotional state. This analyzes emotional information detected from the user's voice, facial expressions, text input, etc. Based on the recognized emotional state, the email content and action list generation are adjusted. For example, if the user is feeling stressed, the server suggests prioritizing high-priority emails.

[1026] Finally, the server compiles the generated summary, action list, and emotional information into a report and sends it to the user's device. The device displays the received report to the user, who then checks the report and performs the requested actions.

[1027] Specific examples

[1028] Email received on October 1, 2023:

[1029] "Work instructions: Create materials for a new project"

[1030] "Customer Contact: Product Inquiry"

[1031] "Information Sharing: Market Trends Report"

[1032] "Company-wide Notice: System Maintenance Notice"

[1033] "Advertisement: Announcement of new product release"

[1034] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[1035] Summary example:

[1036] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1037] Summary of customer email: "I need more information about product features"

[1038] Summary of the information sharing email: "Certain sectors are experiencing rapid growth as a market trend."

[1039] Summary of company-wide email: "System maintenance is scheduled for this weekend."

[1040] Promotional email summary: "New product launching next month"

[1041] List of actions to be taken:

[1042] 1. Creating materials for a new project

[1043] 2. Detailed explanation of product features to customers

[1044] 3. Prepare for system maintenance this weekend

[1045] Emotion recognition:

[1046] If the user is stressed, shorten the action list to reduce stress and focus on the most important tasks.

[1047] Example of an input prompt for a generative AI model:

[1048] Analyze the content of received emails and categorize them into the following categories: work instructions, customer contact, information sharing, company-wide announcements, and promotional advertisements. Also, assign an importance score to each email and generate a summary. Then, create an action list and provide tailored results based on sentiment recognition.

[1049] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1050] Step 1:

[1051] Email collection

[1052] server:

[1053] Every night, the server connects to a mail server using the IMAP protocol, for example, to retrieve incoming emails from the Gmail server.

[1054] The retrieved emails are stored in a database (e.g., MySQL), and metadata for each email (sender, recipient, subject, body, date and time) is stored.

[1055] Input: Received email data from the mail server.

[1056] Output: Email data and metadata stored in a database.

[1057] Step 2:

[1058] Email Classification

[1059] server:

[1060] The server retrieves unclassified emails from the database and analyzes their content using natural language processing techniques (e.g., Google NLP API).

[1061] Based on the analysis results, the information is sorted into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1062] The classification results are stored in the database again.

[1063] Input: Unclassified email data from the database.

[1064] Output: Email data categorized by category.

[1065] Step 3:

[1066] Importance rating

[1067] server:

[1068] The server calculates an importance score for each email based on specific keywords (e.g., "urgent" or "urgent") and a list of specific senders.

[1069] An importance score is stored in the database for each email.

[1070] Input: Classified email data.

[1071] Output: Email data with importance scores assigned.

[1072] Step 4:

[1073] Summary Generation

[1074] server:

[1075] The server uses machine learning and natural language generation techniques (e.g., OpenAI GPT) to generate a summary of each email.

[1076] The summary extracts the key information from the email body and presents it in a concise format of a few lines.

[1077] The generated summaries are stored in a database.

[1078] Input: Email data with an importance score.

[1079] Output: The email data with the summary generated.

[1080] Step 5:

[1081] Action List Creation

[1082] server:

[1083] The server generates a list of actions for the user to perform based on the generated summary and importance score.

[1084] Action lists are prioritized in order of importance and stored in a database.

[1085] Input: The email data from which the summary was generated and the importance score.

[1086] Output: A prioritized list of actions.

[1087] Step 6:

[1088] emotion recognition

[1089] server:

[1090] The server uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotional state.

[1091] Emotional information is analyzed from the user's voice, facial expressions, text input, etc.

[1092] Based on the recognized emotional state, adjustments are made when generating email content and action lists.

[1093] Input: User voice, facial expressions, and text input.

[1094] Output: A tailored action list and summary.

[1095] Step 7:

[1096] Results distribution

[1097] server:

[1098] The server compiles the generated summary, action list, and report reflecting the emotional information into a single document.

[1099] Send the document to the user's device.

[1100] Input: Tailored action list and summary.

[1101] Output: Report sent to user terminal.

[1102] Device:

[1103] The terminal presents the received report to the user.

[1104] The user reviews the report and takes the indicated action.

[1105] Input: The report sent by the server.

[1106] Output: The report as displayed to the user.

[1107] (Application example 2)

[1108] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1109] With the spread of autonomous vehicles, drivers are required to efficiently and safely obtain important information while driving. Conventional systems require drivers to manually check emails, determine their importance, and decide on an action, which can distract them from driving. Furthermore, the lack of appropriate information provided takes into account the driver's emotional state, which can increase stress and fatigue. The present invention solves these problems, enabling drivers to safely and efficiently obtain email information and take appropriate action.

[1110] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1111] In this invention, the server includes means for collecting emails, means for classifying the collected emails into a plurality of categories, means for evaluating the importance of each email, means for summarizing the contents of the emails for each category, means for generating a list of actions to be performed by the user, means for delivering the summaries and the action list to the user, means for detecting the driver's state, and means for adjusting the action list based on emotional information. This allows the driver to quickly identify important emails while driving, and to sense stress and fatigue and respond efficiently based on the action list with adjusted priorities.

[1112] "Email" is a digital message sent and received over a computer network.

[1113] "Means of collection" refers to the communication protocols and software functions for obtaining email from a specific mail server.

[1114] A "classification means" is an algorithm or software function that sorts collected emails into categories based on their content.

[1115] A "means for assessing importance" is an algorithm or system that scores importance based on factors such as the content of an email and the sender.

[1116] "Summarization means" refers to natural language processing techniques and software functions that extract key information from lengthy emails and present it in a short form.

[1117] The "means for generating an action list" is a function that creates a list of specific tasks that the user should perform based on the summarized content and importance rating of the email.

[1118] "Means for distribution" refers to the communication protocol and software functions for transferring the generated summary and action list to the user's terminal and displaying or notifying them.

[1119] The "means for adjusting the action list based on emotional information" is a function that analyzes the driver's emotional state and automatically adjusts the priority and content of the action list according to that state.

[1120] "Means for sensing driver state" refers to sensors and software functions that analyze the driver's voice, facial expressions, and behavior to detect their emotional state.

[1121] This system allows drivers to safely and efficiently manage their emails. It collects emails, automatically classifies them, evaluates their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. The system is primarily composed of a server, a terminal, and users.

[1122] System Configuration

[1123] The server uses the IMAP protocol to collect emails from the mail server. The collected emails are stored in a database, and metadata (sender, recipient, subject, body, date and time) is saved. The server then applies natural language processing (NLP) technology to classify each email into multiple categories, such as work instructions, customer contact, information sharing, company-wide notices, and promotional advertisements. The classified emails are then scored using an importance assessment algorithm.

[1124] Based on the importance rating, the server extracts key information from the body of each email and uses natural language generation technology to generate a concise summary, helping drivers quickly grasp the information they need.

[1125] The server then generates an action list containing specific tasks the user needs to perform, prioritizing them in order of importance.The emotion engine also recognizes emotions from the user's voice, facial expressions, and text input, and reflects them in the action list.

[1126] Finally, the server compiles this information into a single report and sends it to the device (such as a smartphone), which notifies the user by voice or on-screen display.

[1127] Hardware and software used

[1128] Hardware

[1129] Server: A high-performance server for collecting emails, classifying them, assessing their importance, generating summaries, recognizing emotions, and creating action lists.

[1130] User device: smartphone or tablet

[1131] software

[1132] IMAP protocol: for collecting emails

[1133] Natural Language Processing (NLP) algorithms: for email classification and summary generation

[1134] Natural language generation technology: for content summarization

[1135] Emotion Engine: Recognizes the driver's emotional state

[1136] Database: To store collected emails

[1137] Specific examples

[1138] For example, a driver receives the following email on October 1, 2023:

[1139] First message: "Work instructions: Create materials for a new project"

[1140] 2nd message: "Contact from customer: Product inquiry"

[1141] Third email: "Information sharing: Market trend report"

[1142] 4th message: "Company-wide notice: System maintenance notice"

[1143] 5th message: "Advertisement: Announcement of new product release"

[1144] The server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, resulting in the following output:

[1145] Summary example

[1146] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1147] Summary of customer email: "I need more information about product features"

[1148] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[1149] Summary of company-wide email: "System maintenance scheduled for this weekend."

[1150] Promotional email summary: "New product coming next month."

[1151] Action list to be taken

[1152] 1. Creating materials for a new project

[1153] 2. Detailed explanation of product features to customers

[1154] 3. Prepare for system maintenance this weekend

[1155] Example prompts for generative AI models

[1156] Summarize the following email: "This is a customer inquiry. They would like more information on how to use the product. It's urgent."

[1157] The system is designed to help drivers respond to important emails while continuing to drive safely. It provides a concise summary of the email's key content and actions, reads them out loud, and uses emotion recognition to suggest the most appropriate response.

[1158] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1159] Step 1:

[1160] The server uses the IMAP protocol to collect all incoming emails from the mail server for a specified period (e.g., one day) and stores them in a database, including their metadata (sender, recipient, subject, body, date and time).

[1161] Input: Mail server account information, period specification

[1162] Output: Email list (including metadata)

[1163] Step 2:

[1164] The server retrieves the collected emails from the database and uses natural language processing (NLP) technology to analyze the email content and classify it into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[1165] Input: Email data collected in Step 1

[1166] Output: A list of emails sorted by category

[1167] Step 3:

[1168] The server calculates the importance score for each email based on the classified email content. For example, emails containing keywords such as "urgent" or "urgent" or emails from specific senders are given a higher score.

[1169] Input: Email data classified in Step 2

[1170] Output: Email data with importance scores assigned

[1171] Step 4:

[1172] The server uses natural language generation technology to generate a summary of each email, extracting key information from the email body and creating a concise summary of a few lines.

[1173] Input: Email data scored in Step 3

[1174] Output: Email data with summary generated

[1175] Step 5:

[1176] The server generates a list of actions for the user to perform based on the importance score and the summary sentence, and prioritizes the list of actions in order of importance.

[1177] Input: Email data summarized in step 4

[1178] Output: A list of actions ordered by importance

[1179] Step 6:

[1180] The server uses an emotion engine to recognize the user's (driver's) emotional state by analyzing data such as voice, facial expressions, and text input. Based on the recognized emotion, it adjusts the action list and notification content.

[1181] Input: Driver's voice, facial expressions, and text data

[1182] Output: Recognized emotional state and adjusted action list

[1183] Step 7:

[1184] The server compiles the generated summary, action list, and a report including adjustments based on the emotion information into a single document and sends it to the user's terminal.

[1185] Input: Summary sentence generated in step 4, Action list created in step 5, Emotion information recognized in step 6

[1186] Output: Report displayed on the user's terminal

[1187] Step 8:

[1188] The user's device will notify the user of the received report via voice or screen display, allowing the driver to quickly grasp important information while driving and efficiently take necessary actions.

[1189] Input: Report sent from the server

[1190] Output: Notification and display to the driver

[1191] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1192] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1193] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1194] [Third embodiment]

[1195] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1196] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1197] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1198] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1199] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1200] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1201] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1202] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1203] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1204] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1205] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1206] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1207] The present invention relates to a system that collects e-mails, automatically classifies them, evaluates their importance, generates summaries, creates action lists, and distributes the results of these processes. This system is composed of the following elements: a server, a terminal, and a user.

[1208] System Overview

[1209] The system helps users manage their email more efficiently by providing a range of features, including:

[1210] 1. Email Collection:

[1211] The server accesses the mail server and collects received emails for a specific period (e.g., one day).

[1212] 2. Email Classification:

[1213] The server analyzes the content of the emails collected and classifies them into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1214] 3. Importance rating:

[1215] The server calculates an importance score based on the content of each email and prioritizes the emails.

[1216] 4. Summary generation:

[1217] The server uses machine learning and natural language generation techniques to generate a summary of each email.

[1218] 5. Create an action list:

[1219] The server creates a list of actions for the user to take based on the importance score and summary.

[1220] 6. Delivery of results:

[1221] The server sends the generated summary and action list to the terminal and displays them to the user.

[1222] Program processing overview

[1223] Email collection

[1224] 1. Server:

[1225] Every night, all received emails for the day are retrieved from the mail server using the IMAP protocol.

[1226] Store the retrieved emails in a database and save the metadata of each email (sender, recipient, subject, body, date and time).

[1227] Email Classification

[1228] 2. Server:

[1229] Each email is extracted from the database and its content is analyzed using natural language processing technology.

[1230] Based on the analysis results, the information is classified into the following categories: work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1231] The classification results are stored in a database.

[1232] Importance rating

[1233] 3. Server:

[1234] Scoring the importance of each email based on a list of keywords and senders.

[1235] Emails containing important keywords (e.g., "urgent," "important") or from specific senders are given a higher score.

[1236] Save the importance score in a database.

[1237] Summary Generation

[1238] 4. Server:

[1239] Natural language generation technology is used to generate a summary of each email.

[1240] Divide each email into a concise, few-line summary (e.g., "New project materials needed to be created").

[1241] The generated summary is stored in a database.

[1242] Action List Creation

[1243] 5. Server:

[1244] Based on the importance score and summary, a list of actions for the user to perform is generated.

[1245] Action lists are prioritized according to importance (e.g., 1. Create documentation for new projects, 2. Respond to customer inquiries).

[1246] Store the action list in a database.

[1247] Results distribution

[1248] 6. Server:

[1249] The generated summary and action list are compiled into a single report.

[1250] Send the report to the user's device.

[1251] 7. Terminal:

[1252] Receive the report and display it to the user (e.g., display it on the notification screen of a PC or smartphone).

[1253] Specific examples

[1254] For example, if the incoming email on October 1, 2023 is:

[1255] First message: "Work instructions: Create materials for a new project"

[1256] 2nd message: "Contact from customer: Product inquiry"

[1257] Third email: "Information sharing: Market trend report"

[1258] 4th message: "Company-wide notice: System maintenance notice"

[1259] 5th message: "Advertisement: Announcement of new product release"

[1260] Once the server has collected the above emails, classified them, ranked them by importance, generated summaries, and created action lists, the output will look something like this:

[1261] Summary example

[1262] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1263] Summary of customer email: "I need more information about product features"

[1264] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[1265] Summary of company-wide email: "System maintenance scheduled for this weekend."

[1266] Promotional email summary: "New product coming next month."

[1267] Action list to be taken

[1268] 1. Creating materials for a new project

[1269] 2. Detailed explanation of product features to customers

[1270] 3. Prepare for system maintenance this weekend

[1271] In this way, the system allows users to quickly grasp important information and efficiently take necessary actions.

[1272] The processing flow will be explained below.

[1273] Step 1:

[1274] The server connects to the mail server and retrieves all incoming emails for a specific period (e.g., one day) using the IMAP protocol. The server stores the retrieved emails in temporary storage.

[1275] Step 2:

[1276] The server reads the emails retrieved from temporary storage and stores them in a database, along with the metadata for each email (sender, recipient, subject, body, date and time).

[1277] Step 3:

[1278] The server retrieves each email stored in the database and analyzes its content using natural language processing (NLP) technology. Based on the analysis results, the email is automatically classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[1279] Step 4:

[1280] For each email that is categorized, the server calculates an importance score using keyword matching and sender lists. For example, emails containing keywords like "urgent" or "urgent" or from specific senders will receive a higher score.

[1281] Step 5:

[1282] The server uses natural language generation (NLG) technology to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[1283] Step 6:

[1284] The server generates a list of actions for the user to perform based on the importance scores and the generated summary, with the list of actions prioritized in order of importance.

[1285] Step 7:

[1286] The server compiles the summary and action list into a single report, which contains a summary of the messages in each category and the corresponding actions.

[1287] Step 8:

[1288] The server sends the compiled report to the user's device, and the report is provided to the user in the form of email, app notification, or dashboard display.

[1289] Step 9:

[1290] The terminal displays the received report to the user, who can then check the report and take the action indicated.

[1291] Example 1

[1292] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1293] In today's business environment, email is an important means of communication, but it can be difficult to efficiently manage the large amount of information and quickly take necessary action. Conventional email management systems lack the functionality to properly evaluate the importance and content of emails and clearly present the actions that users should take. As a result, users often miss important information or respond late. There is a need to solve this problem and improve user work efficiency.

[1294] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1295] In this invention, the server includes means for analyzing the metadata of emails and storing it in a database, means for analyzing the content of emails using natural language processing technology, and means for generating summaries of emails using natural language generation technology, which enable accurate evaluation of the importance of emails and specific suggestions for actions to be taken by the user.

[1296] "Email" means a digital message sent or received over the Internet or other communications network.

[1297] "Metadata" refers to information attached to an email, such as the sender, recipient, subject, body, date and time.

[1298] A "database" is a collection of digital data that systematically stores and manages collected emails and metadata.

[1299] "Natural language processing technology" refers to the technology of analyzing and understanding human language using a computer, and includes techniques such as tokenizing words and analyzing sentence structure.

[1300] "Natural language generation technology" refers to a technology that allows a computer to generate text in natural human language, and is used to summarize emails and extract information.

[1301] An "abstract" is a short sentence that succinctly summarizes the main points of the email.

[1302] The "importance score" is a number that indicates the importance of an email, calculated based on the email content and sender list.

[1303] An "action list" is a list of specific tasks or action items that a user must perform.

[1304] "Priority" refers to the ordering of multiple tasks or action items based on their importance.

[1305] "Report" refers to information provided to a user in the form of a document summarizing emails and / or a list of actions.

[1306] This invention relates to a system that collects, automatically classifies, evaluates the importance of, summarizes, creates an action list, and distributes e-mails, and is composed of elements of a server, a terminal, and a user. A specific embodiment of this system is shown below.

[1307] System Configuration

[1308] The system mainly consists of the following components:

[1309] Server: Use a Linux server or cloud server (e.g. AWS EC2).

[1310] Database: Use a relational database such as MySQL or PostgreSQL.

[1311] Natural Language Processing (NLP) techniques: Use Python's NLTK and SpaCy libraries.

[1312] Natural Language Generation (NLG) technology: Uses OpenAI's GPT-3 or the latest GPT-4 model.

[1313] Terminal: A user device such as a PC or smartphone.

[1314] System Features

[1315] The system helps users manage their email more efficiently by providing the following features:

[1316] 1. Email Collection:

[1317] The server accesses the mail server and uses the IMAP protocol to collect incoming emails for a specific period (e.g., one day).

[1318] The collected email metadata (sender, recipient, subject, body, date and time) is stored in a database.

[1319] 2. Email Classification:

[1320] The server retrieves each stored email and analyzes its content using natural language processing technology.

[1321] Based on the analysis results, the data is classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising, and stored in a database.

[1322] 3. Importance rating:

[1323] The server calculates an importance score for each email based on its content, keywords, and sender list.

[1324] Save the importance score to the database and update the importance field of the email.

[1325] 4. Summary generation:

[1326] The server uses natural language generation techniques to generate a summary of each email.

[1327] The prompt sentence "Generate a summary of the following email: Email content" is input into the model.

[1328] The generated summary is stored in a database and the summary field corresponding to each email is updated.

[1329] 5. Create an action list:

[1330] The server lists action items for the user to perform based on the importance scores and the generated summary.

[1331] The action list is prioritized based on importance and stored in a database.

[1332] 6. Delivery of results:

[1333] The server compiles the generated summary and action list into a single report and sends it to the user's terminal.

[1334] The device displays the report received from the server to the user on a PC or smartphone.

[1335] Specific examples

[1336] For example, if the incoming email on October 1, 2023 is:

[1337] First message: "Work instructions: Create materials for a new project"

[1338] 2nd message: "Contact from customer: Product inquiry"

[1339] Third email: "Information sharing: Market trend report"

[1340] 4th message: "Company-wide notice: System maintenance notice"

[1341] 5th message: "Advertisement: Announcement of new product release"

[1342] Once the server has collected these emails, sorted them, ranked them by importance, generated summaries, and created action lists, the result will look something like this:

[1343] Summary example

[1344] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1345] Summary of customer email: "I need more information about product features"

[1346] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[1347] Summary of company-wide email: "System maintenance scheduled for this weekend."

[1348] Promotional email summary: "New product coming next month."

[1349] Action list to be taken

[1350] 1. Creating materials for a new project

[1351] 2. Detailed explanation of product features to customers

[1352] 3. Prepare for system maintenance this weekend

[1353] Prompt Sentence Examples

[1354] An example of a prompt sentence to input to the generative AI model is "Generate a summary of the following email: Work instructions: Create materials for a new project."

[1355] In this way, by using the system of the present invention, the user can quickly grasp important information and efficiently take the necessary actions.

[1356] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1357] Specific flow of program processing

[1358] Each processing step is described in detail below.

[1359] Step 1: Email Collection

[1360] server:

[1361] Input: Connect to a mail server (for example, using the IMAP protocol) at a fixed time every night.

[1362] Specific operation: The server retrieves unread emails and newly received emails for a specified period (e.g., the past 24 hours).

[1363] Data processing: Analyze the metadata of the acquired emails (sender, recipient, subject, body, date and time).

[1364] Output: Save the email metadata and body to a database.

[1365] Step 2: Email Classification

[1366] server:

[1367] Input: Collected emails stored in a database.

[1368] What it does: The server analyzes the content of each email using natural language processing (NLP) techniques, such as Python's NLTK or SpaCy.

[1369] Data processing: Based on the analysis results, data is classified into the following categories: work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1370] Output: Store the email classification results in a database and assign each email a corresponding category tag.

[1371] Step 3: Severity Assessment

[1372] server:

[1373] Input: Classified email.

[1374] What it does: The server calculates an importance score for each email based on its content, keywords, and sender list.

[1375] Data processing: For example, emails containing keywords such as "urgent" or "important" or emails from specific senders (such as your boss or important customers) are given a higher score.

[1376] Output: Save the importance score to the database and update the importance field of the email.

[1377] Step 4: Summary generation

[1378] server:

[1379] Input: Emails with importance scores.

[1380] What it does: It uses natural language generation technology (NLG technology, e.g., OpenAI's GPT-3 or GPT-4 models) to convert the content of each email into a concise summary.

[1381] Data processing: The prompt sentence "Generate a summary of the following email: Email content" is input into the model.

[1382] Output: Store the generated summaries in a database and update the summary field for each email.

[1383] Step 5: Create an Action List

[1384] server:

[1385] Input: Email with summary and importance score.

[1386] Specific Action: Based on the importance score and the generated summary, a list of action items for the user to take is generated.

[1387] Data manipulation: Prioritize action items based on importance.

[1388] Output: Stores the action list in the database and updates the user's action list field.

[1389] Step 6: Delivering results

[1390] server:

[1391] Input: The data from which the action list and summary are generated.

[1392] Specific behavior: Compile the generated summary and action list into a single report, for example, generate a report in PDF or HTML format.

[1393] Output: Sends the report to the user's terminal.

[1394] Device:

[1395] Input: The report received from the server.

[1396] Specific operation: The device displays the report to the user on a PC or smartphone, and notifies the user in real time using the notification function.

[1397] Output: User reviews the report and decides what action to take next.

[1398] By implementing this system's series of processes, users can quickly grasp important information and efficiently take the necessary actions.

[1399] (Application example 1)

[1400] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1401] In modern factories, a great deal of information is communicated via email, but it is difficult to efficiently manage the vast volume of emails and quickly take necessary actions. It is also difficult to distinguish between important and unimportant emails, which increases the risk of overlooking important information. This reduces factory operational efficiency and has a negative impact on productivity. To solve these problems, proper email management, automatic classification, importance assessment, rapid summary generation, and the creation and distribution of action lists are required.

[1402] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1403] In this invention, the server includes means for collecting emails, means for classifying the collected emails into multiple categories, means for rating the importance of each email, means for summarizing the content of the emails for each category, means for generating a list of actions to be taken by the user, means for distributing the summaries and the action list to the user, and means for automatically processing emails related to factory work and notifying the results to the manager's terminal, thereby enabling the factory manager to efficiently manage a huge amount of emails, quickly grasp important information, and immediately take necessary actions.

[1404] "Means for collecting email" refers to a system or software that connects to a mail server via the Internet and obtains emails sent to a specific account.

[1405] A "multiple categorization means" is an algorithm or software that automatically sorts retrieved emails into predefined categories based on their content.

[1406] The "means for assessing importance" is a system or software that analyzes the content of each email and calculates and scores the importance of the email.

[1407] The "summarization means" refers to a system or software that extracts key points from the body of an email and generates a concise summary using a machine learning algorithm or similar.

[1408] The "means for generating an action list" is a system or software that lists and prioritizes tasks that a user should perform based on the summary and importance score of each email.

[1409] The "distribution means" is a system or software for sending the generated summary and action list to the user's terminal and notifying them.

[1410] "Means for automatically processing emails related to factory operations and notifying the results to the manager's terminal" refers to a system or software that collects and analyzes emails containing business communications, instructions, progress reports, etc. in the factory, and delivers the results to the manager in real time.

[1411] This invention relates to a system that collects emails, automatically classifies them, evaluates their importance, creates summaries and action lists, and distributes the results. This system aims to efficiently manage business emails, particularly in factories, reducing the burden on managers and enabling them to quickly grasp important information.

[1412] The system is mainly composed of the server, terminal, and user elements. The role of each element and the specific processing method are explained below.

[1413] Server Roles and Operations

[1414] The server plays a central role in collecting, classifying, assessing the importance of emails, generating summaries, and creating and distributing action lists.

[1415] Email collection

[1416] Every night, the server retrieves incoming emails from the mail server using the IMAP protocol, and stores all email metadata (sender, recipient, subject, body, date and time) in a database.

[1417] Email Classification

[1418] The server retrieves emails from the database and analyzes their contents using natural language processing (NLP) technology, categorizing them into categories such as work instructions, production progress, emergency contacts, maintenance information, and advertisements. A keyword-based classification algorithm is used for the specific analysis.

[1419] Importance rating

[1420] The importance of each email is scored based on keywords and senders. Emails containing important keywords (e.g., "urgent" or "important") or from specific senders are given a higher score. This importance score is stored in a database.

[1421] Summary Generation

[1422] We use natural language generation techniques to generate summaries of each email, specifically by extracting a concise, few-line summary from the body of the email using the gensim library.

[1423] Action List Creation

[1424] Based on the importance score and summary, a list of actions for the user to perform is generated, prioritized according to importance.

[1425] Results distribution

[1426] The generated summary and action list are compiled and delivered to the user's terminal. The delivered notification is displayed on the administrator's terminal.

[1427] Device Role

[1428] The user's device receives the summary and action list sent from the server and notifies the administrator. The application on the device displays the notification content in an easy-to-understand manner, allowing the administrator to take action immediately.

[1429] User Roles

[1430] The administrator (user) can quickly take necessary action based on the provided summary and action list. For example, they can prioritize tasks of high importance, such as creating a production plan for a new project or responding to machine failures.

[1431] Specific examples

[1432] For example, if the following email is received on October 1, 2023:

[1433] Work instruction: "Create a production plan for a newly ordered project."

[1434] Manufacturing Progress: "Report on the current progress of the production line"

[1435] Emergency call: "There is a machine breakdown and we need to fix it quickly."

[1436] Maintenance information: "Check the scheduled maintenance schedule"

[1437] Advertisement: "Sales announcement for new materials from suppliers"

[1438] The server then categorizes and rates each email, generating a summary and action list like this:

[1439] Summary example

[1440] Summary of work instruction email: "Create a production plan for the new project this week"

[1441] Summary of manufacturing progress email: "Production line progress is on schedule"

[1442] Emergency message summary: "Machine breakdown, immediate repair required."

[1443] Maintenance information summary: "Scheduled maintenance scheduled for next week"

[1444] Summary of the advertising email: "Information on new materials"

[1445] Action list to be taken

[1446] 1. Creating production plans for new projects

[1447] 2. Response to machine breakdowns

[1448] 3. Preparation for scheduled maintenance

[1449] 4. Checking production progress

[1450] Example prompts for generative AI models

[1451] "Analyze the following email and generate a summary:

[1452] Subject: Create a production plan for a new project

[1453] Body: We need to create a production plan for a newly ordered project this week. Please confirm the detailed specifications and required resources and create an appropriate schedule.

[1454] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1455] Step 1:

[1456] The server establishes a connection to the mail server using the IMAP protocol and retrieves all emails received within a specific period of time. The server stores the metadata of the retrieved emails (sender, recipient, subject, body, date and time) in a database, making all incoming emails available as target input data.

[1457] Step 2:

[1458] The server retrieves each email from the database and analyzes its content using natural language processing technology. Based on the content of the email, the server categorizes it into categories such as work instructions, production progress, emergency contacts, maintenance information, and advertisements. The analyzed data is then stored in the database as output data categorized by category.

[1459] Step 3:

[1460] The server calculates an importance score based on the classified emails. Emails containing important keywords (e.g., "urgent" or "important") or from specific senders are given a higher score. This weights the importance of each email. The score is calculated based on the input data and added to the output data as the importance of each email.

[1461] Step 4:

[1462] The server uses natural language generation techniques, such as the gensim library, to generate a summary of each email's content. The body of each email is summarized into a concise, few-line summary, which is then stored in a database as the output of the analysis.

[1463] Step 5:

[1464] Based on the importance scores and summaries, the server generates a list of actions for the user to perform. The action list is prioritized by importance and identifies tasks to be performed. This action list is stored in a database as part of the output data.

[1465] Step 6:

[1466] The server compiles the generated summary and action list into a single report and sends it to the user's device, allowing the administrator to receive notifications on their smartphone or other device.

[1467] Step 7:

[1468] The terminal displays the received report and notifies the user. Specifically, the terminal's notification function is used to display a summary and action list, allowing the user to quickly take necessary actions based on this.

[1469] Prompt Sentence Examples

[1470] For example, the prompt to verify the generated summary is:

[1471] "Analyze the following email and generate a summary:

[1472] Subject: Create a production plan for a new project

[1473] Body: We need to create a production plan for a newly ordered project this week. Please confirm the detailed specifications and required resources and create an appropriate schedule.

[1474] Through the flow of steps, a system is realized that allows servers, terminals, and users to work together to significantly improve factory operational efficiency.

[1475] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1476] This invention relates to a system that collects emails, automatically classifies them, assesses their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. This system is composed of a server, a terminal, and users, and by combining it with an emotion engine, it provides information and generates action lists that take into account the user's emotional state.

[1477] System Overview

[1478] The system will help users streamline email management and emotion recognition by providing a series of features, including:

[1479] 1. Email Collection:

[1480] The server accesses the mail server and collects all incoming emails for a specific period (e.g., one day).

[1481] 2. Email Classification:

[1482] The server analyzes the content of the emails collected and classifies them into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1483] 3. Importance rating:

[1484] The server calculates an importance score based on the content of each email and prioritizes it.

[1485] 4. Summary generation:

[1486] The server uses machine learning and natural language generation techniques to generate a summary of each email.

[1487] 5. Create an action list:

[1488] The server generates a list of actions for the user to perform based on the importance score and summary text.

[1489] 6. Emotion recognition:

[1490] The server uses an emotion engine that recognizes the user's emotions, and reflects the emotion information when generating the email contents and action list.

[1491] 7. Delivery of results:

[1492] The server generates a report including the summary, action list, and emotion-based adjustments and sends it to the terminal for display to the user.

[1493] Program processing overview

[1494] Email collection

[1495] 1. Server:

[1496] Every night, all received emails for the day are retrieved from the mail server using the IMAP protocol.

[1497] Store the retrieved emails in a database and save the metadata of each email (sender, recipient, subject, body, date and time).

[1498] Email Classification

[1499] 2. Server:

[1500] The email is retrieved from the database and its content is analyzed using natural language processing techniques.

[1501] Based on the analysis results, the messages are sorted into the following categories: work instructions, customer contact, information sharing, company-wide announcements, and advertising.

[1502] The classification results are stored in a database.

[1503] Importance rating

[1504] 3. Server:

[1505] The importance of each email is scored based on a list of keywords and senders. For example, emails containing keywords such as "urgent" or "urgent" or from specific senders are given a higher score.

[1506] Save the importance score in a database.

[1507] Summary Generation

[1508] 4. Server:

[1509] Natural language generation technology is used to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[1510] The generated summaries are stored in a database.

[1511] Action List Creation

[1512] 5. Server:

[1513] Based on the importance score and summary, a list of actions to be performed by the user is generated. The action list is prioritized in order of importance (e.g., 1. Create documents for a new project, 2. Respond to customer inquiries).

[1514] Store the action list in a database.

[1515] emotion recognition

[1516] 6. Server:

[1517] Recognize the user's emotional state using an emotion engine, which analyzes emotional information detected from the user's voice, facial expressions, text input, etc.

[1518] The system adjusts email content and action list generation based on the user's recognized emotional state. For example, if the user is feeling stressed, it suggests prioritizing high-priority emails.

[1519] Results distribution

[1520] 7. Server:

[1521] The generated summary, action list, and report reflecting the sentiment information are compiled into a single document.

[1522] Send the report to the user's device.

[1523] 8. Terminal:

[1524] The received report is displayed to the user, who then reviews the report and takes the action indicated.

[1525] Specific examples

[1526] For example, if the incoming email on October 1, 2023 is:

[1527] First message: "Work instructions: Create materials for a new project"

[1528] 2nd message: "Contact from customer: Product inquiry"

[1529] Third email: "Information sharing: Market trend report"

[1530] 4th message: "Company-wide notice: System maintenance notice"

[1531] 5th message: "Advertisement: Announcement of new product release"

[1532] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[1533] Summary example

[1534] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1535] Summary of customer email: "I need more information about product features"

[1536] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[1537] Summary of company-wide email: "System maintenance scheduled for this weekend."

[1538] Promotional email summary: "New product coming next month."

[1539] Action list to be taken

[1540] 1. Creating materials for a new project

[1541] 2. Detailed explanation of product features to customers

[1542] 3. Prepare for system maintenance this weekend

[1543] emotion recognition

[1544] If the user is stressed, adjust the action list to be shorter and focus on the most important tasks to reduce stress.

[1545] In this way, the system allows users to quickly grasp important information and take necessary actions efficiently.By combining emotion engines, it provides adaptive support according to the user's emotional state.

[1546] The processing flow will be explained below.

[1547] Step 1:

[1548] The server connects to the mail server and retrieves all incoming emails for a specific period (e.g., one day) using the IMAP protocol. The server stores the retrieved emails in temporary storage.

[1549] Step 2:

[1550] The server reads the emails retrieved from temporary storage and stores them in a database, along with the metadata for each email (sender, recipient, subject, body, date and time).

[1551] Step 3:

[1552] The server retrieves each email stored in the database and analyzes its content using natural language processing (NLP) technology. Based on the analysis results, the email is automatically classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[1553] Step 4:

[1554] For each email that is categorized, the server calculates an importance score using keyword matching and sender lists. For example, emails containing keywords like "urgent" or "urgent" or from specific senders will be given a higher score.

[1555] Step 5:

[1556] The server uses natural language generation (NLG) technology to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[1557] Step 6:

[1558] The server generates a list of actions for the user to perform based on the importance scores and the generated summary, with the list of actions prioritized in order of importance.

[1559] Step 7:

[1560] The server uses an emotion engine to recognize the user's emotional state, which analyzes emotional information from the user's voice, facial expressions, text input, etc.

[1561] Step 8:

[1562] The server adjusts the action list or summary based on the perceived emotional state, for example, if the user is feeling stressed, prioritizing important tasks to reduce stress.

[1563] Step 9:

[1564] The server compiles the summary and action list into a single report, which contains a summary of the messages in each category and the corresponding actions.

[1565] Step 10:

[1566] The server sends the compiled report to the user's device, and the report is provided to the user in the form of email, app notification, or dashboard display.

[1567] Step 11:

[1568] The terminal displays the received report to the user, who then checks the report and takes the action indicated.

[1569] Step 12:

[1570] The user performs the actions indicated and inputs the results as feedback to the system, which allows the system to provide a more accurate action list and summary.

[1571] For example, if the email you received on October 1, 2023 is:

[1572] First message: "Work instructions: Create materials for a new project"

[1573] 2nd message: "Contact from customer: Product inquiry"

[1574] Third email: "Information sharing: Market trend report"

[1575] 4th message: "Company-wide notice: System maintenance notice"

[1576] 5th message: "Advertisement: Announcement of new product release"

[1577] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[1578] Summary example

[1579] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1580] Summary of customer email: "I need more information about product features"

[1581] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[1582] Summary of company-wide email: "System maintenance scheduled for this weekend."

[1583] Promotional email summary: "New product coming next month."

[1584] Action list to be taken

[1585] 1. Creating materials for a new project

[1586] 2. Detailed explanation of product features to customers

[1587] 3. Prepare for system maintenance this weekend

[1588] emotion recognition

[1589] If users are stressed, shorten the action list and adjust it to focus on the most important tasks.

[1590] In this way, the system allows users to quickly grasp important information and take necessary actions efficiently.By combining emotion engines, it provides adaptive support according to the user's emotional state.

[1591] Example 2

[1592] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1593] In conventional email management systems, received emails are classified, their importance is assessed, summaries are generated, and action lists are created individually, without taking into account the user's emotional state. As a result, users can become overwhelmed by the volume of emails and may miss important emails or tasks. Furthermore, when users are under stress, it is difficult for them to process emails appropriately.

[1594] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting emails, means for classifying the collected emails into a plurality of categories, means for evaluating the importance of each email, means for summarizing the contents of the emails for each category, means for generating a list of actions to be performed by the user, means for delivering the summaries and action list to the user, means for recognizing the user's emotional state, and means for adjusting the action list and summaries according to the emotional state. This enables the user to quickly grasp important information and receive adaptive support according to their emotional state.

[1595] "Email" means a digital message sent or received over the Internet or other digital network.

[1596] "Category" refers to a group of emails classified based on their contents, and examples include work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[1597] An "importance score" is a number used to rate the importance of an email and is calculated based on specific keywords and sender lists.

[1598] "Natural language generation technology" refers to technology that generates content in natural language that humans can understand, and includes machine learning and AI technologies.

[1599] A "summary" is a short sentence that extracts the main points from a longer text and summarizes them concisely.

[1600] An "action list" is a list of specific tasks or actions that a user must perform, prioritized based on importance.

[1601] "Emotional state" refers to the user's psychological state, and includes emotions such as stress, joy, and impatience.

[1602] An "emotion engine" is hardware or software used to recognize a user's emotions, and can obtain emotional information by analyzing voice, facial expressions, text input, etc.

[1603] "User" means an individual or organization that uses this system.

[1604] MODE FOR CARRYING OUT THE INVENTION

[1605] This invention relates to a system that collects emails, automatically classifies them, assesses their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. This system is composed of a server, a terminal, and users, and by combining it with an emotion engine, it provides information and generates action lists that take into account the user's emotional state.

[1606] The system configuration is as follows: first, the server accesses the mail server and collects all received emails for a specific period (e.g., one day). At this stage, the IMAP protocol is used to retrieve the latest emails from the mail server (e.g., Gmail server) and store them in a database (e.g., MySQL). Email metadata (sender, recipient, subject, body, date and time) is also saved at the same time.

[1607] The server then retrieves the email from the database and analyzes its content using natural language processing technology (e.g., Google NLP API). Based on this analysis, the email is classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements. The classification results are also stored in the database.

[1608] The server also assigns a score to each email based on its importance, based on keywords (e.g., "urgent" or "urgent") and specific sender lists. This importance score is stored in a database for each email.

[1609] The server then uses machine learning and natural language generation techniques (e.g., OpenAI GPT) to generate a summary of each email. The summary extracts key information from the email body and presents it in a concise format of a few lines. The generated summary is also stored in a database.

[1610] Based on the generated summary and the importance score, the server generates a list of actions for the user to perform. The list of actions is prioritized in order of importance and stored in a database.

[1611] Furthermore, the server uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotional state. This analyzes emotional information detected from the user's voice, facial expressions, text input, etc. Based on the recognized emotional state, the email content and action list generation are adjusted. For example, if the user is feeling stressed, the server suggests prioritizing high-priority emails.

[1612] Finally, the server compiles the generated summary, action list, and emotional information into a report and sends it to the user's device. The device displays the received report to the user, who then checks the report and performs the requested actions.

[1613] Specific examples

[1614] Email received on October 1, 2023:

[1615] "Work instructions: Create materials for a new project"

[1616] "Customer Contact: Product Inquiry"

[1617] "Information Sharing: Market Trends Report"

[1618] "Company-wide Notice: System Maintenance Notice"

[1619] "Advertisement: Announcement of new product release"

[1620] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[1621] Summary example:

[1622] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1623] Summary of customer email: "I need more information about product features"

[1624] Summary of the information sharing email: "Certain sectors are experiencing rapid growth as a market trend."

[1625] Summary of company-wide email: "System maintenance is scheduled for this weekend."

[1626] Promotional email summary: "New product launching next month"

[1627] List of actions to be taken:

[1628] 1. Creating materials for a new project

[1629] 2. Detailed explanation of product features to customers

[1630] 3. Prepare for system maintenance this weekend

[1631] Emotion recognition:

[1632] If the user is stressed, shorten the action list to reduce stress and focus on the most important tasks.

[1633] Example of an input prompt for a generative AI model:

[1634] Analyze the content of received emails and categorize them into the following categories: work instructions, customer contact, information sharing, company-wide announcements, and promotional advertisements. Also, assign an importance score to each email and generate a summary. Then, create an action list and provide tailored results based on sentiment recognition.

[1635] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1636] Step 1:

[1637] Email collection

[1638] server:

[1639] Every night, the server connects to a mail server using the IMAP protocol, for example, to retrieve incoming emails from the Gmail server.

[1640] The retrieved emails are stored in a database (e.g., MySQL), and metadata for each email (sender, recipient, subject, body, date and time) is stored.

[1641] Input: Received email data from the mail server.

[1642] Output: Email data and metadata stored in a database.

[1643] Step 2:

[1644] Email Classification

[1645] server:

[1646] The server retrieves unclassified emails from the database and analyzes their content using natural language processing techniques (e.g., Google NLP API).

[1647] Based on the analysis results, the information is sorted into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1648] The classification results are stored in the database again.

[1649] Input: Unclassified email data from the database.

[1650] Output: Email data categorized by category.

[1651] Step 3:

[1652] Importance rating

[1653] server:

[1654] The server calculates an importance score for each email based on specific keywords (e.g., "urgent" or "urgent") and a list of specific senders.

[1655] An importance score is stored in the database for each email.

[1656] Input: Classified email data.

[1657] Output: Email data with importance scores assigned.

[1658] Step 4:

[1659] Summary Generation

[1660] server:

[1661] The server uses machine learning and natural language generation techniques (e.g., OpenAI GPT) to generate a summary of each email.

[1662] The summary extracts the key information from the email body and presents it in a concise format of a few lines.

[1663] The generated summaries are stored in a database.

[1664] Input: Email data with an importance score.

[1665] Output: The email data with the summary generated.

[1666] Step 5:

[1667] Action List Creation

[1668] server:

[1669] The server generates a list of actions for the user to perform based on the generated summary and importance score.

[1670] Action lists are prioritized in order of importance and stored in a database.

[1671] Input: The email data from which the summary was generated and the importance score.

[1672] Output: A prioritized list of actions.

[1673] Step 6:

[1674] emotion recognition

[1675] server:

[1676] The server uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotional state.

[1677] Emotional information is analyzed from the user's voice, facial expressions, text input, etc.

[1678] Based on the recognized emotional state, adjustments are made when generating email content and action lists.

[1679] Input: User voice, facial expressions, and text input.

[1680] Output: A tailored action list and summary.

[1681] Step 7:

[1682] Results distribution

[1683] server:

[1684] The server compiles the generated summary, action list, and report reflecting the emotional information into a single document.

[1685] Send the document to the user's device.

[1686] Input: Tailored action list and summary.

[1687] Output: Report sent to user terminal.

[1688] Device:

[1689] The terminal presents the received report to the user.

[1690] The user reviews the report and takes the indicated action.

[1691] Input: The report sent by the server.

[1692] Output: The report as displayed to the user.

[1693] (Application example 2)

[1694] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1695] With the spread of autonomous vehicles, drivers are required to efficiently and safely obtain important information while driving. Conventional systems require drivers to manually check emails, determine their importance, and decide on an action, which can distract them from driving. Furthermore, the lack of appropriate information provided takes into account the driver's emotional state, which can increase stress and fatigue. The present invention solves these problems, enabling drivers to safely and efficiently obtain email information and take appropriate action.

[1696] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1697] In this invention, the server includes means for collecting emails, means for classifying the collected emails into a plurality of categories, means for evaluating the importance of each email, means for summarizing the contents of the emails for each category, means for generating a list of actions to be performed by the user, means for delivering the summaries and the action list to the user, means for detecting the driver's state, and means for adjusting the action list based on emotional information. This allows the driver to quickly identify important emails while driving, and to sense stress and fatigue and respond efficiently based on the action list with adjusted priorities.

[1698] "Email" is a digital message sent and received over a computer network.

[1699] "Means of collection" refers to the communication protocols and software functions for obtaining email from a specific mail server.

[1700] A "classification means" is an algorithm or software function that sorts collected emails into categories based on their content.

[1701] A "means for assessing importance" is an algorithm or system that scores importance based on factors such as the content of an email and the sender.

[1702] "Summarization means" refers to natural language processing techniques and software functions that extract key information from lengthy emails and present it in a short form.

[1703] The "means for generating an action list" is a function that creates a list of specific tasks that the user should perform based on the summarized content and importance rating of the email.

[1704] "Means for distribution" refers to the communication protocol and software functions for transferring the generated summary and action list to the user's terminal and displaying or notifying them.

[1705] The "means for adjusting the action list based on emotional information" is a function that analyzes the driver's emotional state and automatically adjusts the priority and content of the action list according to that state.

[1706] "Means for sensing driver state" refers to sensors and software functions that analyze the driver's voice, facial expressions, and behavior to detect their emotional state.

[1707] This system allows drivers to safely and efficiently manage their emails. It collects emails, automatically classifies them, evaluates their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. The system is primarily composed of a server, a terminal, and users.

[1708] System Configuration

[1709] The server uses the IMAP protocol to collect emails from the mail server. The collected emails are stored in a database, and metadata (sender, recipient, subject, body, date and time) is saved. The server then applies natural language processing (NLP) technology to classify each email into multiple categories, such as work instructions, customer contact, information sharing, company-wide notices, and promotional advertisements. The classified emails are then scored using an importance assessment algorithm.

[1710] Based on the importance rating, the server extracts key information from the body of each email and uses natural language generation technology to generate a concise summary, helping drivers quickly grasp the information they need.

[1711] The server then generates an action list containing specific tasks the user needs to perform, prioritizing them in order of importance.The emotion engine also recognizes emotions from the user's voice, facial expressions, and text input, and reflects them in the action list.

[1712] Finally, the server compiles this information into a single report and sends it to the device (such as a smartphone), which notifies the user by voice or on-screen display.

[1713] Hardware and software used

[1714] Hardware

[1715] Server: A high-performance server for collecting emails, classifying them, assessing their importance, generating summaries, recognizing emotions, and creating action lists.

[1716] User device: smartphone or tablet

[1717] software

[1718] IMAP protocol: for collecting emails

[1719] Natural Language Processing (NLP) algorithms: for email classification and summary generation

[1720] Natural language generation technology: for content summarization

[1721] Emotion Engine: Recognizes the driver's emotional state

[1722] Database: To store collected emails

[1723] Specific examples

[1724] For example, a driver receives the following email on October 1, 2023:

[1725] First message: "Work instructions: Create materials for a new project"

[1726] 2nd message: "Contact from customer: Product inquiry"

[1727] Third email: "Information sharing: Market trend report"

[1728] 4th message: "Company-wide notice: System maintenance notice"

[1729] 5th message: "Advertisement: Announcement of new product release"

[1730] The server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, resulting in the following output:

[1731] Summary example

[1732] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1733] Summary of customer email: "I need more information about product features"

[1734] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[1735] Summary of company-wide email: "System maintenance scheduled for this weekend."

[1736] Promotional email summary: "New product coming next month."

[1737] Action list to be taken

[1738] 1. Creating materials for a new project

[1739] 2. Detailed explanation of product features to customers

[1740] 3. Prepare for system maintenance this weekend

[1741] Example prompts for generative AI models

[1742] Summarize the following email: "This is a customer inquiry. They would like more information on how to use the product. It's urgent."

[1743] The system is designed to help drivers respond to important emails while continuing to drive safely. It provides a concise summary of the email's key content and actions, reads them out loud, and uses emotion recognition to suggest the most appropriate response.

[1744] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1745] Step 1:

[1746] The server uses the IMAP protocol to collect all incoming emails from the mail server for a specified period (e.g., one day) and stores them in a database, including their metadata (sender, recipient, subject, body, date and time).

[1747] Input: Mail server account information, period specification

[1748] Output: Email list (including metadata)

[1749] Step 2:

[1750] The server retrieves the collected emails from the database and uses natural language processing (NLP) technology to analyze the email content and classify it into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[1751] Input: Email data collected in Step 1

[1752] Output: A list of emails sorted by category

[1753] Step 3:

[1754] The server calculates the importance score for each email based on the classified email content. For example, emails containing keywords such as "urgent" or "urgent" or emails from specific senders are given a higher score.

[1755] Input: Email data classified in Step 2

[1756] Output: Email data with importance scores assigned

[1757] Step 4:

[1758] The server uses natural language generation technology to generate a summary of each email, extracting key information from the email body and creating a concise summary of a few lines.

[1759] Input: Email data scored in Step 3

[1760] Output: Email data with summary generated

[1761] Step 5:

[1762] The server generates a list of actions for the user to perform based on the importance score and the summary sentence, and prioritizes the list of actions in order of importance.

[1763] Input: Email data summarized in step 4

[1764] Output: A list of actions ordered by importance

[1765] Step 6:

[1766] The server uses an emotion engine to recognize the user's (driver's) emotional state by analyzing data such as voice, facial expressions, and text input. Based on the recognized emotion, it adjusts the action list and notification content.

[1767] Input: Driver's voice, facial expressions, and text data

[1768] Output: Recognized emotional state and adjusted action list

[1769] Step 7:

[1770] The server compiles the generated summary, action list, and a report including adjustments based on the emotion information into a single document and sends it to the user's terminal.

[1771] Input: Summary sentence generated in step 4, Action list created in step 5, Emotion information recognized in step 6

[1772] Output: Report displayed on the user's terminal

[1773] Step 8:

[1774] The user's device will notify the user of the received report via voice or screen display, allowing the driver to quickly grasp important information while driving and efficiently take necessary actions.

[1775] Input: Report sent from the server

[1776] Output: Notification and display to the driver

[1777] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1778] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1779] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1780] [Fourth embodiment]

[1781] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1782] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1783] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1784] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1785] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1786] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1787] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1788] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1789] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1790] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1791] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1792] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1793] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1794] The present invention relates to a system that collects e-mails, automatically classifies them, evaluates their importance, generates summaries, creates action lists, and distributes the results of these processes. This system is composed of the following elements: a server, a terminal, and a user.

[1795] System Overview

[1796] The system helps users manage their email more efficiently by providing a range of features, including:

[1797] 1. Email Collection:

[1798] The server accesses the mail server and collects received emails for a specific period (e.g., one day).

[1799] 2. Email Classification:

[1800] The server analyzes the content of the emails collected and classifies them into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1801] 3. Importance rating:

[1802] The server calculates an importance score based on the content of each email and prioritizes the emails.

[1803] 4. Summary generation:

[1804] The server uses machine learning and natural language generation techniques to generate a summary of each email.

[1805] 5. Create an action list:

[1806] The server creates a list of actions for the user to take based on the importance score and summary.

[1807] 6. Delivery of results:

[1808] The server sends the generated summary and action list to the terminal and displays them to the user.

[1809] Program processing overview

[1810] Email collection

[1811] 1. Server:

[1812] Every night, all received emails for the day are retrieved from the mail server using the IMAP protocol.

[1813] Store the retrieved emails in a database and save the metadata of each email (sender, recipient, subject, body, date and time).

[1814] Email Classification

[1815] 2. Server:

[1816] Each email is extracted from the database and its content is analyzed using natural language processing technology.

[1817] Based on the analysis results, the information is classified into the following categories: work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1818] The classification results are stored in a database.

[1819] Importance rating

[1820] 3. Server:

[1821] Scoring the importance of each email based on a list of keywords and senders.

[1822] Emails containing important keywords (e.g., "urgent," "important") or from specific senders are given a higher score.

[1823] Save the importance score in a database.

[1824] Summary Generation

[1825] 4. Server:

[1826] Natural language generation technology is used to generate a summary of each email.

[1827] Divide each email into a concise, few-line summary (e.g., "New project materials needed to be created").

[1828] The generated summary is stored in a database.

[1829] Action List Creation

[1830] 5. Server:

[1831] Based on the importance score and summary, a list of actions for the user to perform is generated.

[1832] Action lists are prioritized according to importance (e.g., 1. Create documentation for new projects, 2. Respond to customer inquiries).

[1833] Store the action list in a database.

[1834] Results distribution

[1835] 6. Server:

[1836] The generated summary and action list are compiled into a single report.

[1837] Send the report to the user's device.

[1838] 7. Terminal:

[1839] Receive the report and display it to the user (e.g., display it on the notification screen of a PC or smartphone).

[1840] Specific examples

[1841] For example, if the incoming email on October 1, 2023 is:

[1842] First message: "Work instructions: Create materials for a new project"

[1843] 2nd message: "Contact from customer: Product inquiry"

[1844] Third email: "Information sharing: Market trend report"

[1845] 4th message: "Company-wide notice: System maintenance notice"

[1846] 5th message: "Advertisement: Announcement of new product release"

[1847] Once the server has collected the above emails, classified them, ranked them by importance, generated summaries, and created action lists, the output will look something like this:

[1848] Summary example

[1849] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1850] Summary of customer email: "I need more information about product features"

[1851] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[1852] Summary of company-wide email: "System maintenance scheduled for this weekend."

[1853] Promotional email summary: "New product coming next month."

[1854] Action list to be taken

[1855] 1. Creating materials for a new project

[1856] 2. Detailed explanation of product features to customers

[1857] 3. Prepare for system maintenance this weekend

[1858] In this way, the system allows users to quickly grasp important information and efficiently take necessary actions.

[1859] The processing flow will be explained below.

[1860] Step 1:

[1861] The server connects to the mail server and retrieves all incoming emails for a specific period (e.g., one day) using the IMAP protocol. The server stores the retrieved emails in temporary storage.

[1862] Step 2:

[1863] The server reads the emails retrieved from temporary storage and stores them in a database, along with the metadata for each email (sender, recipient, subject, body, date and time).

[1864] Step 3:

[1865] The server retrieves each email stored in the database and analyzes its content using natural language processing (NLP) technology. Based on the analysis results, the email is automatically classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[1866] Step 4:

[1867] For each email that is categorized, the server calculates an importance score using keyword matching and sender lists. For example, emails containing keywords like "urgent" or "urgent" or from specific senders will receive a higher score.

[1868] Step 5:

[1869] The server uses natural language generation (NLG) technology to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[1870] Step 6:

[1871] The server generates a list of actions for the user to perform based on the importance scores and the generated summary, with the list of actions prioritized in order of importance.

[1872] Step 7:

[1873] The server compiles the summary and action list into a single report, which contains a summary of the messages in each category and the corresponding actions.

[1874] Step 8:

[1875] The server sends the compiled report to the user's device, and the report is provided to the user in the form of email, app notification, or dashboard display.

[1876] Step 9:

[1877] The terminal displays the received report to the user, who can then check the report and take the action indicated.

[1878] Example 1

[1879] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1880] In today's business environment, email is an important means of communication, but it can be difficult to efficiently manage the large amount of information and quickly take necessary action. Conventional email management systems lack the functionality to properly evaluate the importance and content of emails and clearly present the actions that users should take. As a result, users often miss important information or respond late. There is a need to solve this problem and improve user work efficiency.

[1881] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1882] In this invention, the server includes means for analyzing the metadata of emails and storing it in a database, means for analyzing the content of emails using natural language processing technology, and means for generating summaries of emails using natural language generation technology, which enable accurate evaluation of the importance of emails and specific suggestions for actions to be taken by the user.

[1883] "Email" means a digital message sent or received over the Internet or other communications network.

[1884] "Metadata" refers to information attached to an email, such as the sender, recipient, subject, body, date and time.

[1885] A "database" is a collection of digital data that systematically stores and manages collected emails and metadata.

[1886] "Natural language processing technology" refers to the technology of analyzing and understanding human language using a computer, and includes techniques such as tokenizing words and analyzing sentence structure.

[1887] "Natural language generation technology" refers to a technology that allows a computer to generate text in natural human language, and is used to summarize emails and extract information.

[1888] An "abstract" is a short sentence that succinctly summarizes the main points of the email.

[1889] The "importance score" is a number that indicates the importance of an email, calculated based on the email content and sender list.

[1890] An "action list" is a list of specific tasks or action items that a user must perform.

[1891] "Priority" refers to the ordering of multiple tasks or action items based on their importance.

[1892] "Report" refers to information provided to a user in the form of a document summarizing emails and / or a list of actions.

[1893] This invention relates to a system that collects, automatically classifies, evaluates the importance of, summarizes, creates an action list, and distributes e-mails, and is composed of elements of a server, a terminal, and a user. A specific embodiment of this system is shown below.

[1894] System Configuration

[1895] The system mainly consists of the following components:

[1896] Server: Use a Linux server or cloud server (e.g. AWS EC2).

[1897] Database: Use a relational database such as MySQL or PostgreSQL.

[1898] Natural Language Processing (NLP) techniques: Use Python's NLTK and SpaCy libraries.

[1899] Natural Language Generation (NLG) technology: Uses OpenAI's GPT-3 or the latest GPT-4 model.

[1900] Terminal: A user device such as a PC or smartphone.

[1901] System Features

[1902] The system helps users manage their email more efficiently by providing the following features:

[1903] 1. Email Collection:

[1904] The server accesses the mail server and uses the IMAP protocol to collect incoming emails for a specific period (e.g., one day).

[1905] The collected email metadata (sender, recipient, subject, body, date and time) is stored in a database.

[1906] 2. Email Classification:

[1907] The server retrieves each stored email and analyzes its content using natural language processing technology.

[1908] Based on the analysis results, the data is classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising, and stored in a database.

[1909] 3. Importance rating:

[1910] The server calculates an importance score for each email based on its content, keywords, and sender list.

[1911] Save the importance score to the database and update the importance field of the email.

[1912] 4. Summary generation:

[1913] The server uses natural language generation techniques to generate a summary of each email.

[1914] The prompt sentence "Generate a summary of the following email: Email content" is input into the model.

[1915] The generated summary is stored in a database and the summary field corresponding to each email is updated.

[1916] 5. Create an action list:

[1917] The server lists action items for the user to perform based on the importance scores and the generated summary.

[1918] The action list is prioritized based on importance and stored in a database.

[1919] 6. Delivery of results:

[1920] The server compiles the generated summary and action list into a single report and sends it to the user's terminal.

[1921] The device displays the report received from the server to the user on a PC or smartphone.

[1922] Specific examples

[1923] For example, if the incoming email on October 1, 2023 is:

[1924] First message: "Work instructions: Create materials for a new project"

[1925] 2nd message: "Contact from customer: Product inquiry"

[1926] Third email: "Information sharing: Market trend report"

[1927] 4th message: "Company-wide notice: System maintenance notice"

[1928] 5th message: "Advertisement: Announcement of new product release"

[1929] Once the server has collected these emails, sorted them, ranked them by importance, generated summaries, and created action lists, the result will look something like this:

[1930] Summary example

[1931] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[1932] Summary of customer email: "I need more information about product features"

[1933] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[1934] Summary of company-wide email: "System maintenance scheduled for this weekend."

[1935] Promotional email summary: "New product coming next month."

[1936] Action list to be taken

[1937] 1. Creating materials for a new project

[1938] 2. Detailed explanation of product features to customers

[1939] 3. Prepare for system maintenance this weekend

[1940] Prompt Sentence Examples

[1941] An example of a prompt sentence to input to the generative AI model is "Generate a summary of the following email: Work instructions: Create materials for a new project."

[1942] In this way, by using the system of the present invention, the user can quickly grasp important information and efficiently take the necessary actions.

[1943] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1944] Specific flow of program processing

[1945] Each processing step is described in detail below.

[1946] Step 1: Email Collection

[1947] server:

[1948] Input: Connect to a mail server (for example, using the IMAP protocol) at a fixed time every night.

[1949] Specific operation: The server retrieves unread emails and newly received emails for a specified period (e.g., the past 24 hours).

[1950] Data processing: Analyze the metadata of the acquired emails (sender, recipient, subject, body, date and time).

[1951] Output: Save the email metadata and body to a database.

[1952] Step 2: Email Classification

[1953] server:

[1954] Input: Collected emails stored in a database.

[1955] What it does: The server analyzes the content of each email using natural language processing (NLP) techniques, such as Python's NLTK or SpaCy.

[1956] Data processing: Based on the analysis results, data is classified into the following categories: work instructions, customer contact, information sharing, company-wide notices, and advertising.

[1957] Output: Store the email classification results in a database and assign each email a corresponding category tag.

[1958] Step 3: Severity Assessment

[1959] server:

[1960] Input: Classified email.

[1961] What it does: The server calculates an importance score for each email based on its content, keywords, and sender list.

[1962] Data processing: For example, emails containing keywords such as "urgent" or "important" or emails from specific senders (such as your boss or important customers) are given a higher score.

[1963] Output: Save the importance score to the database and update the importance field of the email.

[1964] Step 4: Summary generation

[1965] server:

[1966] Input: Emails with importance scores.

[1967] What it does: It uses natural language generation technology (NLG technology, e.g., OpenAI's GPT-3 or GPT-4 models) to convert the content of each email into a concise summary.

[1968] Data processing: The prompt sentence "Generate a summary of the following email: Email content" is input into the model.

[1969] Output: Store the generated summaries in a database and update the summary field for each email.

[1970] Step 5: Create an Action List

[1971] server:

[1972] Input: Email with summary and importance score.

[1973] Specific Action: Based on the importance score and the generated summary, a list of action items for the user to take is generated.

[1974] Data manipulation: Prioritize action items based on importance.

[1975] Output: Stores the action list in the database and updates the user's action list field.

[1976] Step 6: Delivering results

[1977] server:

[1978] Input: The data from which the action list and summary are generated.

[1979] Specific behavior: Compile the generated summary and action list into a single report, for example, generate a report in PDF or HTML format.

[1980] Output: Sends the report to the user's terminal.

[1981] Device:

[1982] Input: The report received from the server.

[1983] Specific operation: The device displays the report to the user on a PC or smartphone, and notifies the user in real time using the notification function.

[1984] Output: User reviews the report and decides what action to take next.

[1985] By implementing this system's series of processes, users can quickly grasp important information and efficiently take the necessary actions.

[1986] (Application example 1)

[1987] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1988] In modern factories, a great deal of information is communicated via email, but it is difficult to efficiently manage the vast volume of emails and quickly take necessary actions. It is also difficult to distinguish between important and unimportant emails, which increases the risk of overlooking important information. This reduces factory operational efficiency and has a negative impact on productivity. To solve these problems, proper email management, automatic classification, importance assessment, rapid summary generation, and the creation and distribution of action lists are required.

[1989] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1990] In this invention, the server includes means for collecting emails, means for classifying the collected emails into multiple categories, means for rating the importance of each email, means for summarizing the content of the emails for each category, means for generating a list of actions to be taken by the user, means for distributing the summaries and the action list to the user, and means for automatically processing emails related to factory work and notifying the results to the manager's terminal, thereby enabling the factory manager to efficiently manage a huge amount of emails, quickly grasp important information, and immediately take necessary actions.

[1991] "Means for collecting email" refers to a system or software that connects to a mail server via the Internet and obtains emails sent to a specific account.

[1992] A "multiple categorization means" is an algorithm or software that automatically sorts retrieved emails into predefined categories based on their content.

[1993] The "means for assessing importance" is a system or software that analyzes the content of each email and calculates and scores the importance of the email.

[1994] The "summarization means" refers to a system or software that extracts key points from the body of an email and generates a concise summary using a machine learning algorithm or similar.

[1995] The "means for generating an action list" is a system or software that lists and prioritizes tasks that a user should perform based on the summary and importance score of each email.

[1996] The "distribution means" is a system or software for sending the generated summary and action list to the user's terminal and notifying them.

[1997] "Means for automatically processing emails related to factory operations and notifying the results to the manager's terminal" refers to a system or software that collects and analyzes emails containing business communications, instructions, progress reports, etc. in the factory, and delivers the results to the manager in real time.

[1998] This invention relates to a system that collects emails, automatically classifies them, evaluates their importance, creates summaries and action lists, and distributes the results. This system aims to efficiently manage business emails, particularly in factories, reducing the burden on managers and enabling them to quickly grasp important information.

[1999] The system is mainly composed of the server, terminal, and user elements. The role of each element and the specific processing method are explained below.

[2000] Server Roles and Operations

[2001] The server plays a central role in collecting, classifying, assessing the importance of emails, generating summaries, and creating and distributing action lists.

[2002] Email collection

[2003] Every night, the server retrieves incoming emails from the mail server using the IMAP protocol, and stores all email metadata (sender, recipient, subject, body, date and time) in a database.

[2004] Email Classification

[2005] The server retrieves emails from the database and analyzes their contents using natural language processing (NLP) technology, categorizing them into categories such as work instructions, production progress, emergency contacts, maintenance information, and advertisements. A keyword-based classification algorithm is used for the specific analysis.

[2006] Importance rating

[2007] The importance of each email is scored based on keywords and senders. Emails containing important keywords (e.g., "urgent" or "important") or from specific senders are given a higher score. This importance score is stored in a database.

[2008] Summary Generation

[2009] We use natural language generation techniques to generate summaries of each email, specifically by extracting a concise, few-line summary from the body of the email using the gensim library.

[2010] Action List Creation

[2011] Based on the importance score and summary, a list of actions for the user to perform is generated, prioritized according to importance.

[2012] Results distribution

[2013] The generated summary and action list are compiled and delivered to the user's terminal. The delivered notification is displayed on the administrator's terminal.

[2014] Device Role

[2015] The user's device receives the summary and action list sent from the server and notifies the administrator. The application on the device displays the notification content in an easy-to-understand manner, allowing the administrator to take action immediately.

[2016] User Roles

[2017] The administrator (user) can quickly take necessary action based on the provided summary and action list. For example, they can prioritize tasks of high importance, such as creating a production plan for a new project or responding to machine failures.

[2018] Specific examples

[2019] For example, if the following email is received on October 1, 2023:

[2020] Work instruction: "Create a production plan for a newly ordered project."

[2021] Manufacturing Progress: "Report on the current progress of the production line"

[2022] Emergency call: "There is a machine breakdown and we need to fix it quickly."

[2023] Maintenance information: "Check the scheduled maintenance schedule"

[2024] Advertisement: "Sales announcement for new materials from suppliers"

[2025] The server then categorizes and rates each email, generating a summary and action list like this:

[2026] Summary example

[2027] Summary of work instruction email: "Create a production plan for the new project this week"

[2028] Summary of manufacturing progress email: "Production line progress is on schedule"

[2029] Emergency message summary: "Machine breakdown, immediate repair required."

[2030] Maintenance information summary: "Scheduled maintenance scheduled for next week"

[2031] Summary of the advertising email: "Information on new materials"

[2032] Action list to be taken

[2033] 1. Creating production plans for new projects

[2034] 2. Response to machine breakdowns

[2035] 3. Preparation for scheduled maintenance

[2036] 4. Checking production progress

[2037] Example prompts for generative AI models

[2038] "Analyze the following email and generate a summary:

[2039] Subject: Create a production plan for a new project

[2040] Body: We need to create a production plan for a newly ordered project this week. Please confirm the detailed specifications and required resources and create an appropriate schedule.

[2041] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2042] Step 1:

[2043] The server establishes a connection to the mail server using the IMAP protocol and retrieves all emails received within a specific period of time. The server stores the metadata of the retrieved emails (sender, recipient, subject, body, date and time) in a database, making all incoming emails available as target input data.

[2044] Step 2:

[2045] The server retrieves each email from the database and analyzes its content using natural language processing technology. Based on the content of the email, the server categorizes it into categories such as work instructions, production progress, emergency contacts, maintenance information, and advertisements. The analyzed data is then stored in the database as output data categorized by category.

[2046] Step 3:

[2047] The server calculates an importance score based on the classified emails. Emails containing important keywords (e.g., "urgent" or "important") or from specific senders are given a higher score. This weights the importance of each email. The score is calculated based on the input data and added to the output data as the importance of each email.

[2048] Step 4:

[2049] The server uses natural language generation techniques, such as the gensim library, to generate a summary of each email's content. The body of each email is summarized into a concise, few-line summary, which is then stored in a database as the output of the analysis.

[2050] Step 5:

[2051] Based on the importance scores and summaries, the server generates a list of actions for the user to perform. The action list is prioritized by importance and identifies tasks to be performed. This action list is stored in a database as part of the output data.

[2052] Step 6:

[2053] The server compiles the generated summary and action list into a single report and sends it to the user's device, allowing the administrator to receive notifications on their smartphone or other device.

[2054] Step 7:

[2055] The terminal displays the received report and notifies the user. Specifically, the terminal's notification function is used to display a summary and action list, allowing the user to quickly take necessary actions based on this.

[2056] Prompt Sentence Examples

[2057] For example, the prompt to verify the generated summary is:

[2058] "Analyze the following email and generate a summary:

[2059] Subject: Create a production plan for a new project

[2060] Body: We need to create a production plan for a newly ordered project this week. Please confirm the detailed specifications and required resources and create an appropriate schedule.

[2061] Through the flow of steps, a system is realized that allows servers, terminals, and users to work together to significantly improve factory operational efficiency.

[2062] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2063] This invention relates to a system that collects emails, automatically classifies them, assesses their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. This system is composed of a server, a terminal, and users, and by combining it with an emotion engine, it provides information and generates action lists that take into account the user's emotional state.

[2064] System Overview

[2065] The system will help users streamline email management and emotion recognition by providing a series of features, including:

[2066] 1. Email Collection:

[2067] The server accesses the mail server and collects all incoming emails for a specific period (e.g., one day).

[2068] 2. Email Classification:

[2069] The server analyzes the content of the emails collected and classifies them into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[2070] 3. Importance rating:

[2071] The server calculates an importance score based on the content of each email and prioritizes it.

[2072] 4. Summary generation:

[2073] The server uses machine learning and natural language generation techniques to generate a summary of each email.

[2074] 5. Create an action list:

[2075] The server generates a list of actions for the user to perform based on the importance score and summary text.

[2076] 6. Emotion recognition:

[2077] The server uses an emotion engine that recognizes the user's emotions, and reflects the emotion information when generating the email contents and action list.

[2078] 7. Delivery of results:

[2079] The server generates a report including the summary, action list, and emotion-based adjustments and sends it to the terminal for display to the user.

[2080] Program processing overview

[2081] Email collection

[2082] 1. Server:

[2083] Every night, all received emails for the day are retrieved from the mail server using the IMAP protocol.

[2084] Store the retrieved emails in a database and save the metadata of each email (sender, recipient, subject, body, date and time).

[2085] Email Classification

[2086] 2. Server:

[2087] The email is retrieved from the database and its content is analyzed using natural language processing techniques.

[2088] Based on the analysis results, the messages are sorted into the following categories: work instructions, customer contact, information sharing, company-wide announcements, and advertising.

[2089] The classification results are stored in a database.

[2090] Importance rating

[2091] 3. Server:

[2092] The importance of each email is scored based on a list of keywords and senders. For example, emails containing keywords such as "urgent" or "urgent" or from specific senders are given a higher score.

[2093] Save the importance score in a database.

[2094] Summary Generation

[2095] 4. Server:

[2096] Natural language generation technology is used to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[2097] The generated summaries are stored in a database.

[2098] Action List Creation

[2099] 5. Server:

[2100] Based on the importance score and summary, a list of actions to be performed by the user is generated. The action list is prioritized in order of importance (e.g., 1. Create documents for a new project, 2. Respond to customer inquiries).

[2101] Store the action list in a database.

[2102] emotion recognition

[2103] 6. Server:

[2104] Recognize the user's emotional state using an emotion engine, which analyzes emotional information detected from the user's voice, facial expressions, text input, etc.

[2105] The system adjusts email content and action list generation based on the user's recognized emotional state. For example, if the user is feeling stressed, it suggests prioritizing high-priority emails.

[2106] Results distribution

[2107] 7. Server:

[2108] The generated summary, action list, and report reflecting the sentiment information are compiled into a single document.

[2109] Send the report to the user's device.

[2110] 8. Terminal:

[2111] The received report is displayed to the user, who then reviews the report and takes the action indicated.

[2112] Specific examples

[2113] For example, if the incoming email on October 1, 2023 is:

[2114] First message: "Work instructions: Create materials for a new project"

[2115] 2nd message: "Contact from customer: Product inquiry"

[2116] Third email: "Information sharing: Market trend report"

[2117] 4th message: "Company-wide notice: System maintenance notice"

[2118] 5th message: "Advertisement: Announcement of new product release"

[2119] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[2120] Summary example

[2121] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[2122] Summary of customer email: "I need more information about product features"

[2123] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[2124] Summary of company-wide email: "System maintenance scheduled for this weekend."

[2125] Promotional email summary: "New product coming next month."

[2126] Action list to be taken

[2127] 1. Creating materials for a new project

[2128] 2. Detailed explanation of product features to customers

[2129] 3. Prepare for system maintenance this weekend

[2130] emotion recognition

[2131] If the user is stressed, adjust the action list to be shorter and focus on the most important tasks to reduce stress.

[2132] In this way, the system allows users to quickly grasp important information and take necessary actions efficiently.By combining emotion engines, it provides adaptive support according to the user's emotional state.

[2133] The processing flow will be explained below.

[2134] Step 1:

[2135] The server connects to the mail server and retrieves all incoming emails for a specific period (e.g., one day) using the IMAP protocol. The server stores the retrieved emails in temporary storage.

[2136] Step 2:

[2137] The server reads the emails retrieved from temporary storage and stores them in a database, along with the metadata for each email (sender, recipient, subject, body, date and time).

[2138] Step 3:

[2139] The server retrieves each email stored in the database and analyzes its content using natural language processing (NLP) technology. Based on the analysis results, the email is automatically classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[2140] Step 4:

[2141] For each email that is categorized, the server calculates an importance score using keyword matching and sender lists. For example, emails containing keywords like "urgent" or "urgent" or from specific senders will be given a higher score.

[2142] Step 5:

[2143] The server uses natural language generation (NLG) technology to generate a summary of each email, which is a concise summary of a few lines that extracts key information from the email body.

[2144] Step 6:

[2145] The server generates a list of actions for the user to perform based on the importance scores and the generated summary, with the list of actions prioritized in order of importance.

[2146] Step 7:

[2147] The server uses an emotion engine to recognize the user's emotional state, which analyzes emotional information from the user's voice, facial expressions, text input, etc.

[2148] Step 8:

[2149] The server adjusts the action list or summary based on the perceived emotional state, for example, if the user is feeling stressed, prioritizing important tasks to reduce stress.

[2150] Step 9:

[2151] The server compiles the summary and action list into a single report, which contains a summary of the messages in each category and the corresponding actions.

[2152] Step 10:

[2153] The server sends the compiled report to the user's device, and the report is provided to the user in the form of email, app notification, or dashboard display.

[2154] Step 11:

[2155] The terminal displays the received report to the user, who then checks the report and takes the action indicated.

[2156] Step 12:

[2157] The user performs the actions indicated and inputs the results as feedback to the system, which allows the system to provide a more accurate action list and summary.

[2158] For example, if the email you received on October 1, 2023 is:

[2159] First message: "Work instructions: Create materials for a new project"

[2160] 2nd message: "Contact from customer: Product inquiry"

[2161] Third email: "Information sharing: Market trend report"

[2162] 4th message: "Company-wide notice: System maintenance notice"

[2163] 5th message: "Advertisement: Announcement of new product release"

[2164] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[2165] Summary example

[2166] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[2167] Summary of customer email: "I need more information about product features"

[2168] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[2169] Summary of company-wide email: "System maintenance scheduled for this weekend."

[2170] Promotional email summary: "New product coming next month."

[2171] Action list to be taken

[2172] 1. Creating materials for a new project

[2173] 2. Detailed explanation of product features to customers

[2174] 3. Prepare for system maintenance this weekend

[2175] emotion recognition

[2176] If users are stressed, shorten the action list and adjust it to focus on the most important tasks.

[2177] In this way, the system allows users to quickly grasp important information and take necessary actions efficiently.By combining emotion engines, it provides adaptive support according to the user's emotional state.

[2178] Example 2

[2179] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2180] In conventional email management systems, received emails are classified, their importance is assessed, summaries are generated, and action lists are created individually, without taking into account the user's emotional state. As a result, users can become overwhelmed by the volume of emails and may miss important emails or tasks. Furthermore, when users are under stress, it is difficult for them to process emails appropriately.

[2181] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting emails, means for classifying the collected emails into a plurality of categories, means for evaluating the importance of each email, means for summarizing the contents of the emails for each category, means for generating a list of actions to be performed by the user, means for delivering the summaries and action list to the user, means for recognizing the user's emotional state, and means for adjusting the action list and summaries according to the emotional state. This enables the user to quickly grasp important information and receive adaptive support according to their emotional state.

[2182] "Email" means a digital message sent or received over the Internet or other digital network.

[2183] "Category" refers to a group of emails classified based on their contents, and examples include work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[2184] An "importance score" is a number used to rate the importance of an email and is calculated based on specific keywords and sender lists.

[2185] "Natural language generation technology" refers to technology that generates content in natural language that humans can understand, and includes machine learning and AI technologies.

[2186] A "summary" is a short sentence that extracts the main points from a longer text and summarizes them concisely.

[2187] An "action list" is a list of specific tasks or actions that a user must perform, prioritized based on importance.

[2188] "Emotional state" refers to the user's psychological state, and includes emotions such as stress, joy, and impatience.

[2189] An "emotion engine" is hardware or software used to recognize a user's emotions, and can obtain emotional information by analyzing voice, facial expressions, text input, etc.

[2190] "User" means an individual or organization that uses this system.

[2191] MODE FOR CARRYING OUT THE INVENTION

[2192] This invention relates to a system that collects emails, automatically classifies them, assesses their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. This system is composed of a server, a terminal, and users, and by combining it with an emotion engine, it provides information and generates action lists that take into account the user's emotional state.

[2193] The system configuration is as follows: first, the server accesses the mail server and collects all received emails for a specific period (e.g., one day). At this stage, the IMAP protocol is used to retrieve the latest emails from the mail server (e.g., Gmail server) and store them in a database (e.g., MySQL). Email metadata (sender, recipient, subject, body, date and time) is also saved at the same time.

[2194] The server then retrieves the email from the database and analyzes its content using natural language processing technology (e.g., Google NLP API). Based on this analysis, the email is classified into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements. The classification results are also stored in the database.

[2195] The server also assigns a score to each email based on its importance, based on keywords (e.g., "urgent" or "urgent") and specific sender lists. This importance score is stored in a database for each email.

[2196] The server then uses machine learning and natural language generation techniques (e.g., OpenAI GPT) to generate a summary of each email. The summary extracts key information from the email body and presents it in a concise format of a few lines. The generated summary is also stored in a database.

[2197] Based on the generated summary and the importance score, the server generates a list of actions for the user to perform. The list of actions is prioritized in order of importance and stored in a database.

[2198] Furthermore, the server uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotional state. This analyzes emotional information detected from the user's voice, facial expressions, text input, etc. Based on the recognized emotional state, the email content and action list generation are adjusted. For example, if the user is feeling stressed, the server suggests prioritizing high-priority emails.

[2199] Finally, the server compiles the generated summary, action list, and emotional information into a report and sends it to the user's device. The device displays the received report to the user, who then checks the report and performs the requested actions.

[2200] Specific examples

[2201] Email received on October 1, 2023:

[2202] "Work instructions: Create materials for a new project"

[2203] "Customer Contact: Product Inquiry"

[2204] "Information Sharing: Market Trends Report"

[2205] "Company-wide Notice: System Maintenance Notice"

[2206] "Advertisement: Announcement of new product release"

[2207] When the server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, the output looks like this:

[2208] Summary example:

[2209] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[2210] Summary of customer email: "I need more information about product features"

[2211] Summary of the information sharing email: "Certain sectors are experiencing rapid growth as a market trend."

[2212] Summary of company-wide email: "System maintenance is scheduled for this weekend."

[2213] Promotional email summary: "New product launching next month"

[2214] List of actions to be taken:

[2215] 1. Creating materials for a new project

[2216] 2. Detailed explanation of product features to customers

[2217] 3. Prepare for system maintenance this weekend

[2218] Emotion recognition:

[2219] If the user is stressed, shorten the action list to reduce stress and focus on the most important tasks.

[2220] Example of an input prompt for a generative AI model:

[2221] Analyze the content of received emails and categorize them into the following categories: work instructions, customer contact, information sharing, company-wide announcements, and promotional advertisements. Also, assign an importance score to each email and generate a summary. Then, create an action list and provide tailored results based on sentiment recognition.

[2222] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2223] Step 1:

[2224] Email collection

[2225] server:

[2226] Every night, the server connects to a mail server using the IMAP protocol, for example, to retrieve incoming emails from the Gmail server.

[2227] The retrieved emails are stored in a database (e.g., MySQL), and metadata for each email (sender, recipient, subject, body, date and time) is stored.

[2228] Input: Received email data from the mail server.

[2229] Output: Email data and metadata stored in a database.

[2230] Step 2:

[2231] Email Classification

[2232] server:

[2233] The server retrieves unclassified emails from the database and analyzes their content using natural language processing techniques (e.g., Google NLP API).

[2234] Based on the analysis results, the information is sorted into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertising.

[2235] The classification results are stored in the database again.

[2236] Input: Unclassified email data from the database.

[2237] Output: Email data categorized by category.

[2238] Step 3:

[2239] Importance rating

[2240] server:

[2241] The server calculates an importance score for each email based on specific keywords (e.g., "urgent" or "urgent") and a list of specific senders.

[2242] An importance score is stored in the database for each email.

[2243] Input: Classified email data.

[2244] Output: Email data with importance scores assigned.

[2245] Step 4:

[2246] Summary Generation

[2247] server:

[2248] The server uses machine learning and natural language generation techniques (e.g., OpenAI GPT) to generate a summary of each email.

[2249] The summary extracts the key information from the email body and presents it in a concise format of a few lines.

[2250] The generated summaries are stored in a database.

[2251] Input: Email data with an importance score.

[2252] Output: The email data with the summary generated.

[2253] Step 5:

[2254] Action List Creation

[2255] server:

[2256] The server generates a list of actions for the user to perform based on the generated summary and importance score.

[2257] Action lists are prioritized in order of importance and stored in a database.

[2258] Input: The email data from which the summary was generated and the importance score.

[2259] Output: A prioritized list of actions.

[2260] Step 6:

[2261] emotion recognition

[2262] server:

[2263] The server uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotional state.

[2264] Emotional information is analyzed from the user's voice, facial expressions, text input, etc.

[2265] Based on the recognized emotional state, adjustments are made when generating email content and action lists.

[2266] Input: User voice, facial expressions, and text input.

[2267] Output: A tailored action list and summary.

[2268] Step 7:

[2269] Results distribution

[2270] server:

[2271] The server compiles the generated summary, action list, and report reflecting the emotional information into a single document.

[2272] Send the document to the user's device.

[2273] Input: Tailored action list and summary.

[2274] Output: Report sent to user terminal.

[2275] Device:

[2276] The terminal presents the received report to the user.

[2277] The user reviews the report and takes the indicated action.

[2278] Input: The report sent by the server.

[2279] Output: The report as displayed to the user.

[2280] (Application example 2)

[2281] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2282] With the spread of autonomous vehicles, drivers are required to efficiently and safely obtain important information while driving. Conventional systems require drivers to manually check emails, determine their importance, and decide on an action, which can distract them from driving. Furthermore, the lack of appropriate information provided takes into account the driver's emotional state, which can increase stress and fatigue. The present invention solves these problems, enabling drivers to safely and efficiently obtain email information and take appropriate action.

[2283] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2284] In this invention, the server includes means for collecting emails, means for classifying the collected emails into a plurality of categories, means for evaluating the importance of each email, means for summarizing the contents of the emails for each category, means for generating a list of actions to be performed by the user, means for delivering the summaries and the action list to the user, means for detecting the driver's state, and means for adjusting the action list based on emotional information. This allows the driver to quickly identify important emails while driving, and to sense stress and fatigue and respond efficiently based on the action list with adjusted priorities.

[2285] "Email" is a digital message sent and received over a computer network.

[2286] "Means of collection" refers to the communication protocols and software functions for obtaining email from a specific mail server.

[2287] A "classification means" is an algorithm or software function that sorts collected emails into categories based on their content.

[2288] A "means for assessing importance" is an algorithm or system that scores importance based on factors such as the content of an email and the sender.

[2289] "Summarization means" refers to natural language processing techniques and software functions that extract key information from lengthy emails and present it in a short form.

[2290] The "means for generating an action list" is a function that creates a list of specific tasks that the user should perform based on the summarized content and importance rating of the email.

[2291] "Means for distribution" refers to the communication protocol and software functions for transferring the generated summary and action list to the user's terminal and displaying or notifying them.

[2292] The "means for adjusting the action list based on emotional information" is a function that analyzes the driver's emotional state and automatically adjusts the priority and content of the action list according to that state.

[2293] "Means for sensing driver state" refers to sensors and software functions that analyze the driver's voice, facial expressions, and behavior to detect their emotional state.

[2294] This system allows drivers to safely and efficiently manage their emails. It collects emails, automatically classifies them, evaluates their importance, generates summaries, creates action lists, recognizes emotions, and distributes the results. The system is primarily composed of a server, a terminal, and users.

[2295] System Configuration

[2296] The server uses the IMAP protocol to collect emails from the mail server. The collected emails are stored in a database, and metadata (sender, recipient, subject, body, date and time) is saved. The server then applies natural language processing (NLP) technology to classify each email into multiple categories, such as work instructions, customer contact, information sharing, company-wide notices, and promotional advertisements. The classified emails are then scored using an importance assessment algorithm.

[2297] Based on the importance rating, the server extracts key information from the body of each email and uses natural language generation technology to generate a concise summary, helping drivers quickly grasp the information they need.

[2298] The server then generates an action list containing specific tasks the user needs to perform, prioritizing them in order of importance.The emotion engine also recognizes emotions from the user's voice, facial expressions, and text input, and reflects them in the action list.

[2299] Finally, the server compiles this information into a single report and sends it to the device (such as a smartphone), which notifies the user by voice or on-screen display.

[2300] Hardware and software used

[2301] Hardware

[2302] Server: A high-performance server for collecting emails, classifying them, assessing their importance, generating summaries, recognizing emotions, and creating action lists.

[2303] User device: smartphone or tablet

[2304] software

[2305] IMAP protocol: for collecting emails

[2306] Natural Language Processing (NLP) algorithms: for email classification and summary generation

[2307] Natural language generation technology: for content summarization

[2308] Emotion Engine: Recognizes the driver's emotional state

[2309] Database: To store collected emails

[2310] Specific examples

[2311] For example, a driver receives the following email on October 1, 2023:

[2312] First message: "Work instructions: Create materials for a new project"

[2313] 2nd message: "Contact from customer: Product inquiry"

[2314] Third email: "Information sharing: Market trend report"

[2315] 4th message: "Company-wide notice: System maintenance notice"

[2316] 5th message: "Advertisement: Announcement of new product release"

[2317] The server collects the above emails and performs classification, importance rating, summary generation, and emotion recognition, resulting in the following output:

[2318] Summary example

[2319] Summary of the work instruction email: "Create materials for the new project by the end of this week."

[2320] Summary of customer email: "I need more information about product features"

[2321] Summary of the information sharing email: "Market trends show that certain sectors are experiencing rapid growth."

[2322] Summary of company-wide email: "System maintenance scheduled for this weekend."

[2323] Promotional email summary: "New product coming next month."

[2324] Action list to be taken

[2325] 1. Creating materials for a new project

[2326] 2. Detailed explanation of product features to customers

[2327] 3. Prepare for system maintenance this weekend

[2328] Example prompts for generative AI models

[2329] Summarize the following email: "This is a customer inquiry. They would like more information on how to use the product. It's urgent."

[2330] The system is designed to help drivers respond to important emails while continuing to drive safely. It provides a concise summary of the email's key content and actions, reads them out loud, and uses emotion recognition to suggest the most appropriate response.

[2331] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2332] Step 1:

[2333] The server uses the IMAP protocol to collect all incoming emails from the mail server for a specified period (e.g., one day) and stores them in a database, including their metadata (sender, recipient, subject, body, date and time).

[2334] Input: Mail server account information, period specification

[2335] Output: Email list (including metadata)

[2336] Step 2:

[2337] The server retrieves the collected emails from the database and uses natural language processing (NLP) technology to analyze the email content and classify it into categories such as work instructions, customer contact, information sharing, company-wide notices, and advertisements.

[2338] Input: Email data collected in Step 1

[2339] Output: A list of emails sorted by category

[2340] Step 3:

[2341] The server calculates the importance score for each email based on the classified email content. For example, emails containing keywords such as "urgent" or "urgent" or emails from specific senders are given a higher score.

[2342] Input: Email data classified in Step 2

[2343] Output: Email data with importance scores assigned

[2344] Step 4:

[2345] The server uses natural language generation technology to generate a summary of each email, extracting key information from the email body and creating a concise summary of a few lines.

[2346] Input: Email data scored in Step 3

[2347] Output: Email data with summary generated

[2348] Step 5:

[2349] The server generates a list of actions for the user to perform based on the importance score and the summary sentence, and prioritizes the list of actions in order of importance.

[2350] Input: Email data summarized in step 4

[2351] Output: A list of actions ordered by importance

[2352] Step 6:

[2353] The server uses an emotion engine to recognize the user's (driver's) emotional state by analyzing data such as voice, facial expressions, and text input. Based on the recognized emotion, it adjusts the action list and notification content.

[2354] Input: Driver's voice, facial expressions, and text data

[2355] Output: Recognized emotional state and adjusted action list

[2356] Step 7:

[2357] The server compiles the generated summary, action list, and a report including adjustments based on the emotion information into a single document and sends it to the user's terminal.

[2358] Input: Summary sentence generated in step 4, Action list created in step 5, Emotion information recognized in step 6

[2359] Output: Report displayed on the user's terminal

[2360] Step 8:

[2361] The user's device will notify the user of the received report via voice or screen display, allowing the driver to quickly grasp important information while driving and efficiently take necessary actions.

[2362] Input: Report sent from the server

[2363] Output: Notification and display to the driver

[2364] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2365] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2366] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2367] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2368] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2369] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2370] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2371] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2372] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2373] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2374] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2375] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2376] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2377] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2378] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2379] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2380] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2381] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2382] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2383] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2384] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2385] The following is further disclosed regarding the above embodiment.

[2386] (Claim 1)

[2387] a means for collecting emails;

[2388] A means of classifying collected emails into multiple categories;

[2389] a means for rating the importance of each email;

[2390] A means of summarizing the content of emails for each category;

[2391] means for generating a list of actions to be performed by the user;

[2392] means for delivering said summary and action list to a user;

[2393] A system including:

[2394] (Claim 2)

[2395] 2. The system of claim 1, wherein the means for assessing the importance of the email calculates an importance score and prioritizes the action list based on the score.

[2396] (Claim 3)

[2397] 2. The system of claim 1, wherein the means for generating a summary uses natural language generation technology to extract key points from the email and generate a concise summary.

[2398] "Example 1"

[2399] (Claim 1)

[2400] a means for collecting emails;

[2401] A means of classifying collected emails into multiple categories;

[2402] a means for rating the importance of each email;

[2403] A means of summarizing the content of emails for each category;

[2404] means for generating a list of actions to be performed by the user;

[2405] means for delivering said summary and action list to a user;

[2406] means for analyzing and storing the metadata of said emails in a database;

[2407] A means for analyzing the content of emails using natural language processing technology;

[2408] means for generating a summary of the email using natural language generation techniques;

[2409] means for prioritizing said action list;

[2410] A system including:

[2411] (Claim 2)

[2412] The means for evaluating the importance of the email calculates an importance score and determines the priority of the action list based on the score.

[2413] 10. The system of claim 1.

[2414] (Claim 3)

[2415] The summary generating means extracts key points from the email using natural language generation technology to generate a concise summary.

[2416] 10. The system of claim 1.

[2417] "Application Example 1"

[2418] (Claim 1)

[2419] a means for collecting emails;

[2420] A means of classifying collected emails into multiple categories;

[2421] a means for rating the importance of each email;

[2422] A means of summarizing the content of emails for each category;

[2423] means for generating a list of actions to be performed by the user;

[2424] means for delivering said summary and action list to a user;

[2425] A means for automatically processing emails related to factory operations and notifying the results to the manager's terminal;

[2426] A system including:

[2427] (Claim 2)

[2428] 2. The system of claim 1, wherein the means for assessing the importance of the email calculates an importance score and prioritizes the action list based on the score.

[2429] (Claim 3)

[2430] 2. The system of claim 1, wherein the means for generating a summary uses natural language generation technology to extract key points from the email and generate a concise summary.

[2431] "Example 2: Combining Emotion Engines"

[2432] (Claim 1)

[2433] a means for collecting emails;

[2434] A means of classifying collected emails into multiple categories;

[2435] a means for rating the importance of each email;

[2436] A means of summarizing the content of emails for each category;

[2437] means for generating a list of actions to be performed by the user;

[2438] means for delivering said summary and action list to a user;

[2439] means for recognizing the emotional state of a user;

[2440] a means of adjusting action lists and summaries depending on emotional state;

[2441] A system including:

[2442] (Claim 2)

[2443] 2. The system of claim 1, wherein the means for assessing the importance of the email calculates an importance score and prioritizes the action list based on the score.

[2444] (Claim 3)

[2445] 2. The system of claim 1, wherein the means for generating a summary uses natural language generation technology to extract key points from the email and generate a concise summary.

[2446] "Application example 2 when combining emotion engines"

[2447] (Claim 1)

[2448] a means for collecting emails;

[2449] A means of classifying collected emails into multiple categories;

[2450] a means for rating the importance of each email;

[2451] A means of summarizing the content of emails for each category;

[2452] means for generating a list of actions to be performed by the user;

[2453] means for delivering said summary and action list to a user;

[2454] means for sensing the dri...

Claims

1. a means for collecting emails; A means of classifying collected emails into multiple categories; a means for rating the importance of each email; A means of summarizing the content of emails for each category; means for generating a list of actions to be performed by the user; means for delivering said summary and action list to a user; A system including:

2. 2. The system of claim 1, wherein the means for evaluating the importance of the email calculates an importance score and prioritizes the action list based on the score.

3. 2. The system of claim 1, wherein the means for generating a summary uses natural language generation technology to extract key points from the email and generate a concise summary.

Images