system

Abstract

We provide the system. [Solution] A means of analyzing electronic communications and determining priorities, Means for generating notifications based on the aforementioned priority, A means of automatically generating replies according to criteria specified by the user, A means for sending the automatically generated reply, An information display device provides means for providing notifications by sound and visual means, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the modern business environment, information communication mainly through e-mails has been continuously increasing, and along with this, the need to process important business e-mails quickly and accurately has been increasing. However, if overwhelmed by the amount of e-mails, there is a risk of missing important messages or delaying responses. Furthermore, daily tasks such as scheduling meetings and task management are very time-consuming when done manually and are factors that reduce business efficiency. Therefore, there is a need to provide a method for improving the efficiency of these tasks and reducing the complexity.

Means for Solving the Problems

[0005] This invention solves this problem by providing a means for analyzing electronic communications and determining priorities. Furthermore, by providing a means for generating notifications based on the priorities, it enables users to grasp important information in real time. In addition, by providing a means for automatically generating replies according to user-specified criteria and sending the automatically generated replies, it achieves rapid and appropriate communication. Moreover, by providing a scheduling suggestion function and a function for classifying electronic communications into multiple categories according to their content, the system reduces the user's workload and improves work efficiency.

[0006] "Electronic communications" refers to the sending and receiving of information via networks such as the Internet, and specifically includes email and messaging services.

[0007] "Priority" refers to a set of criteria for determining the order in which tasks and information are processed, based on their importance and urgency.

[0008] "Notification" refers to a message or signal that a system uses to inform a user of the receipt or update of information.

[0009] "Automatic generation" means that the system generates replies or information based on predetermined algorithms and datasets with minimal user intervention.

[0010] "Transmission" refers to the act of sending information that has been created or received to another device or network.

[0011] "Schedule adjustment" is the process of considering multiple appointments, selecting an appropriate time, and confirming the schedule with all parties involved.

[0012] "Classification" refers to the process of sorting and organizing information into different categories or folders according to its content and characteristics. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention provides a virtual assistant system that streamlines daily business operations. This system operates through the collaborative efforts of three main players: a server, a terminal, and a user.

[0035] First, the server accesses the configured email account and receives new emails. The email content is analyzed by a generative AI model to assess its importance and urgency. This assessment is based on keywords in the email, the sender's address, and past message exchange history. If the email is deemed important, the server immediately generates a notification and sends it to the device.

[0036] The device receives notifications sent from the server and presents them to the user. These notifications include an overview of the email and important information, allowing the user to quickly understand the situation and choose the necessary action. For example, if the user chooses to reply to the email, the AI ​​automatically generates a suggested response using templates and past reply patterns. This generated response can then be reviewed and easily modified by the user.

[0037] Furthermore, the server accesses the user's calendar and adjusts the schedule based on the content of the received email. If a meeting request is included, the server analyzes the user's availability and suggests the best meeting time. The suggestion is notified to the user via their device, and once the user approves, the appointment is automatically added to their calendar.

[0038] This system significantly reduces the burden of task management, meeting scheduling, and email communication for users. This allows them to dedicate more time and resources to important strategic tasks.

[0039] As a concrete example, a user receives a meeting request from an important client via email, and the server determines that the email is important. The device immediately notifies the user, and when the user instructs the AI ​​to respond, the server generates and completes an appropriate reply. This entire process enables appropriate and effective responses while minimizing the user's effort.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server periodically accesses the user's email account to check for new mail. When new mail is received, the server saves it to its database.

[0043] Step 2:

[0044] The server applies a generative AI model to analyze the stored emails. It analyzes the email content, sender, subject, etc., and evaluates their importance and urgency. Based on this evaluation, it classifies the emails into "high," "medium," and "low" priority levels.

[0045] Step 3:

[0046] The server generates notifications for terminals for emails with a "high" priority. These notifications include an email summary and key points, along with partial information to encourage the user to take prompt action.

[0047] Step 4:

[0048] The device receives notifications from the server and displays them in the user interface. The user can then view the email content further by checking the notification and opening the email details.

[0049] Step 5:

[0050] The user selects an AI-powered automated reply to the notification email. Based on this selection, the server generates a draft of an appropriate reply based on templates and past reply history.

[0051] Step 6:

[0052] The server sends a draft of the generated reply to the user's device for review. The device displays the draft, and the user can approve or edit the content.

[0053] Step 7:

[0054] Once the user approves the reply, the device sends the final email to the server. The server then relays the email to the sender under the user's name.

[0055] Step 8:

[0056] When the server detects in an incoming email that requires scheduling adjustments, it checks the user's calendar to identify available time slots. It then lists possible meeting dates and times and presents them to the user.

[0057] Step 9:

[0058] Once the user approves, adjusts, and confirms the proposed schedule, the device automatically adds the details to the calendar. It then sends confirmation emails or invitations to the relevant parties.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] In current electronic communication methods, users require considerable time and effort to manually assess the importance and urgency of a large volume of emails and respond appropriately. Furthermore, scheduling is often done manually, which is inefficient. These factors contribute to decreased work efficiency.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes means for receiving electronic communications and analyzing their content using natural language processing technology, means for evaluating the importance and urgency of communications based on the analysis results, and means for generating notifications to users in accordance with the evaluation. This makes it possible to quickly and automatically determine the importance of electronic communications and immediately take necessary actions.

[0064] "Electronic communications" is a general term for information sent and received in digital format, including email and instant messages.

[0065] "Natural language processing technology" refers to technologies that enable computers to understand, analyze, and generate human language, and are used to analyze and process the meaning of text data.

[0066] A "notification" is a message or alert from a system that informs a user of a situation where specific information or action is required.

[0067] A "template" is a set of standardized sentences or formats tailored to a specific purpose, and is used efficiently in automated replies and email composition.

[0068] "Schedule information" refers to data related to a user's schedule and time management, including meetings and events recorded in the calendar.

[0069] The "method for suggesting schedules" refers to a function that optimizes the user's schedule based on received electronic communications and presents the user with recommended dates and times.

[0070] This virtual assistant system is designed to streamline daily business operations. It functions through the collaborative efforts of three entities: the server, the terminal, and the user.

[0071] The server accesses the electronic communication account specified by the user and receives new communications. The software used includes protocols commonly used by email servers (e.g., IMAP, SMTP). The received communications are analyzed by a generative AI model that utilizes natural language processing technology (e.g., a GPT-based model). This analysis evaluates the importance and urgency of the communication content and determines the priority of each communication.

[0072] If a communication is determined to be important, the server generates a notification summarizing its contents. This notification is sent from the server to the terminal and presented to the user. Terminals include information devices that users use on a daily basis, such as smartphones and PCs. The terminal displays the information from the received notification on its screen, prompting the user to make a quick decision.

[0073] When a user chooses to reply to a communication, the server automatically generates a response. Here too, a generation AI model is utilized, using past reply history and templates to construct an appropriate response. This process enables users to communicate effectively in a short amount of time.

[0074] Furthermore, based on the communication content, the server analyzes and suggests the optimal meeting date and time by referring to the user's schedule information. For example, if a meeting request is included in an email, it uses the Calendar API to detect the user's availability and presents the best schedule. This suggestion is notified to the device, and once the user approves, it is automatically reflected in the calendar.

[0075] For example, if a user receives a meeting request from an important business partner, the server will evaluate this communication as important and send a notification to the user's device. If the user instructs the AI ​​to create a reply, the server will generate an appropriate response. Furthermore, a series of processes to optimize the meeting schedule and add it to the calendar are performed with minimal user intervention.

[0076] A typical example of a prompt message would be something like, "Analyze this important email and generate a reply. The email content is as follows," which would be input to the AI ​​model. Such a system significantly reduces the burden of daily tasks for users, allowing for more efficient use of their time.

[0077] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0078] Step 1:

[0079] The server accesses the user-specified email account and receives new emails. Account information and authentication information are required as input. The server receives communications using IMAP or SMTP protocols and stores them in data storage. The output is a data list of new emails. Specifically, the server periodically connects to the mail server to check for new messages.

[0080] Step 2:

[0081] The server sends the content of received electronic communications to a generating AI model for analysis. The input for this step is the content data of the new communications. The server utilizes natural language processing techniques to evaluate the importance and urgency of the communications. Specifically, the data processing involves breaking down the communication text into tokens and detecting specific keywords and contextual patterns. The output of this analysis is a score indicating the importance and urgency of each communication.

[0082] Step 3:

[0083] The server generates notifications for communications deemed important based on the analysis results. The input for this step is importance and urgency scores. The server uses these scores to select communications to notify the user and creates a summarized notification message. The output is the notification data displayed to the user. Specifically, the server converts the generated notification content into a message format and uses a protocol to send push notifications to the terminal.

[0084] Step 4:

[0085] The device receives notifications from the server and displays them to the user. The input for this step is the notification data sent from the server. The device displays the notification as a pop-up on the screen and provides the user with options for action. The output is user action data. Specifically, the device displays the notification details as a tappable link, providing an interface that allows the user to reply or view details as needed.

[0086] Step 5:

[0087] When a user selects a reply option, the server automatically generates a reply using a generative AI model. The input for this step is the user's reply selection information and past reply data. Based on the selected template and past reply history, the AI ​​suggests the most appropriate reply. The output is the generated reply. Specifically, the server constructs the document using AI-powered natural language generation technology and presents it to the user in an editable format.

[0088] Step 6:

[0089] The server adjusts the schedule based on the content of the electronic communication. The input for this step is the meeting request data and the user's calendar information. The server uses the calendar API to retrieve the user's schedule and proposes the optimal meeting time. The output is the optimized meeting date and time information. Specifically, the server sends the proposed meeting date and time to the terminal in a notification format and automatically adds it to the calendar after obtaining the user's approval.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] In today's information society, business professionals must manage a massive volume of electronic communications, quickly grasp important information, and take appropriate action. However, the burden of managing less important information and schedules can hinder the allocation of resources to strategic activities. Furthermore, receiving and responding to information immediately is extremely difficult while on the go or when hands are occupied. This necessitates improvements in information processing efficiency and a reduction in the user's workload.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes means for analyzing electronic communications and determining priorities, means for generating notifications based on the priorities, means for automatically generating replies according to user-specified criteria, and means for providing notifications in audio and visual form on an information display device. This enables business people to obtain necessary information in a timely manner and to efficiently respond and manage their schedules even while on the go.

[0095] "Electronic communication" refers to messages and information sent and received via electronic means, and specifically includes communication methods such as email and online messaging services.

[0096] "Priority" refers to the criteria or order used to evaluate the importance and urgency of received information and determine the priority of response.

[0097] "Notifications" refer to alerts or messages that inform users of the arrival of information or important matters, and are provided visually or audibly.

[0098] "Automatic generation" refers to a process in which content is formed by programs or algorithms without the need for human intervention.

[0099] An "information display device" is a device or apparatus used to display information to a user visually or audibly, and includes wearable devices and mobile terminals.

[0100] "Schedule management" refers to the processes and systems used to efficiently arrange and manage appointments and tasks.

[0101] The system implementing this invention involves the cooperative operation of three elements: a server, a terminal, and a user. First, the server accesses the email account and retrieves newly received electronic communications. The retrieved emails are analyzed by a generative AI model to evaluate their importance and urgency. This evaluation is based on keywords in the email, the sender's address, and the history of past messages. If the email is determined to be important, the server generates a notification and sends it to the terminal.

[0102] The terminal receives notifications sent from the server and presents them to the user through an information display device. This information display device is implemented as smart glasses or other wearable devices and provides notifications via voice and visual means. The notification includes an overview of the email and important information, allowing the user to quickly understand the situation and choose the necessary action. If the user chooses to reply to the email, the server uses AI to automatically generate a recommended reply based on templates and past reply patterns. The generated reply can be reviewed by the user and made minor corrections as needed.

[0103] Furthermore, the server accesses the user's calendar information and adjusts the schedule based on the content of incoming emails. For example, if an email contains a meeting request, the server analyzes the user's availability and suggests the best meeting date and time, either verbally or visually. Once the user accepts the suggestion, the appointment is automatically added to the calendar.

[0104] Through the process described above, this system significantly reduces the burden of task management, meeting scheduling, and email communication on users, allowing them to allocate more resources to important strategic tasks. For example, if a user receives a meeting request from an important client via email, and the server determines the email is important, the terminal will quickly notify the user. The user can then instruct the AI ​​to generate an appropriate reply. This entire process minimizes user effort while enabling appropriate and effective responses.

[0105] Examples of prompts to input into a generative AI model are as follows:

[0106] "New email received: {Summary of email content}. Sender: {Sender's name}. Evaluate the importance of this email and generate the most appropriate action suggestion."

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] The server accesses the user's email account and retrieves new mail. This is done using protocols such as IMAP. It uses email account information as input and outputs data of new mail.

[0110] Step 2:

[0111] The server applies a generative AI model to the acquired email data to analyze the email content. This analysis includes keyword extraction and contextual understanding, and evaluates importance and urgency. Using email content as input, it generates email importance information as output.

[0112] Step 3:

[0113] The server generates and sends notifications to user terminals for emails deemed to be of high importance. These notifications include a summary of the email's important information. The server uses email data with assessed importance as input and obtains user notifications as output.

[0114] Step 4:

[0115] The terminal presents notifications received from the server to the user through an information display device. In this process, it uses the display and speakers of smart glasses or other devices to provide information visually and audibly. It uses the server's notification information as input and obtains audiovisual information for the user as output.

[0116] Step 5:

[0117] The user chooses to reply to the email based on the presented notification. Once the user instructs to reply, the reply request is sent to the server. The user's action is used as input, and the reply request to the server is generated as output.

[0118] Step 6:

[0119] The server receives a user's reply request and automatically generates an appropriate reply using a generative AI model. This reply is generated by referring to past reply patterns and templates. The user's reply request is used as input, and the automatically generated reply is obtained as output.

[0120] Step 7:

[0121] The server presents the user with an automatically generated reply and requests final confirmation. If the user reviews and makes corrections, the server receives those changes and constructs the final reply. The server uses the automatically generated reply and the user's corrections as input to generate the final reply content as output.

[0122] Step 8:

[0123] The server sends the final reply to the recipient. Using the confirmed final reply as input, the sent email is obtained as output.

[0124] Step 9:

[0125] The server accesses the user's calendar, analyzes meeting requests included in emails, and adjusts the user's available schedule. A generative AI model suggests the optimal meeting date and time, and notifies the user's device. Meeting requests and calendar data are used as input, and meeting date and time suggestions are obtained as output.

[0126] Step 10:

[0127] The terminal presents the user with a suggested meeting date and time and awaits approval. If approved, the information is sent to the server and automatically registered in the calendar. The server's meeting suggestions are used as input, and the schedule change after user approval is obtained as output.

[0128] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0129] This invention is a virtual assistant system designed to improve operational efficiency in business, and in particular includes an emotion engine that recognizes user emotions and optimizes responses. This system functions collaboratively with a server, a terminal, and a user.

[0130] The server accesses the user's email account to receive and store new emails. The received emails are analyzed using a generative AI model to determine their importance and urgency. This analysis utilizes the email's text, metadata, and past communication history. Once the email priorities are determined, the server generates appropriate notifications based on this information and sends them to the user's device.

[0131] Furthermore, the server uses an emotion engine to analyze the underlying emotions expressed by the email sender, as well as emotions based on the user's past behavior history. This emotion data is used to adjust the tone of notifications and replies to the user.

[0132] The device receives notifications sent from the server and displays them to the user on the screen. Based on these notifications, the user can decide on a response, taking into account emotional information. Specifically, immediate replies are recommended for emails that are deemed urgent or have a high level of emotional impact.

[0133] When a user replies, the AI ​​generates an automated draft response that takes context into account based on sentiment data. This draft uses polite language and is emotionally sensitive. Users can review the draft, make minor edits, and then send it.

[0134] For example, if a user receives a thank-you email from a customer, the server recognizes this emotion as "joy" and suggests an appropriate reply template. Based on the suggested reply, the user can send a response that reflects their gratitude. This system not only streamlines email communication but also enhances the effectiveness of communication and helps build better relationships.

[0135] As described above, the present invention details the configuration of a system that streamlines information processing in electronic communications and provides a richer user experience by utilizing emotional data.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The server periodically accesses the configured email account and retrieves new emails. The retrieved emails are stored in a database.

[0139] Step 2:

[0140] The server analyzes stored emails using a generation AI model to determine their importance and urgency based on their content, metadata, and past communication history. Based on these results, emails are categorized into different priority groups.

[0141] Step 3:

[0142] The server uses an emotion engine to analyze the sender's emotions from the content of the email. It detects emotions such as "joy," "anger," and "surprise" from the writing style and keywords in the email, and records this information in a database.

[0143] Step 4:

[0144] The server generates notifications based on emotion and priority information. For high-priority emails, such as those that are "urgent" and contain the emotion "anger," the notification will include a note indicating that immediate action is required.

[0145] Step 5:

[0146] The device displays the generated notification to the user. The notification includes a summary of the email, its importance, and even emotionally-based suggestions for action. The user uses this information to decide whether or not to check the email.

[0147] Step 6:

[0148] Users can check notifications, open email details if necessary, and reply. They can also utilize AI-generated, emotion-sensitive reply templates.

[0149] Step 7:

[0150] The server generates an emotionally adaptive automated response draft based on the user's selection. This draft is composed of an appropriate writing style, such as a warm or cool tone, that reflects the user's emotional data.

[0151] Step 8:

[0152] The user reviews the draft reply and edits it as needed. After editing is complete, the device sends the approved email to the server.

[0153] Step 9:

[0154] The server relays the final email to the recipient. At the same time, it saves the sending history to a database for use in future processing and analysis.

[0155] This process can improve users' work efficiency while promoting more human-centered communication.

[0156] (Example 2)

[0157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] In today's business environment, a massive volume of electronic communications is generated, requiring the rapid and efficient identification and handling of critical information. However, many systems merely provide email sorting and notification functions, failing to offer detailed responses that consider the sender's sentiments or the prioritization of the content. As a result, users may overlook important information or engage in inappropriate communication.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0160] In this invention, the server includes means for analyzing electronic communications and determining priorities and emotions; means for generating notifications based on the determined priorities and emotions; and means for automatically generating replies according to criteria and emotion data specified by the user. This enables users to communicate efficiently and with consideration for emotions.

[0161] "Electronic communications" refers to digital messages and data sent and received via the internet.

[0162] "Priority" is a criterion for determining the order in which tasks and information are processed, based on their importance and urgency.

[0163] "Emotions" refer to the feelings and psychological states that the emotion engine analyzes and identifies from the content of emails and messages.

[0164] A "notification" is a message or alert that immediately informs a user of important information or an action.

[0165] "Methods for automatically generating replies" refers to technologies and methods that use AI to automatically create response texts to messages received by users.

[0166] "Editable" means that the user can manually modify or add to the automatically generated reply text.

[0167] This invention is a system for efficiently and emotionally processing electronic communications. This system primarily operates through the collaboration of three parties: a server, a terminal, and a user. The server, with the user's permission, accesses their email account and checks for and stores newly received emails at regular intervals. This requires a server computer and a database system as hardware components.

[0168] The server uses a generative AI model to analyze stored electronic communications. Specifically, it leverages natural language processing techniques and employs a general generative AI framework as the model (for example, a widely used AI framework that does not specify names of people or companies). This analysis determines the importance and urgency of emails. This process comprehensively evaluates the email body, metadata, and past communication history.

[0169] The server then uses an emotion engine to analyze the emotions inherent in the email. The results of this emotion analysis are used as useful information for the user in the notification generation and automated reply generation processes. An example of a prompt at this stage would be: "Analyze the content of this email to determine its urgency and the sender's emotions. Also, suggest an appropriate reply to this thank-you email."

[0170] The device receives notifications sent from the server and immediately displays them on the user's screen. The device places emphasis on UI design to enable users to take appropriate action immediately based on the content of the notification. For example, emails judged to have a "high" emotional impact or those that are urgent are highlighted on the screen using special icons or colors.

[0171] This system allows users to efficiently handle emails. When replying, users receive a draft reply automatically generated by a generative AI model, edit its content, and send it. This enables users to complete responses quickly and in an emotionally sensitive manner.

[0172] As described above, this invention significantly improves the efficiency of electronic communication processing and makes it possible to provide users with a comfortable and meaningful communication experience.

[0173] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0174] Step 1:

[0175] The server accesses the user's email account and retrieves new emails. The input is the user's account information, and the output is the newly received email data. The server stores these emails in a database and prepares the email text and metadata for analysis.

[0176] Step 2:

[0177] The server uses a generated AI model to analyze the content of stored emails. The input is the email text data and associated metadata, and the output is information about the importance and urgency of the emails. This analysis uses natural language processing techniques to extract keywords and phrases contained in the emails and determine their urgency.

[0178] Step 3:

[0179] The server uses an emotion engine to analyze the sender's emotions within incoming emails. The input is the text data of the email, and the output is the type and intensity of the emotion. For example, emotions such as gratitude, anger, and expectation are identified, and based on this, the server provides guidance on how the user should respond.

[0180] Step 4:

[0181] The server generates a notification and sends it to the device. The input is the importance, urgency, and sentiment data obtained in the previous step, and the output is a notification message for the user. This notification is immediately received by the device and displayed appropriately on the screen. The user decides whether to check or reply to the email based on this notification.

[0182] Step 5:

[0183] The device activates an AI-powered automatic reply function when the user indicates their intention to reply to an email. Inputs include email content, sentiment data, and user-specified reply criteria, while output is an automatically generated draft reply. The generated reply appropriately considers polite language and emotional expression.

[0184] Step 6:

[0185] The user reviews the automatically generated reply and edits it as needed. The input is the reply generated by the AI, and the output is the final, edited reply. The user can then send this reply via the server, ensuring the appropriate email is delivered to the recipient.

[0186] (Application Example 2)

[0187] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0188] Traditional electronic communication management systems simply prioritized emails based on their content, sending notifications and replies in a uniform format, making it difficult to communicate in a way that suited the user's emotions and circumstances. As a result, users risked missing important emails or replying in an inappropriate tone. Furthermore, the burden of manually processing individual emails in daily work was significant, creating a need for improved communication efficiency and quality.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0190] In this invention, the server includes means for analyzing electronic communications and determining priorities, means for generating automatic replies based on emotional data, and means for adjusting emotional tone. This enables prioritization and replies to be adjusted according to the user's emotions, resulting in efficient and effective communication tailored to individual needs.

[0191] "Electronic communication" refers to messages and information that are sent and received electronically.

[0192] "Analysis" is the act of analyzing data in order to understand its content and to classify or assign meaning to it.

[0193] "Priority" refers to the ranking used to determine the order in which tasks or actions are handled.

[0194] A "notification" is a message used to inform a user of important information.

[0195] "Emotional data" refers to information that indicates the emotional state of the user or sender.

[0196] An "automatic reply" is a response message generated by a machine based on pre-set criteria.

[0197] "Emotional tone" refers to the way emotions are expressed and the overall atmosphere in communication.

[0198] "Schedule adjustment" means rationally combining appointments and schedules.

[0199] "Metadata" refers to information that includes additional information about the data.

[0200] "Classification" is the process of grouping data based on specific attributes.

[0201] In this invention, the system functions collaboratively with three parties: a server, a terminal, and a user. The server is responsible for receiving and analyzing electronic communications, determining priorities, and transmitting that information to the terminal. Specifically, it uses the content and metadata of electronic communications to extract sentiment data using a generative AI model and generates information necessary for prioritizing replies and notifications.

[0202] The server uses the Python programming language and sentiment analysis libraries (e.g., TextBlob, NLTK) to analyze electronic communications. The analyzed data, along with priority scores and sentiment tones, is sent to the terminal. A cloud-based data server is utilized to enable real-time data processing.

[0203] The terminal provides appropriate notifications to the user based on data received from the server. These terminals may include smart home devices or robots, equipped with voice devices and displays that consider received priority and sentiment scores to deliver notifications to the user.

[0204] Users select actions based on information received via their device. For example, they can immediately acknowledge or reply to particularly important notifications. Based on the user's selection, the server again utilizes AI to generate appropriate reply templates.

[0205] As a concrete example, consider a scenario where a robot is reading the news in the morning when an email regarding an important meeting arrives. In this case, the system would classify the email as "high priority" and suggest a reply in voice, reflecting the appropriate tone of voice. The user can then review the suggested reply and quickly complete the communication by easily approving or editing it.

[0206] An example of a prompt to input into the generative AI model is, "Based on the user's recent behavior history, suggest the optimal tone for the next email reply." Using this prompt, the AI ​​generates a reply that takes the user's emotional state into account.

[0207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0208] Step 1:

[0209] The server receives emails from the user's telecommunications provider. It retrieves the text data, metadata (date and time, sender information, etc.), and past communication history of the received emails. This information is aggregated on the server as input data. The server analyzes this data using a sentiment analysis library to calculate the importance and sentiment score of the emails. As a result of this analysis, a priority score and sentiment data are output.

[0210] Step 2:

[0211] Based on the analysis results, the server generates a notification tailored to the user using a generative AI model. The priority score and sentiment data calculated in step 1 are used as input data. The server sends the generated notification message to the device. The generated notification message is forwarded to the device as output.

[0212] Step 3:

[0213] The device processes notification messages received from the server and displays notifications to the user as alerts or voice guidance. Notification content includes email type, priority, and recommended actions based on sentiment score. The device functions as an interface to communicate with the user on specific devices (smartphones, smart speakers, etc.). The user receives notifications visually or audibly as output.

[0214] Step 4:

[0215] The user chooses whether or not to reply to the received notification. Based on the user's choice, the server generates an automated reply that takes emotional tone into account. The user's choice and emotional data are used as input data. The server utilizes a generative AI model to create a prompt for the reply message and proposes it to the user. As output, the proposed reply message is presented to the user.

[0216] Step 5:

[0217] The user reviews the suggested reply and edits it as needed. After editing, they select the final reply and send it. The server reviews the edited reply again and sends the email to the recipient's address. The user's edited reply is used as input data. The sent email is recorded as output.

[0218] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0219] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0220] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0221] [Second Embodiment]

[0222] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0223] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0224] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0225] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0226] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0228] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0229] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0230] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0231] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0232] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0233] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0234] This invention provides a virtual assistant system that streamlines daily business operations. This system operates through the collaborative efforts of three main players: a server, a terminal, and a user.

[0235] First, the server accesses the configured email account and receives new emails. The email content is analyzed by a generative AI model to assess its importance and urgency. This assessment is based on keywords in the email, the sender's address, and past message exchange history. If the email is deemed important, the server immediately generates a notification and sends it to the device.

[0236] The device receives notifications sent from the server and presents them to the user. These notifications include an overview of the email and important information, allowing the user to quickly understand the situation and choose the necessary action. For example, if the user chooses to reply to the email, the AI ​​automatically generates a suggested response using templates and past reply patterns. This generated response can then be reviewed and easily modified by the user.

[0237] Furthermore, the server accesses the user's calendar and adjusts the schedule based on the content of the received email. If a meeting request is included, the server analyzes the user's availability and suggests the best meeting time. The suggestion is notified to the user via their device, and once the user approves, the appointment is automatically added to their calendar.

[0238] This system significantly reduces the burden of task management, meeting scheduling, and email communication for users. This allows them to dedicate more time and resources to important strategic tasks.

[0239] As a concrete example, a user receives a meeting request from an important client via email, and the server determines that the email is important. The device immediately notifies the user, and when the user instructs the AI ​​to respond, the server generates and completes an appropriate reply. This entire process enables appropriate and effective responses while minimizing the user's effort.

[0240] The following describes the processing flow.

[0241] Step 1:

[0242] The server periodically accesses the user's email account to check for new mail. When new mail is received, the server saves it to its database.

[0243] Step 2:

[0244] The server applies a generative AI model to analyze the stored emails. It analyzes the email content, sender, subject, etc., and evaluates their importance and urgency. Based on this evaluation, it classifies the emails into "high," "medium," and "low" priority levels.

[0245] Step 3:

[0246] The server generates notifications for terminals for emails with a "high" priority. These notifications include an email summary and key points, along with partial information to encourage the user to take prompt action.

[0247] Step 4:

[0248] The device receives notifications from the server and displays them in the user interface. The user can then view the email content further by checking the notification and opening the email details.

[0249] Step 5:

[0250] The user selects an AI-powered automated reply to the notification email. Based on this selection, the server generates a draft of an appropriate reply based on templates and past reply history.

[0251] Step 6:

[0252] The server sends a draft of the generated reply to the user's device for review. The device displays the draft, and the user can approve or edit the content.

[0253] Step 7:

[0254] Once the user approves the reply, the device sends the final email to the server. The server then relays the email to the sender under the user's name.

[0255] Step 8:

[0256] When the server detects in an incoming email that requires scheduling adjustments, it checks the user's calendar to identify available time slots. It then lists possible meeting dates and times and presents them to the user.

[0257] Step 9:

[0258] Once the user approves, adjusts, and confirms the proposed schedule, the device automatically adds the details to the calendar. It then sends confirmation emails or invitations to the relevant parties.

[0259] (Example 1)

[0260] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0261] In current electronic communication methods, users require considerable time and effort to manually assess the importance and urgency of a large volume of emails and respond appropriately. Furthermore, scheduling is often done manually, which is inefficient. These factors contribute to decreased work efficiency.

[0262] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0263] In this invention, the server includes means for receiving electronic communications and analyzing their content using natural language processing technology, means for evaluating the importance and urgency of communications based on the analysis results, and means for generating notifications to users in accordance with the evaluation. This makes it possible to quickly and automatically determine the importance of electronic communications and immediately take necessary actions.

[0264] "Electronic communications" is a general term for information sent and received in digital format, including email and instant messages.

[0265] "Natural language processing technology" refers to technologies that enable computers to understand, analyze, and generate human language, and are used to analyze and process the meaning of text data.

[0266] A "notification" is a message or alert from a system that informs a user of a situation where specific information or action is required.

[0267] A "template" is a set of standardized sentences or formats tailored to a specific purpose, and is used efficiently in automated replies and email composition.

[0268] "Schedule information" refers to data related to a user's schedule and time management, including meetings and events recorded in the calendar.

[0269] The "method for suggesting schedules" refers to a function that optimizes the user's schedule based on received electronic communications and presents the user with recommended dates and times.

[0270] This virtual assistant system is designed to streamline daily business operations. It functions through the collaborative efforts of three entities: the server, the terminal, and the user.

[0271] The server accesses the electronic communication account specified by the user and receives new communications. The software used includes protocols commonly used by email servers (e.g., IMAP, SMTP). The received communications are analyzed by a generative AI model that utilizes natural language processing technology (e.g., a GPT-based model). This analysis evaluates the importance and urgency of the communication content and determines the priority of each communication.

[0272] If a communication is determined to be important, the server generates a notification summarizing its contents. This notification is sent from the server to the terminal and presented to the user. Terminals include information devices that users use on a daily basis, such as smartphones and PCs. The terminal displays the information from the received notification on its screen, prompting the user to make a quick decision.

[0273] When a user chooses to reply to a communication, the server automatically generates a response. Here too, a generation AI model is utilized, using past reply history and templates to construct an appropriate response. This process enables users to communicate effectively in a short amount of time.

[0274] Furthermore, based on the communication content, the server analyzes and suggests the optimal meeting date and time by referring to the user's schedule information. For example, if a meeting request is included in an email, it uses the Calendar API to detect the user's availability and presents the best schedule. This suggestion is notified to the device, and once the user approves, it is automatically reflected in the calendar.

[0275] For example, if a user receives a meeting request from an important business partner, the server will evaluate this communication as important and send a notification to the user's device. If the user instructs the AI ​​to create a reply, the server will generate an appropriate response. Furthermore, a series of processes to optimize the meeting schedule and add it to the calendar are performed with minimal user intervention.

[0276] A typical example of a prompt message would be something like, "Analyze this important email and generate a reply. The email content is as follows," which would be input to the AI ​​model. Such a system significantly reduces the burden of daily tasks for users, allowing for more efficient use of their time.

[0277] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0278] Step 1:

[0279] The server accesses the electronic communication account specified by the user and receives incoming electronic communications. Account information and authentication information are required as inputs. The server receives communications using IMAP or SMTP protocols and stores them in data storage. The output is a data list of incoming electronic communications. As a specific operation, the server periodically connects to the mail server to check for new messages.

[0280] Step 2:

[0281] The server sends the content of the received electronic communication to the generative AI model for analysis. The input for this step is the content data of the incoming communication. The server utilizes natural language processing technology to evaluate the importance and urgency from the communication content. As specific data processing, the communication text is decomposed into tokens to detect specific keywords and context patterns. The output of this analysis is a score indicating the importance and urgency for each communication.

[0282] Step 3:

[0283] Based on the analysis results, the server generates notifications for communications determined to be important. The input for this step is the scores of importance and urgency. The server selects the communications to notify the user using these scores and creates a summarized notification text. The output is the notification data displayed to the user. As a specific operation, the server converts the generated notification content into a message format and uses the protocol to send push notifications to the terminal.

[0284] Step 4:

[0285] The terminal receives notifications from the server and displays them to the user. The input for this step is the notification data sent from the server. The terminal displays the notification as a pop-up on the screen and provides options for the user to respond. The output is the user's action data. As a specific operation, the terminal displays the details of the notification as tappable links and provides an interface that allows the user to reply or view details as needed.

[0286] Step 5:

[0287] When the user selects the reply option, the server uses a generative AI model to automatically generate the reply content. The input for this step is the user's reply selection information and past reply data. The AI proposes an optimal reply text based on the selected template and past reply history. The output is the generated reply text. As a specific operation, the server constructs a document using natural language generation technology by the AI and presents it to the user in an editable format.

[0288] Step 6:

[0289] The server adjusts the schedule based on the content of the electronic communication. The input for this step is the meeting request data and the user's calendar information. The server uses the calendar API to obtain the user's schedule and proposes an optimal meeting time. The output is the optimized meeting date and time information. As a specific operation, the server sends the proposed meeting date and time to the terminal in a notification format and automatically adds it to the calendar after obtaining the user's approval.

[0290] (Application Example 1)

[0291] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0292] In today's information society, business professionals must manage a massive volume of electronic communications, quickly grasp important information, and take appropriate action. However, the burden of managing less important information and schedules can hinder the allocation of resources to strategic activities. Furthermore, receiving and responding to information immediately is extremely difficult while on the go or when hands are occupied. This necessitates improvements in information processing efficiency and a reduction in the user's workload.

[0293] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0294] In this invention, the server includes means for analyzing electronic communications and determining priorities, means for generating notifications based on the priorities, means for automatically generating replies according to user-specified criteria, and means for providing notifications in audio and visual form on an information display device. This enables business people to obtain necessary information in a timely manner and to efficiently respond and manage their schedules even while on the go.

[0295] "Electronic communication" refers to messages and information sent and received via electronic means, and specifically includes communication methods such as email and online messaging services.

[0296] "Priority" refers to the criteria or order used to evaluate the importance and urgency of received information and determine the priority of response.

[0297] "Notifications" refer to alerts or messages that inform users of the arrival of information or important matters, and are provided visually or audibly.

[0298] "Automatic generation" refers to a process in which content is formed by programs or algorithms without the need for human intervention.

[0299] An "information display device" is a device or apparatus used to display information to a user visually or audibly, and includes wearable devices and mobile terminals.

[0300] "Schedule management" refers to the processes and systems used to efficiently arrange and manage appointments and tasks.

[0301] The system implementing this invention involves the cooperative operation of three elements: a server, a terminal, and a user. First, the server accesses the email account and retrieves newly received electronic communications. The retrieved emails are analyzed by a generative AI model to evaluate their importance and urgency. This evaluation is based on keywords in the email, the sender's address, and the history of past messages. If the email is determined to be important, the server generates a notification and sends it to the terminal.

[0302] The terminal receives notifications sent from the server and presents them to the user through an information display device. This information display device is implemented as smart glasses or other wearable devices and provides notifications via voice and visual means. The notification includes an overview of the email and important information, allowing the user to quickly understand the situation and choose the necessary action. If the user chooses to reply to the email, the server uses AI to automatically generate a recommended reply based on templates and past reply patterns. The generated reply can be reviewed by the user and made minor corrections as needed.

[0303] Furthermore, the server accesses the user's calendar information and adjusts the schedule based on the content of incoming emails. For example, if an email contains a meeting request, the server analyzes the user's availability and suggests the best meeting date and time, either verbally or visually. Once the user accepts the suggestion, the appointment is automatically added to the calendar.

[0304] Through the above process, this system can significantly reduce the burden of the user's task management, meeting scheduling, and communication via email, making it possible to allocate a lot of resources to important strategic operations. As a specific example, when a user receives a meeting request from an important customer via email and the server determines that the email is important, the terminal quickly notifies the user. When the user instructs an AI reply, the server generates an appropriate reply. This series of processes enables appropriate and effective responses while minimizing the user's effort.

[0305] Examples of the prompt text input to the generation AI model are as follows:

[0306] "New email received: {summary of email content}. The sender is {sender's name}. Please evaluate the importance of this email and generate a proposal for the optimal action."

[0307] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0308] Step 1:

[0309] The server accesses the user's email account and retrieves new emails. At this time, a protocol such as the IMAP protocol is used. The email account information is used as input, and the new email data is obtained as output.

[0310] Step 2:

[0311] The server applies the generation AI model to the retrieved email data and analyzes the email content. In this analysis, keyword extraction and context understanding are performed to evaluate the importance and urgency. The email content is used as input, and the email importance information is generated as output.

[0312] Step 3:

[0313] The server generates and sends notifications to user terminals for emails deemed to be of high importance. These notifications include a summary of the email's important information. The server uses email data with assessed importance as input and obtains user notifications as output.

[0314] Step 4:

[0315] The terminal presents notifications received from the server to the user through an information display device. In this process, it uses the display and speakers of smart glasses or other devices to provide information visually and audibly. It uses the server's notification information as input and obtains audiovisual information for the user as output.

[0316] Step 5:

[0317] The user chooses to reply to the email based on the presented notification. Once the user instructs to reply, the reply request is sent to the server. The user's action is used as input, and the reply request to the server is generated as output.

[0318] Step 6:

[0319] The server receives a user's reply request and automatically generates an appropriate reply using a generative AI model. This reply is generated by referring to past reply patterns and templates. The user's reply request is used as input, and the automatically generated reply is obtained as output.

[0320] Step 7:

[0321] The server presents the user with an automatically generated reply and requests final confirmation. If the user reviews and makes corrections, the server receives those changes and constructs the final reply. The server uses the automatically generated reply and the user's corrections as input to generate the final reply content as output.

[0322] Step 8:

[0323] The server sends the final reply to the recipient. Using the confirmed final reply as input, the sent email is obtained as output.

[0324] Step 9:

[0325] The server accesses the user's calendar, analyzes meeting requests included in emails, and adjusts the user's available schedule. A generative AI model suggests the optimal meeting date and time, and notifies the user's device. Meeting requests and calendar data are used as input, and meeting date and time suggestions are obtained as output.

[0326] Step 10:

[0327] The terminal presents the user with a suggested meeting date and time and awaits approval. If approved, the information is sent to the server and automatically registered in the calendar. The server's meeting suggestions are used as input, and the schedule change after user approval is obtained as output.

[0328] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0329] This invention is a virtual assistant system designed to improve operational efficiency in business, and in particular includes an emotion engine that recognizes user emotions and optimizes responses. This system functions collaboratively with a server, a terminal, and a user.

[0330] The server accesses the user's email account to receive and store new emails. The received emails are analyzed using a generative AI model to determine their importance and urgency. This analysis utilizes the email's text, metadata, and past communication history. Once the email priorities are determined, the server generates appropriate notifications based on this information and sends them to the user's device.

[0331] Furthermore, the server uses an emotion engine to analyze the underlying emotions expressed by the email sender, as well as emotions based on the user's past behavior history. This emotion data is used to adjust the tone of notifications and replies to the user.

[0332] The device receives notifications sent from the server and displays them to the user on the screen. Based on these notifications, the user can decide on a response, taking into account emotional information. Specifically, immediate replies are recommended for emails that are deemed urgent or have a high level of emotional impact.

[0333] When a user replies, the AI ​​generates an automated draft response that takes context into account based on sentiment data. This draft uses polite language and is emotionally sensitive. Users can review the draft, make minor edits, and then send it.

[0334] For example, if a user receives a thank-you email from a customer, the server recognizes this emotion as "joy" and suggests an appropriate reply template. Based on the suggested reply, the user can send a response that reflects their gratitude. This system not only streamlines email communication but also enhances the effectiveness of communication and helps build better relationships.

[0335] As described above, the present invention details the configuration of a system that streamlines information processing in electronic communications and provides a richer user experience by utilizing emotional data.

[0336] The following describes the processing flow.

[0337] Step 1:

[0338] The server periodically accesses the configured email account and retrieves new emails. The retrieved emails are stored in a database.

[0339] Step 2:

[0340] The server analyzes stored emails using a generation AI model to determine their importance and urgency based on their content, metadata, and past communication history. Based on these results, emails are categorized into different priority groups.

[0341] Step 3:

[0342] The server uses an emotion engine to analyze the sender's emotions from the content of the email. It detects emotions such as "joy," "anger," and "surprise" from the writing style and keywords in the email, and records this information in a database.

[0343] Step 4:

[0344] The server generates notifications based on emotion and priority information. For high-priority emails, such as those that are "urgent" and contain the emotion "anger," the notification will include a note indicating that immediate action is required.

[0345] Step 5:

[0346] The device displays the generated notification to the user. The notification includes a summary of the email, its importance, and even emotionally-based suggestions for action. The user uses this information to decide whether or not to check the email.

[0347] Step 6:

[0348] Users can check notifications, open email details if necessary, and reply. They can also utilize AI-generated, emotion-sensitive reply templates.

[0349] Step 7:

[0350] The server generates an emotionally adaptive automated response draft based on the user's selection. This draft is composed of an appropriate writing style, such as a warm or cool tone, that reflects the user's emotional data.

[0351] Step 8:

[0352] The user reviews the draft reply and edits it as needed. After editing is complete, the device sends the approved email to the server.

[0353] Step 9:

[0354] The server relays the final email to the recipient. At the same time, it saves the sending history to a database for use in future processing and analysis.

[0355] This process can improve users' work efficiency while promoting more human-centered communication.

[0356] (Example 2)

[0357] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0358] In today's business environment, a massive volume of electronic communications is generated, requiring the rapid and efficient identification and handling of critical information. However, many systems merely provide email sorting and notification functions, failing to offer detailed responses that consider the sender's sentiments or the prioritization of the content. As a result, users may overlook important information or engage in inappropriate communication.

[0359] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0360] In this invention, the server includes means for analyzing electronic communications and determining priorities and emotions; means for generating notifications based on the determined priorities and emotions; and means for automatically generating replies according to criteria and emotion data specified by the user. This enables users to communicate efficiently and with consideration for emotions.

[0361] "Electronic communications" refers to digital messages and data sent and received via the internet.

[0362] "Priority" is a criterion for determining the order in which tasks and information are processed, based on their importance and urgency.

[0363] "Emotions" refer to the feelings and psychological states that the emotion engine analyzes and identifies from the content of emails and messages.

[0364] A "notification" is a message or alert that immediately informs a user of important information or an action.

[0365] "Methods for automatically generating replies" refers to technologies and methods that use AI to automatically create response texts to messages received by users.

[0366] "Editable" means that the user can manually modify or add to the automatically generated reply text.

[0367] This invention is a system for efficiently and emotionally processing electronic communications. This system primarily operates through the collaboration of three parties: a server, a terminal, and a user. The server, with the user's permission, accesses their email account and checks for and stores newly received emails at regular intervals. This requires a server computer and a database system as hardware components.

[0368] The server uses a generative AI model to analyze stored electronic communications. Specifically, it leverages natural language processing techniques and employs a general generative AI framework as the model (for example, a widely used AI framework that does not specify names of people or companies). This analysis determines the importance and urgency of emails. This process comprehensively evaluates the email body, metadata, and past communication history.

[0369] The server then uses an emotion engine to analyze the emotions inherent in the email. The results of this emotion analysis are used as useful information for the user in the notification generation and automated reply generation processes. An example of a prompt at this stage would be: "Analyze the content of this email to determine its urgency and the sender's emotions. Also, suggest an appropriate reply to this thank-you email."

[0370] The device receives notifications sent from the server and immediately displays them on the user's screen. The device places emphasis on UI design to enable users to take appropriate action immediately based on the content of the notification. For example, emails judged to have a "high" emotional impact or those that are urgent are highlighted on the screen using special icons or colors.

[0371] This system allows users to efficiently handle emails. When replying, users receive a draft reply automatically generated by a generative AI model, edit its content, and send it. This enables users to complete responses quickly and in an emotionally sensitive manner.

[0372] As described above, this invention significantly improves the efficiency of electronic communication processing and makes it possible to provide users with a comfortable and meaningful communication experience.

[0373] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0374] Step 1:

[0375] The server accesses the user's email account and retrieves new emails. The input is the user's account information, and the output is the newly received email data. The server stores these emails in a database and prepares the email text and metadata for analysis.

[0376] Step 2:

[0377] The server uses a generated AI model to analyze the content of stored emails. The input is the email text data and associated metadata, and the output is information about the importance and urgency of the emails. This analysis uses natural language processing techniques to extract keywords and phrases contained in the emails and determine their urgency.

[0378] Step 3:

[0379] The server uses an emotion engine to analyze the sender's emotions within incoming emails. The input is the text data of the email, and the output is the type and intensity of the emotion. For example, emotions such as gratitude, anger, and expectation are identified, and based on this, the server provides guidance on how the user should respond.

[0380] Step 4:

[0381] The server generates a notification and sends it to the device. The input is the importance, urgency, and sentiment data obtained in the previous step, and the output is a notification message for the user. This notification is immediately received by the device and displayed appropriately on the screen. The user decides whether to check or reply to the email based on this notification.

[0382] Step 5:

[0383] The device activates an AI-powered automatic reply function when the user indicates their intention to reply to an email. Inputs include email content, sentiment data, and user-specified reply criteria, while output is an automatically generated draft reply. The generated reply appropriately considers polite language and emotional expression.

[0384] Step 6:

[0385] The user reviews the automatically generated reply and edits it as needed. The input is the reply generated by the AI, and the output is the final, edited reply. The user can then send this reply via the server, ensuring the appropriate email is delivered to the recipient.

[0386] (Application Example 2)

[0387] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0388] Traditional electronic communication management systems simply prioritized emails based on their content, sending notifications and replies in a uniform format, making it difficult to communicate in a way that suited the user's emotions and circumstances. As a result, users risked missing important emails or replying in an inappropriate tone. Furthermore, the burden of manually processing individual emails in daily work was significant, creating a need for improved communication efficiency and quality.

[0389] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0390] In this invention, the server includes means for analyzing electronic communications and determining priorities, means for generating automatic replies based on emotional data, and means for adjusting emotional tone. This enables prioritization and replies to be adjusted according to the user's emotions, resulting in efficient and effective communication tailored to individual needs.

[0391] "Electronic communication" refers to messages and information that are sent and received electronically.

[0392] "Analysis" is the act of analyzing data in order to understand its content and to classify or assign meaning to it.

[0393] "Priority" refers to the ranking used to determine the order in which tasks or actions are handled.

[0394] A "notification" is a message used to inform a user of important information.

[0395] "Emotional data" refers to information that indicates the emotional state of the user or sender.

[0396] An "automatic reply" is a response message generated by a machine based on pre-set criteria.

[0397] "Emotional tone" refers to the way emotions are expressed and the overall atmosphere in communication.

[0398] "Schedule adjustment" means rationally combining appointments and schedules.

[0399] "Metadata" refers to information that includes additional information about the data.

[0400] "Classification" is the process of grouping data based on specific attributes.

[0401] In this invention, the system functions collaboratively with three parties: a server, a terminal, and a user. The server is responsible for receiving and analyzing electronic communications, determining priorities, and transmitting that information to the terminal. Specifically, it uses the content and metadata of electronic communications to extract sentiment data using a generative AI model and generates information necessary for prioritizing replies and notifications.

[0402] The server uses the Python programming language and sentiment analysis libraries (e.g., TextBlob, NLTK) to analyze electronic communications. The analyzed data, along with priority scores and sentiment tones, is sent to the terminal. A cloud-based data server is utilized to enable real-time data processing.

[0403] The terminal provides appropriate notifications to the user based on data received from the server. These terminals may include smart home devices or robots, equipped with voice devices and displays that consider received priority and sentiment scores to deliver notifications to the user.

[0404] Users select actions based on information received via their device. For example, they can immediately acknowledge or reply to particularly important notifications. Based on the user's selection, the server again utilizes AI to generate appropriate reply templates.

[0405] As a concrete example, consider a scenario where a robot is reading the news in the morning when an email regarding an important meeting arrives. In this case, the system would classify the email as "high priority" and suggest a reply in voice, reflecting the appropriate tone of voice. The user can then review the suggested reply and quickly complete the communication by easily approving or editing it.

[0406] An example of a prompt to input into the generative AI model is, "Based on the user's recent behavior history, suggest the optimal tone for the next email reply." Using this prompt, the AI ​​generates a reply that takes the user's emotional state into account.

[0407] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0408] Step 1:

[0409] The server receives emails from the user's telecommunications provider. It retrieves the text data, metadata (date and time, sender information, etc.), and past communication history of the received emails. This information is aggregated on the server as input data. The server analyzes this data using a sentiment analysis library to calculate the importance and sentiment score of the emails. As a result of this analysis, a priority score and sentiment data are output.

[0410] Step 2:

[0411] Based on the analysis results, the server generates a notification tailored to the user using a generative AI model. The priority score and sentiment data calculated in step 1 are used as input data. The server sends the generated notification message to the device. The generated notification message is forwarded to the device as output.

[0412] Step 3:

[0413] The device processes notification messages received from the server and displays notifications to the user as alerts or voice guidance. Notification content includes email type, priority, and recommended actions based on sentiment score. The device functions as an interface to communicate with the user on specific devices (smartphones, smart speakers, etc.). The user receives notifications visually or audibly as output.

[0414] Step 4:

[0415] The user chooses whether or not to reply to the received notification. Based on the user's choice, the server generates an automated reply that takes emotional tone into account. The user's choice and emotional data are used as input data. The server utilizes a generative AI model to create a prompt for the reply message and proposes it to the user. As output, the proposed reply message is presented to the user.

[0416] Step 5:

[0417] The user reviews the suggested reply and edits it as needed. After editing, they select the final reply and send it. The server reviews the edited reply again and sends the email to the recipient's address. The user's edited reply is used as input data. The sent email is recorded as output.

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0419] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0421] [Third Embodiment]

[0422] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0423] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0424] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0426] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0428] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0429] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0430] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0433] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0434] This invention provides a virtual assistant system that streamlines daily business operations. This system operates through the collaborative efforts of three main players: a server, a terminal, and a user.

[0435] First, the server accesses the configured email account and receives new emails. The email content is analyzed by a generative AI model to assess its importance and urgency. This assessment is based on keywords in the email, the sender's address, and past message exchange history. If the email is deemed important, the server immediately generates a notification and sends it to the device.

[0436] The device receives notifications sent from the server and presents them to the user. These notifications include an overview of the email and important information, allowing the user to quickly understand the situation and choose the necessary action. For example, if the user chooses to reply to the email, the AI ​​automatically generates a suggested response using templates and past reply patterns. This generated response can then be reviewed and easily modified by the user.

[0437] Furthermore, the server accesses the user's calendar and adjusts the schedule based on the content of the received email. If a meeting request is included, the server analyzes the user's availability and suggests the best meeting time. The suggestion is notified to the user via their device, and once the user approves, the appointment is automatically added to their calendar.

[0438] This system significantly reduces the burden of task management, meeting scheduling, and email communication for users. This allows them to dedicate more time and resources to important strategic tasks.

[0439] As a concrete example, a user receives a meeting request from an important client via email, and the server determines that the email is important. The device immediately notifies the user, and when the user instructs the AI ​​to respond, the server generates and completes an appropriate reply. This entire process enables appropriate and effective responses while minimizing the user's effort.

[0440] The following describes the processing flow.

[0441] Step 1:

[0442] The server periodically accesses the user's email account to check for new mail. When new mail is received, the server saves it to its database.

[0443] Step 2:

[0444] The server applies a generative AI model to analyze the stored emails. It analyzes the email content, sender, subject, etc., and evaluates their importance and urgency. Based on this evaluation, it classifies the emails into "high," "medium," and "low" priority levels.

[0445] Step 3:

[0446] The server generates notifications for terminals for emails with a "high" priority. These notifications include an email summary and key points, along with partial information to encourage the user to take prompt action.

[0447] Step 4:

[0448] The device receives notifications from the server and displays them in the user interface. The user can then view the email content further by checking the notification and opening the email details.

[0449] Step 5:

[0450] The user selects an AI-powered automated reply to the notification email. Based on this selection, the server generates a draft of an appropriate reply based on templates and past reply history.

[0451] Step 6:

[0452] The server sends a draft of the generated reply to the user's device for review. The device displays the draft, and the user can approve or edit the content.

[0453] Step 7:

[0454] Once the user approves the reply, the device sends the final email to the server. The server then relays the email to the sender under the user's name.

[0455] Step 8:

[0456] When the server detects in an incoming email that requires scheduling adjustments, it checks the user's calendar to identify available time slots. It then lists possible meeting dates and times and presents them to the user.

[0457] Step 9:

[0458] Once the user approves, adjusts, and confirms the proposed schedule, the device automatically adds the details to the calendar. It then sends confirmation emails or invitations to the relevant parties.

[0459] (Example 1)

[0460] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0461] In current electronic communication methods, users require considerable time and effort to manually assess the importance and urgency of a large volume of emails and respond appropriately. Furthermore, scheduling is often done manually, which is inefficient. These factors contribute to decreased work efficiency.

[0462] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0463] In this invention, the server includes means for receiving electronic communications and analyzing their content using natural language processing technology, means for evaluating the importance and urgency of communications based on the analysis results, and means for generating notifications to users in accordance with the evaluation. This makes it possible to quickly and automatically determine the importance of electronic communications and immediately take necessary actions.

[0464] "Electronic communications" is a general term for information sent and received in digital format, including email and instant messages.

[0465] "Natural language processing technology" refers to technologies that enable computers to understand, analyze, and generate human language, and are used to analyze and process the meaning of text data.

[0466] A "notification" is a message or alert from a system that informs a user of a situation where specific information or action is required.

[0467] A "template" is a set of standardized sentences or formats tailored to a specific purpose, and is used efficiently in automated replies and email composition.

[0468] "Schedule information" refers to data related to a user's schedule and time management, including meetings and events recorded in the calendar.

[0469] The "method for suggesting schedules" refers to a function that optimizes the user's schedule based on received electronic communications and presents the user with recommended dates and times.

[0470] This virtual assistant system is designed to streamline daily business operations. It functions through the collaborative efforts of three entities: the server, the terminal, and the user.

[0471] The server accesses the electronic communication account specified by the user and receives new communications. The software used includes protocols commonly used by email servers (e.g., IMAP, SMTP). The received communications are analyzed by a generative AI model that utilizes natural language processing technology (e.g., a GPT-based model). This analysis evaluates the importance and urgency of the communication content and determines the priority of each communication.

[0472] If a communication is determined to be important, the server generates a notification summarizing its contents. This notification is sent from the server to the terminal and presented to the user. Terminals include information devices that users use on a daily basis, such as smartphones and PCs. The terminal displays the information from the received notification on its screen, prompting the user to make a quick decision.

[0473] When a user chooses to reply to a communication, the server automatically generates a response. Here too, a generation AI model is utilized, using past reply history and templates to construct an appropriate response. This process enables users to communicate effectively in a short amount of time.

[0474] Furthermore, based on the communication content, the server analyzes and suggests the optimal meeting date and time by referring to the user's schedule information. For example, if a meeting request is included in an email, it uses the Calendar API to detect the user's availability and presents the best schedule. This suggestion is notified to the device, and once the user approves, it is automatically reflected in the calendar.

[0475] For example, if a user receives a meeting request from an important business partner, the server will evaluate this communication as important and send a notification to the user's device. If the user instructs the AI ​​to create a reply, the server will generate an appropriate response. Furthermore, a series of processes to optimize the meeting schedule and add it to the calendar are performed with minimal user intervention.

[0476] A typical example of a prompt message would be something like, "Analyze this important email and generate a reply. The email content is as follows," which would be input to the AI ​​model. Such a system significantly reduces the burden of daily tasks for users, allowing for more efficient use of their time.

[0477] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0478] Step 1:

[0479] The server accesses the user-specified email account and receives new emails. Account information and authentication information are required as input. The server receives communications using IMAP or SMTP protocols and stores them in data storage. The output is a data list of new emails. Specifically, the server periodically connects to the mail server to check for new messages.

[0480] Step 2:

[0481] The server sends the content of received electronic communications to a generating AI model for analysis. The input for this step is the content data of the new communications. The server utilizes natural language processing techniques to evaluate the importance and urgency of the communications. Specifically, the data processing involves breaking down the communication text into tokens and detecting specific keywords and contextual patterns. The output of this analysis is a score indicating the importance and urgency of each communication.

[0482] Step 3:

[0483] The server generates notifications for communications deemed important based on the analysis results. The input for this step is importance and urgency scores. The server uses these scores to select communications to notify the user and creates a summarized notification message. The output is the notification data displayed to the user. Specifically, the server converts the generated notification content into a message format and uses a protocol to send push notifications to the terminal.

[0484] Step 4:

[0485] The device receives notifications from the server and displays them to the user. The input for this step is the notification data sent from the server. The device displays the notification as a pop-up on the screen and provides the user with options for action. The output is user action data. Specifically, the device displays the notification details as a tappable link, providing an interface that allows the user to reply or view details as needed.

[0486] Step 5:

[0487] When a user selects a reply option, the server automatically generates a reply using a generative AI model. The input for this step is the user's reply selection information and past reply data. Based on the selected template and past reply history, the AI ​​suggests the most appropriate reply. The output is the generated reply. Specifically, the server constructs the document using AI-powered natural language generation technology and presents it to the user in an editable format.

[0488] Step 6:

[0489] The server adjusts the schedule based on the content of the electronic communication. The input for this step is the meeting request data and the user's calendar information. The server uses the calendar API to retrieve the user's schedule and proposes the optimal meeting time. The output is the optimized meeting date and time information. Specifically, the server sends the proposed meeting date and time to the terminal in a notification format and automatically adds it to the calendar after obtaining the user's approval.

[0490] (Application Example 1)

[0491] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0492] In today's information society, business professionals must manage a massive volume of electronic communications, quickly grasp important information, and take appropriate action. However, the burden of managing less important information and schedules can hinder the allocation of resources to strategic activities. Furthermore, receiving and responding to information immediately is extremely difficult while on the go or when hands are occupied. This necessitates improvements in information processing efficiency and a reduction in the user's workload.

[0493] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0494] In this invention, the server includes means for analyzing electronic communications and determining priorities, means for generating notifications based on the priorities, means for automatically generating replies according to user-specified criteria, and means for providing notifications in audio and visual form on an information display device. This enables business people to obtain necessary information in a timely manner and to efficiently respond and manage their schedules even while on the go.

[0495] "Electronic communication" refers to messages and information sent and received via electronic means, and specifically includes communication methods such as email and online messaging services.

[0496] "Priority" refers to the criteria or order used to evaluate the importance and urgency of received information and determine the priority of response.

[0497] "Notifications" refer to alerts or messages that inform users of the arrival of information or important matters, and are provided visually or audibly.

[0498] "Automatic generation" refers to a process in which content is formed by programs or algorithms without the need for human intervention.

[0499] An "information display device" is a device or apparatus used to display information to a user visually or audibly, and includes wearable devices and mobile terminals.

[0500] "Schedule management" refers to the processes and systems used to efficiently arrange and manage appointments and tasks.

[0501] The system implementing this invention involves the cooperative operation of three elements: a server, a terminal, and a user. First, the server accesses the email account and retrieves newly received electronic communications. The retrieved emails are analyzed by a generative AI model to evaluate their importance and urgency. This evaluation is based on keywords in the email, the sender's address, and the history of past messages. If the email is determined to be important, the server generates a notification and sends it to the terminal.

[0502] The terminal receives notifications sent from the server and presents them to the user through an information display device. This information display device is implemented as smart glasses or other wearable devices and provides notifications via voice and visual means. The notification includes an overview of the email and important information, allowing the user to quickly understand the situation and choose the necessary action. If the user chooses to reply to the email, the server uses AI to automatically generate a recommended reply based on templates and past reply patterns. The generated reply can be reviewed by the user and made minor corrections as needed.

[0503] Furthermore, the server accesses the user's calendar information and adjusts the schedule based on the content of incoming emails. For example, if an email contains a meeting request, the server analyzes the user's availability and suggests the best meeting date and time, either verbally or visually. Once the user accepts the suggestion, the appointment is automatically added to the calendar.

[0504] Through the process described above, this system significantly reduces the burden of task management, meeting scheduling, and email communication on users, allowing them to allocate more resources to important strategic tasks. For example, if a user receives a meeting request from an important client via email, and the server determines the email is important, the terminal will quickly notify the user. The user can then instruct the AI ​​to generate an appropriate reply. This entire process minimizes user effort while enabling appropriate and effective responses.

[0505] Examples of prompts to input into a generative AI model are as follows:

[0506] "New email received: {Summary of email content}. Sender: {Sender's name}. Evaluate the importance of this email and generate the most appropriate action suggestion."

[0507] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0508] Step 1:

[0509] The server accesses the user's email account and retrieves new mail. This is done using protocols such as IMAP. It uses email account information as input and outputs data of new mail.

[0510] Step 2:

[0511] The server applies a generative AI model to the acquired email data to analyze the email content. This analysis includes keyword extraction and contextual understanding, and evaluates importance and urgency. Using email content as input, it generates email importance information as output.

[0512] Step 3:

[0513] The server generates and sends notifications to user terminals for emails deemed to be of high importance. These notifications include a summary of the email's important information. The server uses email data with assessed importance as input and obtains user notifications as output.

[0514] Step 4:

[0515] The terminal presents notifications received from the server to the user through an information display device. In this process, it uses the display and speakers of smart glasses or other devices to provide information visually and audibly. It uses the server's notification information as input and obtains audiovisual information for the user as output.

[0516] Step 5:

[0517] The user chooses to reply to the email based on the presented notification. Once the user instructs to reply, the reply request is sent to the server. The user's action is used as input, and the reply request to the server is generated as output.

[0518] Step 6:

[0519] The server receives a user's reply request and automatically generates an appropriate reply using a generative AI model. This reply is generated by referring to past reply patterns and templates. The user's reply request is used as input, and the automatically generated reply is obtained as output.

[0520] Step 7:

[0521] The server presents the user with an automatically generated reply and requests final confirmation. If the user reviews and makes corrections, the server receives those changes and constructs the final reply. The server uses the automatically generated reply and the user's corrections as input to generate the final reply content as output.

[0522] Step 8:

[0523] The server sends the final reply to the recipient. Using the confirmed final reply as input, the sent email is obtained as output.

[0524] Step 9:

[0525] The server accesses the user's calendar, analyzes meeting requests included in emails, and adjusts the user's available schedule. A generative AI model suggests the optimal meeting date and time, and notifies the user's device. Meeting requests and calendar data are used as input, and meeting date and time suggestions are obtained as output.

[0526] Step 10:

[0527] The terminal presents the user with a suggested meeting date and time and awaits approval. If approved, the information is sent to the server and automatically registered in the calendar. The server's meeting suggestions are used as input, and the schedule change after user approval is obtained as output.

[0528] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0529] This invention is a virtual assistant system designed to improve operational efficiency in business, and in particular includes an emotion engine that recognizes user emotions and optimizes responses. This system functions collaboratively with a server, a terminal, and a user.

[0530] The server accesses the user's email account to receive and store new emails. The received emails are analyzed using a generative AI model to determine their importance and urgency. This analysis utilizes the email's text, metadata, and past communication history. Once the email priorities are determined, the server generates appropriate notifications based on this information and sends them to the user's device.

[0531] Furthermore, the server uses an emotion engine to analyze the underlying emotions expressed by the email sender, as well as emotions based on the user's past behavior history. This emotion data is used to adjust the tone of notifications and replies to the user.

[0532] The device receives notifications sent from the server and displays them to the user on the screen. Based on these notifications, the user can decide on a response, taking into account emotional information. Specifically, immediate replies are recommended for emails that are deemed urgent or have a high level of emotional impact.

[0533] When a user replies, the AI ​​generates an automated draft response that takes context into account based on sentiment data. This draft uses polite language and is emotionally sensitive. Users can review the draft, make minor edits, and then send it.

[0534] For example, if a user receives a thank-you email from a customer, the server recognizes this emotion as "joy" and suggests an appropriate reply template. Based on the suggested reply, the user can send a response that reflects their gratitude. This system not only streamlines email communication but also enhances the effectiveness of communication and helps build better relationships.

[0535] As described above, the present invention details the configuration of a system that streamlines information processing in electronic communications and provides a richer user experience by utilizing emotional data.

[0536] The following describes the processing flow.

[0537] Step 1:

[0538] The server periodically accesses the configured email account and retrieves new emails. The retrieved emails are stored in a database.

[0539] Step 2:

[0540] The server analyzes stored emails using a generation AI model to determine their importance and urgency based on their content, metadata, and past communication history. Based on these results, emails are categorized into different priority groups.

[0541] Step 3:

[0542] The server uses an emotion engine to analyze the sender's emotions from the content of the email. It detects emotions such as "joy," "anger," and "surprise" from the writing style and keywords in the email, and records this information in a database.

[0543] Step 4:

[0544] The server generates notifications based on emotion and priority information. For high-priority emails, such as those that are "urgent" and contain the emotion "anger," the notification will include a note indicating that immediate action is required.

[0545] Step 5:

[0546] The device displays the generated notification to the user. The notification includes a summary of the email, its importance, and even emotionally-based suggestions for action. The user uses this information to decide whether or not to check the email.

[0547] Step 6:

[0548] Users can check notifications, open email details if necessary, and reply. They can also utilize AI-generated, emotion-sensitive reply templates.

[0549] Step 7:

[0550] The server generates an emotionally adaptive automated response draft based on the user's selection. This draft is composed of an appropriate writing style, such as a warm or cool tone, that reflects the user's emotional data.

[0551] Step 8:

[0552] The user reviews the draft reply and edits it as needed. After editing is complete, the device sends the approved email to the server.

[0553] Step 9:

[0554] The server relays the final email to the recipient. At the same time, it saves the sending history to a database for use in future processing and analysis.

[0555] This process can improve users' work efficiency while promoting more human-centered communication.

[0556] (Example 2)

[0557] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0558] In today's business environment, a massive volume of electronic communications is generated, requiring the rapid and efficient identification and handling of critical information. However, many systems merely provide email sorting and notification functions, failing to offer detailed responses that consider the sender's sentiments or the prioritization of the content. As a result, users may overlook important information or engage in inappropriate communication.

[0559] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0560] In this invention, the server includes means for analyzing electronic communications and determining priorities and emotions; means for generating notifications based on the determined priorities and emotions; and means for automatically generating replies according to criteria and emotion data specified by the user. This enables users to communicate efficiently and with consideration for emotions.

[0561] "Electronic communications" refers to digital messages and data sent and received via the internet.

[0562] "Priority" is a criterion for determining the order in which tasks and information are processed, based on their importance and urgency.

[0563] "Emotions" refer to the feelings and psychological states that the emotion engine analyzes and identifies from the content of emails and messages.

[0564] A "notification" is a message or alert that immediately informs a user of important information or an action.

[0565] "Methods for automatically generating replies" refers to technologies and methods that use AI to automatically create response texts to messages received by users.

[0566] "Editable" means that the user can manually modify or add to the automatically generated reply text.

[0567] This invention is a system for efficiently and emotionally processing electronic communications. This system primarily operates through the collaboration of three parties: a server, a terminal, and a user. The server, with the user's permission, accesses their email account and checks for and stores newly received emails at regular intervals. This requires a server computer and a database system as hardware components.

[0568] The server uses a generative AI model to analyze stored electronic communications. Specifically, it leverages natural language processing techniques and employs a general generative AI framework as the model (for example, a widely used AI framework that does not specify names of people or companies). This analysis determines the importance and urgency of emails. This process comprehensively evaluates the email body, metadata, and past communication history.

[0569] The server then uses an emotion engine to analyze the emotions inherent in the email. The results of this emotion analysis are used as useful information for the user in the notification generation and automated reply generation processes. An example of a prompt at this stage would be: "Analyze the content of this email to determine its urgency and the sender's emotions. Also, suggest an appropriate reply to this thank-you email."

[0570] The device receives notifications sent from the server and immediately displays them on the user's screen. The device places emphasis on UI design to enable users to take appropriate action immediately based on the content of the notification. For example, emails judged to have a "high" emotional impact or those that are urgent are highlighted on the screen using special icons or colors.

[0571] This system allows users to efficiently handle emails. When replying, users receive a draft reply automatically generated by a generative AI model, edit its content, and send it. This enables users to complete responses quickly and in an emotionally sensitive manner.

[0572] As described above, this invention significantly improves the efficiency of electronic communication processing and makes it possible to provide users with a comfortable and meaningful communication experience.

[0573] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0574] Step 1:

[0575] The server accesses the user's email account and retrieves new emails. The input is the user's account information, and the output is the newly received email data. The server stores these emails in a database and prepares the email text and metadata for analysis.

[0576] Step 2:

[0577] The server uses a generated AI model to analyze the content of stored emails. The input is the email text data and associated metadata, and the output is information about the importance and urgency of the emails. This analysis uses natural language processing techniques to extract keywords and phrases contained in the emails and determine their urgency.

[0578] Step 3:

[0579] The server uses an emotion engine to analyze the sender's emotions within incoming emails. The input is the text data of the email, and the output is the type and intensity of the emotion. For example, emotions such as gratitude, anger, and expectation are identified, and based on this, the server provides guidance on how the user should respond.

[0580] Step 4:

[0581] The server generates a notification and sends it to the device. The input is the importance, urgency, and sentiment data obtained in the previous step, and the output is a notification message for the user. This notification is immediately received by the device and displayed appropriately on the screen. The user decides whether to check or reply to the email based on this notification.

[0582] Step 5:

[0583] The device activates an AI-powered automatic reply function when the user indicates their intention to reply to an email. Inputs include email content, sentiment data, and user-specified reply criteria, while output is an automatically generated draft reply. The generated reply appropriately considers polite language and emotional expression.

[0584] Step 6:

[0585] The user reviews the automatically generated reply and edits it as needed. The input is the reply generated by the AI, and the output is the final, edited reply. The user can then send this reply via the server, ensuring the appropriate email is delivered to the recipient.

[0586] (Application Example 2)

[0587] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0588] Traditional electronic communication management systems simply prioritized emails based on their content, sending notifications and replies in a uniform format, making it difficult to communicate in a way that suited the user's emotions and circumstances. As a result, users risked missing important emails or replying in an inappropriate tone. Furthermore, the burden of manually processing individual emails in daily work was significant, creating a need for improved communication efficiency and quality.

[0589] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0590] In this invention, the server includes means for analyzing electronic communications and determining priorities, means for generating automatic replies based on emotional data, and means for adjusting emotional tone. This enables prioritization and replies to be adjusted according to the user's emotions, resulting in efficient and effective communication tailored to individual needs.

[0591] "Electronic communication" refers to messages and information that are sent and received electronically.

[0592] "Analysis" is the act of analyzing data in order to understand its content and to classify or assign meaning to it.

[0593] "Priority" refers to the ranking used to determine the order in which tasks or actions are handled.

[0594] A "notification" is a message used to inform a user of important information.

[0595] "Emotional data" refers to information that indicates the emotional state of the user or sender.

[0596] An "automatic reply" is a response message generated by a machine based on pre-set criteria.

[0597] "Emotional tone" refers to the way emotions are expressed and the overall atmosphere in communication.

[0598] "Schedule adjustment" means rationally combining appointments and schedules.

[0599] "Metadata" refers to information that includes additional information about the data.

[0600] "Classification" is the process of grouping data based on specific attributes.

[0601] In this invention, the system functions collaboratively with three parties: a server, a terminal, and a user. The server is responsible for receiving and analyzing electronic communications, determining priorities, and transmitting that information to the terminal. Specifically, it uses the content and metadata of electronic communications to extract sentiment data using a generative AI model and generates information necessary for prioritizing replies and notifications.

[0602] The server uses the Python programming language and sentiment analysis libraries (e.g., TextBlob, NLTK) to analyze electronic communications. The analyzed data, along with priority scores and sentiment tones, is sent to the terminal. A cloud-based data server is utilized to enable real-time data processing.

[0603] The terminal provides appropriate notifications to the user based on data received from the server. These terminals may include smart home devices or robots, equipped with voice devices and displays that consider received priority and sentiment scores to deliver notifications to the user.

[0604] Users select actions based on information received via their device. For example, they can immediately acknowledge or reply to particularly important notifications. Based on the user's selection, the server again utilizes AI to generate appropriate reply templates.

[0605] As a concrete example, consider a scenario where a robot is reading the news in the morning when an email regarding an important meeting arrives. In this case, the system would classify the email as "high priority" and suggest a reply in voice, reflecting the appropriate tone of voice. The user can then review the suggested reply and quickly complete the communication by easily approving or editing it.

[0606] An example of a prompt to input into the generative AI model is, "Based on the user's recent behavior history, suggest the optimal tone for the next email reply." Using this prompt, the AI ​​generates a reply that takes the user's emotional state into account.

[0607] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0608] Step 1:

[0609] The server receives emails from the user's telecommunications provider. It retrieves the text data, metadata (date and time, sender information, etc.), and past communication history of the received emails. This information is aggregated on the server as input data. The server analyzes this data using a sentiment analysis library to calculate the importance and sentiment score of the emails. As a result of this analysis, a priority score and sentiment data are output.

[0610] Step 2:

[0611] Based on the analysis results, the server generates a notification tailored to the user using a generative AI model. The priority score and sentiment data calculated in step 1 are used as input data. The server sends the generated notification message to the device. The generated notification message is forwarded to the device as output.

[0612] Step 3:

[0613] The device processes notification messages received from the server and displays notifications to the user as alerts or voice guidance. Notification content includes email type, priority, and recommended actions based on sentiment score. The device functions as an interface to communicate with the user on specific devices (smartphones, smart speakers, etc.). The user receives notifications visually or audibly as output.

[0614] Step 4:

[0615] The user chooses whether or not to reply to the received notification. Based on the user's choice, the server generates an automated reply that takes emotional tone into account. The user's choice and emotional data are used as input data. The server utilizes a generative AI model to create a prompt for the reply message and proposes it to the user. As output, the proposed reply message is presented to the user.

[0616] Step 5:

[0617] The user reviews the suggested reply and edits it as needed. After editing, they select the final reply and send it. The server reviews the edited reply again and sends the email to the recipient's address. The user's edited reply is used as input data. The sent email is recorded as output.

[0618] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0619] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0620] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0621] [Fourth Embodiment]

[0622] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0623] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0624] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0625] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0626] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0627] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0628] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0629] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0630] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0631] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0632] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0633] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0634] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] This invention provides a virtual assistant system that streamlines daily business operations. This system operates through the collaborative efforts of three main players: a server, a terminal, and a user.

[0636] First, the server accesses the configured email account and receives new emails. The email content is analyzed by a generative AI model to assess its importance and urgency. This assessment is based on keywords in the email, the sender's address, and past message exchange history. If the email is deemed important, the server immediately generates a notification and sends it to the device.

[0637] The device receives notifications sent from the server and presents them to the user. These notifications include an overview of the email and important information, allowing the user to quickly understand the situation and choose the necessary action. For example, if the user chooses to reply to the email, the AI ​​automatically generates a suggested response using templates and past reply patterns. This generated response can then be reviewed and easily modified by the user.

[0638] Furthermore, the server accesses the user's calendar and adjusts the schedule based on the content of the received email. If a meeting request is included, the server analyzes the user's availability and suggests the best meeting time. The suggestion is notified to the user via their device, and once the user approves, the appointment is automatically added to their calendar.

[0639] This system significantly reduces the burden of task management, meeting scheduling, and email communication for users. This allows them to dedicate more time and resources to important strategic tasks.

[0640] As a concrete example, a user receives a meeting request from an important client via email, and the server determines that the email is important. The device immediately notifies the user, and when the user instructs the AI ​​to respond, the server generates and completes an appropriate reply. This entire process enables appropriate and effective responses while minimizing the user's effort.

[0641] The following describes the processing flow.

[0642] Step 1:

[0643] The server periodically accesses the user's email account to check for new mail. When new mail is received, the server saves it to its database.

[0644] Step 2:

[0645] The server applies a generative AI model to analyze the stored emails. It analyzes the email content, sender, subject, etc., and evaluates their importance and urgency. Based on this evaluation, it classifies the emails into "high," "medium," and "low" priority levels.

[0646] Step 3:

[0647] The server generates notifications for terminals for emails with a "high" priority. These notifications include an email summary and key points, along with partial information to encourage the user to take prompt action.

[0648] Step 4:

[0649] The device receives notifications from the server and displays them in the user interface. The user can then view the email content further by checking the notification and opening the email details.

[0650] Step 5:

[0651] The user selects an AI-powered automated reply to the notification email. Based on this selection, the server generates a draft of an appropriate reply based on templates and past reply history.

[0652] Step 6:

[0653] The server sends a draft of the generated reply to the user's device for review. The device displays the draft, and the user can approve or edit the content.

[0654] Step 7:

[0655] Once the user approves the reply, the device sends the final email to the server. The server then relays the email to the sender under the user's name.

[0656] Step 8:

[0657] When the server detects in an incoming email that requires scheduling adjustments, it checks the user's calendar to identify available time slots. It then lists possible meeting dates and times and presents them to the user.

[0658] Step 9:

[0659] Once the user approves, adjusts, and confirms the proposed schedule, the device automatically adds the details to the calendar. It then sends confirmation emails or invitations to the relevant parties.

[0660] (Example 1)

[0661] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0662] In current electronic communication methods, users require considerable time and effort to manually assess the importance and urgency of a large volume of emails and respond appropriately. Furthermore, scheduling is often done manually, which is inefficient. These factors contribute to decreased work efficiency.

[0663] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0664] In this invention, the server includes means for receiving electronic communications and analyzing their content using natural language processing technology, means for evaluating the importance and urgency of communications based on the analysis results, and means for generating notifications to users in accordance with the evaluation. This makes it possible to quickly and automatically determine the importance of electronic communications and immediately take necessary actions.

[0665] "Electronic communications" is a general term for information sent and received in digital format, including email and instant messages.

[0666] "Natural language processing technology" refers to technologies that enable computers to understand, analyze, and generate human language, and are used to analyze and process the meaning of text data.

[0667] A "notification" is a message or alert from a system that informs a user of a situation where specific information or action is required.

[0668] A "template" is a set of standardized sentences or formats tailored to a specific purpose, and is used efficiently in automated replies and email composition.

[0669] "Schedule information" refers to data related to a user's schedule and time management, including meetings and events recorded in the calendar.

[0670] The "method for suggesting schedules" refers to a function that optimizes the user's schedule based on received electronic communications and presents the user with recommended dates and times.

[0671] This virtual assistant system is designed to streamline daily business operations. It functions through the collaborative efforts of three entities: the server, the terminal, and the user.

[0672] The server accesses the electronic communication account specified by the user and receives new communications. The software used includes protocols commonly used by email servers (e.g., IMAP, SMTP). The received communications are analyzed by a generative AI model that utilizes natural language processing technology (e.g., a GPT-based model). This analysis evaluates the importance and urgency of the communication content and determines the priority of each communication.

[0673] If a communication is determined to be important, the server generates a notification summarizing its contents. This notification is sent from the server to the terminal and presented to the user. Terminals include information devices that users use on a daily basis, such as smartphones and PCs. The terminal displays the information from the received notification on its screen, prompting the user to make a quick decision.

[0674] When a user chooses to reply to a communication, the server automatically generates a response. Here too, a generation AI model is utilized, using past reply history and templates to construct an appropriate response. This process enables users to communicate effectively in a short amount of time.

[0675] Furthermore, based on the communication content, the server analyzes and suggests the optimal meeting date and time by referring to the user's schedule information. For example, if a meeting request is included in an email, it uses the Calendar API to detect the user's availability and presents the best schedule. This suggestion is notified to the device, and once the user approves, it is automatically reflected in the calendar.

[0676] For example, if a user receives a meeting request from an important business partner, the server will evaluate this communication as important and send a notification to the user's device. If the user instructs the AI ​​to create a reply, the server will generate an appropriate response. Furthermore, a series of processes to optimize the meeting schedule and add it to the calendar are performed with minimal user intervention.

[0677] A typical example of a prompt message would be something like, "Analyze this important email and generate a reply. The email content is as follows," which would be input to the AI ​​model. Such a system significantly reduces the burden of daily tasks for users, allowing for more efficient use of their time.

[0678] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0679] Step 1:

[0680] The server accesses the user-specified email account and receives new emails. Account information and authentication information are required as input. The server receives communications using IMAP or SMTP protocols and stores them in data storage. The output is a data list of new emails. Specifically, the server periodically connects to the mail server to check for new messages.

[0681] Step 2:

[0682] The server sends the content of received electronic communications to a generating AI model for analysis. The input for this step is the content data of the new communications. The server utilizes natural language processing techniques to evaluate the importance and urgency of the communications. Specifically, the data processing involves breaking down the communication text into tokens and detecting specific keywords and contextual patterns. The output of this analysis is a score indicating the importance and urgency of each communication.

[0683] Step 3:

[0684] The server generates notifications for communications deemed important based on the analysis results. The input for this step is importance and urgency scores. The server uses these scores to select communications to notify the user and creates a summarized notification message. The output is the notification data displayed to the user. Specifically, the server converts the generated notification content into a message format and uses a protocol to send push notifications to the terminal.

[0685] Step 4:

[0686] The device receives notifications from the server and displays them to the user. The input for this step is the notification data sent from the server. The device displays the notification as a pop-up on the screen and provides the user with options for action. The output is user action data. Specifically, the device displays the notification details as a tappable link, providing an interface that allows the user to reply or view details as needed.

[0687] Step 5:

[0688] When a user selects a reply option, the server automatically generates a reply using a generative AI model. The input for this step is the user's reply selection information and past reply data. Based on the selected template and past reply history, the AI ​​suggests the most appropriate reply. The output is the generated reply. Specifically, the server constructs the document using AI-powered natural language generation technology and presents it to the user in an editable format.

[0689] Step 6:

[0690] The server adjusts the schedule based on the content of the electronic communication. The input for this step is the meeting request data and the user's calendar information. The server uses the calendar API to retrieve the user's schedule and proposes the optimal meeting time. The output is the optimized meeting date and time information. Specifically, the server sends the proposed meeting date and time to the terminal in a notification format and automatically adds it to the calendar after obtaining the user's approval.

[0691] (Application Example 1)

[0692] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0693] In today's information society, business professionals must manage a massive volume of electronic communications, quickly grasp important information, and take appropriate action. However, the burden of managing less important information and schedules can hinder the allocation of resources to strategic activities. Furthermore, receiving and responding to information immediately is extremely difficult while on the go or when hands are occupied. This necessitates improvements in information processing efficiency and a reduction in the user's workload.

[0694] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0695] In this invention, the server includes means for analyzing electronic communications and determining priorities, means for generating notifications based on the priorities, means for automatically generating replies according to user-specified criteria, and means for providing notifications in audio and visual form on an information display device. This enables business people to obtain necessary information in a timely manner and to efficiently respond and manage their schedules even while on the go.

[0696] "Electronic communication" refers to messages and information sent and received via electronic means, and specifically includes communication methods such as email and online messaging services.

[0697] "Priority" refers to the criteria or order used to evaluate the importance and urgency of received information and determine the priority of response.

[0698] "Notifications" refer to alerts or messages that inform users of the arrival of information or important matters, and are provided visually or audibly.

[0699] "Automatic generation" refers to a process in which content is formed by programs or algorithms without the need for human intervention.

[0700] An "information display device" is a device or apparatus used to display information to a user visually or audibly, and includes wearable devices and mobile terminals.

[0701] "Schedule management" refers to the processes and systems used to efficiently arrange and manage appointments and tasks.

[0702] The system implementing this invention involves the cooperative operation of three elements: a server, a terminal, and a user. First, the server accesses the email account and retrieves newly received electronic communications. The retrieved emails are analyzed by a generative AI model to evaluate their importance and urgency. This evaluation is based on keywords in the email, the sender's address, and the history of past messages. If the email is determined to be important, the server generates a notification and sends it to the terminal.

[0703] The terminal receives notifications sent from the server and presents them to the user through an information display device. This information display device is implemented as smart glasses or other wearable devices and provides notifications via voice and visual means. The notification includes an overview of the email and important information, allowing the user to quickly understand the situation and choose the necessary action. If the user chooses to reply to the email, the server uses AI to automatically generate a recommended reply based on templates and past reply patterns. The generated reply can be reviewed by the user and made minor corrections as needed.

[0704] Furthermore, the server accesses the user's calendar information and adjusts the schedule based on the content of incoming emails. For example, if an email contains a meeting request, the server analyzes the user's availability and suggests the best meeting date and time, either verbally or visually. Once the user accepts the suggestion, the appointment is automatically added to the calendar.

[0705] Through the process described above, this system significantly reduces the burden of task management, meeting scheduling, and email communication on users, allowing them to allocate more resources to important strategic tasks. For example, if a user receives a meeting request from an important client via email, and the server determines the email is important, the terminal will quickly notify the user. The user can then instruct the AI ​​to generate an appropriate reply. This entire process minimizes user effort while enabling appropriate and effective responses.

[0706] Examples of prompts to input into a generative AI model are as follows:

[0707] "New email received: {Summary of email content}. Sender: {Sender's name}. Evaluate the importance of this email and generate the most appropriate action suggestion."

[0708] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0709] Step 1:

[0710] The server accesses the user's email account and retrieves new mail. This is done using protocols such as IMAP. It uses email account information as input and outputs data of new mail.

[0711] Step 2:

[0712] The server applies a generative AI model to the acquired email data to analyze the email content. This analysis includes keyword extraction and contextual understanding, and evaluates importance and urgency. Using email content as input, it generates email importance information as output.

[0713] Step 3:

[0714] The server generates and sends notifications to user terminals for emails deemed to be of high importance. These notifications include a summary of the email's important information. The server uses email data with assessed importance as input and obtains user notifications as output.

[0715] Step 4:

[0716] The terminal presents notifications received from the server to the user through an information display device. In this process, it uses the display and speakers of smart glasses or other devices to provide information visually and audibly. It uses the server's notification information as input and obtains audiovisual information for the user as output.

[0717] Step 5:

[0718] The user chooses to reply to the email based on the presented notification. Once the user instructs to reply, the reply request is sent to the server. The user's action is used as input, and the reply request to the server is generated as output.

[0719] Step 6:

[0720] The server receives a user's reply request and automatically generates an appropriate reply using a generative AI model. This reply is generated by referring to past reply patterns and templates. The user's reply request is used as input, and the automatically generated reply is obtained as output.

[0721] Step 7:

[0722] The server presents the user with an automatically generated reply and requests final confirmation. If the user reviews and makes corrections, the server receives those changes and constructs the final reply. The server uses the automatically generated reply and the user's corrections as input to generate the final reply content as output.

[0723] Step 8:

[0724] The server sends the final reply to the recipient. Using the confirmed final reply as input, the sent email is obtained as output.

[0725] Step 9:

[0726] The server accesses the user's calendar, analyzes meeting requests included in emails, and adjusts the user's available schedule. A generative AI model suggests the optimal meeting date and time, and notifies the user's device. Meeting requests and calendar data are used as input, and meeting date and time suggestions are obtained as output.

[0727] Step 10:

[0728] The terminal presents the user with a suggested meeting date and time and awaits approval. If approved, the information is sent to the server and automatically registered in the calendar. The server's meeting suggestions are used as input, and the schedule change after user approval is obtained as output.

[0729] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0730] This invention is a virtual assistant system designed to improve operational efficiency in business, and in particular includes an emotion engine that recognizes user emotions and optimizes responses. This system functions collaboratively with a server, a terminal, and a user.

[0731] The server accesses the user's email account to receive and store new emails. The received emails are analyzed using a generative AI model to determine their importance and urgency. This analysis utilizes the email's text, metadata, and past communication history. Once the email priorities are determined, the server generates appropriate notifications based on this information and sends them to the user's device.

[0732] Furthermore, the server uses an emotion engine to analyze the underlying emotions expressed by the email sender, as well as emotions based on the user's past behavior history. This emotion data is used to adjust the tone of notifications and replies to the user.

[0733] The device receives notifications sent from the server and displays them to the user on the screen. Based on these notifications, the user can decide on a response, taking into account emotional information. Specifically, immediate replies are recommended for emails that are deemed urgent or have a high level of emotional impact.

[0734] When a user replies, the AI ​​generates an automated draft response that takes context into account based on sentiment data. This draft uses polite language and is emotionally sensitive. Users can review the draft, make minor edits, and then send it.

[0735] For example, if a user receives a thank-you email from a customer, the server recognizes this emotion as "joy" and suggests an appropriate reply template. Based on the suggested reply, the user can send a response that reflects their gratitude. This system not only streamlines email communication but also enhances the effectiveness of communication and helps build better relationships.

[0736] As described above, the present invention details the configuration of a system that streamlines information processing in electronic communications and provides a richer user experience by utilizing emotional data.

[0737] The following describes the processing flow.

[0738] Step 1:

[0739] The server periodically accesses the configured email account and retrieves new emails. The retrieved emails are stored in a database.

[0740] Step 2:

[0741] The server analyzes stored emails using a generation AI model to determine their importance and urgency based on their content, metadata, and past communication history. Based on these results, emails are categorized into different priority groups.

[0742] Step 3:

[0743] The server uses an emotion engine to analyze the sender's emotions from the content of the email. It detects emotions such as "joy," "anger," and "surprise" from the writing style and keywords in the email, and records this information in a database.

[0744] Step 4:

[0745] The server generates notifications based on emotion and priority information. For high-priority emails, such as those that are "urgent" and contain the emotion "anger," the notification will include a note indicating that immediate action is required.

[0746] Step 5:

[0747] The device displays the generated notification to the user. The notification includes a summary of the email, its importance, and even emotionally-based suggestions for action. The user uses this information to decide whether or not to check the email.

[0748] Step 6:

[0749] Users can check notifications, open email details if necessary, and reply. They can also utilize AI-generated, emotion-sensitive reply templates.

[0750] Step 7:

[0751] The server generates an emotionally adaptive automated response draft based on the user's selection. This draft is composed of an appropriate writing style, such as a warm or cool tone, that reflects the user's emotional data.

[0752] Step 8:

[0753] The user reviews the draft reply and edits it as needed. After editing is complete, the device sends the approved email to the server.

[0754] Step 9:

[0755] The server relays the final email to the recipient. At the same time, it saves the sending history to a database for use in future processing and analysis.

[0756] This process can improve users' work efficiency while promoting more human-centered communication.

[0757] (Example 2)

[0758] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0759] In today's business environment, a massive volume of electronic communications is generated, requiring the rapid and efficient identification and handling of critical information. However, many systems merely provide email sorting and notification functions, failing to offer detailed responses that consider the sender's sentiments or the prioritization of the content. As a result, users may overlook important information or engage in inappropriate communication.

[0760] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0761] In this invention, the server includes means for analyzing electronic communications and determining priorities and emotions; means for generating notifications based on the determined priorities and emotions; and means for automatically generating replies according to criteria and emotion data specified by the user. This enables users to communicate efficiently and with consideration for emotions.

[0762] "Electronic communications" refers to digital messages and data sent and received via the internet.

[0763] "Priority" is a criterion for determining the order in which tasks and information are processed, based on their importance and urgency.

[0764] "Emotions" refer to the feelings and psychological states that the emotion engine analyzes and identifies from the content of emails and messages.

[0765] A "notification" is a message or alert that immediately informs a user of important information or an action.

[0766] "Methods for automatically generating replies" refers to technologies and methods that use AI to automatically create response texts to messages received by users.

[0767] "Editable" means that the user can manually modify or add to the automatically generated reply text.

[0768] This invention is a system for efficiently and emotionally processing electronic communications. This system primarily operates through the collaboration of three parties: a server, a terminal, and a user. The server, with the user's permission, accesses their email account and checks for and stores newly received emails at regular intervals. This requires a server computer and a database system as hardware components.

[0769] The server uses a generative AI model to analyze stored electronic communications. Specifically, it leverages natural language processing techniques and employs a general generative AI framework as the model (for example, a widely used AI framework that does not specify names of people or companies). This analysis determines the importance and urgency of emails. This process comprehensively evaluates the email body, metadata, and past communication history.

[0770] The server then uses an emotion engine to analyze the emotions inherent in the email. The results of this emotion analysis are used as useful information for the user in the notification generation and automated reply generation processes. An example of a prompt at this stage would be: "Analyze the content of this email to determine its urgency and the sender's emotions. Also, suggest an appropriate reply to this thank-you email."

[0771] The device receives notifications sent from the server and immediately displays them on the user's screen. The device places emphasis on UI design to enable users to take appropriate action immediately based on the content of the notification. For example, emails judged to have a "high" emotional impact or those that are urgent are highlighted on the screen using special icons or colors.

[0772] This system allows users to efficiently handle emails. When replying, users receive a draft reply automatically generated by a generative AI model, edit its content, and send it. This enables users to complete responses quickly and in an emotionally sensitive manner.

[0773] As described above, this invention significantly improves the efficiency of electronic communication processing and makes it possible to provide users with a comfortable and meaningful communication experience.

[0774] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0775] Step 1:

[0776] The server accesses the user's email account and retrieves new emails. The input is the user's account information, and the output is the newly received email data. The server stores these emails in a database and prepares the email text and metadata for analysis.

[0777] Step 2:

[0778] The server uses a generated AI model to analyze the content of stored emails. The input is the email text data and associated metadata, and the output is information about the importance and urgency of the emails. This analysis uses natural language processing techniques to extract keywords and phrases contained in the emails and determine their urgency.

[0779] Step 3:

[0780] The server uses an emotion engine to analyze the sender's emotions within incoming emails. The input is the text data of the email, and the output is the type and intensity of the emotion. For example, emotions such as gratitude, anger, and expectation are identified, and based on this, the server provides guidance on how the user should respond.

[0781] Step 4:

[0782] The server generates a notification and sends it to the device. The input is the importance, urgency, and sentiment data obtained in the previous step, and the output is a notification message for the user. This notification is immediately received by the device and displayed appropriately on the screen. The user decides whether to check or reply to the email based on this notification.

[0783] Step 5:

[0784] The device activates an AI-powered automatic reply function when the user indicates their intention to reply to an email. Inputs include email content, sentiment data, and user-specified reply criteria, while output is an automatically generated draft reply. The generated reply appropriately considers polite language and emotional expression.

[0785] Step 6:

[0786] The user reviews the automatically generated reply and edits it as needed. The input is the reply generated by the AI, and the output is the final, edited reply. The user can then send this reply via the server, ensuring the appropriate email is delivered to the recipient.

[0787] (Application Example 2)

[0788] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0789] Traditional electronic communication management systems simply prioritized emails based on their content, sending notifications and replies in a uniform format, making it difficult to communicate in a way that suited the user's emotions and circumstances. As a result, users risked missing important emails or replying in an inappropriate tone. Furthermore, the burden of manually processing individual emails in daily work was significant, creating a need for improved communication efficiency and quality.

[0790] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0791] In this invention, the server includes means for analyzing electronic communications and determining priorities, means for generating automatic replies based on emotional data, and means for adjusting emotional tone. This enables prioritization and replies to be adjusted according to the user's emotions, resulting in efficient and effective communication tailored to individual needs.

[0792] "Electronic communication" refers to messages and information that are sent and received electronically.

[0793] "Analysis" is the act of analyzing data in order to understand its content and to classify or assign meaning to it.

[0794] "Priority" refers to the ranking used to determine the order in which tasks or actions are handled.

[0795] A "notification" is a message used to inform a user of important information.

[0796] "Emotional data" refers to information that indicates the emotional state of the user or sender.

[0797] An "automatic reply" is a response message generated by a machine based on pre-set criteria.

[0798] "Emotional tone" refers to the way emotions are expressed and the overall atmosphere in communication.

[0799] "Schedule adjustment" means rationally combining appointments and schedules.

[0800] "Metadata" refers to information that includes additional information about the data.

[0801] "Classification" is the process of grouping data based on specific attributes.

[0802] In this invention, the system functions collaboratively with three parties: a server, a terminal, and a user. The server is responsible for receiving and analyzing electronic communications, determining priorities, and transmitting that information to the terminal. Specifically, it uses the content and metadata of electronic communications to extract sentiment data using a generative AI model and generates information necessary for prioritizing replies and notifications.

[0803] The server uses the Python programming language and sentiment analysis libraries (e.g., TextBlob, NLTK) to analyze electronic communications. The analyzed data, along with priority scores and sentiment tones, is sent to the terminal. A cloud-based data server is utilized to enable real-time data processing.

[0804] The terminal provides appropriate notifications to the user based on data received from the server. These terminals may include smart home devices or robots, equipped with voice devices and displays that consider received priority and sentiment scores to deliver notifications to the user.

[0805] Users select actions based on information received via their device. For example, they can immediately acknowledge or reply to particularly important notifications. Based on the user's selection, the server again utilizes AI to generate appropriate reply templates.

[0806] As a concrete example, consider a scenario where a robot is reading the news in the morning when an email regarding an important meeting arrives. In this case, the system would classify the email as "high priority" and suggest a reply in voice, reflecting the appropriate tone of voice. The user can then review the suggested reply and quickly complete the communication by easily approving or editing it.

[0807] An example of a prompt to input into the generative AI model is, "Based on the user's recent behavior history, suggest the optimal tone for the next email reply." Using this prompt, the AI ​​generates a reply that takes the user's emotional state into account.

[0808] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0809] Step 1:

[0810] The server receives emails from the user's telecommunications provider. It retrieves the text data, metadata (date and time, sender information, etc.), and past communication history of the received emails. This information is aggregated on the server as input data. The server analyzes this data using a sentiment analysis library to calculate the importance and sentiment score of the emails. As a result of this analysis, a priority score and sentiment data are output.

[0811] Step 2:

[0812] Based on the analysis results, the server generates a notification tailored to the user using a generative AI model. The priority score and sentiment data calculated in step 1 are used as input data. The server sends the generated notification message to the device. The generated notification message is forwarded to the device as output.

[0813] Step 3:

[0814] The device processes notification messages received from the server and displays notifications to the user as alerts or voice guidance. Notification content includes email type, priority, and recommended actions based on sentiment score. The device functions as an interface to communicate with the user on specific devices (smartphones, smart speakers, etc.). The user receives notifications visually or audibly as output.

[0815] Step 4:

[0816] The user chooses whether or not to reply to the received notification. Based on the user's choice, the server generates an automated reply that takes emotional tone into account. The user's choice and emotional data are used as input data. The server utilizes a generative AI model to create a prompt for the reply message and proposes it to the user. As output, the proposed reply message is presented to the user.

[0817] Step 5:

[0818] The user reviews the suggested reply and edits it as needed. After editing, they select the final reply and send it. The server reviews the edited reply again and sends the email to the recipient's address. The user's edited reply is used as input data. The sent email is recorded as output.

[0819] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0820] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0821] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0822] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0823] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0824] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0825] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0826] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0827] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0828] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0829] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0830] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0831] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0832] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0833] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0834] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0835] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0836] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0837] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0838] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0839] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0840] The following is further disclosed regarding the embodiments described above.

[0841] (Claim 1)

[0842] A means of analyzing electronic communications and determining priorities,

[0843] Means for generating notifications based on the aforementioned priority,

[0844] A means of automatically generating replies according to criteria specified by the user,

[0845] A means for sending the automatically generated reply,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, further comprising means for automatically proposing a schedule based on the content of the aforementioned electronic communication.

[0849] (Claim 3)

[0850] The system according to claim 1, further comprising means for classifying received electronic communications into multiple categories based on their content and metadata.

[0851] "Example 1"

[0852] (Claim 1)

[0853] A means for receiving electronic communications and analyzing their content using natural language processing technology,

[0854] Based on the aforementioned analysis results, a means for evaluating the importance and urgency of the communication,

[0855] A means for generating a notification to the user in accordance with the aforementioned evaluation,

[0856] A means of automatically generating email reply content based on user instructions, while referring to past reply history and templates,

[0857] A means of analyzing schedule information based on electronic communication requirements and proposing an appropriate schedule,

[0858] ...

[0859] A system that includes this.

[0860] (Claim 2)

[0861] The system according to claim 1, further comprising means for automatically determining the importance of a communication based on the content and transmission history of the received electronic communication in the aforementioned analysis.

[0862] (Claim 3)

[0863] The system according to claim 1, further comprising means for analyzing the content of communications and automatically adjusting the user's schedule in the aforementioned analysis.

[0864] "Application Example 1"

[0865] (Claim 1)

[0866] A means of analyzing electronic communications and determining priorities,

[0867] Means for generating notifications based on the aforementioned priority,

[0868] A means of automatically generating replies according to criteria specified by the user,

[0869] A means for sending the automatically generated reply,

[0870] An information display device provides means for providing notifications by sound and visual means,

[0871] A system that includes this.

[0872] (Claim 2)

[0873] The system according to claim 1, further comprising means for automatically proposing a schedule adjustment based on the content of the aforementioned electronic communication and displaying it on a presentation device.

[0874] (Claim 3)

[0875] The system according to claim 1, further comprising means for classifying received electronic communications into multiple information classifications based on their content and metadata.

[0876] "Example 2 of combining an emotion engine"

[0877] (Claim 1)

[0878] A means of analyzing electronic communications to determine priorities and emotions,

[0879] A means for generating a notification based on the determined priority and emotion,

[0880] A means for automatically generating replies based on user-specified criteria and sentiment data,

[0881] A means for making the automatically generated reply editable and sending the edited result,

[0882] A system that includes this.

[0883] (Claim 2)

[0884] The system according to claim 1, further comprising means for classifying received electronic communications into multiple categories based on content, metadata, and past communication history.

[0885] (Claim 3)

[0886] The system according to claim 1, further comprising means for supporting user decision-making by utilizing displayed notifications and taking sentiment analysis data into consideration.

[0887] "Application example 2 when combining with an emotional engine"

[0888] (Claim 1)

[0889] A means of analyzing electronic communications and determining priorities,

[0890] Means for generating notifications based on the aforementioned priority,

[0891] A means for users to generate automated replies based on sentiment data,

[0892] A means for sending the automatically generated reply,

[0893] A means for adjusting the emotional tone based on the emotional analysis of the aforementioned electronic communication,

[0894] A system that includes this.

[0895] (Claim 2)

[0896] The system according to claim 1, further comprising means for automatically proposing a schedule based on the content of the aforementioned electronic communication.

[0897] (Claim 3)

[0898] The system according to claim 1, further comprising means for classifying received electronic communications into multiple categories based on their content and metadata, and optimizing the classification results based on sentiment data. [Explanation of Symbols]

[0899] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of analyzing electronic communications and determining priorities, Means for generating notifications based on the aforementioned priority, A means of automatically generating replies according to criteria specified by the user, A means for sending the automatically generated reply, An information display device provides means for providing notifications by sound and visual means, A system that includes this.

2. The system according to claim 1, further comprising means for automatically proposing a schedule adjustment based on the content of the electronic communication and displaying it on a presentation device.

3. The system according to claim 1, further comprising means for classifying received electronic communications into multiple information classifications based on their content and metadata.

Images