system

Abstract

Provide a system. [Solution] A means for calling an API of an email account and a communication tool to collect new messages; A means for analyzing the content of the collected messages and extracting sender, sending time, content, tag, and mention information; a means for scoring the importance of messages based on the analysis results and determining their priority; means for sorting messages based on priority and converting them into a data format for notification to a user terminal; means for receiving notifications from user terminals and displaying messages in order of priority; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's business environment, a huge number of messages arrive constantly via email and various communication tools. Among these messages are important messages that require prompt responses, as well as low-priority messages. Many users must manually determine the priority of messages, which can result in important messages being buried. This situation can lead to business delays and overlooking important communications, significantly reducing efficiency. To solve this problem, the present invention aims to provide a system that automatically determines the priority of messages and quickly delivers important messages to users. [Means for solving the problem]

[0005] The present invention provides a system that includes the following means. First, a means is provided for calling an API of an email account and a communication tool to collect new messages. Next, a means is provided for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information. Next, a means is provided for scoring the importance of messages based on the analysis results and determining their priority. Next, a means is provided for sorting messages based on priority and converting them into a data format for notification to a user terminal. Finally, a means is provided for receiving notifications from the user terminal and displaying messages in order of priority. This allows users to quickly and efficiently check important messages.

[0006] "New messages" are messages that have been newly received through an email account or communication tool.

[0007] An "email account" is an account of an email service that a user uses to send and receive emails.

[0008] "Communication tools" are software or platforms that enable sending and receiving messages and sharing information.

[0009] "API" stands for Application Programming Interface, an interface that enables communication and data exchange between software programs.

[0010] "Originator" refers to the person or system that sent the message.

[0011] "Time of sending" refers to the date and time the message was sent.

[0012] "Content" is a general term for information such as text, images, and links contained in a message.

[0013] A "tag" refers to classification information or an identifier that is added to a message.

[0014] "Mention" refers to the act of mentioning or notifying a specific user on a communication tool.

[0015] "Analysis" is the process of analyzing the content and metadata of a message to extract that information.

[0016] "Scoring" is the process of quantifying the importance of a message based on the analysis results.

[0017] "Priority" is an indicator that indicates the importance or urgency of a message.

[0018] "Data format" refers to a uniform structure or format for storing, representing, and transmitting data.

[0019] A "notification" is an alert or message that notifies the user of some information.

[0020] "Display" refers to the act of outputting information to a monitor or display in a form that can be viewed by a user. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention relates to a system that organizes messages received from a user's email account or multiple communication tools in order of priority and promptly notifies the user of important messages. To implement this invention, the system is constructed according to the following procedure.

[0043] System configuration

[0044] The system consists of the following main components:

[0045] 1. Server

[0046] Message Collection Module

[0047] Message Analysis Module

[0048] Priority Determination Algorithm

[0049] Message Organizer Module

[0050] 2. User Device

[0051] Program processing

[0052] Message Collection

[0053] The server first periodically retrieves new messages using the APIs of the email accounts and communication tools linked to the user. For example, the server operates the Gmail API to fetch new emails from the user's mailbox. Similarly, the server calls the APIs of Slack and MICROSOFT® TEAMS® to retrieve the user's chat messages.

[0054] Message Parsing

[0055] The captured messages are then analyzed by the server's message analysis module. During the analysis, information about the sender, time of message, content, tags, and mentions is extracted. For example, mentions related to a specific project or messages containing keywords such as "urgent" or "important" are extracted.

[0056] Priority determination

[0057] The server then runs a priority algorithm based on the information obtained from the message analysis. This algorithm scores each message based on data such as mentions, the presence of important keywords, and the importance of the sender. For example, an email from a boss containing the keyword "urgent" would be given a high score.

[0058] Message organization

[0059] Based on the determined score, the server sorts the messages in order of priority, then converts them into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[0060] User Notification and Display

[0061] The user terminal receives the message list from the server and notifies the user of messages in descending order of priority. For example, the terminal displays a pop-up notification for high-priority messages and displays a list of medium-priority messages on a dashboard.

[0062] Specific examples

[0063] 1. Get new emails

[0064] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[0065] 2. Message Analysis

[0066] The server analyzes the emails and extracts the sender (boss), the time of sending, and the keyword "urgent." It also calls the Slack API to retrieve project progress report messages in which the user is mentioned.

[0067] 3. Priority determination

[0068] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it also takes into account mention information in Slack messages to set priorities.

[0069] 4. Message organization and notifications

[0070] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[0071] 5. User Notices

[0072] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[0073] 6. User Support

[0074] Users receive notifications and can check and respond to messages with higher priority.

[0075] This allows users to quickly and efficiently view and respond to important messages.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] Get new messages

[0079] The server periodically calls the APIs of email accounts and communication tools that the user has registered in advance. For example, the server uses the Gmail API to obtain new emails and the Slack API to collect new chat messages.

[0080] Step 2:

[0081] Preparing for Message Analysis

[0082] The server prepares the retrieved message data for analysis by loading a portion of the email or chat message into memory and extracting the information needed for analysis (such as sender, time of message, and content).

[0083] Step 3:

[0084] Extracting caller information

[0085] The server extracts originator information from message metadata, for example, identifying the "From" field in an email or the sender ID in a chat message.

[0086] Step 4:

[0087] Extracting the call time

[0088] The server obtains the message's sending time by checking the "Date" field in the email or the timestamp of the chat message.

[0089] Step 5:

[0090] Message content analysis

[0091] The server analyzes the message content to detect specific keywords or phrases, such as "urgent," "important," or "meeting."

[0092] Step 6:

[0093] Detecting tag and mention information

[0094] The server detects tags and mentions in messages by parsing @mentions in Slack messages and CC / BCC fields in emails.

[0095] Step 7:

[0096] Priority Scoring

[0097] The server then scores the importance of each message based on the analysis results, taking into account whether it contains specific keywords, whether it is from an important sender, whether it is mentioned, and so on.

[0098] Step 8:

[0099] Sorting by priority

[0100] The server sorts messages by priority based on the scoring results, with messages with higher scores coming to the top.

[0101] Step 9:

[0102] Converting to data format

[0103] The server converts the sorted message list into an appropriate data format (for example, JSON format) for transmission to the user terminal.

[0104] Step 10:

[0105] Send to user terminal

[0106] The server sends the sorted message list to the user's device and transfers the data using an API endpoint.

[0107] Step 11:

[0108] Displaying pop-up notifications

[0109] The device analyzes the message data received from the server and displays pop-up notifications for the most important messages, such as emails that require immediate attention.

[0110] Step 12:

[0111] Display on the dashboard

[0112] The user terminal displays a list of messages sorted by priority on the dashboard, in a list format for easy user access.

[0113] Step 13:

[0114] User message confirmation

[0115] The user receives notifications from the device and checks the messages in order of priority. First, open the most important message and take the necessary action.

[0116] Step 14:

[0117] Taking corresponding actions

[0118] Users can respond immediately to important messages, such as replying to an "urgent" email from their boss or checking the progress of a project.

[0119] Step 15:

[0120] Organizing confirmed messages

[0121] The device will archive or move messages that you have acknowledged to another folder, remove the acknowledged message from the list, and display the next important message.

[0122] By the above processing steps, the system of the present invention can promptly and efficiently present important messages to the user and prompt them to take action.

[0123] Example 1

[0124] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0125] Nowadays, many users use multiple messaging methods (e.g., email and chat apps). As a result, there is a high possibility that important messages will be buried and overlooked. This leads to problems such as reduced work efficiency and delayed important notifications. The present invention aims to centrally collect messages from different messaging methods, organize and notify them according to importance, and enable users to quickly understand important messages and respond appropriately.

[0126] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0127] In this invention, the server includes means for calling an API of a messaging account and a communication tool to collect new messages, means for analyzing the contents of the collected messages and extracting sender, sending time, content, tag, and mention information, means for scoring the importance of the messages based on the analysis results and determining their priority, means for sorting the messages based on their priority and converting them into a data format for notification to a user device, and means for receiving notifications from the user device and displaying the messages in order of priority. This makes it possible to centrally manage and organize messages from different messaging means and quickly notify users of important messages.

[0128] A "new message" is the most recent message received in a user's messaging account or communication tool.

[0129] A "messaging account" is one or more email accounts or chat application accounts used by a user.

[0130] "Communication tools" are platforms and applications (e.g., email, chat applications) that users use to send and receive messages.

[0131] "API" is an abbreviation for Application Programming Interface, a set of rules and tools for exchanging data between different software programs.

[0132] "Means for collection" refers to the server's functionality for obtaining new messages using the API of messaging accounts and communication tools.

[0133] The "analyzing means" is a function of the server for analyzing the content of collected messages and extracting sender, sending time, content, tag, and mention information.

[0134] An "originator" is a person or entity that sends a message.

[0135] "Time of sending" is the date and time the message was sent.

[0136] "Content" refers to the message body and the information contained therein.

[0137] A "tag" is an identifier or keyword that is added to a message.

[0138] "Mention information" is information in which a specific user is mentioned or named in a message.

[0139] The "scoring means" is a server function for quantifying the importance of messages based on the analysis results and setting priorities.

[0140] A "sorting means" is a server's capability to order messages based on priority.

[0141] The "means for converting into a data format" is a function of the server that converts the sorted messages into a format appropriate for notifying the user device.

[0142] A "user device" is an electronic device (e.g., smartphone, tablet, computer) that a user uses to receive and view messages.

[0143] The "means for receiving notifications" is a function that enables the user device to receive notifications from the server.

[0144] The "means for displaying" is a function that allows the user device to visually present messages to the user in order of priority.

[0145] The present invention relates to a system that automatically collects new messages from a user's multiple messaging accounts and communication tools, analyzes and scores them, and then promptly notifies the user of high-priority messages. The system for implementing this invention is composed of a server and a user terminal.

[0146] Overall structure

[0147] Server: Message collection module, message analysis module, priority determination algorithm, message organization module

[0148] User terminal: message display module, notification module

[0149] The specific hardware and software used

[0150] Server: We recommend a physical server or a cloud-based virtual server with high-performance processors, RAM, and storage, such as AWS (Amazon Web Services) or Google (Google Cloud Platform).

[0151] Communication tool APIs: Gmail API, Slack API, Microsoft Teams API, etc. are used.

[0152] Natural language processing software: spaCy, NLTK, Google Cloud Natural Language API, etc. are used for message analysis.

[0153] Data format: Convert to JSON format and send / receive messages.

[0154] Message Collection

[0155] The server first periodically retrieves new messages by calling the APIs of the email accounts and communication tools that the user has linked. For example, it uses the Gmail API to retrieve new emails from Gmail. Similarly, it calls the APIs of Slack and Microsoft Teams to retrieve chat messages.

[0156] Specifically, the server calls the Gmail API every minute to retrieve new emails for the user. An example of the API request used is:

[0157] "GET https: / / www.googleapis.com / gmail / v1 / users / {userId} / messages"

[0158] Message Parsing

[0159] Messages retrieved by the server are analyzed by the message analysis module. During the analysis process, information about the sender, time of sending, content, tags, and mentions is extracted. In particular, natural language processing technology is used to extract important keywords (e.g., "urgent" and "important") from the message body.

[0160] Specifically, analysis is performed using the Google Cloud Natural Language API to extract the keyword "urgent" from the email content.

[0161] Message Priority Determination

[0162] Based on the analyzed information, the server uses a priority determination algorithm to score the importance of the message. This algorithm takes into account the importance of the sender, keywords in the message content, mention information, etc. For example, an email from a boss containing the keyword "urgent" will be given the highest score.

[0163] An example of a specific scoring rule:

[0164] if "urgent" in message and sender == "boss": priority_score += 10

[0165] Message organization and notifications

[0166] Based on the priority score, the server sorts the messages in ascending or descending order and sends them to the user's device in JSON format, so that the sorted messages are notified to the user's device.

[0167] User Notification and Display

[0168] The user device analyzes the received message list and notifies the user in order of priority, displaying a pop-up notification for high-priority messages and displaying a list of medium-priority messages on the dashboard.

[0169] Specifically, high-priority messages are displayed on the user's device screen in the following format:

[0170] "[Urgent Notification] Supervisor: Please respond immediately"

[0171] User Support

[0172] Users receive notifications and can view and respond to messages with higher priority, allowing them to quickly and efficiently view and respond to important messages.

[0173] This allows users to prevent messages from being missed and quickly grasp important information, thereby improving work efficiency.

[0174] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0175] Step 1:

[0176] The server retrieves new messages by calling the API of the messaging account and communication tool linked by the user. Specifically, it fetches new messages using the Gmail API, Slack API, Microsoft Teams API, etc. It sends an API request and receives the new messages in JSON format as a response. For example, in the case of the Gmail API, the request looks like this:

[0177] GET https: / / www.googleapis.com / gmail / v1 / users / {userId} / messages

[0178] Input: User's API credentials

[0179] Output: New message data in JSON format

[0180] What happens: The server sends an API request every minute to fetch new messages.

[0181] Step 2:

[0182] The server analyzes the collected message content and extracts information about the sender, time of sending, content, tags, and mentions. This analysis is performed using natural language processing software (e.g., spaCy, NLTK, Google Cloud Natural Language API). Important keywords and flags (e.g., "urgent" and "important") are extracted from the message body.

[0183] Input: New message data in JSON format

[0184] Output: Analysis results (sender, time of posting, content, tags, mention information)

[0185] What it does: The server uses natural language processing software to extract important keywords from the message body.

[0186] Step 3:

[0187] The server then runs a priority determination algorithm based on the analysis results to score the importance of the message. The algorithm takes into account the importance of the sender, keywords in the message content, mention information, etc. For example, an email from a superior containing the keyword "urgent" will be given a high score.

[0188] Input: Analysis results

[0189] Output: Priority score for each message

[0190] What happens: The server scores using the following rules:

[0191] if "urgent" in message and sender == "boss": priority_score += 10

[0192] Step 4:

[0193] The server sorts messages based on their priority scores, converts the sorted messages into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[0194] Input: A message and its priority score

[0195] Output: A sorted list of messages in JSON format

[0196] What happens: The server sorts the messages using an algorithm such as quicksort and serializes them into JSON format.

[0197] Step 5:

[0198] The user device analyzes the message list received from the server and notifies the user of messages in order of priority, displaying a pop-up notification for high-priority messages and displaying a list of medium-priority messages on the dashboard.

[0199] Input: A sorted list of messages in JSON format

[0200] Output: Visual notifications and messages

[0201] What happens: The user device generates a pop-up notification to immediately display high priority messages.

[0202] Step 6:

[0203] Users receive notifications and can check and respond to messages in order of priority, allowing them to quickly and efficiently check and respond to important messages.

[0204] Input: Notification from user device

[0205] Output: User action (acknowledgement and response to message)

[0206] What happens: The user clicks on the pop-up notification to view the message details and take the necessary action.

[0207] This allows the user to respond efficiently without missing any important messages.

[0208] (Application example 1)

[0209] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0210] In modern society, many users use not only email and chat tools but also various content distribution services. However, these messages and content are often received in large quantities at once, putting users at risk of missing important information. Furthermore, the lack of an efficient way to quickly find important content that is relevant to users' interests further exacerbates information overload. Therefore, there is a need for a system that can efficiently organize multiple messages and content and prioritize notifications of important content.

[0211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0212] In this invention, the server includes means for calling an API of an email account and a communication tool to collect new messages, means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information, means for scoring the importance of the messages and determining their priority based on the analysis results, means for sorting the messages based on their priority and converting them into a data format for notification to a user terminal, means for receiving notifications from the user terminal and displaying the messages in order of priority, means for collecting new content from a content distribution service and extracting keywords for the sending time, author, title, and content, means for scoring the importance of the content based on categories of interest to the user or specific keywords and determining its priority, and means for notifying the user of the content sorted in order of priority. This allows users to quickly and efficiently check important information from multiple communication tools and content distribution services.

[0213] "New Messages" means the most recent messages received through your email account and communication tools.

[0214] "Means for calling an API" refers to a method for using an application program interface (API) to obtain necessary data from external services or databases.

[0215] "Means for analyzing the contents of a message" refers to a method for extracting and analyzing internal information of a received message (such as sender, time of sending, content, tags, and mention information).

[0216] The "means for scoring and determining priority" is a method for evaluating the importance of each message or content numerically based on the analyzed information and determining its priority.

[0217] The "means for converting into a data format" refers to a method for converting the organized messages or content into an appropriate data format (e.g., JSON format) for transfer to a user terminal.

[0218] The "means for notifying the user terminal" is a method for quickly notifying the user terminal of high-priority messages or content.

[0219] The "means for displaying messages in order of priority" is a method for displaying messages or content sorted according to importance in an orderly manner on a user terminal.

[0220] "New content" refers to information such as the latest news, articles, and videos obtained from content distribution services.

[0221] The "means for scoring the importance of content based on categories or specific keywords" is a method for evaluating the importance of each piece of content based on the presence or absence of interest categories or keywords set by the user.

[0222] The "means for notifying the user of the content sorted in order of priority" is a method for ranking the rated content according to the user's level of interest and notifying the user of the content in that order.

[0223] To specifically implement this invention, it is necessary to build a system according to the following detailed description. This system consists of two main components: a server and a user terminal. Below, we will explain how these components work together to process and calculate data.

[0224] Server configuration and operation

[0225] Hardware and Software

[0226] The server uses a cloud service such as AWS, and the software running on the server includes Python, Flask (a web framework), feedparser (an RSS analysis library), Firebase (a cloud messaging service), and so on.

[0227] Message and content collection

[0228] The server periodically collects new messages and content by calling the APIs of the email accounts and communication tools registered by the user. For example, it uses RSS feeds and APIs to obtain the latest news and articles. It also collects emails and chat messages using APIs such as Gmail and Slack.

[0229] Analysis and Scoring

[0230] The collected messages and content are analyzed on the server. During this analysis, information about the sender, time of posting, content, tags, and mentions is extracted. In the case of content, the category, author, title, and keywords are also extracted. Next, a scoring algorithm is run based on the analysis results to score the importance of each message or piece of content. This takes into account the user's interests and set keywords.

[0231] Determining priority and converting to data format

[0232] The scored data is sorted by priority, converted into an appropriate data format (e.g., JSON format) for notification to the user device, and sent to the user device via an API endpoint.

[0233] Configuration and operation of user terminal

[0234] Hardware and Software

[0235] The user device is typically a smartphone with a dedicated app installed, which receives notifications from the server via Firebase Cloud Messaging and processes them.

[0236] Notifications and Displays

[0237] The user device analyzes the data received from the server and displays messages and content in order of priority. For example, high-priority messages and content are displayed as pop-up notifications, while medium-priority messages and content are displayed as a list on the app's dashboard.

[0238] Examples of specific examples and prompts

[0239] For example, if a user wants to be notified of news containing keywords such as "important" or "urgent" in a news app, the user might input the following prompt sentence to the server:

[0240] Collect the latest news in categories that interest you (e.g., "Technology," "Health," "Business") and prioritize news that contains specific keywords (e.g., "Important," "Urgent").

[0241] In this way, users can efficiently view important messages and content that are relevant to their interests.

[0242] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0243] Step 1:

[0244] The server calls the APIs of the email accounts and communication tools registered by the user to collect new messages. At this time, the server uses APIs such as Gmail and Slack to periodically obtain data at specified intervals. The input for this process is the user's API authentication information, and the output is a list of new messages.

[0245] Step 2:

[0246] The server analyzes the collected new messages. Specifically, it extracts the sender, time, content, tags, and mention information for each message. In this step, it uses natural language processing technology to analyze the message text and extract the necessary information. The input is a list of new messages, and the output is the analyzed message data.

[0247] Step 3:

[0248] The server runs a scoring algorithm based on the parsed message data to determine the importance of each message. For example, a message from a superior containing the keyword "urgent" would be given a high score. The input to this process is the parsed message data, and the output is a scored message list.

[0249] Step 4:

[0250] The server sorts the scored messages based on their priority, then converts them into a data format (e.g., JSON) for notification to the user device. The input of this step is the scored message list, and the output is the sorted message data.

[0251] Step 5:

[0252] The server sends the aligned message data to the user terminal through the API endpoint. The input of this process is the aligned message data, and the output is the notification data sent to the user terminal.

[0253] Step 6:

[0254] The device receives and analyzes the sorted message data received from the server. It then displays the messages in order of priority. High-priority messages are displayed as pop-up notifications, and medium-priority messages are listed on the in-app dashboard. The input of this step is the notification data sent from the server, and the output is the message list displayed to the user.

[0255] Step 7:

[0256] The server periodically collects new content in categories that the user is interested in. For example, it uses an RSS feed or an API to get the latest news and articles. The input for this step is a feed URL or an API endpoint, and the output is a list of new content.

[0257] Step 8:

[0258] The server analyzes the collected new content and extracts keywords related to publication time, author, title, and content. Natural language processing and text analysis techniques are used in this step. The input is a list of new content, and the output is the analyzed content data.

[0259] Step 9:

[0260] The server runs a scoring algorithm based on the analyzed content data to determine the importance of each piece of content, taking into account the user's interest categories and specific keywords. The input is the analyzed content data, and the output is a scored content list.

[0261] Step 10:

[0262] The server sorts the content based on priority, converts it into a data format, and then notifies the user device through an API endpoint. The input of this step is a scored content list, and the output is the sorted content data.

[0263] Step 11:

[0264] The terminal displays the sorted content data received from the server. High-priority content is notified immediately, and low-priority content is displayed in a list on the dashboard. The input is the notification data sent from the server, and the output is the content list displayed to the user.

[0265] In this way, users can quickly and efficiently review important messages and content.

[0266] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0267] This invention is a system incorporating an emotion engine that recognizes the user's emotions, and is particularly intended to organize messages received from email accounts and multiple communication tools in order of priority and promptly notify the user of important messages. To implement this invention, the system is constructed according to the following procedure.

[0268] System configuration

[0269] The system consists of the following main components:

[0270] 1. Server

[0271] Message Collection Module

[0272] Message Analysis Module

[0273] Priority Determination Algorithm

[0274] Message Organizer Module

[0275] Emotion Engine

[0276] 2. User Device

[0277] Program processing

[0278] Message Collection

[0279] The server periodically retrieves new messages using the APIs of the email accounts and communication tools that the user has linked. For example, the server operates the Gmail API to fetch new emails from the user's mailbox. Similarly, the server calls the APIs of Slack, Microsoft Teams, etc. to retrieve the user's chat messages.

[0280] Message Parsing

[0281] The acquired messages are analyzed by the server's message analysis module. During the analysis, the sender, time of sending, content, tags, and mention information are extracted. In addition, an emotion engine recognizes the user's emotions from the message content and identifies negative emotions.

[0282] Priority determination

[0283] The server then runs a priority determination algorithm based on the information obtained from the message analysis. This algorithm scores the importance of each message, taking into account the presence or absence of mentions and important keywords, the importance of the sender, and the emotional information detected by the emotional engine. For example, an email from a superior containing the keyword "urgent" and also detecting negative emotions will be given a higher score.

[0284] Message organization

[0285] Based on the determined score, the server sorts the messages in order of priority, then converts them into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[0286] User Notification and Display

[0287] The user terminal receives the message list from the server and notifies the user of messages in descending order of priority. For example, the terminal displays a pop-up notification for high-priority messages and displays a list of medium-priority messages on a dashboard.

[0288] Specific examples

[0289] 1. Get new emails

[0290] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[0291] 2. Message Analysis

[0292] The server analyzes the received emails and extracts the sender (boss), time of sending, and the keyword "urgent." It also calls Slack's API to retrieve project progress report messages in which the user is mentioned. It then uses an emotion engine to recognize the user's emotions from the message content and identifies messages with negative emotions.

[0293] 3. Priority determination

[0294] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion engine.

[0295] 4. Message organization and notifications

[0296] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[0297] 5. User Notices

[0298] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[0299] 6. User Response

[0300] Users receive notifications and respond to messages with high priority. First, they check the "urgent" email from their boss, which is the most important, and respond or reply to it. Then, they check and respond to messages in Slack.

[0301] This allows users to quickly and efficiently review important messages and take appropriate action. The emotion engine allows for quicker responses to messages that are particularly urgent or emotionally important.

[0302] The processing flow will be explained below.

[0303] Step 1:

[0304] Get new messages

[0305] The server periodically calls the APIs of the email accounts and communication tools that the user has previously linked. For example, the server uses the Gmail API to retrieve new emails and the Slack API to collect new chat messages.

[0306] Step 2:

[0307] Preparing for Message Analysis

[0308] The server prepares the retrieved message data for analysis by loading the contents of the email or chat message into memory and extracting the information needed for analysis (such as sender, time of sending, and content).

[0309] Step 3:

[0310] Extracting caller information

[0311] The server extracts sender information from message metadata, for example by parsing the "From" field in emails or the sender ID of chat messages.

[0312] Step 4:

[0313] Extracting the call time

[0314] The server obtains the message's sending time by checking the "Date" field in the email or the timestamp of the chat message.

[0315] Step 5:

[0316] Message content analysis

[0317] The server analyzes the message content to detect specific keywords or phrases, such as "urgent," "important," or "meeting."

[0318] Step 6:

[0319] Detecting tag and mention information

[0320] The server detects tags and mentions in messages by parsing @mentions in Slack messages and CC / BCC fields in emails.

[0321] Step 7:

[0322] Emotion analysis using an emotion engine

[0323] The server uses an emotion engine to recognize the user's emotions from the message content, and identifies negative emotions using text and voice analysis.

[0324] Step 8:

[0325] Priority Scoring

[0326] The server then scores the importance of each message based on the analysis results, taking into account factors such as whether the message contains specific keywords, whether it is from an important sender, and whether the emotion engine detects negative emotions.

[0327] Step 9:

[0328] Sorting by priority

[0329] The server sorts messages by priority based on the scoring results, with messages with higher scores coming to the top.

[0330] Step 10:

[0331] Converting to data format

[0332] The server converts the sorted message list into an appropriate data format (for example, JSON format) for transmission to the user terminal.

[0333] Step 11:

[0334] Send to user terminal

[0335] The server sends the sorted message list to the user's device and transfers the data using an API endpoint.

[0336] Step 12:

[0337] Displaying pop-up notifications

[0338] The device analyzes the message data received from the server and displays pop-up notifications for the most important messages, such as emails that require immediate attention.

[0339] Step 13:

[0340] Display on the dashboard

[0341] The user terminal displays a list of messages sorted by priority on the dashboard, in a list format for easy user access.

[0342] Step 14:

[0343] User message confirmation

[0344] The user receives notifications from the device and checks the messages in order of priority. First, open the most important message and take the necessary action.

[0345] Step 15:

[0346] Taking corresponding actions

[0347] Users can respond immediately to important messages, such as replying to an "urgent" email from their boss or checking the progress of a project.

[0348] Step 16:

[0349] Organizing confirmed messages

[0350] The device will archive or move messages that you have acknowledged to another folder, remove the acknowledged message from the list, and display the next important message.

[0351] This allows users to quickly and efficiently view and respond to important messages, and the emotion engine allows for quick responses to messages that are particularly urgent or emotionally important.

[0352] Example 2

[0353] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0354] In today's business environment, users receive a large volume of messages from multiple electronic communication channels, and they need to be able to quickly and efficiently identify and respond to important messages. However, doing this manually is extremely time-consuming, so a system is needed that can automatically identify high-priority messages and notify users immediately. It is also necessary to be able to determine the emotional content of messages and prioritize messages with specific emotions.

[0355] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for calling an API of an electronic communication means for collecting new messages, a means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information, a means for scoring the importance of the messages based on the analysis results and determining their priority, a means for recognizing emotional information from the message content using an emotion engine and reflecting it in the priority determination, a means for sorting messages based on their priority and converting them into a data format for notification to a user terminal, and a means for receiving notifications from the user terminal and displaying messages in order of priority. This allows users to quickly and efficiently identify important or emotionally significant messages and respond appropriately.

[0356] "Electronic communication means" refers to communication means for sending and receiving messages, such as email and chat tools.

[0357] "API" stands for Application Programming Interface, an interface for exchanging information and commands between different software programs.

[0358] An "emotion engine" is an algorithm or software module that analyzes a user's emotions from the content of a message and assigns an emotion score such as positive, negative, or neutral.

[0359] "Scoring" is the process of quantifying importance and priority based on analyzed message information.

[0360] "Priority" is an index of importance for determining the processing order of messages.

[0361] "Mention information" is information in which a specific user or group is mentioned in a message.

[0362] A "tag" is a keyword or classification information that is added to a message and indicates the content or category of the message.

[0363] A "data format" is a standard that defines the structure and form of data, such as JSON or XML.

[0364] A "user terminal" is a device used by a user, such as a computer, smartphone, or tablet.

[0365] This invention is a system incorporating an emotion engine that recognizes the user's emotions, and in particular, is designed to automatically determine the importance of new messages and promptly notify the user. To implement this invention, the system is constructed according to the following procedure.

[0366] System configuration

[0367] The system consists of the following main components:

[0368] 1. Server

[0369] Message collection module: Calls the API of the electronic communication means to collect messages.

[0370] Message analysis module: Analyzes the content of collected messages and extracts sender, time of sending, content, tags, and mention information.

[0371] Priority determination algorithm: Based on the analysis results, the importance of the message is scored and the priority is determined.

[0372] Emotion engine: Recognizes emotional information from the content of messages and reflects it in determining priority.

[0373] Message sorting module: Sorts messages based on priority and converts them into a data format for notification to the user terminal.

[0374] 2. User Device

[0375] Notification system: Receives notifications from the server and displays messages based on priority.

[0376] System processing flow

[0377] Message Collection

[0378] The server periodically calls the API of the electronic communication means linked by the user to obtain new messages. Specifically, the server collects messages using the Gmail API, Slack API, Microsoft Teams API, etc.

[0379] Message Parsing

[0380] The acquired messages are analyzed by the server's message analysis module. This analysis extracts information about the sender, time of sending, content, tags, and mentions. An emotion engine is also used to recognize emotional information from the message content. Specifically, an emotional score such as positive, negative, or neutral is generated.

[0381] Priority determination

[0382] Based on the analyzed information, the server runs a priority determination algorithm. This algorithm scores the importance of each message by combining mention information, the presence or absence of important keywords, the importance of the sender, and emotional information. For example, an email from a boss containing the keyword "urgent" will be given a high score. Messages containing negative emotions will also be given a high score.

[0383] Message organization and notifications

[0384] Scored messages are sorted by priority and converted into a data format (e.g., JSON format) by the server, and then sent to the user's device via an API endpoint.

[0385] User Notification and Display

[0386] The user terminal receives the message list from the server and notifies the user of messages in order of priority. Specifically, the terminal displays a pop-up notification, and displays a list of messages with medium priority on the dashboard.

[0387] User response

[0388] After receiving a notification, users can check and respond to messages with higher priority. For example, they can check and respond to the most important "urgent" email from their boss. Then, they can check and respond to messages in Slack.

[0389] This allows users to quickly and efficiently check important messages and take appropriate action. The emotion engine also enables quick responses to messages that are particularly urgent or emotionally important.

[0390] Specific examples

[0391] 1. Get new emails

[0392] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[0393] 2. Message Analysis

[0394] The server analyzes the received emails and extracts the sender (boss), time of sending, and the keyword "urgent." It also calls Slack's API to retrieve project progress report messages in which the user is mentioned. It then uses an emotion engine to recognize emotional information from the message content and identify messages with negative emotions.

[0395] 3. Priority determination

[0396] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion engine.

[0397] 4. Message organization and notifications

[0398] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[0399] 5. User Notices

[0400] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[0401] 6. User Response

[0402] Users receive notifications and respond to messages with high priority. First, they check the "urgent" email from their boss, which is the most important, and respond or reply to it. Then, they check and respond to messages in Slack.

[0403] Prompt Sentence Examples

[0404] An example of a prompt for a generative AI model might be, "If I receive an urgent message from my boss, please advise me on how I should respond first."

[0405] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0406] Step 1: Message Collection

[0407] The server periodically calls the API of the electronic communication method linked by the user to obtain new messages. Specifically, the server collects messages using the Gmail API, Slack API, Microsoft Teams API, etc. The input required is the user's authentication information and API call parameters. The output is a list of collected unread messages. This list is stored in the server's database.

[0408] Step 2: Message analysis

[0409] The server passes the collected messages to a message analysis module, which uses a natural language processing (NLP) library (such as NLTK or SpaCy) to analyze the messages. The input is a list of unanalyzed messages. During the analysis process, sender, time of message, content, tags, and mention information are extracted. The output is a list of analyzed messages with this information, along with an emotion score for each message generated by the emotion engine.

[0410] Step 3: Prioritization

[0411] The server runs a priority determination algorithm using the parsed message list. The parsed message list, including the sender, time of sending, tags, mention information, and sentiment information, is required as input. The priority determination algorithm scores messages based on important keywords, the importance of the sender, and sentiment information. For example, a message from a boss containing the keyword "urgent" is assigned a high score. The output is a list containing the priority score for each message.

[0412] Step 4: Message organization and notifications

[0413] The server sorts messages based on their priority scores and converts them into a data format (e.g., JSON format) for notification to the user device. A message list with priority scores is required as input. The server converts this into JSON format and sends it to the user device via an API endpoint. The output is the notification data sent to the user device.

[0414] Step 5: User Notification and Display

[0415] The user terminal parses the JSON formatted notification data received from the server. Notification data is required as input. The user terminal analyzes this data and notifies the user of messages in order of priority. Specifically, a pop-up notification is displayed for high-priority messages, and messages with medium priority are listed on the dashboard. The output is a message that is visually displayed to the user.

[0416] Step 6: User Action

[0417] The user receives notifications from their device and checks and responds to messages in order of priority. The input requires notification information from the user's device. The user checks the most important "urgent" email from their boss and replies or responds to it. Then, the user checks the message in Slack and replies to it. The output is the reply or response sent by the user, which is sent again to the server and the message status is updated.

[0418] Through these steps, users can quickly and efficiently review important messages and respond appropriately. The emotion engine allows users to respond quickly to especially urgent or emotionally significant messages.

[0419] (Application example 2)

[0420] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0421] Conventional email and communication tool message management systems simply retrieve messages and determine their importance. However, they were unable to prioritize or notify messages taking into account the user's emotions, making it difficult to respond quickly to important messages, especially those with negative sentiment. This made it difficult for users to quickly understand and respond to truly important messages. Furthermore, in physical stores, it was not possible to understand customer emotions in real time and respond appropriately. This resulted in a decline in service quality and hindered improvements in customer satisfaction.

[0422] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0423] In this invention, the server includes: a means for calling an email account and a communication tool interface for collecting new messages; a means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and reference information; a means for ranking the importance of the messages and determining their priority based on the analysis results; a means for sorting the messages based on priority and converting them into a data format for notification to the user terminal; a means for receiving notifications from the user terminal and displaying the messages in order of priority; and a means for incorporating an emotion recognition engine that recognizes the user's emotions into the interface, analyzing the collected emotion data, and assigning a higher priority to messages with negative emotions. This allows users to quickly identify and respond to important messages, especially those with negative emotions. It also enables real-time monitoring of customer emotions in physical stores and prompt appropriate responses.

[0424] "New Messages" are the most recent messages received through your email account or communication tool.

[0425] An "email account" is an online account that a user uses to send and receive email.

[0426] A "communication tool" is a platform or application that users use to send and receive messages to each other.

[0427] An "interface" is a connection point or interface where parts of a system communicate and exchange data with each other.

[0428] "Analysis" is the act of examining the contents of collected messages in detail and extracting necessary information.

[0429] "Originator" is the user or account that sent the message.

[0430] "Time of Sent" is the exact date and time the message was sent.

[0431] "Content" is the text or information contained in the message.

[0432] A "tag" is a specific keyword or label added to a message that is used to categorize the message.

[0433] "Reference information" is metadata that indicates the related information referenced by a message or the relevance of a message.

[0434] "Ranking" refers to classifying the importance of a message into a numerical value or category based on analyzed information.

[0435] "Priority" is an index that indicates the importance or urgency of a message.

[0436] A "data format" is a format that structures message data in a specific format to facilitate storage and transmission.

[0437] An "emotion recognition engine" is a software component that analyzes the emotions of users and customers and recognizes their emotional state.

[0438] "Negative emotions" are negative emotional states such as anger, sadness, and fear.

[0439] This invention is a system that includes an emotion recognition engine that recognizes user emotions, and is particularly intended to organize messages received from email accounts and multiple communication tools in order of priority and promptly notify users of important messages. It can also be applied to applications that perform real-time emotion recognition in customer service at brick-and-mortar stores to improve the quality of service.

[0440] System configuration

[0441] The system consists of the following main components:

[0442] 1. Server

[0443] Message collection module: Calls the interface of the email account and communication tool and periodically retrieves new messages.

[0444] Message Analysis Module: Analyzes collected messages and extracts sender, time of sending, content, tags, and reference information.

[0445] Priority determination algorithm: Based on the analysis results, the importance of messages is ranked and their priority is determined.

[0446] Message sorting module: Sorts messages based on priority and converts them into a data format for notification to the user terminal.

[0447] Emotion Recognition Engine: Analyzes emotional data and prioritizes messages with negative emotions.

[0448] 2. User Device

[0449] Notification receiving component: Receives and displays a list of messages received from the server, organized by priority.

[0450] Emotion-aware interface: Analyze customer emotions in real time and respond appropriately.

[0451] Program processing

[0452] Message Collection

[0453] The server periodically retrieves new messages using the APIs of the email account and communication tool. For example, the server operates a common email API or chat tool API to retrieve and collect new emails and chat messages.

[0454] Message Parsing

[0455] The server analyzes messages acquired by the server using a message analysis module. During the analysis process, the sender, time of message, content, tags, and reference information are extracted. Furthermore, an emotion recognition engine recognizes the emotions of users and customers from the message content and identifies negative emotions.

[0456] Priority determination

[0457] The server then runs a priority algorithm based on the analyzed information to rank the importance of each message. For example, messages from superiors or messages containing keywords like "urgent" are given a higher score. The priority is further adjusted based on negative emotions detected by an emotion recognition engine.

[0458] Message organization

[0459] Based on the determined score, the server sorts the messages in order of priority and then converts them into an appropriate data format for notification to the user device. The sorted message list is sent to the user device through an API endpoint.

[0460] User Notification and Display

[0461] The user device receives a message list from the server and notifies them in order of priority. For example, the device displays a pop-up notification for high-priority messages and displays a list of other messages. It also monitors customer sentiment in real time in physical stores and sends notifications to promptly take appropriate action.

[0462] Specific examples

[0463] 1. Get new emails

[0464] The server uses a general-purpose email API to retrieve new emails from the user's mailbox. For example, you might receive an email from your boss marked "urgent."

[0465] 2. Message Analysis

[0466] The server analyzes the received email and extracts the sender, time of sending, and the keyword "urgent." It then uses an emotion recognition engine to recognize the user's emotions from the message content and identify negative emotions.

[0467] 3. Priority determination

[0468] The server then runs a scoring algorithm based on the analysis results, assigning a high score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion recognition engine.

[0469] 4. Message organization and notifications

[0470] The server sorts the messages by priority and sends them to the user terminal in the appropriate data format.

[0471] 5. User Notices

[0472] The user device analyzes the received messages and notifies the user of high-priority messages via a pop-up notification. In addition, in physical stores, the system detects negative customer sentiment and notifies the user to take prompt action.

[0473] Prompt Sentence Examples

[0474] Below are some example prompts to input to a generative AI model:

[0475] "A customer changes their facial expression at the counter. Develop a system that recognizes the customer's emotions from image and audio data and notifies an employee if negative emotions (anger, sadness, fear) are detected. The hardware used is smart glasses, and the software used is TENSORFLOW(R) / Keras."

[0476] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0477] Step 1:

[0478] Message collection

[0479] The server calls the interfaces of the email account and communication tool to periodically retrieve new messages. Specifically, the server periodically collects new emails and chat messages using the API of each interface. The input is the API information of the email account and chat tool, and the output is a list of collected unread messages.

[0480] Step 2:

[0481] Message Parsing

[0482] The server analyzes the collected message content. Specifically, it uses a message analysis module to extract sender, time of message, content, tags, and reference information. It then uses an emotion recognition engine to extract emotional data from the message content and identify negative emotions. The input is a list of newly arrived messages, and the output is structured data and emotional data of the messages.

[0483] Step 3:

[0484] Determining Priority

[0485] The server runs a priority determination algorithm based on the analyzed information to rank the importance of messages. Specifically, it evaluates important keywords, the message sender, the time of sending, emotional data, etc., and assigns a score to each message. The input is the structured data and emotional data of the message, and the output is a list of scored messages.

[0486] Step 4:

[0487] Organizing messages

[0488] The server sorts the messages in order of priority based on the determined scores. Specifically, it sorts the array based on the score of each message and then converts it into a data format for notification. The input is a list of scored messages, and the output is a list of messages sorted by priority.

[0489] Step 5:

[0490] Message notifications

[0491] The user terminal receives and displays the message list sorted by priority received from the server. Specifically, the terminal displays a pop-up notification for high-priority messages and displays the list for other messages. The input is the list of messages sorted by priority, and the output is the notification to the user and the messages displayed.

[0492] Step 6:

[0493] Customer Sentiment Detection

[0494] The user terminal (smart glasses) captures the customer's facial expressions and voice in real time and inputs them into the emotion recognition engine. Specifically, the smart glasses use a camera and microphone to send the captured image and voice data to the emotion recognition engine. The input is image and voice data, and the output is detected emotion data.

[0495] Step 7:

[0496] Customer response notification

[0497] The user device (smart glasses) identifies customers with negative emotions based on the emotion data received from the emotion recognition engine and notifies employees. Specifically, a warning message is displayed on the smart glasses' display, urging employees to take prompt action. The input is the detected emotion data, and the output is a warning notification to employees.

[0498] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0499] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0500] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0501] [Second embodiment]

[0502] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0503] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0504] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0505] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0506] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0507] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0508] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0509] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0510] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0511] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0512] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0513] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0514] This invention relates to a system that organizes messages received from a user's email account or multiple communication tools in order of priority and promptly notifies the user of important messages. To implement this invention, the system is constructed according to the following procedure.

[0515] System configuration

[0516] The system consists of the following main components:

[0517] 1. Server

[0518] Message Collection Module

[0519] Message Analysis Module

[0520] Priority Determination Algorithm

[0521] Message Organizer Module

[0522] 2. User Device

[0523] Program processing

[0524] Message Collection

[0525] The server first periodically retrieves new messages using the APIs of the email accounts and communication tools linked to the user. For example, the server operates the Gmail API to fetch new emails from the user's mailbox. Similarly, the server calls the APIs of Slack, Microsoft Teams, and other services to retrieve the user's chat messages.

[0526] Message Parsing

[0527] The captured messages are then analyzed by the server's message analysis module. During the analysis, information about the sender, time of message, content, tags, and mentions is extracted. For example, mentions related to a specific project or messages containing keywords such as "urgent" or "important" are extracted.

[0528] Priority determination

[0529] The server then runs a priority algorithm based on the information obtained from the message analysis. This algorithm scores each message based on data such as mentions, the presence of important keywords, and the importance of the sender. For example, an email from a boss containing the keyword "urgent" would be given a high score.

[0530] Message organization

[0531] Based on the determined score, the server sorts the messages in order of priority, then converts them into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[0532] User Notification and Display

[0533] The user terminal receives the message list from the server and notifies the user of messages in descending order of priority. For example, the terminal displays a pop-up notification for high-priority messages and displays a list of medium-priority messages on a dashboard.

[0534] Specific examples

[0535] 1. Get new emails

[0536] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[0537] 2. Message Analysis

[0538] The server analyzes the emails and extracts the sender (boss), the time of sending, and the keyword "urgent." It also calls the Slack API to retrieve project progress report messages in which the user is mentioned.

[0539] 3. Priority determination

[0540] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it also takes into account mention information in Slack messages to set priorities.

[0541] 4. Message organization and notifications

[0542] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[0543] 5. User Notices

[0544] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[0545] 6. User Support

[0546] Users receive notifications and can check and respond to messages with higher priority.

[0547] This allows users to quickly and efficiently view and respond to important messages.

[0548] The processing flow will be explained below.

[0549] Step 1:

[0550] Get new messages

[0551] The server periodically calls the APIs of email accounts and communication tools that the user has registered in advance. For example, the server uses the Gmail API to obtain new emails and the Slack API to collect new chat messages.

[0552] Step 2:

[0553] Preparing for Message Analysis

[0554] The server prepares the retrieved message data for analysis by loading a portion of the email or chat message into memory and extracting the information needed for analysis (such as sender, time of message, and content).

[0555] Step 3:

[0556] Extracting caller information

[0557] The server extracts originator information from message metadata, for example, identifying the "From" field in an email or the sender ID in a chat message.

[0558] Step 4:

[0559] Extracting the call time

[0560] The server obtains the message's sending time by checking the "Date" field in the email or the timestamp of the chat message.

[0561] Step 5:

[0562] Message content analysis

[0563] The server analyzes the message content to detect specific keywords or phrases, such as "urgent," "important," or "meeting."

[0564] Step 6:

[0565] Detecting tag and mention information

[0566] The server detects tags and mentions in messages by parsing @mentions in Slack messages and CC / BCC fields in emails.

[0567] Step 7:

[0568] Priority Scoring

[0569] The server then scores the importance of each message based on the analysis results, taking into account whether it contains specific keywords, whether it is from an important sender, whether it is mentioned, and so on.

[0570] Step 8:

[0571] Sorting by priority

[0572] The server sorts messages by priority based on the scoring results, with messages with higher scores coming to the top.

[0573] Step 9:

[0574] Converting to data format

[0575] The server converts the sorted message list into an appropriate data format (for example, JSON format) for transmission to the user terminal.

[0576] Step 10:

[0577] Send to user terminal

[0578] The server sends the sorted message list to the user's device and transfers the data using an API endpoint.

[0579] Step 11:

[0580] Displaying pop-up notifications

[0581] The device analyzes the message data received from the server and displays pop-up notifications for the most important messages, such as emails that require immediate attention.

[0582] Step 12:

[0583] Display on the dashboard

[0584] The user terminal displays a list of messages sorted by priority on the dashboard, in a list format for easy user access.

[0585] Step 13:

[0586] User message confirmation

[0587] The user receives notifications from the device and checks the messages in order of priority. First, open the most important message and take the necessary action.

[0588] Step 14:

[0589] Taking corresponding actions

[0590] Users can respond immediately to important messages, such as replying to an "urgent" email from their boss or checking the progress of a project.

[0591] Step 15:

[0592] Organizing confirmed messages

[0593] The device will archive or move messages that you have acknowledged to another folder, remove the acknowledged message from the list, and display the next important message.

[0594] By the above processing steps, the system of the present invention can promptly and efficiently present important messages to the user and prompt them to take action.

[0595] Example 1

[0596] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0597] Nowadays, many users use multiple messaging methods (e.g., email and chat apps). As a result, there is a high possibility that important messages will be buried and overlooked. This leads to problems such as reduced work efficiency and delayed important notifications. The present invention aims to centrally collect messages from different messaging methods, organize and notify them according to importance, and enable users to quickly understand important messages and respond appropriately.

[0598] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0599] In this invention, the server includes means for calling an API of a messaging account and a communication tool to collect new messages, means for analyzing the contents of the collected messages and extracting sender, sending time, content, tag, and mention information, means for scoring the importance of the messages based on the analysis results and determining their priority, means for sorting the messages based on their priority and converting them into a data format for notification to a user device, and means for receiving notifications from the user device and displaying the messages in order of priority. This makes it possible to centrally manage and organize messages from different messaging means and quickly notify users of important messages.

[0600] A "new message" is the most recent message received in a user's messaging account or communication tool.

[0601] A "messaging account" is one or more email accounts or chat application accounts used by a user.

[0602] "Communication tools" are platforms and applications (e.g., email, chat applications) that users use to send and receive messages.

[0603] "API" is an abbreviation for Application Programming Interface, a set of rules and tools for exchanging data between different software programs.

[0604] "Means for collection" refers to the server's functionality for obtaining new messages using the API of messaging accounts and communication tools.

[0605] The "analyzing means" is a function of the server for analyzing the content of collected messages and extracting sender, sending time, content, tag, and mention information.

[0606] An "originator" is a person or entity that sends a message.

[0607] "Time of sending" is the date and time the message was sent.

[0608] "Content" refers to the message body and the information contained therein.

[0609] A "tag" is an identifier or keyword that is added to a message.

[0610] "Mention information" is information in which a specific user is mentioned or named in a message.

[0611] The "scoring means" is a server function for quantifying the importance of messages based on the analysis results and setting priorities.

[0612] A "sorting means" is a server's capability to order messages based on priority.

[0613] The "means for converting into a data format" is a function of the server that converts the sorted messages into a format appropriate for notifying the user device.

[0614] A "user device" is an electronic device (e.g., smartphone, tablet, computer) that a user uses to receive and view messages.

[0615] The "means for receiving notifications" is a function that enables the user device to receive notifications from the server.

[0616] The "means for displaying" is a function that allows the user device to visually present messages to the user in order of priority.

[0617] The present invention relates to a system that automatically collects new messages from a user's multiple messaging accounts and communication tools, analyzes and scores them, and then promptly notifies the user of high-priority messages. The system for implementing this invention is composed of a server and a user terminal.

[0618] Overall structure

[0619] Server: Message collection module, message analysis module, priority determination algorithm, message organization module

[0620] User terminal: message display module, notification module

[0621] The specific hardware and software used

[0622] Server: We recommend a physical server or a cloud-based virtual server with high-performance processors, RAM, and storage, such as Amazon Web Services (AWS) or Google Cloud Platform.

[0623] Communication tool APIs: Gmail API, Slack API, Microsoft Teams API, etc. are used.

[0624] Natural language processing software: spaCy, NLTK, Google Cloud Natural Language API, etc. are used for message analysis.

[0625] Data format: Convert to JSON format and send / receive messages.

[0626] Message Collection

[0627] The server first periodically retrieves new messages by calling the APIs of the email accounts and communication tools that the user has linked. For example, it uses the Gmail API to retrieve new emails from Gmail. Similarly, it calls the APIs of Slack and Microsoft Teams to retrieve chat messages.

[0628] Specifically, the server calls the Gmail API every minute to retrieve new emails for the user. An example of the API request used is:

[0629] "GET https: / / www.googleapis.com / gmail / v1 / users / {userId} / messages"

[0630] Message Parsing

[0631] Messages retrieved by the server are analyzed by the message analysis module. During the analysis process, information about the sender, time of sending, content, tags, and mentions is extracted. In particular, natural language processing technology is used to extract important keywords (e.g., "urgent" and "important") from the message body.

[0632] Specifically, analysis is performed using the Google Cloud Natural Language API to extract the keyword "urgent" from the email content.

[0633] Message Priority Determination

[0634] Based on the analyzed information, the server uses a priority determination algorithm to score the importance of the message. This algorithm takes into account the importance of the sender, keywords in the message content, mention information, etc. For example, an email from a boss containing the keyword "urgent" will be given the highest score.

[0635] An example of a specific scoring rule:

[0636] if "urgent" in message and sender == "boss": priority_score += 10

[0637] Message organization and notifications

[0638] Based on the priority score, the server sorts the messages in ascending or descending order and sends them to the user's device in JSON format, so that the sorted messages are notified to the user's device.

[0639] User Notification and Display

[0640] The user device analyzes the received message list and notifies the user in order of priority, displaying a pop-up notification for high-priority messages and displaying a list of medium-priority messages on the dashboard.

[0641] Specifically, high-priority messages are displayed on the user's device screen in the following format:

[0642] "[Urgent Notification] Supervisor: Please respond immediately"

[0643] User Support

[0644] Users receive notifications and can view and respond to messages with higher priority, allowing them to quickly and efficiently view and respond to important messages.

[0645] This allows users to prevent messages from being missed and quickly grasp important information, thereby improving work efficiency.

[0646] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0647] Step 1:

[0648] The server retrieves new messages by calling the API of the messaging account and communication tool linked by the user. Specifically, it fetches new messages using the Gmail API, Slack API, Microsoft Teams API, etc. It sends an API request and receives the new messages in JSON format as a response. For example, in the case of the Gmail API, the request looks like this:

[0649] GET https: / / www.googleapis.com / gmail / v1 / users / {userId} / messages

[0650] Input: User's API credentials

[0651] Output: New message data in JSON format

[0652] What happens: The server sends an API request every minute to fetch new messages.

[0653] Step 2:

[0654] The server analyzes the collected message content and extracts information about the sender, time of sending, content, tags, and mentions. This analysis is performed using natural language processing software (e.g., spaCy, NLTK, Google Cloud Natural Language API). Important keywords and flags (e.g., "urgent" and "important") are extracted from the message body.

[0655] Input: New message data in JSON format

[0656] Output: Analysis results (sender, time of posting, content, tags, mention information)

[0657] What it does: The server uses natural language processing software to extract important keywords from the message body.

[0658] Step 3:

[0659] The server then runs a priority determination algorithm based on the analysis results to score the importance of the message. The algorithm takes into account the importance of the sender, keywords in the message content, mention information, etc. For example, an email from a superior containing the keyword "urgent" will be given a high score.

[0660] Input: Analysis results

[0661] Output: Priority score for each message

[0662] What happens: The server scores using the following rules:

[0663] if "urgent" in message and sender == "boss": priority_score += 10

[0664] Step 4:

[0665] The server sorts messages based on their priority scores, converts the sorted messages into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[0666] Input: A message and its priority score

[0667] Output: A sorted list of messages in JSON format

[0668] What happens: The server sorts the messages using an algorithm such as quicksort and serializes them into JSON format.

[0669] Step 5:

[0670] The user device analyzes the message list received from the server and notifies the user of messages in order of priority, displaying a pop-up notification for high-priority messages and displaying a list of medium-priority messages on the dashboard.

[0671] Input: A sorted list of messages in JSON format

[0672] Output: Visual notifications and messages

[0673] What happens: The user device generates a pop-up notification to immediately display high priority messages.

[0674] Step 6:

[0675] Users receive notifications and can check and respond to messages in order of priority, allowing them to quickly and efficiently check and respond to important messages.

[0676] Input: Notification from user device

[0677] Output: User action (acknowledgement and response to message)

[0678] What happens: The user clicks on the pop-up notification to view the message details and take the necessary action.

[0679] This allows the user to respond efficiently without missing any important messages.

[0680] (Application example 1)

[0681] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0682] In modern society, many users use not only email and chat tools but also various content distribution services. However, these messages and content are often received in large quantities at once, putting users at risk of missing important information. Furthermore, the lack of an efficient way to quickly find important content that is relevant to users' interests further exacerbates information overload. Therefore, there is a need for a system that can efficiently organize multiple messages and content and prioritize notifications of important content.

[0683] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0684] In this invention, the server includes means for calling an API of an email account and a communication tool to collect new messages, means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information, means for scoring the importance of the messages and determining their priority based on the analysis results, means for sorting the messages based on their priority and converting them into a data format for notification to a user terminal, means for receiving notifications from the user terminal and displaying the messages in order of priority, means for collecting new content from a content distribution service and extracting keywords for the sending time, author, title, and content, means for scoring the importance of the content based on categories of interest to the user or specific keywords and determining its priority, and means for notifying the user of the content sorted in order of priority. This allows users to quickly and efficiently check important information from multiple communication tools and content distribution services.

[0685] "New Messages" means the most recent messages received through your email account and communication tools.

[0686] "Means for calling an API" refers to a method for using an application program interface (API) to obtain necessary data from external services or databases.

[0687] "Means for analyzing the contents of a message" refers to a method for extracting and analyzing internal information of a received message (such as sender, time of sending, content, tags, and mention information).

[0688] The "means for scoring and determining priority" is a method for evaluating the importance of each message or content numerically based on the analyzed information and determining its priority.

[0689] The "means for converting into a data format" refers to a method for converting the organized messages or content into an appropriate data format (e.g., JSON format) for transfer to a user terminal.

[0690] The "means for notifying the user terminal" is a method for quickly notifying the user terminal of high-priority messages or content.

[0691] The "means for displaying messages in order of priority" is a method for displaying messages or content sorted according to importance in an orderly manner on a user terminal.

[0692] "New content" refers to information such as the latest news, articles, and videos obtained from content distribution services.

[0693] The "means for scoring the importance of content based on categories or specific keywords" is a method for evaluating the importance of each piece of content based on the presence or absence of interest categories or keywords set by the user.

[0694] The "means for notifying the user of the content sorted in order of priority" is a method for ranking the rated content according to the user's level of interest and notifying the user of the content in that order.

[0695] To specifically implement this invention, it is necessary to build a system according to the following detailed description. This system consists of two main components: a server and a user terminal. Below, we will explain how these components work together to process and calculate data.

[0696] Server configuration and operation

[0697] Hardware and Software

[0698] The server uses a cloud service such as AWS, and the software running on the server includes Python, Flask (a web framework), feedparser (an RSS analysis library), Firebase (a cloud messaging service), and so on.

[0699] Message and content collection

[0700] The server periodically collects new messages and content by calling the APIs of the email accounts and communication tools registered by the user. For example, it uses RSS feeds and APIs to obtain the latest news and articles. It also collects emails and chat messages using APIs such as Gmail and Slack.

[0701] Analysis and Scoring

[0702] The collected messages and content are analyzed on the server. During this analysis, information about the sender, time of posting, content, tags, and mentions is extracted. In the case of content, the category, author, title, and keywords are also extracted. Next, a scoring algorithm is run based on the analysis results to score the importance of each message or piece of content. This takes into account the user's interests and set keywords.

[0703] Determining priority and converting to data format

[0704] The scored data is sorted by priority, converted into an appropriate data format (e.g., JSON format) for notification to the user device, and sent to the user device via an API endpoint.

[0705] Configuration and operation of user terminal

[0706] Hardware and Software

[0707] The user device is typically a smartphone with a dedicated app installed, which receives notifications from the server via Firebase Cloud Messaging and processes them.

[0708] Notifications and Displays

[0709] The user device analyzes the data received from the server and displays messages and content in order of priority. For example, high-priority messages and content are displayed as pop-up notifications, while medium-priority messages and content are displayed as a list on the app's dashboard.

[0710] Examples of specific examples and prompts

[0711] For example, if a user wants to be notified of news containing keywords such as "important" or "urgent" in a news app, the user might input the following prompt sentence to the server:

[0712] Collect the latest news in categories that interest you (e.g., "Technology," "Health," "Business") and prioritize news that contains specific keywords (e.g., "Important," "Urgent").

[0713] In this way, users can efficiently view important messages and content that are relevant to their interests.

[0714] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0715] Step 1:

[0716] The server calls the APIs of the email accounts and communication tools registered by the user to collect new messages. At this time, the server uses APIs such as Gmail and Slack to periodically obtain data at specified intervals. The input for this process is the user's API authentication information, and the output is a list of new messages.

[0717] Step 2:

[0718] The server analyzes the collected new messages. Specifically, it extracts the sender, time, content, tags, and mention information for each message. In this step, it uses natural language processing technology to analyze the message text and extract the necessary information. The input is a list of new messages, and the output is the analyzed message data.

[0719] Step 3:

[0720] The server runs a scoring algorithm based on the parsed message data to determine the importance of each message. For example, a message from a superior containing the keyword "urgent" would be given a high score. The input to this process is the parsed message data, and the output is a scored message list.

[0721] Step 4:

[0722] The server sorts the scored messages based on their priority, then converts them into a data format (e.g., JSON) for notification to the user device. The input of this step is the scored message list, and the output is the sorted message data.

[0723] Step 5:

[0724] The server sends the aligned message data to the user terminal through the API endpoint. The input of this process is the aligned message data, and the output is the notification data sent to the user terminal.

[0725] Step 6:

[0726] The device receives and analyzes the sorted message data received from the server. It then displays the messages in order of priority. High-priority messages are displayed as pop-up notifications, and medium-priority messages are listed on the in-app dashboard. The input of this step is the notification data sent from the server, and the output is the message list displayed to the user.

[0727] Step 7:

[0728] The server periodically collects new content in categories that the user is interested in. For example, it uses an RSS feed or an API to get the latest news and articles. The input for this step is a feed URL or an API endpoint, and the output is a list of new content.

[0729] Step 8:

[0730] The server analyzes the collected new content and extracts keywords related to publication time, author, title, and content. Natural language processing and text analysis techniques are used in this step. The input is a list of new content, and the output is the analyzed content data.

[0731] Step 9:

[0732] The server runs a scoring algorithm based on the analyzed content data to determine the importance of each piece of content, taking into account the user's interest categories and specific keywords. The input is the analyzed content data, and the output is a scored content list.

[0733] Step 10:

[0734] The server sorts the content based on priority, converts it into a data format, and then notifies the user device through an API endpoint. The input of this step is a scored content list, and the output is the sorted content data.

[0735] Step 11:

[0736] The terminal displays the sorted content data received from the server. High-priority content is notified immediately, and low-priority content is displayed in a list on the dashboard. The input is the notification data sent from the server, and the output is the content list displayed to the user.

[0737] In this way, users can quickly and efficiently review important messages and content.

[0738] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0739] This invention is a system incorporating an emotion engine that recognizes the user's emotions, and is particularly intended to organize messages received from email accounts and multiple communication tools in order of priority and promptly notify the user of important messages. To implement this invention, the system is constructed according to the following procedure.

[0740] System configuration

[0741] The system consists of the following main components:

[0742] 1. Server

[0743] Message Collection Module

[0744] Message Analysis Module

[0745] Priority Determination Algorithm

[0746] Message Organizer Module

[0747] Emotion Engine

[0748] 2. User Device

[0749] Program processing

[0750] Message Collection

[0751] The server periodically retrieves new messages using the APIs of the email accounts and communication tools that the user has linked. For example, the server operates the Gmail API to fetch new emails from the user's mailbox. Similarly, the server calls the APIs of Slack, Microsoft Teams, etc. to retrieve the user's chat messages.

[0752] Message Parsing

[0753] The acquired messages are analyzed by the server's message analysis module. During the analysis, the sender, time of sending, content, tags, and mention information are extracted. In addition, an emotion engine recognizes the user's emotions from the message content and identifies negative emotions.

[0754] Priority determination

[0755] The server then runs a priority determination algorithm based on the information obtained from the message analysis. This algorithm scores the importance of each message, taking into account the presence or absence of mentions and important keywords, the importance of the sender, and the emotional information detected by the emotional engine. For example, an email from a superior containing the keyword "urgent" and also detecting negative emotions will be given a higher score.

[0756] Message organization

[0757] Based on the determined score, the server sorts the messages in order of priority, then converts them into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[0758] User Notification and Display

[0759] The user terminal receives the message list from the server and notifies the user of messages in descending order of priority. For example, the terminal displays a pop-up notification for high-priority messages and displays a list of medium-priority messages on a dashboard.

[0760] Specific examples

[0761] 1. Get new emails

[0762] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[0763] 2. Message Analysis

[0764] The server analyzes the received emails and extracts the sender (boss), time of sending, and the keyword "urgent." It also calls Slack's API to retrieve project progress report messages in which the user is mentioned. It then uses an emotion engine to recognize the user's emotions from the message content and identifies messages with negative emotions.

[0765] 3. Priority determination

[0766] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion engine.

[0767] 4. Message organization and notifications

[0768] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[0769] 5. User Notices

[0770] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[0771] 6. User Response

[0772] Users receive notifications and respond to messages with high priority. First, they check the "urgent" email from their boss, which is the most important, and respond or reply to it. Then, they check and respond to messages in Slack.

[0773] This allows users to quickly and efficiently review important messages and take appropriate action. The emotion engine allows for quicker responses to messages that are particularly urgent or emotionally important.

[0774] The processing flow will be explained below.

[0775] Step 1:

[0776] Get new messages

[0777] The server periodically calls the APIs of the email accounts and communication tools that the user has previously linked. For example, the server uses the Gmail API to retrieve new emails and the Slack API to collect new chat messages.

[0778] Step 2:

[0779] Preparing for Message Analysis

[0780] The server prepares the retrieved message data for analysis by loading the contents of the email or chat message into memory and extracting the information needed for analysis (such as sender, time of sending, and content).

[0781] Step 3:

[0782] Extracting caller information

[0783] The server extracts sender information from message metadata, for example by parsing the "From" field in emails or the sender ID of chat messages.

[0784] Step 4:

[0785] Extracting the call time

[0786] The server obtains the message's sending time by checking the "Date" field in the email or the timestamp of the chat message.

[0787] Step 5:

[0788] Message content analysis

[0789] The server analyzes the message content to detect specific keywords or phrases, such as "urgent," "important," or "meeting."

[0790] Step 6:

[0791] Detecting tag and mention information

[0792] The server detects tags and mentions in messages by parsing @mentions in Slack messages and CC / BCC fields in emails.

[0793] Step 7:

[0794] Emotion analysis using an emotion engine

[0795] The server uses an emotion engine to recognize the user's emotions from the message content, and identifies negative emotions using text and voice analysis.

[0796] Step 8:

[0797] Priority Scoring

[0798] The server then scores the importance of each message based on the analysis results, taking into account factors such as whether the message contains specific keywords, whether it is from an important sender, and whether the emotion engine detects negative emotions.

[0799] Step 9:

[0800] Sorting by priority

[0801] The server sorts messages by priority based on the scoring results, with messages with higher scores coming to the top.

[0802] Step 10:

[0803] Converting to data format

[0804] The server converts the sorted message list into an appropriate data format (for example, JSON format) for transmission to the user terminal.

[0805] Step 11:

[0806] Send to user terminal

[0807] The server sends the sorted message list to the user's device and transfers the data using an API endpoint.

[0808] Step 12:

[0809] Displaying pop-up notifications

[0810] The device analyzes the message data received from the server and displays pop-up notifications for the most important messages, such as emails that require immediate attention.

[0811] Step 13:

[0812] Display on the dashboard

[0813] The user terminal displays a list of messages sorted by priority on the dashboard, in a list format for easy user access.

[0814] Step 14:

[0815] User message confirmation

[0816] The user receives notifications from the device and checks the messages in order of priority. First, open the most important message and take the necessary action.

[0817] Step 15:

[0818] Taking corresponding actions

[0819] Users can respond immediately to important messages, such as replying to an "urgent" email from their boss or checking the progress of a project.

[0820] Step 16:

[0821] Organizing confirmed messages

[0822] The device will archive or move messages that you have acknowledged to another folder, remove the acknowledged message from the list, and display the next important message.

[0823] This allows users to quickly and efficiently view and respond to important messages, and the emotion engine allows for quick responses to messages that are particularly urgent or emotionally important.

[0824] Example 2

[0825] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0826] In today's business environment, users receive a large volume of messages from multiple electronic communication channels, and they need to be able to quickly and efficiently identify and respond to important messages. However, doing this manually is extremely time-consuming, so a system is needed that can automatically identify high-priority messages and notify users immediately. It is also necessary to be able to determine the emotional content of messages and prioritize messages with specific emotions.

[0827] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for calling an API of an electronic communication means for collecting new messages, a means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information, a means for scoring the importance of the messages based on the analysis results and determining their priority, a means for recognizing emotional information from the message content using an emotion engine and reflecting it in the priority determination, a means for sorting messages based on their priority and converting them into a data format for notification to a user terminal, and a means for receiving notifications from the user terminal and displaying messages in order of priority. This allows users to quickly and efficiently identify important or emotionally significant messages and respond appropriately.

[0828] "Electronic communication means" refers to communication means for sending and receiving messages, such as email and chat tools.

[0829] "API" stands for Application Programming Interface, an interface for exchanging information and commands between different software programs.

[0830] An "emotion engine" is an algorithm or software module that analyzes a user's emotions from the content of a message and assigns an emotion score such as positive, negative, or neutral.

[0831] "Scoring" is the process of quantifying importance and priority based on analyzed message information.

[0832] "Priority" is an index of importance for determining the processing order of messages.

[0833] "Mention information" is information in which a specific user or group is mentioned in a message.

[0834] A "tag" is a keyword or classification information that is added to a message and indicates the content or category of the message.

[0835] A "data format" is a standard that defines the structure and form of data, such as JSON or XML.

[0836] A "user terminal" is a device used by a user, such as a computer, smartphone, or tablet.

[0837] This invention is a system incorporating an emotion engine that recognizes the user's emotions, and in particular, is designed to automatically determine the importance of new messages and promptly notify the user. To implement this invention, the system is constructed according to the following procedure.

[0838] System configuration

[0839] The system consists of the following main components:

[0840] 1. Server

[0841] Message collection module: Calls the API of the electronic communication means to collect messages.

[0842] Message analysis module: Analyzes the content of collected messages and extracts sender, time of sending, content, tags, and mention information.

[0843] Priority determination algorithm: Based on the analysis results, the importance of the message is scored and the priority is determined.

[0844] Emotion engine: Recognizes emotional information from the content of messages and reflects it in determining priority.

[0845] Message sorting module: Sorts messages based on priority and converts them into a data format for notification to the user terminal.

[0846] 2. User Device

[0847] Notification system: Receives notifications from the server and displays messages based on priority.

[0848] System processing flow

[0849] Message Collection

[0850] The server periodically calls the API of the electronic communication means linked by the user to obtain new messages. Specifically, the server collects messages using the Gmail API, Slack API, Microsoft Teams API, etc.

[0851] Message Parsing

[0852] The acquired messages are analyzed by the server's message analysis module. This analysis extracts information about the sender, time of sending, content, tags, and mentions. An emotion engine is also used to recognize emotional information from the message content. Specifically, an emotional score such as positive, negative, or neutral is generated.

[0853] Priority determination

[0854] Based on the analyzed information, the server runs a priority determination algorithm. This algorithm scores the importance of each message by combining mention information, the presence or absence of important keywords, the importance of the sender, and emotional information. For example, an email from a boss containing the keyword "urgent" will be given a high score. Messages containing negative emotions will also be given a high score.

[0855] Message organization and notifications

[0856] Scored messages are sorted by priority and converted into a data format (e.g., JSON format) by the server, and then sent to the user's device via an API endpoint.

[0857] User Notification and Display

[0858] The user terminal receives the message list from the server and notifies the user of messages in order of priority. Specifically, the terminal displays a pop-up notification, and displays a list of messages with medium priority on the dashboard.

[0859] User response

[0860] After receiving a notification, users can check and respond to messages with higher priority. For example, they can check and respond to the most important "urgent" email from their boss. Then, they can check and respond to messages in Slack.

[0861] This allows users to quickly and efficiently check important messages and take appropriate action. The emotion engine also enables quick responses to messages that are particularly urgent or emotionally important.

[0862] Specific examples

[0863] 1. Get new emails

[0864] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[0865] 2. Message Analysis

[0866] The server analyzes the received emails and extracts the sender (boss), time of sending, and the keyword "urgent." It also calls Slack's API to retrieve project progress report messages in which the user is mentioned. It then uses an emotion engine to recognize emotional information from the message content and identify messages with negative emotions.

[0867] 3. Priority determination

[0868] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion engine.

[0869] 4. Message organization and notifications

[0870] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[0871] 5. User Notices

[0872] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[0873] 6. User Response

[0874] Users receive notifications and respond to messages with high priority. First, they check the "urgent" email from their boss, which is the most important, and respond or reply to it. Then, they check and respond to messages in Slack.

[0875] Prompt Sentence Examples

[0876] An example of a prompt for a generative AI model might be, "If I receive an urgent message from my boss, please advise me on how I should respond first."

[0877] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0878] Step 1: Message Collection

[0879] The server periodically calls the API of the electronic communication method linked by the user to obtain new messages. Specifically, the server collects messages using the Gmail API, Slack API, Microsoft Teams API, etc. The input required is the user's authentication information and API call parameters. The output is a list of collected unread messages. This list is stored in the server's database.

[0880] Step 2: Message analysis

[0881] The server passes the collected messages to a message analysis module, which uses a natural language processing (NLP) library (such as NLTK or SpaCy) to analyze the messages. The input is a list of unanalyzed messages. During the analysis process, sender, time of message, content, tags, and mention information are extracted. The output is a list of analyzed messages with this information, along with an emotion score for each message generated by the emotion engine.

[0882] Step 3: Prioritization

[0883] The server runs a priority determination algorithm using the parsed message list. The parsed message list, including the sender, time of sending, tags, mention information, and sentiment information, is required as input. The priority determination algorithm scores messages based on important keywords, the importance of the sender, and sentiment information. For example, a message from a boss containing the keyword "urgent" is assigned a high score. The output is a list containing the priority score for each message.

[0884] Step 4: Message organization and notifications

[0885] The server sorts messages based on their priority scores and converts them into a data format (e.g., JSON format) for notification to the user device. A message list with priority scores is required as input. The server converts this into JSON format and sends it to the user device via an API endpoint. The output is the notification data sent to the user device.

[0886] Step 5: User Notification and Display

[0887] The user terminal parses the JSON formatted notification data received from the server. Notification data is required as input. The user terminal analyzes this data and notifies the user of messages in order of priority. Specifically, a pop-up notification is displayed for high-priority messages, and messages with medium priority are listed on the dashboard. The output is a message that is visually displayed to the user.

[0888] Step 6: User Action

[0889] The user receives notifications from their device and checks and responds to messages in order of priority. The input requires notification information from the user's device. The user checks the most important "urgent" email from their boss and replies or responds to it. Then, the user checks the message in Slack and replies to it. The output is the reply or response sent by the user, which is sent again to the server and the message status is updated.

[0890] Through these steps, users can quickly and efficiently review important messages and respond appropriately. The emotion engine allows users to respond quickly to especially urgent or emotionally significant messages.

[0891] (Application example 2)

[0892] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0893] Conventional email and communication tool message management systems simply retrieve messages and determine their importance. However, they were unable to prioritize or notify messages taking into account the user's emotions, making it difficult to respond quickly to important messages, especially those with negative sentiment. This made it difficult for users to quickly understand and respond to truly important messages. Furthermore, in physical stores, it was not possible to understand customer emotions in real time and respond appropriately. This resulted in a decline in service quality and hindered improvements in customer satisfaction.

[0894] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0895] In this invention, the server includes: a means for calling an email account and a communication tool interface for collecting new messages; a means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and reference information; a means for ranking the importance of the messages and determining their priority based on the analysis results; a means for sorting the messages based on priority and converting them into a data format for notification to the user terminal; a means for receiving notifications from the user terminal and displaying the messages in order of priority; and a means for incorporating an emotion recognition engine that recognizes the user's emotions into the interface, analyzing the collected emotion data, and assigning a higher priority to messages with negative emotions. This allows users to quickly identify and respond to important messages, especially those with negative emotions. It also enables real-time monitoring of customer emotions in physical stores and prompt appropriate responses.

[0896] "New Messages" are the most recent messages received through your email account or communication tool.

[0897] An "email account" is an online account that a user uses to send and receive email.

[0898] A "communication tool" is a platform or application that users use to send and receive messages to each other.

[0899] An "interface" is a connection point or interface where parts of a system communicate and exchange data with each other.

[0900] "Analysis" is the act of examining the contents of collected messages in detail and extracting necessary information.

[0901] "Originator" is the user or account that sent the message.

[0902] "Time of Sent" is the exact date and time the message was sent.

[0903] "Content" is the text or information contained in the message.

[0904] A "tag" is a specific keyword or label added to a message that is used to categorize the message.

[0905] "Reference information" is metadata that indicates the related information referenced by a message or the relevance of a message.

[0906] "Ranking" refers to classifying the importance of a message into a numerical value or category based on analyzed information.

[0907] "Priority" is an index that indicates the importance or urgency of a message.

[0908] A "data format" is a format that structures message data in a specific format to facilitate storage and transmission.

[0909] An "emotion recognition engine" is a software component that analyzes the emotions of users and customers and recognizes their emotional state.

[0910] "Negative emotions" are negative emotional states such as anger, sadness, and fear.

[0911] This invention is a system that includes an emotion recognition engine that recognizes user emotions, and is particularly intended to organize messages received from email accounts and multiple communication tools in order of priority and promptly notify users of important messages. It can also be applied to applications that perform real-time emotion recognition in customer service at brick-and-mortar stores to improve the quality of service.

[0912] System configuration

[0913] The system consists of the following main components:

[0914] 1. Server

[0915] Message collection module: Calls the interface of the email account and communication tool and periodically retrieves new messages.

[0916] Message Analysis Module: Analyzes collected messages and extracts sender, time of sending, content, tags, and reference information.

[0917] Priority determination algorithm: Based on the analysis results, the importance of messages is ranked and their priority is determined.

[0918] Message sorting module: Sorts messages based on priority and converts them into a data format for notification to the user terminal.

[0919] Emotion Recognition Engine: Analyzes emotional data and prioritizes messages with negative emotions.

[0920] 2. User Device

[0921] Notification receiving component: Receives and displays a list of messages received from the server, organized by priority.

[0922] Emotion-aware interface: Analyze customer emotions in real time and respond appropriately.

[0923] Program processing

[0924] Message Collection

[0925] The server periodically retrieves new messages using the APIs of the email account and communication tool. For example, the server operates a common email API or chat tool API to retrieve and collect new emails and chat messages.

[0926] Message Parsing

[0927] The server analyzes messages acquired by the server using a message analysis module. During the analysis process, the sender, time of message, content, tags, and reference information are extracted. Furthermore, an emotion recognition engine recognizes the emotions of users and customers from the message content and identifies negative emotions.

[0928] Priority determination

[0929] The server then runs a priority algorithm based on the analyzed information to rank the importance of each message. For example, messages from superiors or messages containing keywords like "urgent" are given a higher score. The priority is further adjusted based on negative emotions detected by an emotion recognition engine.

[0930] Message organization

[0931] Based on the determined score, the server sorts the messages in order of priority and then converts them into an appropriate data format for notification to the user device. The sorted message list is sent to the user device through an API endpoint.

[0932] User Notification and Display

[0933] The user device receives a message list from the server and notifies them in order of priority. For example, the device displays a pop-up notification for high-priority messages and displays a list of other messages. It also monitors customer sentiment in real time in physical stores and sends notifications to promptly take appropriate action.

[0934] Specific examples

[0935] 1. Get new emails

[0936] The server uses a general-purpose email API to retrieve new emails from the user's mailbox. For example, you might receive an email from your boss marked "urgent."

[0937] 2. Message Analysis

[0938] The server analyzes the received email and extracts the sender, time of sending, and the keyword "urgent." It then uses an emotion recognition engine to recognize the user's emotions from the message content and identify negative emotions.

[0939] 3. Priority determination

[0940] The server then runs a scoring algorithm based on the analysis results, assigning a high score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion recognition engine.

[0941] 4. Message organization and notifications

[0942] The server sorts the messages by priority and sends them to the user terminal in the appropriate data format.

[0943] 5. User Notices

[0944] The user device analyzes the received messages and notifies the user of high-priority messages via a pop-up notification. In addition, in physical stores, the system detects negative customer sentiment and notifies the user to take prompt action.

[0945] Prompt Sentence Examples

[0946] Below are some example prompts to input to a generative AI model:

[0947] "A customer changes their facial expression at the counter. Develop a system that recognizes the customer's emotions from image and audio data and notifies an employee if a negative emotion (anger, sadness, fear) is detected. The hardware used is smart glasses, and the software used is TensorFlow / Keras."

[0948] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0949] Step 1:

[0950] Message collection

[0951] The server calls the interfaces of the email account and communication tool to periodically retrieve new messages. Specifically, the server periodically collects new emails and chat messages using the API of each interface. The input is the API information of the email account and chat tool, and the output is a list of collected unread messages.

[0952] Step 2:

[0953] Message Parsing

[0954] The server analyzes the collected message content. Specifically, it uses a message analysis module to extract sender, time of message, content, tags, and reference information. It then uses an emotion recognition engine to extract emotional data from the message content and identify negative emotions. The input is a list of newly arrived messages, and the output is structured data and emotional data of the messages.

[0955] Step 3:

[0956] Determining Priority

[0957] The server runs a priority determination algorithm based on the analyzed information to rank the importance of messages. Specifically, it evaluates important keywords, the message sender, the time of sending, emotional data, etc., and assigns a score to each message. The input is the structured data and emotional data of the message, and the output is a list of scored messages.

[0958] Step 4:

[0959] Organizing messages

[0960] The server sorts the messages in order of priority based on the determined scores. Specifically, it sorts the array based on the score of each message and then converts it into a data format for notification. The input is a list of scored messages, and the output is a list of messages sorted by priority.

[0961] Step 5:

[0962] Message notifications

[0963] The user terminal receives and displays the message list sorted by priority received from the server. Specifically, the terminal displays a pop-up notification for high-priority messages and displays the list for other messages. The input is the list of messages sorted by priority, and the output is the notification to the user and the messages displayed.

[0964] Step 6:

[0965] Customer Sentiment Detection

[0966] The user terminal (smart glasses) captures the customer's facial expressions and voice in real time and inputs them into the emotion recognition engine. Specifically, the smart glasses use a camera and microphone to send the captured image and voice data to the emotion recognition engine. The input is image and voice data, and the output is detected emotion data.

[0967] Step 7:

[0968] Customer response notification

[0969] The user device (smart glasses) identifies customers with negative emotions based on the emotion data received from the emotion recognition engine and notifies employees. Specifically, a warning message is displayed on the smart glasses' display, urging employees to take prompt action. The input is the detected emotion data, and the output is a warning notification to employees.

[0970] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0971] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0972] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0973] [Third embodiment]

[0974] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0975] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0976] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0977] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0978] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0979] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0980] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0981] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0982] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0983] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0984] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0985] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0986] This invention relates to a system that organizes messages received from a user's email account or multiple communication tools in order of priority and promptly notifies the user of important messages. To implement this invention, the system is constructed according to the following procedure.

[0987] System configuration

[0988] The system consists of the following main components:

[0989] 1. Server

[0990] Message Collection Module

[0991] Message Analysis Module

[0992] Priority Determination Algorithm

[0993] Message Organizer Module

[0994] 2. User Device

[0995] Program processing

[0996] Message Collection

[0997] The server first periodically retrieves new messages using the APIs of the email accounts and communication tools linked to the user. For example, the server operates the Gmail API to fetch new emails from the user's mailbox. Similarly, the server calls the APIs of Slack, Microsoft Teams, and other services to retrieve the user's chat messages.

[0998] Message Parsing

[0999] The captured messages are then analyzed by the server's message analysis module. During the analysis, information about the sender, time of message, content, tags, and mentions is extracted. For example, mentions related to a specific project or messages containing keywords such as "urgent" or "important" are extracted.

[1000] Priority determination

[1001] The server then runs a priority algorithm based on the information obtained from the message analysis. This algorithm scores each message based on data such as mentions, the presence of important keywords, and the importance of the sender. For example, an email from a boss containing the keyword "urgent" would be given a high score.

[1002] Message organization

[1003] Based on the determined score, the server sorts the messages in order of priority, then converts them into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[1004] User Notification and Display

[1005] The user terminal receives the message list from the server and notifies the user of messages in descending order of priority. For example, the terminal displays a pop-up notification for high-priority messages and displays a list of medium-priority messages on a dashboard.

[1006] Specific examples

[1007] 1. Get new emails

[1008] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[1009] 2. Message Analysis

[1010] The server analyzes the emails and extracts the sender (boss), the time of sending, and the keyword "urgent." It also calls the Slack API to retrieve project progress report messages in which the user is mentioned.

[1011] 3. Priority determination

[1012] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it also takes into account mention information in Slack messages to set priorities.

[1013] 4. Message organization and notifications

[1014] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[1015] 5. User Notices

[1016] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[1017] 6. User Support

[1018] Users receive notifications and can check and respond to messages with higher priority.

[1019] This allows users to quickly and efficiently view and respond to important messages.

[1020] The processing flow will be explained below.

[1021] Step 1:

[1022] Get new messages

[1023] The server periodically calls the APIs of email accounts and communication tools that the user has registered in advance. For example, the server uses the Gmail API to obtain new emails and the Slack API to collect new chat messages.

[1024] Step 2:

[1025] Preparing for Message Analysis

[1026] The server prepares the retrieved message data for analysis by loading a portion of the email or chat message into memory and extracting the information needed for analysis (such as sender, time of message, and content).

[1027] Step 3:

[1028] Extracting caller information

[1029] The server extracts originator information from message metadata, for example, identifying the "From" field in an email or the sender ID in a chat message.

[1030] Step 4:

[1031] Extracting the call time

[1032] The server obtains the message's sending time by checking the "Date" field in the email or the timestamp of the chat message.

[1033] Step 5:

[1034] Message content analysis

[1035] The server analyzes the message content to detect specific keywords or phrases, such as "urgent," "important," or "meeting."

[1036] Step 6:

[1037] Detecting tag and mention information

[1038] The server detects tags and mentions in messages by parsing @mentions in Slack messages and CC / BCC fields in emails.

[1039] Step 7:

[1040] Priority Scoring

[1041] The server then scores the importance of each message based on the analysis results, taking into account whether it contains specific keywords, whether it is from an important sender, whether it is mentioned, and so on.

[1042] Step 8:

[1043] Sorting by priority

[1044] The server sorts messages by priority based on the scoring results, with messages with higher scores coming to the top.

[1045] Step 9:

[1046] Converting to data format

[1047] The server converts the sorted message list into an appropriate data format (for example, JSON format) for transmission to the user terminal.

[1048] Step 10:

[1049] Send to user terminal

[1050] The server sends the sorted message list to the user's device and transfers the data using an API endpoint.

[1051] Step 11:

[1052] Displaying pop-up notifications

[1053] The device analyzes the message data received from the server and displays pop-up notifications for the most important messages, such as emails that require immediate attention.

[1054] Step 12:

[1055] Display on the dashboard

[1056] The user terminal displays a list of messages sorted by priority on the dashboard, in a list format for easy user access.

[1057] Step 13:

[1058] User message confirmation

[1059] The user receives notifications from the device and checks the messages in order of priority. First, open the most important message and take the necessary action.

[1060] Step 14:

[1061] Taking corresponding actions

[1062] Users can respond immediately to important messages, such as replying to an "urgent" email from their boss or checking the progress of a project.

[1063] Step 15:

[1064] Organizing confirmed messages

[1065] The device will archive or move messages that you have acknowledged to another folder, remove the acknowledged message from the list, and display the next important message.

[1066] By the above processing steps, the system of the present invention can promptly and efficiently present important messages to the user and prompt them to take action.

[1067] Example 1

[1068] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1069] Nowadays, many users use multiple messaging methods (e.g., email and chat apps). As a result, there is a high possibility that important messages will be buried and overlooked. This leads to problems such as reduced work efficiency and delayed important notifications. The present invention aims to centrally collect messages from different messaging methods, organize and notify them according to importance, and enable users to quickly understand important messages and respond appropriately.

[1070] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1071] In this invention, the server includes means for calling an API of a messaging account and a communication tool to collect new messages, means for analyzing the contents of the collected messages and extracting sender, sending time, content, tag, and mention information, means for scoring the importance of the messages based on the analysis results and determining their priority, means for sorting the messages based on their priority and converting them into a data format for notification to a user device, and means for receiving notifications from the user device and displaying the messages in order of priority. This makes it possible to centrally manage and organize messages from different messaging means and quickly notify users of important messages.

[1072] A "new message" is the most recent message received in a user's messaging account or communication tool.

[1073] A "messaging account" is one or more email accounts or chat application accounts used by a user.

[1074] "Communication tools" are platforms and applications (e.g., email, chat applications) that users use to send and receive messages.

[1075] "API" is an abbreviation for Application Programming Interface, a set of rules and tools for exchanging data between different software programs.

[1076] "Means for collection" refers to the server's functionality for obtaining new messages using the API of messaging accounts and communication tools.

[1077] The "analyzing means" is a function of the server for analyzing the content of collected messages and extracting sender, sending time, content, tag, and mention information.

[1078] An "originator" is a person or entity that sends a message.

[1079] "Time of sending" is the date and time the message was sent.

[1080] "Content" refers to the message body and the information contained therein.

[1081] A "tag" is an identifier or keyword that is added to a message.

[1082] "Mention information" is information in which a specific user is mentioned or named in a message.

[1083] The "scoring means" is a server function for quantifying the importance of messages based on the analysis results and setting priorities.

[1084] A "sorting means" is a server's capability to order messages based on priority.

[1085] The "means for converting into a data format" is a function of the server that converts the sorted messages into a format appropriate for notifying the user device.

[1086] A "user device" is an electronic device (e.g., smartphone, tablet, computer) that a user uses to receive and view messages.

[1087] The "means for receiving notifications" is a function that enables the user device to receive notifications from the server.

[1088] The "means for displaying" is a function that allows the user device to visually present messages to the user in order of priority.

[1089] The present invention relates to a system that automatically collects new messages from a user's multiple messaging accounts and communication tools, analyzes and scores them, and then promptly notifies the user of high-priority messages. The system for implementing this invention is composed of a server and a user terminal.

[1090] Overall structure

[1091] Server: Message collection module, message analysis module, priority determination algorithm, message organization module

[1092] User terminal: message display module, notification module

[1093] The specific hardware and software used

[1094] Server: We recommend a physical server or a cloud-based virtual server with high-performance processors, RAM, and storage, such as Amazon Web Services (AWS) or Google Cloud Platform.

[1095] Communication tool APIs: Gmail API, Slack API, Microsoft Teams API, etc. are used.

[1096] Natural language processing software: spaCy, NLTK, Google Cloud Natural Language API, etc. are used for message analysis.

[1097] Data format: Convert to JSON format and send / receive messages.

[1098] Message Collection

[1099] The server first periodically retrieves new messages by calling the APIs of the email accounts and communication tools that the user has linked. For example, it uses the Gmail API to retrieve new emails from Gmail. Similarly, it calls the APIs of Slack and Microsoft Teams to retrieve chat messages.

[1100] Specifically, the server calls the Gmail API every minute to retrieve new emails for the user. An example of the API request used is:

[1101] "GET https: / / www.googleapis.com / gmail / v1 / users / {userId} / messages"

[1102] Message Parsing

[1103] Messages retrieved by the server are analyzed by the message analysis module. During the analysis process, information about the sender, time of sending, content, tags, and mentions is extracted. In particular, natural language processing technology is used to extract important keywords (e.g., "urgent" and "important") from the message body.

[1104] Specifically, analysis is performed using the Google Cloud Natural Language API to extract the keyword "urgent" from the email content.

[1105] Message Priority Determination

[1106] Based on the analyzed information, the server uses a priority determination algorithm to score the importance of the message. This algorithm takes into account the importance of the sender, keywords in the message content, mention information, etc. For example, an email from a boss containing the keyword "urgent" will be given the highest score.

[1107] An example of a specific scoring rule:

[1108] if "urgent" in message and sender == "boss": priority_score += 10

[1109] Message organization and notifications

[1110] Based on the priority score, the server sorts the messages in ascending or descending order and sends them to the user's device in JSON format, so that the sorted messages are notified to the user's device.

[1111] User Notification and Display

[1112] The user device analyzes the received message list and notifies the user in order of priority, displaying a pop-up notification for high-priority messages and displaying a list of medium-priority messages on the dashboard.

[1113] Specifically, high-priority messages are displayed on the user's device screen in the following format:

[1114] "[Urgent Notification] Supervisor: Please respond immediately"

[1115] User Support

[1116] Users receive notifications and can view and respond to messages with higher priority, allowing them to quickly and efficiently view and respond to important messages.

[1117] This allows users to prevent messages from being missed and quickly grasp important information, thereby improving work efficiency.

[1118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1119] Step 1:

[1120] The server retrieves new messages by calling the API of the messaging account and communication tool linked by the user. Specifically, it fetches new messages using the Gmail API, Slack API, Microsoft Teams API, etc. It sends an API request and receives the new messages in JSON format as a response. For example, in the case of the Gmail API, the request looks like this:

[1121] GET https: / / www.googleapis.com / gmail / v1 / users / {userId} / messages

[1122] Input: User's API credentials

[1123] Output: New message data in JSON format

[1124] What happens: The server sends an API request every minute to fetch new messages.

[1125] Step 2:

[1126] The server analyzes the collected message content and extracts information about the sender, time of sending, content, tags, and mentions. This analysis is performed using natural language processing software (e.g., spaCy, NLTK, Google Cloud Natural Language API). Important keywords and flags (e.g., "urgent" and "important") are extracted from the message body.

[1127] Input: New message data in JSON format

[1128] Output: Analysis results (sender, time of posting, content, tags, mention information)

[1129] What it does: The server uses natural language processing software to extract important keywords from the message body.

[1130] Step 3:

[1131] The server then runs a priority determination algorithm based on the analysis results to score the importance of the message. The algorithm takes into account the importance of the sender, keywords in the message content, mention information, etc. For example, an email from a superior containing the keyword "urgent" will be given a high score.

[1132] Input: Analysis results

[1133] Output: Priority score for each message

[1134] What happens: The server scores using the following rules:

[1135] if "urgent" in message and sender == "boss": priority_score += 10

[1136] Step 4:

[1137] The server sorts messages based on their priority scores, converts the sorted messages into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[1138] Input: A message and its priority score

[1139] Output: A sorted list of messages in JSON format

[1140] What happens: The server sorts the messages using an algorithm such as quicksort and serializes them into JSON format.

[1141] Step 5:

[1142] The user device analyzes the message list received from the server and notifies the user of messages in order of priority, displaying a pop-up notification for high-priority messages and displaying a list of medium-priority messages on the dashboard.

[1143] Input: A sorted list of messages in JSON format

[1144] Output: Visual notifications and messages

[1145] What happens: The user device generates a pop-up notification to immediately display high priority messages.

[1146] Step 6:

[1147] Users receive notifications and can check and respond to messages in order of priority, allowing them to quickly and efficiently check and respond to important messages.

[1148] Input: Notification from user device

[1149] Output: User action (acknowledgement and response to message)

[1150] What happens: The user clicks on the pop-up notification to view the message details and take the necessary action.

[1151] This allows the user to respond efficiently without missing any important messages.

[1152] (Application example 1)

[1153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1154] In modern society, many users use not only email and chat tools but also various content distribution services. However, these messages and content are often received in large quantities at once, putting users at risk of missing important information. Furthermore, the lack of an efficient way to quickly find important content that is relevant to users' interests further exacerbates information overload. Therefore, there is a need for a system that can efficiently organize multiple messages and content and prioritize notifications of important content.

[1155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1156] In this invention, the server includes means for calling an API of an email account and a communication tool to collect new messages, means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information, means for scoring the importance of the messages and determining their priority based on the analysis results, means for sorting the messages based on their priority and converting them into a data format for notification to a user terminal, means for receiving notifications from the user terminal and displaying the messages in order of priority, means for collecting new content from a content distribution service and extracting keywords for the sending time, author, title, and content, means for scoring the importance of the content based on categories of interest to the user or specific keywords and determining its priority, and means for notifying the user of the content sorted in order of priority. This allows users to quickly and efficiently check important information from multiple communication tools and content distribution services.

[1157] "New Messages" means the most recent messages received through your email account and communication tools.

[1158] "Means for calling an API" refers to a method for using an application program interface (API) to obtain necessary data from external services or databases.

[1159] "Means for analyzing the contents of a message" refers to a method for extracting and analyzing internal information of a received message (such as sender, time of sending, content, tags, and mention information).

[1160] The "means for scoring and determining priority" is a method for evaluating the importance of each message or content numerically based on the analyzed information and determining its priority.

[1161] The "means for converting into a data format" refers to a method for converting the organized messages or content into an appropriate data format (e.g., JSON format) for transfer to a user terminal.

[1162] The "means for notifying the user terminal" is a method for quickly notifying the user terminal of high-priority messages or content.

[1163] The "means for displaying messages in order of priority" is a method for displaying messages or content sorted according to importance in an orderly manner on a user terminal.

[1164] "New content" refers to information such as the latest news, articles, and videos obtained from content distribution services.

[1165] The "means for scoring the importance of content based on categories or specific keywords" is a method for evaluating the importance of each piece of content based on the presence or absence of interest categories or keywords set by the user.

[1166] The "means for notifying the user of the content sorted in order of priority" is a method for ranking the rated content according to the user's level of interest and notifying the user of the content in that order.

[1167] To specifically implement this invention, it is necessary to build a system according to the following detailed description. This system consists of two main components: a server and a user terminal. Below, we will explain how these components work together to process and calculate data.

[1168] Server configuration and operation

[1169] Hardware and Software

[1170] The server uses a cloud service such as AWS, and the software running on the server includes Python, Flask (a web framework), feedparser (an RSS analysis library), Firebase (a cloud messaging service), and so on.

[1171] Message and content collection

[1172] The server periodically collects new messages and content by calling the APIs of the email accounts and communication tools registered by the user. For example, it uses RSS feeds and APIs to obtain the latest news and articles. It also collects emails and chat messages using APIs such as Gmail and Slack.

[1173] Analysis and Scoring

[1174] The collected messages and content are analyzed on the server. During this analysis, information about the sender, time of posting, content, tags, and mentions is extracted. In the case of content, the category, author, title, and keywords are also extracted. Next, a scoring algorithm is run based on the analysis results to score the importance of each message or piece of content. This takes into account the user's interests and set keywords.

[1175] Determining priority and converting to data format

[1176] The scored data is sorted by priority, converted into an appropriate data format (e.g., JSON format) for notification to the user device, and sent to the user device via an API endpoint.

[1177] Configuration and operation of user terminal

[1178] Hardware and Software

[1179] The user device is typically a smartphone with a dedicated app installed, which receives notifications from the server via Firebase Cloud Messaging and processes them.

[1180] Notifications and Displays

[1181] The user device analyzes the data received from the server and displays messages and content in order of priority. For example, high-priority messages and content are displayed as pop-up notifications, while medium-priority messages and content are displayed as a list on the app's dashboard.

[1182] Examples of specific examples and prompts

[1183] For example, if a user wants to be notified of news containing keywords such as "important" or "urgent" in a news app, the user might input the following prompt sentence to the server:

[1184] Collect the latest news in categories that interest you (e.g., "Technology," "Health," "Business") and prioritize news that contains specific keywords (e.g., "Important," "Urgent").

[1185] In this way, users can efficiently view important messages and content that are relevant to their interests.

[1186] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1187] Step 1:

[1188] The server calls the APIs of the email accounts and communication tools registered by the user to collect new messages. At this time, the server uses APIs such as Gmail and Slack to periodically obtain data at specified intervals. The input for this process is the user's API authentication information, and the output is a list of new messages.

[1189] Step 2:

[1190] The server analyzes the collected new messages. Specifically, it extracts the sender, time, content, tags, and mention information for each message. In this step, it uses natural language processing technology to analyze the message text and extract the necessary information. The input is a list of new messages, and the output is the analyzed message data.

[1191] Step 3:

[1192] The server runs a scoring algorithm based on the parsed message data to determine the importance of each message. For example, a message from a superior containing the keyword "urgent" would be given a high score. The input to this process is the parsed message data, and the output is a scored message list.

[1193] Step 4:

[1194] The server sorts the scored messages based on their priority, then converts them into a data format (e.g., JSON) for notification to the user device. The input of this step is the scored message list, and the output is the sorted message data.

[1195] Step 5:

[1196] The server sends the aligned message data to the user terminal through the API endpoint. The input of this process is the aligned message data, and the output is the notification data sent to the user terminal.

[1197] Step 6:

[1198] The device receives and analyzes the sorted message data received from the server. It then displays the messages in order of priority. High-priority messages are displayed as pop-up notifications, and medium-priority messages are listed on the in-app dashboard. The input of this step is the notification data sent from the server, and the output is the message list displayed to the user.

[1199] Step 7:

[1200] The server periodically collects new content in categories that the user is interested in. For example, it uses an RSS feed or an API to get the latest news and articles. The input for this step is a feed URL or an API endpoint, and the output is a list of new content.

[1201] Step 8:

[1202] The server analyzes the collected new content and extracts keywords related to publication time, author, title, and content. Natural language processing and text analysis techniques are used in this step. The input is a list of new content, and the output is the analyzed content data.

[1203] Step 9:

[1204] The server runs a scoring algorithm based on the analyzed content data to determine the importance of each piece of content, taking into account the user's interest categories and specific keywords. The input is the analyzed content data, and the output is a scored content list.

[1205] Step 10:

[1206] The server sorts the content based on priority, converts it into a data format, and then notifies the user device through an API endpoint. The input of this step is a scored content list, and the output is the sorted content data.

[1207] Step 11:

[1208] The terminal displays the sorted content data received from the server. High-priority content is notified immediately, and low-priority content is displayed in a list on the dashboard. The input is the notification data sent from the server, and the output is the content list displayed to the user.

[1209] In this way, users can quickly and efficiently review important messages and content.

[1210] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1211] This invention is a system incorporating an emotion engine that recognizes the user's emotions, and is particularly intended to organize messages received from email accounts and multiple communication tools in order of priority and promptly notify the user of important messages. To implement this invention, the system is constructed according to the following procedure.

[1212] System configuration

[1213] The system consists of the following main components:

[1214] 1. Server

[1215] Message Collection Module

[1216] Message Analysis Module

[1217] Priority Determination Algorithm

[1218] Message Organizer Module

[1219] Emotion Engine

[1220] 2. User Device

[1221] Program processing

[1222] Message Collection

[1223] The server periodically retrieves new messages using the APIs of the email accounts and communication tools that the user has linked. For example, the server operates the Gmail API to fetch new emails from the user's mailbox. Similarly, the server calls the APIs of Slack, Microsoft Teams, etc. to retrieve the user's chat messages.

[1224] Message Parsing

[1225] The acquired messages are analyzed by the server's message analysis module. During the analysis, the sender, time of sending, content, tags, and mention information are extracted. In addition, an emotion engine recognizes the user's emotions from the message content and identifies negative emotions.

[1226] Priority determination

[1227] The server then runs a priority determination algorithm based on the information obtained from the message analysis. This algorithm scores the importance of each message, taking into account the presence or absence of mentions and important keywords, the importance of the sender, and the emotional information detected by the emotional engine. For example, an email from a superior containing the keyword "urgent" and also detecting negative emotions will be given a higher score.

[1228] Message organization

[1229] Based on the determined score, the server sorts the messages in order of priority, then converts them into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[1230] User Notification and Display

[1231] The user terminal receives the message list from the server and notifies the user of messages in descending order of priority. For example, the terminal displays a pop-up notification for high-priority messages and displays a list of medium-priority messages on a dashboard.

[1232] Specific examples

[1233] 1. Get new emails

[1234] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[1235] 2. Message Analysis

[1236] The server analyzes the received emails and extracts the sender (boss), time of sending, and the keyword "urgent." It also calls Slack's API to retrieve project progress report messages in which the user is mentioned. It then uses an emotion engine to recognize the user's emotions from the message content and identifies messages with negative emotions.

[1237] 3. Priority determination

[1238] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion engine.

[1239] 4. Message organization and notifications

[1240] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[1241] 5. User Notices

[1242] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[1243] 6. User Response

[1244] Users receive notifications and respond to messages with high priority. First, they check the "urgent" email from their boss, which is the most important, and respond or reply to it. Then, they check and respond to messages in Slack.

[1245] This allows users to quickly and efficiently review important messages and take appropriate action. The emotion engine allows for quicker responses to messages that are particularly urgent or emotionally important.

[1246] The processing flow will be explained below.

[1247] Step 1:

[1248] Get new messages

[1249] The server periodically calls the APIs of the email accounts and communication tools that the user has previously linked. For example, the server uses the Gmail API to retrieve new emails and the Slack API to collect new chat messages.

[1250] Step 2:

[1251] Preparing for Message Analysis

[1252] The server prepares the retrieved message data for analysis by loading the contents of the email or chat message into memory and extracting the information needed for analysis (such as sender, time of sending, and content).

[1253] Step 3:

[1254] Extracting caller information

[1255] The server extracts sender information from message metadata, for example by parsing the "From" field in emails or the sender ID of chat messages.

[1256] Step 4:

[1257] Extracting the call time

[1258] The server obtains the message's sending time by checking the "Date" field in the email or the timestamp of the chat message.

[1259] Step 5:

[1260] Message content analysis

[1261] The server analyzes the message content to detect specific keywords or phrases, such as "urgent," "important," or "meeting."

[1262] Step 6:

[1263] Detecting tag and mention information

[1264] The server detects tags and mentions in messages by parsing @mentions in Slack messages and CC / BCC fields in emails.

[1265] Step 7:

[1266] Emotion analysis using an emotion engine

[1267] The server uses an emotion engine to recognize the user's emotions from the message content, and identifies negative emotions using text and voice analysis.

[1268] Step 8:

[1269] Priority Scoring

[1270] The server then scores the importance of each message based on the analysis results, taking into account factors such as whether the message contains specific keywords, whether it is from an important sender, and whether the emotion engine detects negative emotions.

[1271] Step 9:

[1272] Sorting by priority

[1273] The server sorts messages by priority based on the scoring results, with messages with higher scores coming to the top.

[1274] Step 10:

[1275] Converting to data format

[1276] The server converts the sorted message list into an appropriate data format (for example, JSON format) for transmission to the user terminal.

[1277] Step 11:

[1278] Send to user terminal

[1279] The server sends the sorted message list to the user's device and transfers the data using an API endpoint.

[1280] Step 12:

[1281] Displaying pop-up notifications

[1282] The device analyzes the message data received from the server and displays pop-up notifications for the most important messages, such as emails that require immediate attention.

[1283] Step 13:

[1284] Display on the dashboard

[1285] The user terminal displays a list of messages sorted by priority on the dashboard, in a list format for easy user access.

[1286] Step 14:

[1287] User message confirmation

[1288] The user receives notifications from the device and checks the messages in order of priority. First, open the most important message and take the necessary action.

[1289] Step 15:

[1290] Taking corresponding actions

[1291] Users can respond immediately to important messages, such as replying to an "urgent" email from their boss or checking the progress of a project.

[1292] Step 16:

[1293] Organizing confirmed messages

[1294] The device will archive or move messages that you have acknowledged to another folder, remove the acknowledged message from the list, and display the next important message.

[1295] This allows users to quickly and efficiently view and respond to important messages, and the emotion engine allows for quick responses to messages that are particularly urgent or emotionally important.

[1296] Example 2

[1297] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1298] In today's business environment, users receive a large volume of messages from multiple electronic communication channels, and they need to be able to quickly and efficiently identify and respond to important messages. However, doing this manually is extremely time-consuming, so a system is needed that can automatically identify high-priority messages and notify users immediately. It is also necessary to be able to determine the emotional content of messages and prioritize messages with specific emotions.

[1299] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for calling an API of an electronic communication means for collecting new messages, a means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information, a means for scoring the importance of the messages based on the analysis results and determining their priority, a means for recognizing emotional information from the message content using an emotion engine and reflecting it in the priority determination, a means for sorting messages based on their priority and converting them into a data format for notification to a user terminal, and a means for receiving notifications from the user terminal and displaying messages in order of priority. This allows users to quickly and efficiently identify important or emotionally significant messages and respond appropriately.

[1300] "Electronic communication means" refers to communication means for sending and receiving messages, such as email and chat tools.

[1301] "API" stands for Application Programming Interface, an interface for exchanging information and commands between different software programs.

[1302] An "emotion engine" is an algorithm or software module that analyzes a user's emotions from the content of a message and assigns an emotion score such as positive, negative, or neutral.

[1303] "Scoring" is the process of quantifying importance and priority based on analyzed message information.

[1304] "Priority" is an index of importance for determining the processing order of messages.

[1305] "Mention information" is information in which a specific user or group is mentioned in a message.

[1306] A "tag" is a keyword or classification information that is added to a message and indicates the content or category of the message.

[1307] A "data format" is a standard that defines the structure and form of data, such as JSON or XML.

[1308] A "user terminal" is a device used by a user, such as a computer, smartphone, or tablet.

[1309] This invention is a system incorporating an emotion engine that recognizes the user's emotions, and in particular, is designed to automatically determine the importance of new messages and promptly notify the user. To implement this invention, the system is constructed according to the following procedure.

[1310] System configuration

[1311] The system consists of the following main components:

[1312] 1. Server

[1313] Message collection module: Calls the API of the electronic communication means to collect messages.

[1314] Message analysis module: Analyzes the content of collected messages and extracts sender, time of sending, content, tags, and mention information.

[1315] Priority determination algorithm: Based on the analysis results, the importance of the message is scored and the priority is determined.

[1316] Emotion engine: Recognizes emotional information from the content of messages and reflects it in determining priority.

[1317] Message sorting module: Sorts messages based on priority and converts them into a data format for notification to the user terminal.

[1318] 2. User Device

[1319] Notification system: Receives notifications from the server and displays messages based on priority.

[1320] System processing flow

[1321] Message Collection

[1322] The server periodically calls the API of the electronic communication means linked by the user to obtain new messages. Specifically, the server collects messages using the Gmail API, Slack API, Microsoft Teams API, etc.

[1323] Message Parsing

[1324] The acquired messages are analyzed by the server's message analysis module. This analysis extracts information about the sender, time of sending, content, tags, and mentions. An emotion engine is also used to recognize emotional information from the message content. Specifically, an emotional score such as positive, negative, or neutral is generated.

[1325] Priority determination

[1326] Based on the analyzed information, the server runs a priority determination algorithm. This algorithm scores the importance of each message by combining mention information, the presence or absence of important keywords, the importance of the sender, and emotional information. For example, an email from a boss containing the keyword "urgent" will be given a high score. Messages containing negative emotions will also be given a high score.

[1327] Message organization and notifications

[1328] Scored messages are sorted by priority and converted into a data format (e.g., JSON format) by the server, and then sent to the user's device via an API endpoint.

[1329] User Notification and Display

[1330] The user terminal receives the message list from the server and notifies the user of messages in order of priority. Specifically, the terminal displays a pop-up notification, and displays a list of messages with medium priority on the dashboard.

[1331] User response

[1332] After receiving a notification, users can check and respond to messages with higher priority. For example, they can check and respond to the most important "urgent" email from their boss. Then, they can check and respond to messages in Slack.

[1333] This allows users to quickly and efficiently check important messages and take appropriate action. The emotion engine also enables quick responses to messages that are particularly urgent or emotionally important.

[1334] Specific examples

[1335] 1. Get new emails

[1336] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[1337] 2. Message Analysis

[1338] The server analyzes the received emails and extracts the sender (boss), time of sending, and the keyword "urgent." It also calls Slack's API to retrieve project progress report messages in which the user is mentioned. It then uses an emotion engine to recognize emotional information from the message content and identify messages with negative emotions.

[1339] 3. Priority determination

[1340] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion engine.

[1341] 4. Message organization and notifications

[1342] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[1343] 5. User Notices

[1344] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[1345] 6. User Response

[1346] Users receive notifications and respond to messages with high priority. First, they check the "urgent" email from their boss, which is the most important, and respond or reply to it. Then, they check and respond to messages in Slack.

[1347] Prompt Sentence Examples

[1348] An example of a prompt for a generative AI model might be, "If I receive an urgent message from my boss, please advise me on how I should respond first."

[1349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1350] Step 1: Message Collection

[1351] The server periodically calls the API of the electronic communication method linked by the user to obtain new messages. Specifically, the server collects messages using the Gmail API, Slack API, Microsoft Teams API, etc. The input required is the user's authentication information and API call parameters. The output is a list of collected unread messages. This list is stored in the server's database.

[1352] Step 2: Message analysis

[1353] The server passes the collected messages to a message analysis module, which uses a natural language processing (NLP) library (such as NLTK or SpaCy) to analyze the messages. The input is a list of unanalyzed messages. During the analysis process, sender, time of message, content, tags, and mention information are extracted. The output is a list of analyzed messages with this information, along with an emotion score for each message generated by the emotion engine.

[1354] Step 3: Prioritization

[1355] The server runs a priority determination algorithm using the parsed message list. The parsed message list, including the sender, time of sending, tags, mention information, and sentiment information, is required as input. The priority determination algorithm scores messages based on important keywords, the importance of the sender, and sentiment information. For example, a message from a boss containing the keyword "urgent" is assigned a high score. The output is a list containing the priority score for each message.

[1356] Step 4: Message organization and notifications

[1357] The server sorts messages based on their priority scores and converts them into a data format (e.g., JSON format) for notification to the user device. A message list with priority scores is required as input. The server converts this into JSON format and sends it to the user device via an API endpoint. The output is the notification data sent to the user device.

[1358] Step 5: User Notification and Display

[1359] The user terminal parses the JSON formatted notification data received from the server. Notification data is required as input. The user terminal analyzes this data and notifies the user of messages in order of priority. Specifically, a pop-up notification is displayed for high-priority messages, and messages with medium priority are listed on the dashboard. The output is a message that is visually displayed to the user.

[1360] Step 6: User Action

[1361] The user receives notifications from their device and checks and responds to messages in order of priority. The input requires notification information from the user's device. The user checks the most important "urgent" email from their boss and replies or responds to it. Then, the user checks the message in Slack and replies to it. The output is the reply or response sent by the user, which is sent again to the server and the message status is updated.

[1362] Through these steps, users can quickly and efficiently review important messages and respond appropriately. The emotion engine allows users to respond quickly to especially urgent or emotionally significant messages.

[1363] (Application example 2)

[1364] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1365] Conventional email and communication tool message management systems simply retrieve messages and determine their importance. However, they were unable to prioritize or notify messages taking into account the user's emotions, making it difficult to respond quickly to important messages, especially those with negative sentiment. This made it difficult for users to quickly understand and respond to truly important messages. Furthermore, in physical stores, it was not possible to understand customer emotions in real time and respond appropriately. This resulted in a decline in service quality and hindered improvements in customer satisfaction.

[1366] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1367] In this invention, the server includes: a means for calling an email account and a communication tool interface for collecting new messages; a means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and reference information; a means for ranking the importance of the messages and determining their priority based on the analysis results; a means for sorting the messages based on priority and converting them into a data format for notification to the user terminal; a means for receiving notifications from the user terminal and displaying the messages in order of priority; and a means for incorporating an emotion recognition engine that recognizes the user's emotions into the interface, analyzing the collected emotion data, and assigning a higher priority to messages with negative emotions. This allows users to quickly identify and respond to important messages, especially those with negative emotions. It also enables real-time monitoring of customer emotions in physical stores and prompt appropriate responses.

[1368] "New Messages" are the most recent messages received through your email account or communication tool.

[1369] An "email account" is an online account that a user uses to send and receive email.

[1370] A "communication tool" is a platform or application that users use to send and receive messages to each other.

[1371] An "interface" is a connection point or interface where parts of a system communicate and exchange data with each other.

[1372] "Analysis" is the act of examining the contents of collected messages in detail and extracting necessary information.

[1373] "Originator" is the user or account that sent the message.

[1374] "Time of Sent" is the exact date and time the message was sent.

[1375] "Content" is the text or information contained in the message.

[1376] A "tag" is a specific keyword or label added to a message that is used to categorize the message.

[1377] "Reference information" is metadata that indicates the related information referenced by a message or the relevance of a message.

[1378] "Ranking" refers to classifying the importance of a message into a numerical value or category based on analyzed information.

[1379] "Priority" is an index that indicates the importance or urgency of a message.

[1380] A "data format" is a format that structures message data in a specific format to facilitate storage and transmission.

[1381] An "emotion recognition engine" is a software component that analyzes the emotions of users and customers and recognizes their emotional state.

[1382] "Negative emotions" are negative emotional states such as anger, sadness, and fear.

[1383] This invention is a system that includes an emotion recognition engine that recognizes user emotions, and is particularly intended to organize messages received from email accounts and multiple communication tools in order of priority and promptly notify users of important messages. It can also be applied to applications that perform real-time emotion recognition in customer service at brick-and-mortar stores to improve the quality of service.

[1384] System configuration

[1385] The system consists of the following main components:

[1386] 1. Server

[1387] Message collection module: Calls the interface of the email account and communication tool and periodically retrieves new messages.

[1388] Message Analysis Module: Analyzes collected messages and extracts sender, time of sending, content, tags, and reference information.

[1389] Priority determination algorithm: Based on the analysis results, the importance of messages is ranked and their priority is determined.

[1390] Message sorting module: Sorts messages based on priority and converts them into a data format for notification to the user terminal.

[1391] Emotion Recognition Engine: Analyzes emotional data and prioritizes messages with negative emotions.

[1392] 2. User Device

[1393] Notification receiving component: Receives and displays a list of messages received from the server, organized by priority.

[1394] Emotion-aware interface: Analyze customer emotions in real time and respond appropriately.

[1395] Program processing

[1396] Message Collection

[1397] The server periodically retrieves new messages using the APIs of the email account and communication tool. For example, the server operates a common email API or chat tool API to retrieve and collect new emails and chat messages.

[1398] Message Parsing

[1399] The server analyzes messages acquired by the server using a message analysis module. During the analysis process, the sender, time of message, content, tags, and reference information are extracted. Furthermore, an emotion recognition engine recognizes the emotions of users and customers from the message content and identifies negative emotions.

[1400] Priority determination

[1401] The server then runs a priority algorithm based on the analyzed information to rank the importance of each message. For example, messages from superiors or messages containing keywords like "urgent" are given a higher score. The priority is further adjusted based on negative emotions detected by an emotion recognition engine.

[1402] Message organization

[1403] Based on the determined score, the server sorts the messages in order of priority and then converts them into an appropriate data format for notification to the user device. The sorted message list is sent to the user device through an API endpoint.

[1404] User Notification and Display

[1405] The user device receives a message list from the server and notifies them in order of priority. For example, the device displays a pop-up notification for high-priority messages and displays a list of other messages. It also monitors customer sentiment in real time in physical stores and sends notifications to promptly take appropriate action.

[1406] Specific examples

[1407] 1. Get new emails

[1408] The server uses a general-purpose email API to retrieve new emails from the user's mailbox. For example, you might receive an email from your boss marked "urgent."

[1409] 2. Message Analysis

[1410] The server analyzes the received email and extracts the sender, time of sending, and the keyword "urgent." It then uses an emotion recognition engine to recognize the user's emotions from the message content and identify negative emotions.

[1411] 3. Priority determination

[1412] The server then runs a scoring algorithm based on the analysis results, assigning a high score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion recognition engine.

[1413] 4. Message organization and notifications

[1414] The server sorts the messages by priority and sends them to the user terminal in the appropriate data format.

[1415] 5. User Notices

[1416] The user device analyzes the received messages and notifies the user of high-priority messages via a pop-up notification. It also detects negative customer sentiment in physical stores and notifies them to take prompt action.

[1417] Prompt Sentence Examples

[1418] Below are some example prompts to input to a generative AI model:

[1419] "A customer changes their facial expression at the counter. Develop a system that recognizes the customer's emotions from image and audio data and notifies an employee if a negative emotion (anger, sadness, fear) is detected. The hardware used is smart glasses, and the software used is TensorFlow / Keras."

[1420] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1421] Step 1:

[1422] Message collection

[1423] The server calls the interfaces of the email account and communication tool to periodically retrieve new messages. Specifically, the server periodically collects new emails and chat messages using the API of each interface. The input is the API information of the email account and chat tool, and the output is a list of collected unread messages.

[1424] Step 2:

[1425] Message Parsing

[1426] The server analyzes the collected message content. Specifically, it uses a message analysis module to extract sender, time of message, content, tags, and reference information. It then uses an emotion recognition engine to extract emotional data from the message content and identify negative emotions. The input is a list of newly arrived messages, and the output is structured data and emotional data of the messages.

[1427] Step 3:

[1428] Determining Priority

[1429] The server runs a priority determination algorithm based on the analyzed information to rank the importance of messages. Specifically, it evaluates important keywords, the message sender, the time of sending, emotional data, etc., and assigns a score to each message. The input is the structured data and emotional data of the message, and the output is a list of scored messages.

[1430] Step 4:

[1431] Organizing messages

[1432] The server sorts the messages in order of priority based on the determined scores. Specifically, it sorts the array based on the score of each message and then converts it into a data format for notification. The input is a list of scored messages, and the output is a list of messages sorted by priority.

[1433] Step 5:

[1434] Message notifications

[1435] The user terminal receives and displays the message list sorted by priority received from the server. Specifically, the terminal displays a pop-up notification for high-priority messages and displays the list for other messages. The input is the list of messages sorted by priority, and the output is the notification to the user and the messages displayed.

[1436] Step 6:

[1437] Customer Sentiment Detection

[1438] The user terminal (smart glasses) captures the customer's facial expressions and voice in real time and inputs them into the emotion recognition engine. Specifically, the smart glasses use a camera and microphone to send the captured image and voice data to the emotion recognition engine. The input is image and voice data, and the output is detected emotion data.

[1439] Step 7:

[1440] Customer response notification

[1441] The user device (smart glasses) identifies customers with negative emotions based on the emotion data received from the emotion recognition engine and notifies employees. Specifically, a warning message is displayed on the smart glasses' display, urging employees to take prompt action. The input is the detected emotion data, and the output is a warning notification to employees.

[1442] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1443] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1444] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1445] [Fourth embodiment]

[1446] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1447] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1448] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1449] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1450] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1451] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1452] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1453] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1454] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1455] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1456] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1457] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1458] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1459] This invention relates to a system that organizes messages received from a user's email account or multiple communication tools in order of priority and promptly notifies the user of important messages. To implement this invention, the system is constructed according to the following procedure.

[1460] System configuration

[1461] The system consists of the following main components:

[1462] 1. Server

[1463] Message Collection Module

[1464] Message Analysis Module

[1465] Priority Determination Algorithm

[1466] Message Organizer Module

[1467] 2. User Device

[1468] Program processing

[1469] Message Collection

[1470] The server first periodically retrieves new messages using the APIs of the email accounts and communication tools linked to the user. For example, the server operates the Gmail API to fetch new emails from the user's mailbox. Similarly, the server calls the APIs of Slack, Microsoft Teams, and other services to retrieve the user's chat messages.

[1471] Message Parsing

[1472] The captured messages are then analyzed by the server's message analysis module. During the analysis, information about the sender, time of message, content, tags, and mentions is extracted. For example, mentions related to a specific project or messages containing keywords such as "urgent" or "important" are extracted.

[1473] Priority determination

[1474] The server then runs a priority algorithm based on the information obtained from the message analysis. This algorithm scores each message based on data such as mentions, the presence of important keywords, and the importance of the sender. For example, an email from a boss containing the keyword "urgent" would be given a high score.

[1475] Message organization

[1476] Based on the determined score, the server sorts the messages in order of priority, then converts them into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[1477] User Notification and Display

[1478] The user terminal receives the message list from the server and notifies the user of messages in descending order of priority. For example, the terminal displays a pop-up notification for high-priority messages and displays a list of medium-priority messages on a dashboard.

[1479] Specific examples

[1480] 1. Get new emails

[1481] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[1482] 2. Message Analysis

[1483] The server analyzes the emails and extracts the sender (boss), the time of sending, and the keyword "urgent." It also calls the Slack API to retrieve project progress report messages in which the user is mentioned.

[1484] 3. Priority determination

[1485] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it also takes into account mention information in Slack messages to set priorities.

[1486] 4. Message organization and notifications

[1487] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[1488] 5. User Notices

[1489] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[1490] 6. User Support

[1491] Users receive notifications and can check and respond to messages with higher priority.

[1492] This allows users to quickly and efficiently view and respond to important messages.

[1493] The processing flow will be explained below.

[1494] Step 1:

[1495] Get new messages

[1496] The server periodically calls the APIs of email accounts and communication tools that the user has registered in advance. For example, the server uses the Gmail API to obtain new emails and the Slack API to collect new chat messages.

[1497] Step 2:

[1498] Preparing for Message Analysis

[1499] The server prepares the retrieved message data for analysis by loading a portion of the email or chat message into memory and extracting the information needed for analysis (such as sender, time of message, and content).

[1500] Step 3:

[1501] Extracting caller information

[1502] The server extracts originator information from message metadata, for example, identifying the "From" field in an email or the sender ID in a chat message.

[1503] Step 4:

[1504] Extracting the call time

[1505] The server obtains the message's sending time by checking the "Date" field in the email or the timestamp of the chat message.

[1506] Step 5:

[1507] Message content analysis

[1508] The server analyzes the message content to detect specific keywords or phrases, such as "urgent," "important," or "meeting."

[1509] Step 6:

[1510] Detecting tag and mention information

[1511] The server detects tags and mentions in messages by parsing @mentions in Slack messages and CC / BCC fields in emails.

[1512] Step 7:

[1513] Priority Scoring

[1514] The server then scores the importance of each message based on the analysis results, taking into account whether it contains specific keywords, whether it is from an important sender, whether it is mentioned, and so on.

[1515] Step 8:

[1516] Sorting by priority

[1517] The server sorts messages by priority based on the scoring results, with messages with higher scores coming to the top.

[1518] Step 9:

[1519] Converting to data format

[1520] The server converts the sorted message list into an appropriate data format (for example, JSON format) for transmission to the user terminal.

[1521] Step 10:

[1522] Send to user terminal

[1523] The server sends the sorted message list to the user's device and transfers the data using an API endpoint.

[1524] Step 11:

[1525] Displaying pop-up notifications

[1526] The device analyzes the message data received from the server and displays pop-up notifications for the most important messages, such as emails that require immediate attention.

[1527] Step 12:

[1528] Display on the dashboard

[1529] The user terminal displays a list of messages sorted by priority on the dashboard, in a list format for easy user access.

[1530] Step 13:

[1531] User message confirmation

[1532] The user receives notifications from the device and checks the messages in order of priority. First, open the most important message and take the necessary action.

[1533] Step 14:

[1534] Taking corresponding actions

[1535] Users can respond immediately to important messages, such as replying to an "urgent" email from their boss or checking the progress of a project.

[1536] Step 15:

[1537] Organizing confirmed messages

[1538] The device will archive or move messages that you have acknowledged to another folder, remove the acknowledged message from the list, and display the next important message.

[1539] By the above processing steps, the system of the present invention can promptly and efficiently present important messages to the user and prompt them to take action.

[1540] Example 1

[1541] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1542] Nowadays, many users use multiple messaging methods (e.g., email and chat apps). As a result, there is a high possibility that important messages will be buried and overlooked. This leads to problems such as reduced work efficiency and delayed important notifications. The present invention aims to centrally collect messages from different messaging methods, organize and notify them according to importance, and enable users to quickly understand important messages and respond appropriately.

[1543] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1544] In this invention, the server includes means for calling an API of a messaging account and a communication tool to collect new messages, means for analyzing the contents of the collected messages and extracting sender, sending time, content, tag, and mention information, means for scoring the importance of the messages based on the analysis results and determining their priority, means for sorting the messages based on their priority and converting them into a data format for notification to a user device, and means for receiving notifications from the user device and displaying the messages in order of priority. This makes it possible to centrally manage and organize messages from different messaging means and quickly notify users of important messages.

[1545] A "new message" is the most recent message received in a user's messaging account or communication tool.

[1546] A "messaging account" is one or more email accounts or chat application accounts used by a user.

[1547] "Communication tools" are platforms and applications (e.g., email, chat applications) that users use to send and receive messages.

[1548] "API" is an abbreviation for Application Programming Interface, a set of rules and tools for exchanging data between different software programs.

[1549] "Means for collection" refers to the server's functionality for obtaining new messages using the API of messaging accounts and communication tools.

[1550] The "analyzing means" is a function of the server for analyzing the content of collected messages and extracting sender, sending time, content, tag, and mention information.

[1551] An "originator" is a person or entity that sends a message.

[1552] "Time of sending" is the date and time the message was sent.

[1553] "Content" refers to the message body and the information contained therein.

[1554] A "tag" is an identifier or keyword that is added to a message.

[1555] "Mention information" is information in which a specific user is mentioned or named in a message.

[1556] The "scoring means" is a server function for quantifying the importance of messages based on the analysis results and setting priorities.

[1557] A "sorting means" is a server's capability to order messages based on priority.

[1558] The "means for converting into a data format" is a function of the server that converts the sorted messages into a format appropriate for notifying the user device.

[1559] A "user device" is an electronic device (e.g., smartphone, tablet, computer) that a user uses to receive and view messages.

[1560] The "means for receiving notifications" is a function that enables the user device to receive notifications from the server.

[1561] The "means for displaying" is a function that allows the user device to visually present messages to the user in order of priority.

[1562] The present invention relates to a system that automatically collects new messages from a user's multiple messaging accounts and communication tools, analyzes and scores them, and then promptly notifies the user of high-priority messages. The system for implementing this invention is composed of a server and a user terminal.

[1563] Overall structure

[1564] Server: Message collection module, message analysis module, priority determination algorithm, message organization module

[1565] User terminal: message display module, notification module

[1566] The specific hardware and software used

[1567] Server: We recommend a physical server or a cloud-based virtual server with high-performance processors, RAM, and storage, such as Amazon Web Services (AWS) or Google Cloud Platform.

[1568] Communication tool APIs: Gmail API, Slack API, Microsoft Teams API, etc. are used.

[1569] Natural language processing software: spaCy, NLTK, Google Cloud Natural Language API, etc. are used for message analysis.

[1570] Data format: Convert to JSON format and send / receive messages.

[1571] Message Collection

[1572] The server first periodically retrieves new messages by calling the APIs of the email accounts and communication tools that the user has linked. For example, it uses the Gmail API to retrieve new emails from Gmail. Similarly, it calls the APIs of Slack and Microsoft Teams to retrieve chat messages.

[1573] Specifically, the server calls the Gmail API every minute to retrieve new emails for the user. An example of the API request used is:

[1574] "GET https: / / www.googleapis.com / gmail / v1 / users / {userId} / messages"

[1575] Message Parsing

[1576] Messages retrieved by the server are analyzed by the message analysis module. During the analysis process, information about the sender, time of sending, content, tags, and mentions is extracted. In particular, natural language processing technology is used to extract important keywords (e.g., "urgent" and "important") from the message body.

[1577] Specifically, analysis is performed using the Google Cloud Natural Language API to extract the keyword "urgent" from the email content.

[1578] Message Priority Determination

[1579] Based on the analyzed information, the server uses a priority determination algorithm to score the importance of the message. This algorithm takes into account the importance of the sender, keywords in the message content, mention information, etc. For example, an email from a boss containing the keyword "urgent" will be given the highest score.

[1580] An example of a specific scoring rule:

[1581] if "urgent" in message and sender == "boss": priority_score += 10

[1582] Message organization and notifications

[1583] Based on the priority score, the server sorts the messages in ascending or descending order and sends them to the user's device in JSON format, so that the sorted messages are notified to the user's device.

[1584] User Notification and Display

[1585] The user device analyzes the received message list and notifies the user in order of priority, displaying a pop-up notification for high-priority messages and displaying a list of medium-priority messages on the dashboard.

[1586] Specifically, high-priority messages are displayed on the user's device screen in the following format:

[1587] "[Urgent Notification] Supervisor: Please respond immediately"

[1588] User Support

[1589] Users receive notifications and can view and respond to messages with higher priority, allowing them to quickly and efficiently view and respond to important messages.

[1590] This allows users to prevent messages from being missed and quickly grasp important information, thereby improving work efficiency.

[1591] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1592] Step 1:

[1593] The server retrieves new messages by calling the API of the messaging account and communication tool linked by the user. Specifically, it fetches new messages using the Gmail API, Slack API, Microsoft Teams API, etc. It sends an API request and receives the new messages in JSON format as a response. For example, in the case of the Gmail API, the request looks like this:

[1594] GET https: / / www.googleapis.com / gmail / v1 / users / {userId} / messages

[1595] Input: User's API credentials

[1596] Output: New message data in JSON format

[1597] What happens: The server sends an API request every minute to fetch new messages.

[1598] Step 2:

[1599] The server analyzes the collected message content and extracts information about the sender, time of sending, content, tags, and mentions. This analysis is performed using natural language processing software (e.g., spaCy, NLTK, Google Cloud Natural Language API). Important keywords and flags (e.g., "urgent" and "important") are extracted from the message body.

[1600] Input: New message data in JSON format

[1601] Output: Analysis results (sender, time of posting, content, tags, mention information)

[1602] What it does: The server uses natural language processing software to extract important keywords from the message body.

[1603] Step 3:

[1604] The server then runs a priority determination algorithm based on the analysis results to score the importance of the message. The algorithm takes into account the importance of the sender, keywords in the message content, mention information, etc. For example, an email from a superior containing the keyword "urgent" will be given a high score.

[1605] Input: Analysis results

[1606] Output: Priority score for each message

[1607] What happens: The server scores using the following rules:

[1608] if "urgent" in message and sender == "boss": priority_score += 10

[1609] Step 4:

[1610] The server sorts messages based on their priority scores, converts the sorted messages into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[1611] Input: A message and its priority score

[1612] Output: A sorted list of messages in JSON format

[1613] What happens: The server sorts the messages using an algorithm such as quicksort and serializes them into JSON format.

[1614] Step 5:

[1615] The user device analyzes the message list received from the server and notifies the user of messages in order of priority, displaying a pop-up notification for high-priority messages and displaying a list of medium-priority messages on the dashboard.

[1616] Input: A sorted list of messages in JSON format

[1617] Output: Visual notifications and messages

[1618] What happens: The user device generates a pop-up notification to immediately display high priority messages.

[1619] Step 6:

[1620] Users receive notifications and can check and respond to messages in order of priority, allowing them to quickly and efficiently check and respond to important messages.

[1621] Input: Notification from user device

[1622] Output: User action (acknowledgement and response to message)

[1623] What happens: The user clicks on the pop-up notification to view the message details and take the necessary action.

[1624] This allows the user to respond efficiently without missing any important messages.

[1625] (Application example 1)

[1626] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1627] In modern society, many users use not only email and chat tools but also various content distribution services. However, these messages and content are often received in large quantities at once, putting users at risk of missing important information. Furthermore, the lack of an efficient way to quickly find important content that is relevant to users' interests further exacerbates information overload. Therefore, there is a need for a system that can efficiently organize multiple messages and content and prioritize notifications of important content.

[1628] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1629] In this invention, the server includes means for calling an API of an email account and a communication tool to collect new messages, means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information, means for scoring the importance of the messages and determining their priority based on the analysis results, means for sorting the messages based on their priority and converting them into a data format for notification to a user terminal, means for receiving notifications from the user terminal and displaying the messages in order of priority, means for collecting new content from a content distribution service and extracting keywords for the sending time, author, title, and content, means for scoring the importance of the content based on categories of interest to the user or specific keywords and determining its priority, and means for notifying the user of the content sorted in order of priority. This allows users to quickly and efficiently check important information from multiple communication tools and content distribution services.

[1630] "New Messages" means the most recent messages received through your email account and communication tools.

[1631] "Means for calling an API" refers to a method for using an application program interface (API) to obtain necessary data from external services or databases.

[1632] "Means for analyzing the contents of a message" refers to a method for extracting and analyzing internal information of a received message (such as sender, time of sending, content, tags, and mention information).

[1633] The "means for scoring and determining priority" is a method for evaluating the importance of each message or content numerically based on the analyzed information and determining its priority.

[1634] The "means for converting into a data format" refers to a method for converting the organized messages or content into an appropriate data format (e.g., JSON format) for transfer to a user terminal.

[1635] The "means for notifying the user terminal" is a method for quickly notifying the user terminal of high-priority messages or content.

[1636] The "means for displaying messages in order of priority" is a method for displaying messages or content sorted according to importance in an orderly manner on a user terminal.

[1637] "New content" refers to information such as the latest news, articles, and videos obtained from content distribution services.

[1638] The "means for scoring the importance of content based on categories or specific keywords" is a method for evaluating the importance of each piece of content based on the presence or absence of interest categories or keywords set by the user.

[1639] The "means for notifying the user of the content sorted in order of priority" is a method for ranking the rated content according to the user's level of interest and notifying the user of the content in that order.

[1640] To specifically implement this invention, it is necessary to build a system according to the following detailed description. This system consists of two main components: a server and a user terminal. Below, we will explain how these components work together to process and calculate data.

[1641] Server configuration and operation

[1642] Hardware and Software

[1643] The server uses a cloud service such as AWS, and the software running on the server includes Python, Flask (a web framework), feedparser (an RSS analysis library), Firebase (a cloud messaging service), and so on.

[1644] Message and content collection

[1645] The server periodically collects new messages and content by calling the APIs of the email accounts and communication tools registered by the user. For example, it uses RSS feeds and APIs to obtain the latest news and articles. It also collects emails and chat messages using APIs such as Gmail and Slack.

[1646] Analysis and Scoring

[1647] The collected messages and content are analyzed on the server. During this analysis, information about the sender, time of posting, content, tags, and mentions is extracted. In the case of content, the category, author, title, and keywords are also extracted. Next, a scoring algorithm is run based on the analysis results to score the importance of each message or piece of content. This takes into account the user's interests and set keywords.

[1648] Determining priority and converting to data format

[1649] The scored data is sorted by priority, converted into an appropriate data format (e.g., JSON format) for notification to the user device, and sent to the user device via an API endpoint.

[1650] Configuration and operation of user terminal

[1651] Hardware and Software

[1652] The user device is typically a smartphone with a dedicated app installed, which receives notifications from the server via Firebase Cloud Messaging and processes them.

[1653] Notifications and Displays

[1654] The user device analyzes the data received from the server and displays messages and content in order of priority. For example, high-priority messages and content are displayed as pop-up notifications, while medium-priority messages and content are displayed as a list on the app's dashboard.

[1655] Examples of specific examples and prompts

[1656] For example, if a user wants to be notified of news containing keywords such as "important" or "urgent" in a news app, the user might input the following prompt sentence to the server:

[1657] Collect the latest news in categories that interest you (e.g., "Technology," "Health," "Business") and prioritize news that contains specific keywords (e.g., "Important," "Urgent").

[1658] In this way, users can efficiently view important messages and content that are relevant to their interests.

[1659] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1660] Step 1:

[1661] The server calls the APIs of the email accounts and communication tools registered by the user to collect new messages. At this time, the server uses APIs such as Gmail and Slack to periodically obtain data at specified intervals. The input for this process is the user's API authentication information, and the output is a list of new messages.

[1662] Step 2:

[1663] The server analyzes the collected new messages. Specifically, it extracts the sender, time, content, tags, and mention information for each message. In this step, it uses natural language processing technology to analyze the message text and extract the necessary information. The input is a list of new messages, and the output is the analyzed message data.

[1664] Step 3:

[1665] The server runs a scoring algorithm based on the parsed message data to determine the importance of each message. For example, a message from a superior containing the keyword "urgent" would be given a high score. The input to this process is the parsed message data, and the output is a scored message list.

[1666] Step 4:

[1667] The server sorts the scored messages based on their priority, then converts them into a data format (e.g., JSON) for notification to the user device. The input of this step is the scored message list, and the output is the sorted message data.

[1668] Step 5:

[1669] The server sends the aligned message data to the user terminal through the API endpoint. The input of this process is the aligned message data, and the output is the notification data sent to the user terminal.

[1670] Step 6:

[1671] The device receives and analyzes the sorted message data received from the server. It then displays the messages in order of priority. High-priority messages are displayed as pop-up notifications, and medium-priority messages are listed on the in-app dashboard. The input of this step is the notification data sent from the server, and the output is the message list displayed to the user.

[1672] Step 7:

[1673] The server periodically collects new content in categories that the user is interested in. For example, it uses an RSS feed or an API to get the latest news and articles. The input for this step is a feed URL or an API endpoint, and the output is a list of new content.

[1674] Step 8:

[1675] The server analyzes the collected new content and extracts keywords related to publication time, author, title, and content. Natural language processing and text analysis techniques are used in this step. The input is a list of new content, and the output is the analyzed content data.

[1676] Step 9:

[1677] The server runs a scoring algorithm based on the analyzed content data to determine the importance of each piece of content, taking into account the user's interest categories and specific keywords. The input is the analyzed content data, and the output is a scored content list.

[1678] Step 10:

[1679] The server sorts the content based on priority, converts it into a data format, and then notifies the user device through an API endpoint. The input of this step is a scored content list, and the output is the sorted content data.

[1680] Step 11:

[1681] The terminal displays the sorted content data received from the server. High-priority content is notified immediately, and low-priority content is displayed in a list on the dashboard. The input is the notification data sent from the server, and the output is the content list displayed to the user.

[1682] In this way, users can quickly and efficiently review important messages and content.

[1683] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1684] This invention is a system incorporating an emotion engine that recognizes the user's emotions, and is particularly intended to organize messages received from email accounts and multiple communication tools in order of priority and promptly notify the user of important messages. To implement this invention, the system is constructed according to the following procedure.

[1685] System configuration

[1686] The system consists of the following main components:

[1687] 1. Server

[1688] Message Collection Module

[1689] Message Analysis Module

[1690] Priority Determination Algorithm

[1691] Message Organizer Module

[1692] Emotion Engine

[1693] 2. User Device

[1694] Program processing

[1695] Message Collection

[1696] The server periodically retrieves new messages using the APIs of the email accounts and communication tools that the user has linked. For example, the server operates the Gmail API to fetch new emails from the user's mailbox. Similarly, the server calls the APIs of Slack, Microsoft Teams, etc. to retrieve the user's chat messages.

[1697] Message Parsing

[1698] The acquired messages are analyzed by the server's message analysis module. During the analysis, the sender, time of sending, content, tags, and mention information are extracted. In addition, an emotion engine recognizes the user's emotions from the message content and identifies negative emotions.

[1699] Priority determination

[1700] The server then runs a priority determination algorithm based on the information obtained from the message analysis. This algorithm scores the importance of each message, taking into account the presence or absence of mentions and important keywords, the importance of the sender, and the emotional information detected by the emotional engine. For example, an email from a superior containing the keyword "urgent" and also detecting negative emotions will be given a higher score.

[1701] Message organization

[1702] Based on the determined score, the server sorts the messages in order of priority, then converts them into a data format (e.g., JSON) for notification to the user device, and sends the sorted message list to the user device via an API endpoint.

[1703] User Notification and Display

[1704] The user terminal receives the message list from the server and notifies the user of messages in descending order of priority. For example, the terminal displays a pop-up notification for high-priority messages and displays a list of medium-priority messages on a dashboard.

[1705] Specific examples

[1706] 1. Get new emails

[1707] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[1708] 2. Message Analysis

[1709] The server analyzes the received emails and extracts the sender (boss), time of sending, and the keyword "urgent." It also calls Slack's API to retrieve project progress report messages in which the user is mentioned. It then uses an emotion engine to recognize the user's emotions from the message content and identifies messages with negative emotions.

[1710] 3. Priority determination

[1711] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion engine.

[1712] 4. Message organization and notifications

[1713] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[1714] 5. User Notices

[1715] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[1716] 6. User Response

[1717] Users receive notifications and respond to messages with high priority. First, they check the "urgent" email from their boss, which is the most important, and respond or reply to it. Then, they check and respond to messages in Slack.

[1718] This allows users to quickly and efficiently review important messages and take appropriate action. The emotion engine allows for quicker responses to messages that are particularly urgent or emotionally important.

[1719] The processing flow will be explained below.

[1720] Step 1:

[1721] Get new messages

[1722] The server periodically calls the APIs of the email accounts and communication tools that the user has previously linked. For example, the server uses the Gmail API to retrieve new emails and the Slack API to collect new chat messages.

[1723] Step 2:

[1724] Preparing for Message Analysis

[1725] The server prepares the retrieved message data for analysis by loading the contents of the email or chat message into memory and extracting the information needed for analysis (such as sender, time of sending, and content).

[1726] Step 3:

[1727] Extracting caller information

[1728] The server extracts sender information from message metadata, for example by parsing the "From" field in emails or the sender ID of chat messages.

[1729] Step 4:

[1730] Extracting the call time

[1731] The server obtains the message's sending time by checking the "Date" field in the email or the timestamp of the chat message.

[1732] Step 5:

[1733] Message content analysis

[1734] The server analyzes the message content to detect specific keywords or phrases, such as "urgent," "important," or "meeting."

[1735] Step 6:

[1736] Detecting tag and mention information

[1737] The server detects tags and mentions in messages by parsing @mentions in Slack messages and CC / BCC fields in emails.

[1738] Step 7:

[1739] Emotion analysis using an emotion engine

[1740] The server uses an emotion engine to recognize the user's emotions from the message content, and identifies negative emotions using text and voice analysis.

[1741] Step 8:

[1742] Priority Scoring

[1743] The server then scores the importance of each message based on the analysis results, taking into account factors such as whether the message contains specific keywords, whether it is from an important sender, and whether the emotion engine detects negative emotions.

[1744] Step 9:

[1745] Sorting by priority

[1746] The server sorts messages by priority based on the scoring results, with messages with higher scores coming to the top.

[1747] Step 10:

[1748] Converting to data format

[1749] The server converts the sorted message list into an appropriate data format (for example, JSON format) for transmission to the user terminal.

[1750] Step 11:

[1751] Send to user terminal

[1752] The server sends the sorted message list to the user's device and transfers the data using an API endpoint.

[1753] Step 12:

[1754] Displaying pop-up notifications

[1755] The device analyzes the message data received from the server and displays pop-up notifications for the most important messages, such as emails that require immediate attention.

[1756] Step 13:

[1757] Display on the dashboard

[1758] The user terminal displays a list of messages sorted by priority on the dashboard, in a list format for easy user access.

[1759] Step 14:

[1760] User message confirmation

[1761] The user receives notifications from the device and checks the messages in order of priority. First, open the most important message and take the necessary action.

[1762] Step 15:

[1763] Taking corresponding actions

[1764] Users can respond immediately to important messages, such as replying to an "urgent" email from their boss or checking the progress of a project.

[1765] Step 16:

[1766] Organizing confirmed messages

[1767] The device will archive or move messages that you have acknowledged to another folder, remove the acknowledged message from the list, and display the next important message.

[1768] This allows users to quickly and efficiently view and respond to important messages, and the emotion engine allows for quick responses to messages that are particularly urgent or emotionally important.

[1769] Example 2

[1770] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1771] In today's business environment, users receive a large volume of messages from multiple electronic communication channels, and they need to be able to quickly and efficiently identify and respond to important messages. However, doing this manually is extremely time-consuming, so a system is needed that can automatically identify high-priority messages and notify users immediately. It is also necessary to be able to determine the emotional content of messages and prioritize messages with specific emotions.

[1772] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for calling an API of an electronic communication means for collecting new messages, a means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and mention information, a means for scoring the importance of the messages based on the analysis results and determining their priority, a means for recognizing emotional information from the message content using an emotion engine and reflecting it in the priority determination, a means for sorting messages based on their priority and converting them into a data format for notification to a user terminal, and a means for receiving notifications from the user terminal and displaying messages in order of priority. This allows users to quickly and efficiently identify important or emotionally significant messages and respond appropriately.

[1773] "Electronic communication means" refers to communication means for sending and receiving messages, such as email and chat tools.

[1774] "API" stands for Application Programming Interface, an interface for exchanging information and commands between different software programs.

[1775] An "emotion engine" is an algorithm or software module that analyzes a user's emotions from the content of a message and assigns an emotion score such as positive, negative, or neutral.

[1776] "Scoring" is the process of quantifying importance and priority based on analyzed message information.

[1777] "Priority" is an index of importance for determining the processing order of messages.

[1778] "Mention information" is information in which a specific user or group is mentioned in a message.

[1779] A "tag" is a keyword or classification information that is added to a message and indicates the content or category of the message.

[1780] A "data format" is a standard that defines the structure and form of data, such as JSON or XML.

[1781] A "user terminal" is a device used by a user, such as a computer, smartphone, or tablet.

[1782] This invention is a system incorporating an emotion engine that recognizes the user's emotions, and in particular, is designed to automatically determine the importance of new messages and promptly notify the user. To implement this invention, the system is constructed according to the following procedure.

[1783] System configuration

[1784] The system consists of the following main components:

[1785] 1. Server

[1786] Message collection module: Calls the API of the electronic communication means to collect messages.

[1787] Message analysis module: Analyzes the content of collected messages and extracts sender, time of sending, content, tags, and mention information.

[1788] Priority determination algorithm: Based on the analysis results, the importance of the message is scored and the priority is determined.

[1789] Emotion engine: Recognizes emotional information from the content of messages and reflects it in determining priority.

[1790] Message sorting module: Sorts messages based on priority and converts them into a data format for notification to the user terminal.

[1791] 2. User Device

[1792] Notification system: Receives notifications from the server and displays messages based on priority.

[1793] System processing flow

[1794] Message Collection

[1795] The server periodically calls the API of the electronic communication means linked by the user to obtain new messages. Specifically, the server collects messages using the Gmail API, Slack API, Microsoft Teams API, etc.

[1796] Message Parsing

[1797] The acquired messages are analyzed by the server's message analysis module. This analysis extracts information about the sender, time of sending, content, tags, and mentions. An emotion engine is also used to recognize emotional information from the message content. Specifically, an emotional score such as positive, negative, or neutral is generated.

[1798] Priority determination

[1799] Based on the analyzed information, the server runs a priority determination algorithm. This algorithm scores the importance of each message by combining mention information, the presence or absence of important keywords, the importance of the sender, and emotional information. For example, an email from a boss containing the keyword "urgent" will be given a high score. Messages containing negative emotions will also be given a high score.

[1800] Message organization and notifications

[1801] Scored messages are sorted by priority and converted into a data format (e.g., JSON format) by the server, and then sent to the user's device via an API endpoint.

[1802] User Notification and Display

[1803] The user terminal receives the message list from the server and notifies the user of messages in order of priority. Specifically, the terminal displays a pop-up notification, and displays a list of messages with medium priority on the dashboard.

[1804] User response

[1805] After receiving a notification, users can check and respond to messages with higher priority. For example, they can check and respond to the most important "urgent" email from their boss. Then, they can check and respond to messages in Slack.

[1806] This allows users to quickly and efficiently check important messages and take appropriate action. The emotion engine also enables quick responses to messages that are particularly urgent or emotionally important.

[1807] Specific examples

[1808] 1. Get new emails

[1809] The server uses the Gmail API to retrieve new emails from the user's mailbox, for example, detecting emails from the boss marked as "urgent."

[1810] 2. Message Analysis

[1811] The server analyzes the received emails and extracts the sender (boss), time of sending, and the keyword "urgent." It also calls Slack's API to retrieve project progress report messages in which the user is mentioned. It then uses an emotion engine to recognize emotional information from the message content and identify messages with negative emotions.

[1812] 3. Priority determination

[1813] Based on the analysis results, the server runs a scoring algorithm and assigns the highest score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion engine.

[1814] 4. Message organization and notifications

[1815] The server sorts messages by priority, generates data in JSON format, and sends it to the user's terminal.

[1816] 5. User Notices

[1817] The user's device analyzes the received message and immediately notifies the user of "urgent" emails from their boss via a pop-up notification. Subsequently, messages with Slack mentions are displayed on the dashboard.

[1818] 6. User Response

[1819] Users receive notifications and respond to messages with high priority. First, they check the "urgent" email from their boss, which is the most important, and respond or reply to it. Then, they check and respond to messages in Slack.

[1820] Prompt Sentence Examples

[1821] An example of a prompt for a generative AI model might be, "If I receive an urgent message from my boss, please advise me on how I should respond first."

[1822] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1823] Step 1: Message Collection

[1824] The server periodically calls the API of the electronic communication method linked by the user to obtain new messages. Specifically, the server collects messages using the Gmail API, Slack API, Microsoft Teams API, etc. The input required is the user's authentication information and API call parameters. The output is a list of collected unread messages. This list is stored in the server's database.

[1825] Step 2: Message analysis

[1826] The server passes the collected messages to a message analysis module, which uses a natural language processing (NLP) library (such as NLTK or SpaCy) to analyze the messages. The input is a list of unanalyzed messages. During the analysis process, sender, time of message, content, tags, and mention information are extracted. The output is a list of analyzed messages with this information, along with an emotion score for each message generated by the emotion engine.

[1827] Step 3: Prioritization

[1828] The server runs a priority determination algorithm using the parsed message list. The parsed message list, including the sender, time of sending, tags, mention information, and sentiment information, is required as input. The priority determination algorithm scores messages based on important keywords, the importance of the sender, and sentiment information. For example, a message from a boss containing the keyword "urgent" is assigned a high score. The output is a list containing the priority score for each message.

[1829] Step 4: Message organization and notifications

[1830] The server sorts messages based on their priority scores and converts them into a data format (e.g., JSON format) for notification to the user device. A message list with priority scores is required as input. The server converts this into JSON format and sends it to the user device via an API endpoint. The output is the notification data sent to the user device.

[1831] Step 5: User Notification and Display

[1832] The user terminal parses the JSON formatted notification data received from the server. Notification data is required as input. The user terminal analyzes this data and notifies the user of messages in order of priority. Specifically, a pop-up notification is displayed for high-priority messages, and messages with medium priority are listed on the dashboard. The output is a message that is visually displayed to the user.

[1833] Step 6: User Action

[1834] The user receives notifications from their device and checks and responds to messages in order of priority. The input requires notification information from the user's device. The user checks the most important "urgent" email from their boss and replies or responds to it. Then, the user checks the message in Slack and replies to it. The output is the reply or response sent by the user, which is sent again to the server and the message status is updated.

[1835] Through these steps, users can quickly and efficiently review important messages and respond appropriately. The emotion engine allows users to respond quickly to especially urgent or emotionally significant messages.

[1836] (Application example 2)

[1837] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1838] Conventional email and communication tool message management systems simply retrieve messages and determine their importance. However, they were unable to prioritize or notify messages taking into account the user's emotions, making it difficult to respond quickly to important messages, especially those with negative sentiment. This made it difficult for users to quickly understand and respond to truly important messages. Furthermore, in physical stores, it was not possible to understand customer emotions in real time and respond appropriately. This resulted in a decline in service quality and hindered improvements in customer satisfaction.

[1839] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1840] In this invention, the server includes: a means for calling an email account and a communication tool interface for collecting new messages; a means for analyzing the content of the collected messages and extracting sender, sending time, content, tags, and reference information; a means for ranking the importance of the messages and determining their priority based on the analysis results; a means for sorting the messages based on priority and converting them into a data format for notification to the user terminal; a means for receiving notifications from the user terminal and displaying the messages in order of priority; and a means for incorporating an emotion recognition engine that recognizes the user's emotions into the interface, analyzing the collected emotion data, and assigning a higher priority to messages with negative emotions. This allows users to quickly identify and respond to important messages, especially those with negative emotions. It also enables real-time monitoring of customer emotions in physical stores and prompt appropriate responses.

[1841] "New Messages" are the most recent messages received through your email account or communication tool.

[1842] An "email account" is an online account that a user uses to send and receive email.

[1843] A "communication tool" is a platform or application that users use to send and receive messages to each other.

[1844] An "interface" is a connection point or interface where parts of a system communicate and exchange data with each other.

[1845] "Analysis" is the act of examining the contents of collected messages in detail and extracting necessary information.

[1846] "Originator" is the user or account that sent the message.

[1847] "Time of Sent" is the exact date and time the message was sent.

[1848] "Content" is the text or information contained in the message.

[1849] A "tag" is a specific keyword or label added to a message that is used to categorize the message.

[1850] "Reference information" is metadata that indicates the related information referenced by a message or the relevance of a message.

[1851] "Ranking" refers to classifying the importance of a message into a numerical value or category based on analyzed information.

[1852] "Priority" is an index that indicates the importance or urgency of a message.

[1853] A "data format" is a format that structures message data in a specific format to facilitate storage and transmission.

[1854] An "emotion recognition engine" is a software component that analyzes the emotions of users and customers and recognizes their emotional state.

[1855] "Negative emotions" are negative emotional states such as anger, sadness, and fear.

[1856] This invention is a system that includes an emotion recognition engine that recognizes user emotions, and is particularly intended to organize messages received from email accounts and multiple communication tools in order of priority and promptly notify users of important messages. It can also be applied to applications that perform real-time emotion recognition in customer service at brick-and-mortar stores to improve the quality of service.

[1857] System configuration

[1858] The system consists of the following main components:

[1859] 1. Server

[1860] Message collection module: Calls the interface of the email account and communication tool and periodically retrieves new messages.

[1861] Message Analysis Module: Analyzes collected messages and extracts sender, time of sending, content, tags, and reference information.

[1862] Priority determination algorithm: Based on the analysis results, the importance of messages is ranked and their priority is determined.

[1863] Message sorting module: Sorts messages based on priority and converts them into a data format for notification to the user terminal.

[1864] Emotion Recognition Engine: Analyzes emotional data and prioritizes messages with negative emotions.

[1865] 2. User Device

[1866] Notification receiving component: Receives and displays a list of messages received from the server, organized by priority.

[1867] Emotion-aware interface: Analyze customer emotions in real time and respond appropriately.

[1868] Program processing

[1869] Message Collection

[1870] The server periodically retrieves new messages using the APIs of the email account and communication tool. For example, the server operates a common email API or chat tool API to retrieve and collect new emails and chat messages.

[1871] Message Parsing

[1872] The server analyzes messages acquired by the server using a message analysis module. During the analysis process, the sender, time of message, content, tags, and reference information are extracted. Furthermore, an emotion recognition engine recognizes the emotions of users and customers from the message content and identifies negative emotions.

[1873] Priority determination

[1874] The server then runs a priority algorithm based on the analyzed information to rank the importance of each message. For example, messages from superiors or messages containing keywords like "urgent" are given a higher score. The priority is further adjusted based on negative emotions detected by an emotion recognition engine.

[1875] Message organization

[1876] Based on the determined score, the server sorts the messages in order of priority and then converts them into an appropriate data format for notification to the user device. The sorted message list is sent to the user device through an API endpoint.

[1877] User Notification and Display

[1878] The user device receives a message list from the server and notifies them in order of priority. For example, the device displays a pop-up notification for high-priority messages and displays a list of other messages. It also monitors customer sentiment in real time in physical stores and sends notifications to promptly take appropriate action.

[1879] Specific examples

[1880] 1. Get new emails

[1881] The server uses a general-purpose email API to retrieve new emails from the user's mailbox. For example, you might receive an email from your boss marked "urgent."

[1882] 2. Message Analysis

[1883] The server analyzes the received email and extracts the sender, time of sending, and the keyword "urgent." It then uses an emotion recognition engine to recognize the user's emotions from the message content and identify negative emotions.

[1884] 3. Priority determination

[1885] The server then runs a scoring algorithm based on the analysis results, assigning a high score to emails from superiors that contain the keyword "urgent." At the same time, it further adjusts the priority by taking into account negative emotional information detected by the emotion recognition engine.

[1886] 4. Message organization and notifications

[1887] The server sorts the messages by priority and sends them to the user terminal in the appropriate data format.

[1888] 5. User Notices

[1889] The user device analyzes the received messages and notifies the user of high-priority messages via a pop-up notification. In addition, in physical stores, the system detects negative customer sentiment and notifies the user to take prompt action.

[1890] Prompt Sentence Examples

[1891] Below are some example prompts to input to a generative AI model:

[1892] "A customer changes their facial expression at the counter. Develop a system that recognizes the customer's emotions from image and audio data and notifies an employee if a negative emotion (anger, sadness, fear) is detected. The hardware used is smart glasses, and the software used is TensorFlow / Keras."

[1893] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1894] Step 1:

[1895] Message collection

[1896] The server calls the interfaces of the email account and communication tool to periodically retrieve new messages. Specifically, the server periodically collects new emails and chat messages using the API of each interface. The input is the API information of the email account and chat tool, and the output is a list of collected unread messages.

[1897] Step 2:

[1898] Message Parsing

[1899] The server analyzes the collected message content. Specifically, it uses a message analysis module to extract sender, time of message, content, tags, and reference information. It then uses an emotion recognition engine to extract emotional data from the message content and identify negative emotions. The input is a list of newly arrived messages, and the output is structured data and emotional data of the messages.

[1900] Step 3:

[1901] Determining Priority

[1902] The server runs a priority determination algorithm based on the analyzed information to rank the importance of messages. Specifically, it evaluates important keywords, the message sender, the time of sending, emotional data, etc., and assigns a score to each message. The input is the structured data and emotional data of the message, and the output is a list of scored messages.

[1903] Step 4:

[1904] Organizing messages

[1905] The server sorts the messages in order of priority based on the determined scores. Specifically, it sorts the array based on the score of each message and then converts it into a data format for notification. The input is a list of scored messages, and the output is a list of messages sorted by priority.

[1906] Step 5:

[1907] Message notifications

[1908] The user terminal receives and displays the message list sorted by priority received from the server. Specifically, the terminal displays a pop-up notification for high-priority messages and displays the list for other messages. The input is the list of messages sorted by priority, and the output is the notification to the user and the messages displayed.

[1909] Step 6:

[1910] Customer Sentiment Detection

[1911] The user terminal (smart glasses) captures the customer's facial expressions and voice in real time and inputs them into the emotion recognition engine. Specifically, the smart glasses use a camera and microphone to send the captured image and voice data to the emotion recognition engine. The input is image and voice data, and the output is detected emotion data.

[1912] Step 7:

[1913] Customer response notification

[1914] The user device (smart glasses) identifies customers with negative emotions based on the emotion data received from the emotion recognition engine and notifies employees. Specifically, a warning message is displayed on the smart glasses' display, urging employees to take prompt action. The input is the detected emotion data, and the output is a warning notification to employees.

[1915] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1916] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1917] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1918] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1919] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1920] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1921] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1922] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1923] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1924] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1925] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1926] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1927] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1928] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1929] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1930] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1931] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1932] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1933] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1934] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1935] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1936] The following is further disclosed regarding the above embodiment.

[1937] (Claim 1)

[1938] A means for calling an API of an email account and a communication tool to collect new messages;

[1939] A means for analyzing the content of the collected messages and extracting sender, sending time, content, tag, and mention information;

[1940] a means for scoring the importance of messages based on the analysis results and determining their priority;

[1941] means for sorting messages based on priority and converting them into a data format for notification to a user terminal;

[1942] means for receiving notifications from user terminals and displaying messages in order of priority;

[1943] A system including:

[1944] (Claim 2)

[1945] 10. The system of claim 1, further comprising means for periodically obtaining new messages using an API of the email account and the communication tool.

[1946] (Claim 3)

[1947] 10. The system of claim 1, further comprising means for searching for specific important keywords within the message to determine importance.

[1948] "Example 1"

[1949] (Claim 1)

[1950] a means for calling an API of a messaging account and a communication tool to collect new messages;

[1951] A means for analyzing the content of the collected messages and extracting sender, sending time, content, tag, and mention information;

[1952] a means for scoring the importance of messages based on the analysis results and determining their priority;

[1953] means for sorting messages based on priority and converting the messages into a data format for notification to a user device;

[1954] means for receiving notifications from the user device and displaying messages in order of priority;

[1955] A system including:

[1956] (Claim 2)

[1957] 10. The system of claim 1, further comprising means for periodically retrieving new messages using an API of a messaging account and a communication tool.

[1958] (Claim 3)

[1959] 10. The system of claim 1, further comprising means for searching for specific important keywords within the message and determining importance using natural language processing techniques.

[1960] "Application Example 1"

[1961] (Claim 1)

[1962] A means for calling an API of an email account and a communication tool to collect new messages;

[1963] A means for analyzing the content of the collected messages and extracting sender, sending time, content, tag, and mention information;

[1964] a means for scoring the importance of messages based on the analysis results and determining their priority;

[1965] means for sorting messages based on priority and converting them into a data format for notification to a user terminal;

[1966] means for receiving notifications from user terminals and displaying messages in order of priority;

[1967] A means for collecting new content from a content distribution service and extracting keywords related to the time of publication, author, title, and content;

[1968] A means for scoring the importance of content based on categories of interest to users or specific keywords and determining priority;

[1969] means for notifying a user of the prioritized content;

[1970] A system including:

[1971] (Claim 2)

[1972] 10. The system of claim 1, further comprising means for periodically obtaining new messages using an API of the email account and the communication tool.

[1973] (Claim 3)

[1974] 10. The system of claim 1, further comprising means for searching for specific important keywords within the message to determine importance.

[1975] "Example 2: Combining Emotion Engines"

[1976] (Claim 1)

[1977] a means for calling an API of the electronic communication means for collecting new messages;

[1978] A means for analyzing the content of the collected messages and extracting sender, sending time, content, tag, and mention information;

[1979] a means for scoring the importance of messages based on the analysis results and determining their priority;

[1980] a means for recognizing emotional information from the contents...

Claims

1. A means for calling an API of an email account and a communication tool to collect new messages; A means for analyzing the content of the collected messages and extracting sender, sending time, content, tag, and mention information; a means for scoring the importance of messages based on the analysis results and determining their priority; means for sorting messages based on priority and converting them into a data format for notification to a user terminal; means for receiving notifications from user terminals and displaying messages in order of priority; A system including:

2. The system according to claim 1 , further comprising means for periodically obtaining new messages using an API of the email account and the communication tool.

3. 2. The system of claim 1, further comprising means for searching for specific important keywords within the message to determine the importance.

Images