System

Abstract

A system is provided.SOLUTION: A system, comprising: means for a user to access a chat room using a communication device; means for the user to tap a chat digest button; means for sensing an operation of the user to acquire a message in the chat room; means for the server to analyze the message using a generative AI model and extract important information; means for the server to transmit a generated summary to the communication device; and means for the communication device to display the summary.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem that the present invention aims to solve is the problem that when a large number of messages are exchanged in a chat application, important information gets buried and users are unable to efficiently grasp the information. This problem is particularly noticeable among business people and active group chat participants, and can lead to reduced communication efficiency and delayed decision-making. [Means for solving the problem]

[0005] The present invention solves this problem by allowing a user to access a conversation chat room using a communication terminal and tap the conversation digest button. This operation causes the communication terminal to acquire message data in the conversation chat room and send it to a server. The server uses a generative AI model to analyze the message data, extract important information, and generate a summary. The generated summary is sent from the server to the communication terminal, which displays it, allowing the user to quickly grasp important information. This enables efficient communication and rapid decision-making.

[0006] A "user" refers to an individual who operates a communication terminal and uses a chat application.

[0007] "Communication terminal" refers to a device that can connect to the Internet and communicate, such as a smartphone, tablet, or personal computer.

[0008] A "conversation talk room" refers to a virtual space within a chat application where multiple users can exchange text messages.

[0009] A "conversation digest button" refers to an interface element that a user taps to obtain a summary of the conversation content.

[0010] "Message Data" refers to data including text messages exchanged within a conversation chat room.

[0011] "Server" refers to a central computer system that processes data over the Internet and interacts with users' communication terminals.

[0012] A "generative AI model" refers to an algorithm or program that uses artificial intelligence techniques to analyze and summarize messages.

[0013] "Analysis" refers to the process of analyzing message data to extract important information.

[0014] The "summary sentence" refers to a sentence that succinctly summarizes important information extracted from the message data. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The following specific example is given as an embodiment for carrying out the present invention: In this system, a user uses a communication terminal to efficiently summarize message data in a conversation talk room via a server.

[0037] System Overview

[0038] Using the Conversation Digest Feature

[0039] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[0040] When a user taps this "Conversation Digest" button, a summary of the messages in the conversation chat room will begin.

[0041] Getting and sending messages

[0042] The device detects user actions and retrieves the latest message data in the conversation chat room. This data typically includes messages from the past 24 hours or a customizable range.

[0043] The acquired message data is temporarily stored on the device and then sent to a server for analysis.

[0044] Message analysis and summarization

[0045] The server processes the received message data and supplies it to the generative AI model.

[0046] The generative AI model analyzes message data, understands the context, and extracts important information. Based on this information, the generative AI model generates a concise summary.

[0047] Sending and displaying abstracts

[0048] The server sends the generated summary to the terminal.

[0049] The terminal receives this summary and displays it in the conversation talk room.

[0050] Users can check the summary and quickly grasp important information.

[0051] Specific examples

[0052] Use in project management group chats

[0053] 1. A user opens a group chat for project management.

[0054] 2. Group chats have a lot of messages exchanged, and important information can get buried.

[0055] 3. The user taps the "Conversation Digest" button.

[0056] 4. The device retrieves message data from the past 24 hours and sends it to the server.

[0057] 5. The server supplies the received message data to the generative AI model and begins analysis.

[0058] 6. The generative AI model summarizes the conversation, including the following message:

[0059] "Person A: Please give me a progress report on the project."

[0060] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[0061] "Person C: The design proposal is complete. Please review it."

[0062] "D: What time is the meeting today?"

[0063] 7. A generative AI model summarizes these messages and generates the following summary:

[0064] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0065] 8. The server sends the summary to the terminal.

[0066] 9. The device receives the summary and displays it in the conversation chat room.

[0067] 10. Users can quickly review the summary and grasp the important information.

[0068] In this way, by using the system of the present invention, users can efficiently extract important information from a large number of messages and make quick decisions, thereby improving the efficiency of business communications and preventing work delays.

[0069] The processing flow will be explained below.

[0070] Step 1:

[0071] A user uses a communication terminal to start a chat application and access a talk room.

[0072] The device retrieves the chat room data and displays it on the screen.

[0073] Step 2:

[0074] The device will display a "Conversation Digest" button on the interface within the chat room.

[0075] Step 3:

[0076] The user taps the "Conversation Digest" button.

[0077] The device detects this tap event and starts the following process.

[0078] Step 4:

[0079] The device retrieves the latest message data in the chat room for a certain range (e.g., the past 24 hours).

[0080] The acquired message data is temporarily stored in storage.

[0081] Step 5:

[0082] The terminal transmits the stored message data to the server.

[0083] The message data includes information such as the sender, the time of sending, and the content.

[0084] Step 6:

[0085] The server starts processing to analyze the received message data.

[0086] Feed the data into a generative AI model.

[0087] Step 7:

[0088] Generative AI models analyze message data, understand context and extract key information.

[0089] A concise summary is generated based on the extracted information.

[0090] Step 8:

[0091] The server transmits the generated summary to the terminal.

[0092] Confirm that the summary has been sent.

[0093] Step 9:

[0094] The summary received by the device is displayed in the chat room.

[0095] Users can check the summary and quickly grasp important information.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] In conventional conversation chat rooms, it has been difficult for users to quickly grasp important information amid the large number of messages exchanged. Furthermore, there is no way to concisely summarize the contents of messages, which reduces the efficiency of communication. In response to this problem, the present invention aims to enable users to easily grasp important information in conversation chat rooms.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes means for analyzing message data using a generative AI model, understanding the context, and extracting important information, means for understanding the specific context from the analyzed message data, identifying key points, and providing a prompt sentence to the generative AI model, and means for transmitting the generated summary sentence to the communication terminal, thereby enabling a user to efficiently extract important information from a large number of messages and grasp it in a short time.

[0101] "User" means an individual or entity that accesses and operates the System.

[0102] A "communication terminal" is a device used by a user, and includes smartphones, tablets, personal computers, etc.

[0103] A "conversation talk room" is a virtual environment for users to exchange messages.

[0104] A "conversation digest button" is a button on the interface that a user taps to initiate the conversation summary function.

[0105] "Message data" is a collection of text information exchanged within a conversation talk room.

[0106] A "server" is a computer system that receives message data sent from a communication terminal and performs analysis processing.

[0107] A "generative AI model" is an artificial intelligence program that analyzes and summarizes text data based on machine learning algorithms.

[0108] A "summary" is a concise sentence that contains key information from the original message data, extracted and generated by a generative AI model.

[0109] A "prompt sentence" is an instruction sentence input into a generative AI model and is used during summarization and analysis.

[0110] "Context" refers to the situation or background information that allows for understanding the meaning and relationships within the message data.

[0111] "Important information" refers to the main content or points in the message data that are deemed necessary for the user to know.

[0112] This invention is a system that allows users to use their communication terminals to efficiently summarize message data in a conversation talk room via a server. This system uses a generative AI model to analyze the message data, extract important information, and generate a summary.

[0113] Hardware Configuration

[0114] The communication device used by the user is a smartphone, tablet, or PC. These communication devices display a "Conversation Digest" button as part of their interface.

[0115] The server is a computer system that receives message data sent from the communication device, supplies the data to the generative AI model, and performs analysis and summarization. The server has the processing power to run Python programs and a network connection to access the generative AI model (e.g., OpenAI's GPT-3).

[0116] Software Configuration

[0117] The software of the communication terminal is an application that controls the user interface. The application has a function to display a conversation digest button on the interface and retrieve message data after the user taps the button.

[0118] The server software receives message data, preprocesses it, feeds prompts to the generative AI model, receives the analysis results, generates summaries, and sends the summaries, including Python scripts for data cleansing and metadata extraction.

[0119] Data processing and calculation

[0120] When a device retrieves message data, it typically retrieves the past 24 hours. This data is converted to JSON format and sent to the server.

[0121] The server preprocesses the received data, using Python scripts to cleanse the data and extract metadata, and then passes the preprocessed data to the generative AI model.

[0122] The generative AI model analyzes the message data based on the prompt, extracts important information, and generates a summary, which is then converted back to JSON format and sent from the server to the device.

[0123] Specific examples

[0124] For example, consider the following messages exchanged in a project management group chat:

[0125] "Person A: Please give me a progress report on the project."

[0126] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[0127] "Person C: The design proposal is complete. Please review it."

[0128] "D: What time is the meeting today?"

[0129] When the user taps the "Conversation Digest" button, the device retrieves these messages and sends them to the server, which uses a generative AI model to extract key information and generate the following summary:

[0130] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0131] The generated summary is sent to the terminal and displayed to the user in the conversation chat room, allowing the user to quickly grasp important information.

[0132] In the above-described form, the present invention provides a means for a user to efficiently extract important information from a large number of messages and make quick decisions.

[0133] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0134] Step 1:

[0135] The user taps the "Conversation Digest" button.

[0136] Input: The user taps the conversation digest button.

[0137] Output: Trigger to start the conversation digest function.

[0138] Specific operation: The conversation digest button is tapped on the communication device interface, which uses a user interface written in JavaScript or a similar language.

[0139] Step 2:

[0140] The terminal retrieves the message data.

[0141] Input: Trigger to start the conversation digest function.

[0142] Output: Message data for the past 24 hours.

[0143] Specific operation: The device accesses a database or local storage and retrieves message data for the past 24 hours using an SQL query, etc.

[0144] Step 3:

[0145] The terminal transmits the message data to the server.

[0146] Input: Message data from the past 24 hours.

[0147] Output: Message data converted to JSON format.

[0148] Specific operation: The acquired message data is converted into JSON format and sent to the server using the HTTPS protocol, using a Python library or HTTP client.

[0149] Step 4:

[0150] The server pre-processes the message data.

[0151] Input: Message data in JSON format.

[0152] Output: Preprocessed data and metadata.

[0153] Specific operation: A Python script parses the received data, performs data cleansing (removing missing values, removing text noise, etc.), and extracts metadata (e.g., speaker, timestamp).

[0154] Step 5:

[0155] The server feeds the preprocessed data to the generative AI model.

[0156] Input: Preprocessed data and metadata.

[0157] Output: The prompt sentence that is fed to the generative AI model.

[0158] Specific operation: A prompt sentence is generated based on the preprocessed data and fed to a generative AI model (e.g., GPT-3). This prompt sentence is structured in text format as follows:

[0159] "Please summarize the following message: 'Message data content'"

[0160] Step 6:

[0161] A generative AI model generates a summary.

[0162] Input: The prompt sentence fed to the generative AI model.

[0163] Output: The generated summary.

[0164] How it works: The generative AI model analyzes message data based on the prompt, extracts key information, and generates a summary, a process that takes anywhere from a few seconds to a few minutes.

[0165] Step 7:

[0166] The server transmits the generated summary to the terminal.

[0167] Input: The generated summary sentence.

[0168] Output: Summary text converted to JSON format.

[0169] Specific operation: The generated summary is converted back into JSON format and sent to the terminal via the HTTPS protocol.

[0170] Step 8:

[0171] The terminal displays the summary.

[0172] Input: Abstract in JSON format.

[0173] Output: A summary statement that is displayed in the user interface.

[0174] Specific operation: The device parses the received summary text and displays it on the interface, using a JavaScript framework (e.g., React or Vue.js).

[0175] Step 9:

[0176] The user checks the summary.

[0177] Input: The summary text displayed in the user interface.

[0178] Output: Important information that the user will understand in a short time.

[0179] Specific operation: The user can check the displayed summary text and quickly grasp important information.

[0180] (Application example 1)

[0181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] While autonomous vehicles are operating, it is difficult for drivers and passengers to quickly and efficiently grasp important traffic-related information, such as traffic conditions and accident information, in real time. Furthermore, important information can get lost in the vast amount of traffic information collected, potentially reducing operational efficiency and safety. There is a need for a system that can solve these issues and support quick decision-making while driving.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0184] In this invention, the server includes: means for a user to access a conversation talk room using a communication terminal; means for the user to tap a conversation digest button; means for detecting the user's operation and acquiring message data in the conversation talk room; means for transmitting the acquired message data to the server; means for the server to analyze the message data using a generative AI model and extract important information; means for the server to transmit the generated summary to the communication terminal; means for the communication terminal to display the summary; means for acquiring traffic-related messages while the autonomous vehicle is in operation and generating a summary; and means for displaying the generated summary on an in-vehicle display, thereby enabling drivers and passengers to quickly grasp important traffic-related information and make appropriate decisions even while the autonomous vehicle is in operation.

[0185] A "communication terminal" is an electronic device that a user uses for communication.

[0186] A "conversation talk room" is a virtual space where users can exchange messages.

[0187] The "conversation digest button" is an operation button for summarizing messages in a chat room.

[0188] "Sensing an operation" means that the communication terminal detects an input action performed by the user.

[0189] "Message data" refers to text information exchanged within a chat room.

[0190] "Server" means a computer system that provides data analysis and storage functions.

[0191] A "generative AI model" is an algorithm that uses artificial intelligence to extract important information from data and generate a summary.

[0192] A "summary" is a sentence that shortens a long sentence or message and summarizes the main points.

[0193] An "autonomous vehicle" is a vehicle that operates autonomously without the need for a human driver.

[0194] A "traffic-related message" is a message that includes information about traffic conditions or traffic.

[0195] An "in-vehicle display" is a screen device that is installed inside a vehicle and displays information.

[0196] The following specific example is provided as an embodiment of the present invention. This system efficiently summarizes traffic-related message data while a user is driving an autonomous vehicle using a communication terminal. The details of this system are described below.

[0197] System Overview

[0198] Using the Traffic Digest Feature

[0199] The user presses the traffic digest button on the in-vehicle display of the autonomous vehicle, which causes the system to retrieve and summarize traffic-related messages from the past 24 hours.

[0200] Getting and sending messages

[0201] The terminal detects user operation and detects that the traffic digest button has been pressed, which causes the communication terminal to obtain traffic-related message data for the past 24 hours from the communication terminal of the autonomous vehicle.

[0202] The acquired message data is temporarily stored on the device and then sent to a server for analysis.

[0203] Message analysis and summarization

[0204] The server processes the received message data and supplies it to a generative AI model, such as OpenAI's GPT-3.

[0205] The generative AI model analyzes message data, understands the context, and extracts important information. Based on this information, the generative AI model generates a concise summary.

[0206] Sending and displaying abstracts

[0207] The server transmits the generated summary to the communication terminal.

[0208] The terminal receives the summary and displays it on the in-vehicle display, allowing the user to quickly understand important information by checking the summary.

[0209] Specific examples

[0210] For example, consider the following traffic-related messages stored in a vehicle's communications terminal:

[0211] "08:00 Traffic congestion occurs, traffic volume increases"

[0212] "10:00 Accident occurred, detour recommended"

[0213] "15:00 Traffic decrease"

[0214] "18:00 Traffic jam occurs again"

[0215] When the user presses the traffic digest button on the in-car display, the system sends these messages to the server and generates a summary like this:

[0216] Traffic information: Traffic increased around 08:00, causing congestion. An accident occurred around 10:00, and a detour was recommended. Traffic decreased around 15:00, and congestion reoccurred around 18:00.

[0217] Example prompt sentence:

[0218] Congestion and increased traffic volume

[0219] Accident occurred. Detour recommended.

[0220] Traffic decrease

[0221] Traffic jams occur again

[0222] This will enable drivers and passengers to quickly grasp important traffic-related information and make appropriate decisions while driving autonomous vehicles, contributing to improved operational efficiency and safety.

[0223] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0224] Step 1:

[0225] The user presses the traffic digest button displayed on the in-vehicle display of the autonomous vehicle. The input is the user's operation, and the output is the terminal detecting the operation. Specifically, the button press is recorded by touching the in-vehicle display.

[0226] Step 2:

[0227] The device detects user actions and retrieves traffic-related message data from the past 24 hours. The input is the user pressing a button, and the output is retrieving traffic-related message data. The device collects the latest traffic-related messages from its internal memory or a cloud service.

[0228] Step 3:

[0229] The acquired message data is sent to the server. The input is the traffic-related message data collected by the device, and the output is sending the data to the server. The device sends the message data as a POST request to the server's API endpoint via an Internet connection.

[0230] Step 4:

[0231] The server processes the received message data and supplies it to the generative AI model. The input is traffic-related message data received from the device, and the output is data to be analyzed and passed to the generative AI model. The server preprocesses the received data, converts it into an appropriate format, and sends it to the model.

[0232] Step 5:

[0233] The generative AI model analyzes the message data and extracts key information. The input is preprocessed traffic-related message data, and the output is the extracted key information. Specifically, the generative AI model analyzes the context of traffic messages and identifies important events and information.

[0234] Step 6:

[0235] The generative AI model generates a concise summary based on the extracted information. The input is the extracted key information, and the output is the generated summary. The model converts the key information into a summary format and creates a summary in an easy-to-understand style.

[0236] Step 7:

[0237] The server sends the generated summary to the communication terminal. The input is the generated summary, and the output is the summary sent to the communication terminal. The server again sends the generated summary to the terminal using the API endpoint.

[0238] Step 8:

[0239] The terminal receives the summary and displays it on the in-vehicle display. The input is the summary sent from the server, and the output is the summary displayed on the screen. The terminal receives the summary and displays it on the display so that the user can easily read it.

[0240] This trend will lead to the realization of a system that summarizes important traffic information in real time within autonomous vehicles and provides it to users.

[0241] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0242] The following specific example is provided as a mode for carrying out the present invention: This system allows a user to use a communication terminal and efficiently summarizes message data in a conversation talk room via a server, and further incorporates an emotion engine that recognizes the user's emotions.

[0243] System Overview

[0244] Using conversation digests and emotion recognition

[0245] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[0246] When a user taps this "Conversation Digest" button, a summary of the messages in the conversation chat room will begin.

[0247] Message capture and sentiment data collection

[0248] The device detects user actions and retrieves the latest message data in the conversation chat room. This data typically includes messages from the past 24 hours or a customizable range.

[0249] The device also simultaneously collects and temporarily stores the user's emotional data.

[0250] Sending message data and emotion data

[0251] The acquired message data and emotion data are temporarily stored on the device and then sent to a server for analysis.

[0252] Message and sentiment analysis

[0253] The server processes the received message data and emotion data and supplies it to the generative AI model.

[0254] Generative AI models analyze message data to understand context and extract key information.

[0255] The emotion engine analyzes the user's emotional data and adjusts the summary content according to the user's emotional state. For example, if the user is feeling stressed, the system generates a more concise summary that focuses on the important information.

[0256] Sending and displaying abstracts

[0257] The server sends the generated summary to the terminal.

[0258] The terminal receives this summary and displays it in the conversation talk room.

[0259] Users can check the summary and quickly grasp important information.

[0260] Specific examples

[0261] Use in stressful project management group chats

[0262] 1. A user opens a group chat for project management.

[0263] 2. Group chats are often filled with messages, and important information can get buried, causing stress for users.

[0264] 3. The user taps the "Conversation Digest" button.

[0265] 4. The device retrieves message data from the past 24 hours, collects emotion data, and sends it to the server.

[0266] 5. The server supplies the received message data and emotion data to the generative AI model and begins analysis.

[0267] 6. The generative AI model summarizes the conversation, including the following message:

[0268] "Person A: Please give me a progress report on the project."

[0269] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[0270] "Person C: The design proposal is complete. Please review it."

[0271] "D: What time is the meeting today?"

[0272] 7. The emotion engine takes into account the user's stress level and generates a concise summary that focuses on the important information.

[0273] 8. The generative AI model generates the following summary:

[0274] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0275] 9. The server sends the summary to the terminal.

[0276] 10. The device receives the summary and displays it in the conversation room.

[0277] 11. Users can quickly review the summary and grasp the important information.

[0278] In this way, the system of the present invention allows users to efficiently extract important information from a large number of messages and make quick decisions. Furthermore, by taking into account the user's emotional state, the system provides a more appropriate summary, contributing to stress reduction.

[0279] The processing flow will be explained below.

[0280] Step 1:

[0281] A user uses a communication terminal to start a chat application and access a talk room.

[0282] The device retrieves the chat room data and displays it on the screen.

[0283] Step 2:

[0284] The device will display a "Conversation Digest" button on the interface within the chat room.

[0285] Step 3:

[0286] The user taps the "Conversation Digest" button.

[0287] The device detects this tap event and starts the following process.

[0288] Step 4:

[0289] The device retrieves the latest message data in the chat room for a certain range (e.g., the past 24 hours).

[0290] At the same time, the device also collects the user's emotional data and temporarily stores it in storage.

[0291] Step 5:

[0292] The message data and emotion data stored on the device are transmitted to the server.

[0293] The message data includes information such as the sender, the time of sending, and the content.

[0294] Step 6:

[0295] The server processes the received message data and emotion data.

[0296] This includes pre-processing the data to ensure accurate analysis of the entire data set.

[0297] Step 7:

[0298] The server uses the generative AI model to analyze the message data.

[0299] Generative AI models understand the context of messages and extract key information.

[0300] Step 8:

[0301] An emotion engine analyzes the user's emotion data to identify the user's emotional state.

[0302] The emotion engine determines whether the user is stressed or relaxed.

[0303] Step 9:

[0304] A concise summary is generated based on the key information extracted by the generative AI model.

[0305] At the same time, the content of the summary is adjusted based on information from the emotion engine.

[0306] For example, if the user is feeling stressed, a more concise and to-the-point summary will be generated.

[0307] Step 10:

[0308] The server transmits the generated summary to the terminal.

[0309] Confirm that the summary has been sent.

[0310] Step 11:

[0311] The summary received by the device is displayed in the chat room.

[0312] Users can check the summary and quickly grasp important information.

[0313] Step 12:

[0314] Collect user feedback and fine-tune the generative AI model and emotion engine algorithms as needed.

[0315] Example 2

[0316] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0317] In today's information society, users are increasingly using communication platforms where a large number of messages are exchanged. However, it is difficult to efficiently extract and understand important information from these numerous messages. The stress users experience while reading messages is also a major problem. Conventional systems simply display messages, without summarizing or adjusting them based on emotions, which results in information being scattered and hinders efficient decision-making. Therefore, a method is needed to enable users to efficiently grasp important information and reduce stress.

[0318] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring message data and user emotion data, means for analyzing the message data using a generative AI model to understand the context and extract important information, and means for analyzing the user emotion data using an emotion engine and adjusting the summary sentence according to the user's emotional state. This allows the user to quickly grasp automatically summarized information from a large number of messages, and furthermore, by adjusting the information according to the user's emotional state, less stressful communication is realized.

[0319] A "communication terminal" is a device that allows a user to connect to the Internet and use various services, and includes smartphones, PCs, tablets, etc.

[0320] "Conversation talk room" refers to a chat room or messaging platform where multiple users can exchange messages.

[0321] "User" refers to an individual who uses a communication terminal to participate in a conversation talk room and send, receive, and view messages.

[0322] "Conversation Digest Button" refers to a button on the interface that activates a function that summarizes messages in a conversation chat room.

[0323] "Message Data" refers to text and media information sent and received by a user within a conversation chat room.

[0324] "Emotion data" refers to information about the user's emotional state obtained from facial expressions, voice, self-input, etc.

[0325] "Server" refers to a central control system for receiving, analyzing, and generating message data and emotion data.

[0326] "Generative AI Model" means an artificial intelligence model used by the Server that includes algorithms that analyze message data and extract key information.

[0327] An "emotion engine" refers to a system that analyzes a user's emotional data and adjusts summary generation according to the user's emotional state.

[0328] This invention relates to a system that allows users to use their communication terminals to efficiently summarize message data in a conversation talk room via a server. It also incorporates an emotion engine that recognizes the user's emotions and adjusts the content of the summary.

[0329] System Overview

[0330] Hardware and software used

[0331] The present invention uses the following hardware and software.

[0332] Communication device: A device that allows users to send and receive messages and use summary functions. This includes smartphones, PCs, tablets, etc.

[0333] Server: A central control system that receives message data and emotion data, analyzes and processes them using generative AI models and emotion engines.

[0334] Generative AI models: Algorithms that analyze message data, understand context, and extract key information.

[0335] Emotion engine: A system that analyzes the user's emotional data and adjusts summary generation according to the user's emotional state.

[0336] Data Flow and Processing

[0337] 1. User Action:

[0338] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[0339] When the user taps this "Conversation Digest" button, the summary process begins.

[0340] 2. Data Acquisition:

[0341] The device detects the user's operations and acquires message data and user emotion data in the conversation talk room.

[0342] This data typically includes messages from the past 24 hours, or a range you configure.

[0343] 3. Data transmission and analysis:

[0344] The message data and emotion data acquired by the terminal are transmitted to a server via the Internet.

[0345] The server formats the received data and feeds it to the generative AI model.

[0346] A generative AI model analyzes message data and extracts key information.

[0347] 4. Use of Emotion Engine:

[0348] The server uses an emotion engine to analyze the user's emotion data and adjusts the summary according to the user's emotional state.

[0349] For example, if a user is feeling stressed, a summary will be generated that is more concise and focuses on the important information.

[0350] 5. Send and display abstracts:

[0351] The server sends the generated summary to the terminal.

[0352] The device receives the summary and displays it in the conversation chat room.

[0353] Users can check the summary and quickly grasp important information.

[0354] Specific examples

[0355] Use in stressful project management group chats

[0356] 1. A user opens a group chat for project management.

[0357] 2. Group chats can be filled with messages, and important information can get lost in the shuffle, causing frustration for users.

[0358] 3. The user taps the "Conversation Digest" button.

[0359] 4. The device retrieves message data from the past 24 hours, collects emotion data, and sends it to the server.

[0360] 5. The server passes the following prompt to the generative AI model:

[0361] "Project status report, task completion, next phase, design proposal submission, review request, meeting time proposal"

[0362] 6. The server adjusts the emotion engine and generates the following summary:

[0363] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0364] 7. The server sends the summary to the terminal.

[0365] 8. The device receives the summary and displays it in the conversation chat room.

[0366] 9. Users can quickly review the summary and grasp the important information.

[0367] This allows users to efficiently extract important information from a large number of messages and make quick decisions.The emotion engine also provides summaries based on the user's emotional state, contributing to stress reduction.

[0368] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0369] Step 1:

[0370] A user accesses a conversation talk room using a communication terminal.

[0371] Users log in to a chat room using a dedicated application or web browser on a device such as a smartphone or PC. The input is the user's access operation, and the output is the display of the chat room interface, allowing the user to send and receive messages.

[0372] Step 2:

[0373] The user taps the "Conversation Digest" button.

[0374] The user taps the "Conversation Digest" button displayed in the chat room to start the process. The input is the button tap event, and the output is that this event is triggered to proceed to the next step.

[0375] Step 3:

[0376] The device acquires message data and user emotion data from within the chat room.

[0377] The device retrieves message data from the server for the past 24 hours or any specified range, and simultaneously acquires user emotion data from the camera and microphone. Input is button tap events and data read from the interface, and output is temporary storage of the retrieved message data and emotion data.

[0378] Step 4:

[0379] The message data and emotion data acquired by the terminal are transmitted to the server.

[0380] The device sends the temporarily stored data to a server via the Internet. The input is the acquired message data and emotion data, and the output is sending this data to the server. It is preferable to use encryption technology such as SSL / TLS for transmission.

[0381] Step 5:

[0382] The server analyzes the received data and feeds it into a generative AI model.

[0383] The server formats the received message data and emotion data into an appropriate format. The input is the data sent from the device, and the output is a set of formatted data. These data are fed into the generative AI model.

[0384] Step 6:

[0385] A generative AI model analyzes message data and extracts key information.

[0386] The generative AI model analyzes the provided message data, understands the context, and extracts key information. The input is the formatted message data, and the output is the extracted key information. Prompt sentences can be used to improve the accuracy of the model.

[0387] Step 7:

[0388] The emotion engine analyzes the user's emotion data and adjusts the summary text.

[0389] The emotion engine analyzes the user's emotional data and adjusts the summary based on the user's emotional state. For example, if the user is feeling stressed, the summary will be concise and focus on the important information. The input is the emotional data and extracted important information, and the output is a summary adjusted according to the user's emotion.

[0390] Step 8:

[0391] The server sends the generated summary to the terminal.

[0392] The server sends the summary adjusted by the emotion engine to the terminal. The input is the adjusted summary, and the output is the summary reaching the terminal. HTTP or WebSocket is generally used as the transmission protocol.

[0393] Step 9:

[0394] The device receives the summary and displays it in the conversation chat room.

[0395] The terminal displays the summary received from the server on the user interface. The input is the summary from the server, and the output is the user checking the summary. This allows the user to grasp important information in a short time.

[0396] This series of processing steps allows users to efficiently extract important information from a large number of messages, realizing stress-free communication.

[0397] (Application example 2)

[0398] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0399] In logistics centers, business communications are routinely carried out via large volumes of messages and voice input, but important information often gets buried, resulting in problems of reduced work efficiency and increased stress.In addition, there is the issue of the quality of communication not improving because information is provided uniformly without considering the emotional state of staff.

[0400] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing message data using a generative AI model and an emotion engine, extracting important information, and adjusting the content of the summary based on the user's emotional state, means for transmitting the summary generated by the server to the communication terminal, means for the communication terminal to display the summary, and means for summarizing and displaying business communications conducted by conversation and voice input. This makes it possible to efficiently extract important information from business communications at a logistics center and provide it in a form that takes into consideration the user's emotional state.

[0401] "User" refers to the staff working at the logistics center.

[0402] "Communication terminal" refers to a device that can connect to the Internet, including wearable devices such as smart glasses or smartphones.

[0403] "Conversation talk room" refers to a chat room or voice conversation space where business communications take place at a logistics center.

[0404] A "conversation digest button" refers to an interface that a user can tap to start a conversation summary.

[0405] "Message data" refers to text and audio data in a conversation talk room obtained from a communication terminal.

[0406] "Emotion data" refers to data that indicates the user's emotional state.

[0407] "Server" refers to the computer system that is equipped with the generative AI model and emotion engine and performs data analysis.

[0408] "Generative AI model" refers to an artificial intelligence model that analyzes acquired message data and extracts important information.

[0409] "Emotion engine" refers to a system for analyzing a user's emotional data and adjusting the content of the summary.

[0410] "Summary" refers to a sentence that summarizes the main points of a message, generated by a generative AI model and emotion engine.

[0411] The present invention is embodied in a system for summarizing business communications in a logistics center, using wearable devices such as smart glasses or smartphones as communication terminals.

[0412] System Configuration and Operation

[0413] 1. A user uses a communication device to access a conversation talk room.

[0414] Smart glasses (e.g., Google Glass, Microsoft HoloLens) are used as communication devices.

[0415] Users can communicate through text chat or voice input within the conversation room.

[0416] 2. Click the conversation digest button.

[0417] When a user taps the conversation digest button, the system retrieves message and emotion data from the past 24 hours. This button appears on the HUD (Head-Up Display).

[0418] 3. Data acquisition and transmission.

[0419] The communication terminal acquires the message data and emotion data and temporarily stores them.

[0420] The data is sent to a server where it is analyzed using a generative AI model (e.g., OpenAI GPT-3) and an emotion engine (e.g., Affectiva Emotion SDK).

[0421] 4. Data analysis and key points extraction.

[0422] The server uses a generative AI model to analyze the message data and extract key information.

[0423] The emotion engine analyzes the user's emotion data and adjusts the content of the summary according to the user's stress level.

[0424] 5. Send and display the summary.

[0425] The server transmits the generated summary to the communication terminal.

[0426] The communication terminal receives the summary and displays it on the glasses' HUD.

[0427] Specific examples

[0428] Usage example 1

[0429] Prompt statement:

[0430] Logistics Center Staff A: "What's the delivery schedule for this week?"

[0431] Logistics Center Staff B: "We have 10 cases to be delivered on Monday and 20 cases to be delivered on Tuesday."

[0432] summary:

[0433] "This week's delivery schedule: 10 cases on Monday, 20 cases on Tuesday."

[0434] Usage example 2

[0435] Prompt statement:

[0436] Logistics Center Staff C: "Which shelf should I place the next item on?"

[0437] Logistics Center Staff D: "Please place item X on shelf A and item Y on shelf B."

[0438] summary:

[0439] "Item X is on shelf A and item Y is on shelf B"

[0440] In this way, by implementing the invention, it is possible to efficiently extract important information from business communications at a logistics center and provide the information while taking into consideration the emotional state of the user, which is expected to improve work efficiency and reduce stress.

[0441] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0442] Step 1:

[0443] A user uses a communication terminal to access a conversation talk room.

[0444] Specific behavior:

[0445] The communication device (smart glasses or smartphone) connects to the server via the Internet and displays a screen of text chat and voice input data within the conversation room.

[0446] Step 2:

[0447] The user taps the conversation digest button.

[0448] Specific behavior:

[0449] The user taps the "Conversation Digest" button displayed on the communication device's HUD. This action triggers the acquisition of message and emotion data from the past 24 hours.

[0450] Input: User taps

[0451] Output: Start of data acquisition process

[0452] Step 3:

[0453] The device acquires the latest message data and emotion data in the conversation chat room and temporarily stores them.

[0454] Specific behavior:

[0455] Software built into the communication device retrieves the latest message data from the server via an API, and uses the camera and microphone to collect emotional data from the user's facial expressions and tone of voice, which is then temporarily stored in local storage.

[0456] Input: Message data in the conversation chat room, user emotion data

[0457] Output: Temporarily save the acquired data

[0458] Step 4:

[0459] The terminal transmits the acquired message data and emotion data to the server.

[0460] Specific behavior:

[0461] The communication device sends the acquired message data and emotion data to the server via an HTTP request. The sent data is encoded in JSON format.

[0462] Input: Temporarily saved message data and emotion data

[0463] Output: Send data to the server

[0464] Step 5:

[0465] The server uses a generative AI model and an emotion engine to analyze message data, extract key information, and adjust the content of the summary based on the user's emotional state.

[0466] Specific behavior:

[0467] The server inputs the received data into a generative AI model (e.g., OpenAI GPT-3), which analyzes the message data and extracts context and important information. At the same time, an emotion engine (e.g., Affectiva Emotion SDK) analyzes the user's emotional data and makes adjustments, such as making the summary more concise, if the user is feeling stressed.

[0468] Input: Message data and emotion data sent to the server

[0469] Output: Adjusted summary

[0470] Step 6:

[0471] The server transmits the generated summary to the communication terminal.

[0472] Specific behavior:

[0473] The server sends the generated summary to the communication terminal via an HTTP response. The response data is encoded in JSON format.

[0474] Input: Adjusted summary

[0475] Output: Send summary to communication terminal

[0476] Step 7:

[0477] The communication terminal receives the summary and displays it in the conversation talk room.

[0478] Specific behavior:

[0479] The communication terminal displays the received summary on a HUD (Head-Up Display) and provides it to the user, allowing the user to quickly grasp important information.

[0480] Input: Abstract sent from the server

[0481] Output: Summary displayed on HUD

[0482] Through the above steps, important information can be efficiently extracted from business communications at a logistics center, and a summary can be provided that takes into consideration the user's emotional state.

[0483] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0484] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0485] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0486] [Second embodiment]

[0487] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0488] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0489] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0490] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0491] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0492] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0493] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0494] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0495] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0496] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0497] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0498] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0499] The following specific example is given as an embodiment for carrying out the present invention: In this system, a user uses a communication terminal to efficiently summarize message data in a conversation talk room via a server.

[0500] System Overview

[0501] Using the Conversation Digest Feature

[0502] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[0503] When a user taps this "Conversation Digest" button, a summary of the messages in the conversation chat room will begin.

[0504] Getting and sending messages

[0505] The device detects user actions and retrieves the latest message data in the conversation chat room. This data typically includes messages from the past 24 hours or a customizable range.

[0506] The acquired message data is temporarily stored on the device and then sent to a server for analysis.

[0507] Message analysis and summarization

[0508] The server processes the received message data and supplies it to the generative AI model.

[0509] The generative AI model analyzes message data, understands the context, and extracts important information. Based on this information, the generative AI model generates a concise summary.

[0510] Sending and displaying abstracts

[0511] The server sends the generated summary to the terminal.

[0512] The terminal receives this summary and displays it in the conversation talk room.

[0513] Users can check the summary and quickly grasp important information.

[0514] Specific examples

[0515] Use in project management group chats

[0516] 1. A user opens a group chat for project management.

[0517] 2. Group chats have a lot of messages exchanged, and important information can get buried.

[0518] 3. The user taps the "Conversation Digest" button.

[0519] 4. The device retrieves message data from the past 24 hours and sends it to the server.

[0520] 5. The server supplies the received message data to the generative AI model and begins analysis.

[0521] 6. The generative AI model summarizes the conversation, including the following message:

[0522] "Person A: Please give me a progress report on the project."

[0523] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[0524] "Person C: The design proposal is complete. Please review it."

[0525] "D: What time is the meeting today?"

[0526] 7. A generative AI model summarizes these messages and generates the following summary:

[0527] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0528] 8. The server sends the summary to the terminal.

[0529] 9. The device receives the summary and displays it in the conversation chat room.

[0530] 10. Users can quickly review the summary and grasp the important information.

[0531] In this way, by using the system of the present invention, users can efficiently extract important information from a large number of messages and make quick decisions, thereby improving the efficiency of business communications and preventing work delays.

[0532] The processing flow will be explained below.

[0533] Step 1:

[0534] A user uses a communication terminal to start a chat application and access a talk room.

[0535] The device retrieves the chat room data and displays it on the screen.

[0536] Step 2:

[0537] The device will display a "Conversation Digest" button on the interface within the chat room.

[0538] Step 3:

[0539] The user taps the "Conversation Digest" button.

[0540] The device detects this tap event and starts the following process.

[0541] Step 4:

[0542] The device retrieves the latest message data in the chat room for a certain range (e.g., the past 24 hours).

[0543] The acquired message data is temporarily stored in storage.

[0544] Step 5:

[0545] The terminal transmits the stored message data to the server.

[0546] The message data includes information such as the sender, the time of sending, and the content.

[0547] Step 6:

[0548] The server starts processing to analyze the received message data.

[0549] Feed the data into a generative AI model.

[0550] Step 7:

[0551] Generative AI models analyze message data, understand context and extract key information.

[0552] A concise summary is generated based on the extracted information.

[0553] Step 8:

[0554] The server transmits the generated summary to the terminal.

[0555] Confirm that the summary has been sent.

[0556] Step 9:

[0557] The summary received by the device is displayed in the chat room.

[0558] Users can check the summary and quickly grasp important information.

[0559] Example 1

[0560] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0561] In conventional conversation chat rooms, it has been difficult for users to quickly grasp important information amid the large number of messages exchanged. Furthermore, there is no way to concisely summarize the contents of messages, which reduces the efficiency of communication. In response to this problem, the present invention aims to enable users to easily grasp important information in conversation chat rooms.

[0562] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0563] In this invention, the server includes means for analyzing message data using a generative AI model, understanding the context, and extracting important information, means for understanding the specific context from the analyzed message data, identifying key points, and providing a prompt sentence to the generative AI model, and means for transmitting the generated summary sentence to the communication terminal, thereby enabling a user to efficiently extract important information from a large number of messages and grasp it in a short time.

[0564] "User" means an individual or entity that accesses and operates the System.

[0565] A "communication terminal" is a device used by a user, and includes smartphones, tablets, personal computers, etc.

[0566] A "conversation talk room" is a virtual environment for users to exchange messages.

[0567] A "conversation digest button" is a button on the interface that a user taps to initiate the conversation summary function.

[0568] "Message data" is a collection of text information exchanged within a conversation talk room.

[0569] A "server" is a computer system that receives message data sent from a communication terminal and performs analysis processing.

[0570] A "generative AI model" is an artificial intelligence program that analyzes and summarizes text data based on machine learning algorithms.

[0571] A "summary" is a concise sentence that contains key information from the original message data, extracted and generated by a generative AI model.

[0572] A "prompt sentence" is an instruction sentence input into a generative AI model and is used during summarization and analysis.

[0573] "Context" refers to the situation or background information that allows for understanding the meaning and relationships within the message data.

[0574] "Important information" refers to the main content or points in the message data that are deemed necessary for the user to know.

[0575] This invention is a system that allows users to use their communication terminals to efficiently summarize message data in a conversation talk room via a server. This system uses a generative AI model to analyze the message data, extract important information, and generate a summary.

[0576] Hardware Configuration

[0577] The communication device used by the user is a smartphone, tablet, or PC. These communication devices display a "Conversation Digest" button as part of their interface.

[0578] The server is a computer system that receives message data sent from the communication device, supplies the data to the generative AI model, and performs analysis and summarization. The server has the processing power to run Python programs and a network connection to access the generative AI model (e.g., OpenAI's GPT-3).

[0579] Software Configuration

[0580] The software of the communication terminal is an application that controls the user interface. The application has a function to display a conversation digest button on the interface and retrieve message data after the user taps the button.

[0581] The server software receives message data, preprocesses it, feeds prompts to the generative AI model, receives the analysis results, generates summaries, and sends the summaries, including Python scripts for data cleansing and metadata extraction.

[0582] Data processing and calculation

[0583] When a device retrieves message data, it typically retrieves the past 24 hours. This data is converted to JSON format and sent to the server.

[0584] The server preprocesses the received data, using Python scripts to cleanse the data and extract metadata, and then passes the preprocessed data to the generative AI model.

[0585] The generative AI model analyzes the message data based on the prompt, extracts important information, and generates a summary, which is then converted back to JSON format and sent from the server to the device.

[0586] Specific examples

[0587] For example, consider the following messages exchanged in a project management group chat:

[0588] "Person A: Please give me a progress report on the project."

[0589] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[0590] "Person C: The design proposal is complete. Please review it."

[0591] "D: What time is the meeting today?"

[0592] When the user taps the "Conversation Digest" button, the device retrieves these messages and sends them to the server, which uses a generative AI model to extract key information and generate the following summary:

[0593] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0594] The generated summary is sent to the terminal and displayed to the user in the conversation chat room, allowing the user to quickly grasp important information.

[0595] In the above-described form, the present invention provides a means for a user to efficiently extract important information from a large number of messages and make quick decisions.

[0596] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0597] Step 1:

[0598] The user taps the "Conversation Digest" button.

[0599] Input: The user taps the conversation digest button.

[0600] Output: Trigger to start the conversation digest function.

[0601] Specific operation: The conversation digest button is tapped on the communication device interface, which uses a user interface written in JavaScript or a similar language.

[0602] Step 2:

[0603] The terminal retrieves the message data.

[0604] Input: Trigger to start the conversation digest function.

[0605] Output: Message data for the past 24 hours.

[0606] Specific operation: The device accesses a database or local storage and retrieves message data for the past 24 hours using an SQL query, etc.

[0607] Step 3:

[0608] The terminal transmits the message data to the server.

[0609] Input: Message data from the past 24 hours.

[0610] Output: Message data converted to JSON format.

[0611] Specific operation: The acquired message data is converted into JSON format and sent to the server using the HTTPS protocol, using a Python library or HTTP client.

[0612] Step 4:

[0613] The server pre-processes the message data.

[0614] Input: Message data in JSON format.

[0615] Output: Preprocessed data and metadata.

[0616] Specific operation: A Python script parses the received data, performs data cleansing (removing missing values, removing text noise, etc.), and extracts metadata (e.g., speaker, timestamp).

[0617] Step 5:

[0618] The server feeds the preprocessed data to the generative AI model.

[0619] Input: Preprocessed data and metadata.

[0620] Output: The prompt sentence that is fed to the generative AI model.

[0621] Specific operation: A prompt sentence is generated based on the preprocessed data and fed to a generative AI model (e.g., GPT-3). This prompt sentence is structured in text format as follows:

[0622] "Please summarize the following message: 'Message data content'"

[0623] Step 6:

[0624] A generative AI model generates a summary.

[0625] Input: The prompt sentence fed to the generative AI model.

[0626] Output: The generated summary.

[0627] How it works: The generative AI model analyzes message data based on the prompt, extracts key information, and generates a summary, a process that takes anywhere from a few seconds to a few minutes.

[0628] Step 7:

[0629] The server transmits the generated summary to the terminal.

[0630] Input: The generated summary sentence.

[0631] Output: Summary text converted to JSON format.

[0632] Specific operation: The generated summary is converted back into JSON format and sent to the terminal via the HTTPS protocol.

[0633] Step 8:

[0634] The terminal displays the summary.

[0635] Input: Abstract in JSON format.

[0636] Output: A summary statement that is displayed in the user interface.

[0637] Specific operation: The device parses the received summary text and displays it on the interface, using a JavaScript framework (e.g., React or Vue.js).

[0638] Step 9:

[0639] The user checks the summary.

[0640] Input: The summary text displayed in the user interface.

[0641] Output: Important information that the user will understand in a short time.

[0642] Specific operation: The user can check the displayed summary text and quickly grasp important information.

[0643] (Application example 1)

[0644] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0645] While autonomous vehicles are operating, it is difficult for drivers and passengers to quickly and efficiently grasp important traffic-related information, such as traffic conditions and accident information, in real time. Furthermore, important information can get lost in the vast amount of traffic information collected, potentially reducing operational efficiency and safety. There is a need for a system that can solve these issues and support quick decision-making while driving.

[0646] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0647] In this invention, the server includes: means for a user to access a conversation talk room using a communication terminal; means for the user to tap a conversation digest button; means for detecting the user's operation and acquiring message data in the conversation talk room; means for transmitting the acquired message data to the server; means for the server to analyze the message data using a generative AI model and extract important information; means for the server to transmit the generated summary to the communication terminal; means for the communication terminal to display the summary; means for acquiring traffic-related messages while the autonomous vehicle is in operation and generating a summary; and means for displaying the generated summary on an in-vehicle display, thereby enabling drivers and passengers to quickly grasp important traffic-related information and make appropriate decisions even while the autonomous vehicle is in operation.

[0648] A "communication terminal" is an electronic device that a user uses for communication.

[0649] A "conversation talk room" is a virtual space where users can exchange messages.

[0650] The "conversation digest button" is an operation button for summarizing messages in a chat room.

[0651] "Sensing an operation" means that the communication terminal detects an input action performed by the user.

[0652] "Message data" refers to text information exchanged within a chat room.

[0653] "Server" means a computer system that provides data analysis and storage functions.

[0654] A "generative AI model" is an algorithm that uses artificial intelligence to extract important information from data and generate a summary.

[0655] A "summary" is a sentence that shortens a long sentence or message and summarizes the main points.

[0656] An "autonomous vehicle" is a vehicle that operates autonomously without the need for a human driver.

[0657] A "traffic-related message" is a message that includes information about traffic conditions or traffic.

[0658] An "in-vehicle display" is a screen device that is installed inside a vehicle and displays information.

[0659] The following specific example is provided as an embodiment of the present invention. This system efficiently summarizes traffic-related message data while a user is driving an autonomous vehicle using a communication terminal. The details of this system are described below.

[0660] System Overview

[0661] Using the Traffic Digest Feature

[0662] The user presses the traffic digest button on the in-vehicle display of the autonomous vehicle, which causes the system to retrieve and summarize traffic-related messages from the past 24 hours.

[0663] Getting and sending messages

[0664] The terminal detects user operation and detects that the traffic digest button has been pressed, which causes the communication terminal to obtain traffic-related message data for the past 24 hours from the communication terminal of the autonomous vehicle.

[0665] The acquired message data is temporarily stored on the device and then sent to a server for analysis.

[0666] Message analysis and summarization

[0667] The server processes the received message data and supplies it to a generative AI model, such as OpenAI's GPT-3.

[0668] The generative AI model analyzes message data, understands the context, and extracts important information. Based on this information, the generative AI model generates a concise summary.

[0669] Sending and displaying abstracts

[0670] The server transmits the generated summary to the communication terminal.

[0671] The terminal receives the summary and displays it on the in-vehicle display, allowing the user to quickly understand important information by checking the summary.

[0672] Specific examples

[0673] For example, consider the following traffic-related messages stored in a vehicle's communications terminal:

[0674] "08:00 Traffic congestion occurs, traffic volume increases"

[0675] "10:00 Accident occurred, detour recommended"

[0676] "15:00 Traffic decrease"

[0677] "18:00 Traffic jam occurs again"

[0678] When the user presses the traffic digest button on the in-car display, the system sends these messages to the server and generates a summary like this:

[0679] Traffic information: Traffic increased around 08:00, causing congestion. An accident occurred around 10:00, and a detour was recommended. Traffic decreased around 15:00, and congestion reoccurred around 18:00.

[0680] Example prompt sentence:

[0681] Congestion and increased traffic volume

[0682] Accident occurred. Detour recommended.

[0683] Traffic decrease

[0684] Traffic jams occur again

[0685] This will enable drivers and passengers to quickly grasp important traffic-related information and make appropriate decisions while driving autonomous vehicles, contributing to improved operational efficiency and safety.

[0686] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0687] Step 1:

[0688] The user presses the traffic digest button displayed on the in-vehicle display of the autonomous vehicle. The input is the user's operation, and the output is the terminal detecting the operation. Specifically, the button press is recorded by touching the in-vehicle display.

[0689] Step 2:

[0690] The device detects user actions and retrieves traffic-related message data from the past 24 hours. The input is the user pressing a button, and the output is retrieving traffic-related message data. The device collects the latest traffic-related messages from its internal memory or a cloud service.

[0691] Step 3:

[0692] The acquired message data is sent to the server. The input is the traffic-related message data collected by the device, and the output is sending the data to the server. The device sends the message data as a POST request to the server's API endpoint via an Internet connection.

[0693] Step 4:

[0694] The server processes the received message data and supplies it to the generative AI model. The input is traffic-related message data received from the device, and the output is data to be analyzed and passed to the generative AI model. The server preprocesses the received data, converts it into an appropriate format, and sends it to the model.

[0695] Step 5:

[0696] The generative AI model analyzes the message data and extracts key information. The input is preprocessed traffic-related message data, and the output is the extracted key information. Specifically, the generative AI model analyzes the context of traffic messages and identifies important events and information.

[0697] Step 6:

[0698] The generative AI model generates a concise summary based on the extracted information. The input is the extracted key information, and the output is the generated summary. The model converts the key information into a summary format and creates a summary in an easy-to-understand style.

[0699] Step 7:

[0700] The server sends the generated summary to the communication terminal. The input is the generated summary, and the output is the summary sent to the communication terminal. The server again sends the generated summary to the terminal using the API endpoint.

[0701] Step 8:

[0702] The terminal receives the summary and displays it on the in-vehicle display. The input is the summary sent from the server, and the output is the summary displayed on the screen. The terminal receives the summary and displays it on the display so that the user can easily read it.

[0703] This trend will lead to the realization of a system that summarizes important traffic information in real time within autonomous vehicles and provides it to users.

[0704] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0705] The following specific example is provided as a mode for carrying out the present invention: This system allows a user to use a communication terminal and efficiently summarizes message data in a conversation talk room via a server, and further incorporates an emotion engine that recognizes the user's emotions.

[0706] System Overview

[0707] Using conversation digests and emotion recognition

[0708] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[0709] When a user taps this "Conversation Digest" button, a summary of the messages in the conversation chat room will begin.

[0710] Message capture and sentiment data collection

[0711] The device detects user actions and retrieves the latest message data in the conversation chat room. This data typically includes messages from the past 24 hours or a customizable range.

[0712] The device also simultaneously collects and temporarily stores the user's emotional data.

[0713] Sending message data and emotion data

[0714] The acquired message data and emotion data are temporarily stored on the device and then sent to a server for analysis.

[0715] Message and sentiment analysis

[0716] The server processes the received message data and emotion data and supplies it to the generative AI model.

[0717] Generative AI models analyze message data to understand context and extract key information.

[0718] The emotion engine analyzes the user's emotional data and adjusts the summary content according to the user's emotional state. For example, if the user is feeling stressed, the system generates a more concise summary that focuses on the important information.

[0719] Sending and displaying abstracts

[0720] The server sends the generated summary to the terminal.

[0721] The terminal receives this summary and displays it in the conversation talk room.

[0722] Users can check the summary and quickly grasp important information.

[0723] Specific examples

[0724] Use in stressful project management group chats

[0725] 1. A user opens a group chat for project management.

[0726] 2. Group chats are often filled with messages, and important information can get buried, causing stress for users.

[0727] 3. The user taps the "Conversation Digest" button.

[0728] 4. The device retrieves message data from the past 24 hours, collects emotion data, and sends it to the server.

[0729] 5. The server supplies the received message data and emotion data to the generative AI model and begins analysis.

[0730] 6. The generative AI model summarizes the conversation, including the following message:

[0731] "Person A: Please give me a progress report on the project."

[0732] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[0733] "Person C: The design proposal is complete. Please review it."

[0734] "D: What time is the meeting today?"

[0735] 7. The emotion engine takes into account the user's stress level and generates a concise summary that focuses on the important information.

[0736] 8. The generative AI model generates the following summary:

[0737] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0738] 9. The server sends the summary to the terminal.

[0739] 10. The device receives the summary and displays it in the conversation room.

[0740] 11. Users can quickly review the summary and grasp the important information.

[0741] In this way, the system of the present invention allows users to efficiently extract important information from a large number of messages and make quick decisions. Furthermore, by taking into account the user's emotional state, the system provides a more appropriate summary, contributing to stress reduction.

[0742] The processing flow will be explained below.

[0743] Step 1:

[0744] A user uses a communication terminal to start a chat application and access a talk room.

[0745] The device retrieves the chat room data and displays it on the screen.

[0746] Step 2:

[0747] The device will display a "Conversation Digest" button on the interface within the chat room.

[0748] Step 3:

[0749] The user taps the "Conversation Digest" button.

[0750] The device detects this tap event and starts the following process.

[0751] Step 4:

[0752] The device retrieves the latest message data in the chat room for a certain range (e.g., the past 24 hours).

[0753] At the same time, the device also collects the user's emotional data and temporarily stores it in storage.

[0754] Step 5:

[0755] The message data and emotion data stored on the device are transmitted to the server.

[0756] The message data includes information such as the sender, the time of sending, and the content.

[0757] Step 6:

[0758] The server processes the received message data and emotion data.

[0759] This includes pre-processing the data to ensure accurate analysis of the entire data set.

[0760] Step 7:

[0761] The server uses the generative AI model to analyze the message data.

[0762] Generative AI models understand the context of messages and extract key information.

[0763] Step 8:

[0764] An emotion engine analyzes the user's emotion data to identify the user's emotional state.

[0765] The emotion engine determines whether the user is stressed or relaxed.

[0766] Step 9:

[0767] A concise summary is generated based on the key information extracted by the generative AI model.

[0768] At the same time, the content of the summary is adjusted based on information from the emotion engine.

[0769] For example, if the user is feeling stressed, a more concise and to-the-point summary will be generated.

[0770] Step 10:

[0771] The server transmits the generated summary to the terminal.

[0772] Confirm that the summary has been sent.

[0773] Step 11:

[0774] The summary received by the device is displayed in the chat room.

[0775] Users can check the summary and quickly grasp important information.

[0776] Step 12:

[0777] Collect user feedback and fine-tune the generative AI model and emotion engine algorithms as needed.

[0778] Example 2

[0779] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0780] In today's information society, users are increasingly using communication platforms where a large number of messages are exchanged. However, it is difficult to efficiently extract and understand important information from these numerous messages. The stress users experience while reading messages is also a major problem. Conventional systems simply display messages, without summarizing or adjusting them based on emotions, which results in information being scattered and hinders efficient decision-making. Therefore, a method is needed to enable users to efficiently grasp important information and reduce stress.

[0781] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring message data and user emotion data, means for analyzing the message data using a generative AI model to understand the context and extract important information, and means for analyzing the user emotion data using an emotion engine and adjusting the summary sentence according to the user's emotional state. This allows the user to quickly grasp automatically summarized information from a large number of messages, and furthermore, by adjusting the information according to the user's emotional state, less stressful communication is realized.

[0782] A "communication terminal" is a device that allows a user to connect to the Internet and use various services, and includes smartphones, PCs, tablets, etc.

[0783] "Conversation talk room" refers to a chat room or messaging platform where multiple users can exchange messages.

[0784] "User" refers to an individual who uses a communication terminal to participate in a conversation talk room and send, receive, and view messages.

[0785] "Conversation Digest Button" refers to a button on the interface that activates a function that summarizes messages in a conversation chat room.

[0786] "Message Data" refers to text and media information sent and received by a user within a conversation chat room.

[0787] "Emotion data" refers to information about the user's emotional state obtained from facial expressions, voice, self-input, etc.

[0788] "Server" refers to a central control system for receiving, analyzing, and generating message data and emotion data.

[0789] "Generative AI Model" means an artificial intelligence model used by the Server that includes algorithms that analyze message data and extract key information.

[0790] An "emotion engine" refers to a system that analyzes a user's emotional data and adjusts summary generation according to the user's emotional state.

[0791] This invention relates to a system that allows users to use their communication terminals to efficiently summarize message data in a conversation talk room via a server. It also incorporates an emotion engine that recognizes the user's emotions and adjusts the content of the summary.

[0792] System Overview

[0793] Hardware and software used

[0794] The present invention uses the following hardware and software.

[0795] Communication device: A device that allows users to send and receive messages and use summary functions. This includes smartphones, PCs, tablets, etc.

[0796] Server: A central control system that receives message data and emotion data, analyzes and processes them using generative AI models and emotion engines.

[0797] Generative AI models: Algorithms that analyze message data, understand context, and extract key information.

[0798] Emotion engine: A system that analyzes the user's emotional data and adjusts summary generation according to the user's emotional state.

[0799] Data Flow and Processing

[0800] 1. User Action:

[0801] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[0802] When the user taps this "Conversation Digest" button, the summary process begins.

[0803] 2. Data Acquisition:

[0804] The device detects the user's operations and acquires message data and user emotion data in the conversation talk room.

[0805] This data typically includes messages from the past 24 hours, or a range you configure.

[0806] 3. Data transmission and analysis:

[0807] The message data and emotion data acquired by the terminal are transmitted to a server via the Internet.

[0808] The server formats the received data and feeds it to the generative AI model.

[0809] A generative AI model analyzes message data and extracts key information.

[0810] 4. Use of Emotion Engine:

[0811] The server uses an emotion engine to analyze the user's emotion data and adjusts the summary according to the user's emotional state.

[0812] For example, if a user is feeling stressed, a summary will be generated that is more concise and focuses on the important information.

[0813] 5. Send and display abstracts:

[0814] The server sends the generated summary to the terminal.

[0815] The device receives the summary and displays it in the conversation chat room.

[0816] Users can check the summary and quickly grasp important information.

[0817] Specific examples

[0818] Use in stressful project management group chats

[0819] 1. A user opens a group chat for project management.

[0820] 2. Group chats can be filled with messages, and important information can get lost in the shuffle, causing frustration for users.

[0821] 3. The user taps the "Conversation Digest" button.

[0822] 4. The device retrieves message data from the past 24 hours, collects emotion data, and sends it to the server.

[0823] 5. The server passes the following prompt to the generative AI model:

[0824] "Project status report, task completion, next phase, design proposal submission, review request, meeting time proposal"

[0825] 6. The server adjusts the emotion engine and generates the following summary:

[0826] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0827] 7. The server sends the summary to the terminal.

[0828] 8. The device receives the summary and displays it in the conversation chat room.

[0829] 9. Users can quickly review the summary and grasp the important information.

[0830] This allows users to efficiently extract important information from a large number of messages and make quick decisions.The emotion engine also provides summaries based on the user's emotional state, contributing to stress reduction.

[0831] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0832] Step 1:

[0833] A user accesses a conversation talk room using a communication terminal.

[0834] Users log in to a chat room using a dedicated application or web browser on a device such as a smartphone or PC. The input is the user's access operation, and the output is the display of the chat room interface, allowing the user to send and receive messages.

[0835] Step 2:

[0836] The user taps the "Conversation Digest" button.

[0837] The user taps the "Conversation Digest" button displayed in the chat room to start the process. The input is the button tap event, and the output is that this event is triggered to proceed to the next step.

[0838] Step 3:

[0839] The device acquires message data and user emotion data from within the chat room.

[0840] The device retrieves message data from the server for the past 24 hours or any specified range, and simultaneously acquires user emotion data from the camera and microphone. Input is button tap events and data read from the interface, and output is temporary storage of the retrieved message data and emotion data.

[0841] Step 4:

[0842] The message data and emotion data acquired by the terminal are transmitted to the server.

[0843] The device sends the temporarily stored data to a server via the Internet. The input is the acquired message data and emotion data, and the output is sending this data to the server. It is preferable to use encryption technology such as SSL / TLS for transmission.

[0844] Step 5:

[0845] The server analyzes the received data and feeds it into a generative AI model.

[0846] The server formats the received message data and emotion data into an appropriate format. The input is the data sent from the device, and the output is a set of formatted data. These data are fed into the generative AI model.

[0847] Step 6:

[0848] A generative AI model analyzes message data and extracts key information.

[0849] The generative AI model analyzes the provided message data, understands the context, and extracts key information. The input is the formatted message data, and the output is the extracted key information. Prompt sentences can be used to improve the accuracy of the model.

[0850] Step 7:

[0851] The emotion engine analyzes the user's emotion data and adjusts the summary text.

[0852] The emotion engine analyzes the user's emotional data and adjusts the summary based on the user's emotional state. For example, if the user is feeling stressed, the summary will be concise and focus on the important information. The input is the emotional data and extracted important information, and the output is a summary adjusted according to the user's emotion.

[0853] Step 8:

[0854] The server sends the generated summary to the terminal.

[0855] The server sends the summary adjusted by the emotion engine to the terminal. The input is the adjusted summary, and the output is the summary reaching the terminal. HTTP or WebSocket is generally used as the transmission protocol.

[0856] Step 9:

[0857] The device receives the summary and displays it in the conversation chat room.

[0858] The terminal displays the summary received from the server on the user interface. The input is the summary from the server, and the output is the user checking the summary. This allows the user to grasp important information in a short time.

[0859] This series of processing steps allows users to efficiently extract important information from a large number of messages, realizing stress-free communication.

[0860] (Application example 2)

[0861] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0862] In logistics centers, business communications are routinely carried out via large volumes of messages and voice input, but important information often gets buried, resulting in problems of reduced work efficiency and increased stress.In addition, there is the issue of the quality of communication not improving because information is provided uniformly without considering the emotional state of staff.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing message data using a generative AI model and an emotion engine, extracting important information, and adjusting the content of the summary based on the user's emotional state, means for transmitting the summary generated by the server to the communication terminal, means for the communication terminal to display the summary, and means for summarizing and displaying business communications conducted by conversation and voice input. This makes it possible to efficiently extract important information from business communications at a logistics center and provide it in a form that takes into consideration the user's emotional state.

[0864] "User" refers to the staff working at the logistics center.

[0865] "Communication terminal" refers to a device that can connect to the Internet, including wearable devices such as smart glasses or smartphones.

[0866] "Conversation talk room" refers to a chat room or voice conversation space where business communications take place at a logistics center.

[0867] A "conversation digest button" refers to an interface that a user can tap to start a conversation summary.

[0868] "Message data" refers to text and audio data in a conversation talk room obtained from a communication terminal.

[0869] "Emotion data" refers to data that indicates the user's emotional state.

[0870] "Server" refers to the computer system that is equipped with the generative AI model and emotion engine and performs data analysis.

[0871] "Generative AI model" refers to an artificial intelligence model that analyzes acquired message data and extracts important information.

[0872] "Emotion engine" refers to a system for analyzing a user's emotional data and adjusting the content of the summary.

[0873] "Summary" refers to a sentence that summarizes the main points of a message, generated by a generative AI model and emotion engine.

[0874] The present invention is embodied in a system for summarizing business communications in a logistics center, using wearable devices such as smart glasses or smartphones as communication terminals.

[0875] System Configuration and Operation

[0876] 1. A user uses a communication device to access a conversation talk room.

[0877] Smart glasses (e.g., Google Glass, Microsoft HoloLens) are used as communication devices.

[0878] Users can communicate through text chat or voice input within the conversation room.

[0879] 2. Click the conversation digest button.

[0880] When a user taps the conversation digest button, the system retrieves message and emotion data from the past 24 hours. This button appears on the HUD (Head-Up Display).

[0881] 3. Data acquisition and transmission.

[0882] The communication terminal acquires the message data and emotion data and temporarily stores them.

[0883] The data is sent to a server where it is analyzed using a generative AI model (e.g., OpenAI GPT-3) and an emotion engine (e.g., Affectiva Emotion SDK).

[0884] 4. Data analysis and key points extraction.

[0885] The server uses a generative AI model to analyze the message data and extract key information.

[0886] The emotion engine analyzes the user's emotion data and adjusts the content of the summary according to the user's stress level.

[0887] 5. Send and display the summary.

[0888] The server transmits the generated summary to the communication terminal.

[0889] The communication terminal receives the summary and displays it on the glasses' HUD.

[0890] Specific examples

[0891] Usage example 1

[0892] Prompt statement:

[0893] Logistics Center Staff A: "What's the delivery schedule for this week?"

[0894] Logistics Center Staff B: "We have 10 cases to be delivered on Monday and 20 cases to be delivered on Tuesday."

[0895] summary:

[0896] "This week's delivery schedule: 10 cases on Monday, 20 cases on Tuesday."

[0897] Usage example 2

[0898] Prompt statement:

[0899] Logistics Center Staff C: "Which shelf should I place the next item on?"

[0900] Logistics Center Staff D: "Please place item X on shelf A and item Y on shelf B."

[0901] summary:

[0902] "Item X is on shelf A and item Y is on shelf B"

[0903] In this way, by implementing the invention, it is possible to efficiently extract important information from business communications at a logistics center and provide the information while taking into consideration the emotional state of the user, which is expected to improve work efficiency and reduce stress.

[0904] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0905] Step 1:

[0906] A user uses a communication terminal to access a conversation talk room.

[0907] Specific behavior:

[0908] The communication device (smart glasses or smartphone) connects to the server via the Internet and displays a screen of text chat and voice input data within the conversation room.

[0909] Step 2:

[0910] The user taps the conversation digest button.

[0911] Specific behavior:

[0912] The user taps the "Conversation Digest" button displayed on the communication device's HUD. This action triggers the acquisition of message and emotion data from the past 24 hours.

[0913] Input: User taps

[0914] Output: Start of data acquisition process

[0915] Step 3:

[0916] The device acquires the latest message data and emotion data in the conversation chat room and temporarily stores them.

[0917] Specific behavior:

[0918] Software built into the communication device retrieves the latest message data from the server via an API, and uses the camera and microphone to collect emotional data from the user's facial expressions and tone of voice, which is then temporarily stored in local storage.

[0919] Input: Message data in the conversation chat room, user emotion data

[0920] Output: Temporarily save the acquired data

[0921] Step 4:

[0922] The terminal transmits the acquired message data and emotion data to the server.

[0923] Specific behavior:

[0924] The communication device sends the acquired message data and emotion data to the server via an HTTP request. The sent data is encoded in JSON format.

[0925] Input: Temporarily saved message data and emotion data

[0926] Output: Send data to the server

[0927] Step 5:

[0928] The server uses a generative AI model and an emotion engine to analyze message data, extract key information, and adjust the content of the summary based on the user's emotional state.

[0929] Specific behavior:

[0930] The server inputs the received data into a generative AI model (e.g., OpenAI GPT-3), which analyzes the message data and extracts context and important information. At the same time, an emotion engine (e.g., Affectiva Emotion SDK) analyzes the user's emotional data and makes adjustments, such as making the summary more concise, if the user is feeling stressed.

[0931] Input: Message data and emotion data sent to the server

[0932] Output: Adjusted summary

[0933] Step 6:

[0934] The server transmits the generated summary to the communication terminal.

[0935] Specific behavior:

[0936] The server sends the generated summary to the communication terminal via an HTTP response. The response data is encoded in JSON format.

[0937] Input: Adjusted summary

[0938] Output: Send summary to communication terminal

[0939] Step 7:

[0940] The communication terminal receives the summary and displays it in the conversation talk room.

[0941] Specific behavior:

[0942] The communication terminal displays the received summary on a HUD (Head-Up Display) and provides it to the user, allowing the user to quickly grasp important information.

[0943] Input: Abstract sent from the server

[0944] Output: Summary displayed on HUD

[0945] Through the above steps, important information can be efficiently extracted from business communications at a logistics center, and a summary can be provided that takes into consideration the user's emotional state.

[0946] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0947] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0948] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0949] [Third embodiment]

[0950] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0951] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0952] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0953] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0954] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0955] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0956] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0957] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0958] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0959] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0960] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0961] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0962] The following specific example is given as an embodiment for carrying out the present invention: In this system, a user uses a communication terminal to efficiently summarize message data in a conversation talk room via a server.

[0963] System Overview

[0964] Using the Conversation Digest Feature

[0965] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[0966] When a user taps this "Conversation Digest" button, a summary of the messages in the conversation chat room will begin.

[0967] Getting and sending messages

[0968] The device detects user actions and retrieves the latest message data in the conversation chat room. This data typically includes messages from the past 24 hours or a customizable range.

[0969] The acquired message data is temporarily stored on the device and then sent to a server for analysis.

[0970] Message analysis and summarization

[0971] The server processes the received message data and supplies it to the generative AI model.

[0972] The generative AI model analyzes message data, understands the context, and extracts important information. Based on this information, the generative AI model generates a concise summary.

[0973] Sending and displaying abstracts

[0974] The server sends the generated summary to the terminal.

[0975] The terminal receives this summary and displays it in the conversation talk room.

[0976] Users can check the summary and quickly grasp important information.

[0977] Specific examples

[0978] Use in project management group chats

[0979] 1. A user opens a group chat for project management.

[0980] 2. Group chats have a lot of messages exchanged, and important information can get buried.

[0981] 3. The user taps the "Conversation Digest" button.

[0982] 4. The device retrieves message data from the past 24 hours and sends it to the server.

[0983] 5. The server supplies the received message data to the generative AI model and begins analysis.

[0984] 6. The generative AI model summarizes the conversation, including the following message:

[0985] "Person A: Please give me a progress report on the project."

[0986] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[0987] "Person C: The design proposal is complete. Please review it."

[0988] "D: What time is the meeting today?"

[0989] 7. A generative AI model summarizes these messages and generates the following summary:

[0990] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[0991] 8. The server sends the summary to the terminal.

[0992] 9. The device receives the summary and displays it in the conversation chat room.

[0993] 10. Users can quickly review the summary and grasp the important information.

[0994] In this way, by using the system of the present invention, users can efficiently extract important information from a large number of messages and make quick decisions, thereby improving the efficiency of business communications and preventing work delays.

[0995] The processing flow will be explained below.

[0996] Step 1:

[0997] A user uses a communication terminal to start a chat application and access a talk room.

[0998] The device retrieves the chat room data and displays it on the screen.

[0999] Step 2:

[1000] The device will display a "Conversation Digest" button on the interface within the chat room.

[1001] Step 3:

[1002] The user taps the "Conversation Digest" button.

[1003] The device detects this tap event and starts the following process.

[1004] Step 4:

[1005] The device retrieves the latest message data in the chat room for a certain range (e.g., the past 24 hours).

[1006] The acquired message data is temporarily stored in storage.

[1007] Step 5:

[1008] The terminal transmits the stored message data to the server.

[1009] The message data includes information such as the sender, the time of sending, and the content.

[1010] Step 6:

[1011] The server starts processing to analyze the received message data.

[1012] Feed the data into a generative AI model.

[1013] Step 7:

[1014] Generative AI models analyze message data, understand context and extract key information.

[1015] A concise summary is generated based on the extracted information.

[1016] Step 8:

[1017] The server transmits the generated summary to the terminal.

[1018] Confirm that the summary has been sent.

[1019] Step 9:

[1020] The summary received by the device is displayed in the chat room.

[1021] Users can check the summary and quickly grasp important information.

[1022] Example 1

[1023] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1024] In conventional conversation chat rooms, it has been difficult for users to quickly grasp important information amid the large number of messages exchanged. Furthermore, there is no way to concisely summarize the contents of messages, which reduces the efficiency of communication. In response to this problem, the present invention aims to enable users to easily grasp important information in conversation chat rooms.

[1025] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1026] In this invention, the server includes means for analyzing message data using a generative AI model, understanding the context, and extracting important information, means for understanding the specific context from the analyzed message data, identifying key points, and providing a prompt sentence to the generative AI model, and means for transmitting the generated summary sentence to the communication terminal, thereby enabling a user to efficiently extract important information from a large number of messages and grasp it in a short time.

[1027] "User" means an individual or entity that accesses and operates the System.

[1028] A "communication terminal" is a device used by a user, and includes smartphones, tablets, personal computers, etc.

[1029] A "conversation talk room" is a virtual environment for users to exchange messages.

[1030] A "conversation digest button" is a button on the interface that a user taps to initiate the conversation summary function.

[1031] "Message data" is a collection of text information exchanged within a conversation talk room.

[1032] A "server" is a computer system that receives message data sent from a communication terminal and performs analysis processing.

[1033] A "generative AI model" is an artificial intelligence program that analyzes and summarizes text data based on machine learning algorithms.

[1034] A "summary" is a concise sentence that contains key information from the original message data, extracted and generated by a generative AI model.

[1035] A "prompt sentence" is an instruction sentence input into a generative AI model and is used during summarization and analysis.

[1036] "Context" refers to the situation or background information that allows for understanding the meaning and relationships within the message data.

[1037] "Important information" refers to the main content or points in the message data that are deemed necessary for the user to know.

[1038] This invention is a system that allows users to use their communication terminals to efficiently summarize message data in a conversation talk room via a server. This system uses a generative AI model to analyze the message data, extract important information, and generate a summary.

[1039] Hardware Configuration

[1040] The communication device used by the user is a smartphone, tablet, or PC. These communication devices display a "Conversation Digest" button as part of their interface.

[1041] The server is a computer system that receives message data sent from the communication device, supplies the data to the generative AI model, and performs analysis and summarization. The server has the processing power to run Python programs and a network connection to access the generative AI model (e.g., OpenAI's GPT-3).

[1042] Software Configuration

[1043] The software of the communication terminal is an application that controls the user interface. The application has a function to display a conversation digest button on the interface and retrieve message data after the user taps the button.

[1044] The server software receives message data, preprocesses it, feeds prompts to the generative AI model, receives the analysis results, generates summaries, and sends the summaries, including Python scripts for data cleansing and metadata extraction.

[1045] Data processing and calculation

[1046] When a device retrieves message data, it typically retrieves the past 24 hours. This data is converted to JSON format and sent to the server.

[1047] The server preprocesses the received data, using Python scripts to cleanse the data and extract metadata, and then passes the preprocessed data to the generative AI model.

[1048] The generative AI model analyzes the message data based on the prompt, extracts important information, and generates a summary, which is then converted back to JSON format and sent from the server to the device.

[1049] Specific examples

[1050] For example, consider the following messages exchanged in a project management group chat:

[1051] "Person A: Please give me a progress report on the project."

[1052] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[1053] "Person C: The design proposal is complete. Please review it."

[1054] "D: What time is the meeting today?"

[1055] When the user taps the "Conversation Digest" button, the device retrieves these messages and sends them to the server, which uses a generative AI model to extract key information and generate the following summary:

[1056] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[1057] The generated summary is sent to the terminal and displayed to the user in the conversation chat room, allowing the user to quickly grasp important information.

[1058] In the above-described form, the present invention provides a means for a user to efficiently extract important information from a large number of messages and make quick decisions.

[1059] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1060] Step 1:

[1061] The user taps the "Conversation Digest" button.

[1062] Input: The user taps the conversation digest button.

[1063] Output: Trigger to start the conversation digest function.

[1064] Specific operation: The conversation digest button is tapped on the communication device interface, which uses a user interface written in JavaScript or a similar language.

[1065] Step 2:

[1066] The terminal retrieves the message data.

[1067] Input: Trigger to start the conversation digest function.

[1068] Output: Message data for the past 24 hours.

[1069] Specific operation: The device accesses a database or local storage and retrieves message data for the past 24 hours using an SQL query, etc.

[1070] Step 3:

[1071] The terminal transmits the message data to the server.

[1072] Input: Message data from the past 24 hours.

[1073] Output: Message data converted to JSON format.

[1074] Specific operation: The acquired message data is converted into JSON format and sent to the server using the HTTPS protocol, using a Python library or HTTP client.

[1075] Step 4:

[1076] The server pre-processes the message data.

[1077] Input: Message data in JSON format.

[1078] Output: Preprocessed data and metadata.

[1079] Specific operation: A Python script parses the received data, performs data cleansing (removing missing values, removing text noise, etc.), and extracts metadata (e.g., speaker, timestamp).

[1080] Step 5:

[1081] The server feeds the preprocessed data to the generative AI model.

[1082] Input: Preprocessed data and metadata.

[1083] Output: The prompt sentence that is fed to the generative AI model.

[1084] Specific operation: A prompt sentence is generated based on the preprocessed data and fed to a generative AI model (e.g., GPT-3). This prompt sentence is structured in text format as follows:

[1085] "Please summarize the following message: 'Message data content'"

[1086] Step 6:

[1087] A generative AI model generates a summary.

[1088] Input: The prompt sentence fed to the generative AI model.

[1089] Output: The generated summary.

[1090] How it works: The generative AI model analyzes message data based on the prompt, extracts key information, and generates a summary, a process that takes anywhere from a few seconds to a few minutes.

[1091] Step 7:

[1092] The server transmits the generated summary to the terminal.

[1093] Input: The generated summary sentence.

[1094] Output: Summary text converted to JSON format.

[1095] Specific operation: The generated summary is converted back into JSON format and sent to the terminal via the HTTPS protocol.

[1096] Step 8:

[1097] The terminal displays the summary.

[1098] Input: Abstract in JSON format.

[1099] Output: A summary statement that is displayed in the user interface.

[1100] Specific operation: The device parses the received summary text and displays it on the interface, using a JavaScript framework (e.g., React or Vue.js).

[1101] Step 9:

[1102] The user checks the summary.

[1103] Input: The summary text displayed in the user interface.

[1104] Output: Important information that the user will understand in a short time.

[1105] Specific operation: The user can check the displayed summary text and quickly grasp important information.

[1106] (Application example 1)

[1107] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1108] While autonomous vehicles are operating, it is difficult for drivers and passengers to quickly and efficiently grasp important traffic-related information, such as traffic conditions and accident information, in real time. Furthermore, important information can get lost in the vast amount of traffic information collected, potentially reducing operational efficiency and safety. There is a need for a system that can solve these issues and support quick decision-making while driving.

[1109] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1110] In this invention, the server includes: means for a user to access a conversation talk room using a communication terminal; means for the user to tap a conversation digest button; means for detecting the user's operation and acquiring message data in the conversation talk room; means for transmitting the acquired message data to the server; means for the server to analyze the message data using a generative AI model and extract important information; means for the server to transmit the generated summary to the communication terminal; means for the communication terminal to display the summary; means for acquiring traffic-related messages while the autonomous vehicle is in operation and generating a summary; and means for displaying the generated summary on an in-vehicle display, thereby enabling drivers and passengers to quickly grasp important traffic-related information and make appropriate decisions even while the autonomous vehicle is in operation.

[1111] A "communication terminal" is an electronic device that a user uses for communication.

[1112] A "conversation talk room" is a virtual space where users can exchange messages.

[1113] The "conversation digest button" is an operation button for summarizing messages in a chat room.

[1114] "Sensing an operation" means that the communication terminal detects an input action performed by the user.

[1115] "Message data" refers to text information exchanged within a chat room.

[1116] "Server" means a computer system that provides data analysis and storage functions.

[1117] A "generative AI model" is an algorithm that uses artificial intelligence to extract important information from data and generate a summary.

[1118] A "summary" is a sentence that shortens a long sentence or message and summarizes the main points.

[1119] An "autonomous vehicle" is a vehicle that operates autonomously without the need for a human driver.

[1120] A "traffic-related message" is a message that includes information about traffic conditions or traffic.

[1121] An "in-vehicle display" is a screen device that is installed inside a vehicle and displays information.

[1122] The following specific example is provided as an embodiment of the present invention. This system efficiently summarizes traffic-related message data while a user is driving an autonomous vehicle using a communication terminal. The details of this system are described below.

[1123] System Overview

[1124] Using the Traffic Digest Feature

[1125] The user presses the traffic digest button on the in-vehicle display of the autonomous vehicle, which causes the system to retrieve and summarize traffic-related messages from the past 24 hours.

[1126] Getting and sending messages

[1127] The terminal detects user operation and detects that the traffic digest button has been pressed, which causes the communication terminal to obtain traffic-related message data for the past 24 hours from the communication terminal of the autonomous vehicle.

[1128] The acquired message data is temporarily stored on the device and then sent to a server for analysis.

[1129] Message analysis and summarization

[1130] The server processes the received message data and supplies it to a generative AI model, such as OpenAI's GPT-3.

[1131] The generative AI model analyzes message data, understands the context, and extracts important information. Based on this information, the generative AI model generates a concise summary.

[1132] Sending and displaying abstracts

[1133] The server transmits the generated summary to the communication terminal.

[1134] The terminal receives the summary and displays it on the in-vehicle display, allowing the user to quickly understand important information by checking the summary.

[1135] Specific examples

[1136] For example, consider the following traffic-related messages stored in a vehicle's communications terminal:

[1137] "08:00 Traffic congestion occurs, traffic volume increases"

[1138] "10:00 Accident occurred, detour recommended"

[1139] "15:00 Traffic decrease"

[1140] "18:00 Traffic jam occurs again"

[1141] When the user presses the traffic digest button on the in-car display, the system sends these messages to the server and generates a summary like this:

[1142] Traffic information: Traffic increased around 08:00, causing congestion. An accident occurred around 10:00, and a detour was recommended. Traffic decreased around 15:00, and congestion reoccurred around 18:00.

[1143] Example prompt sentence:

[1144] Congestion and increased traffic volume

[1145] Accident occurred. Detour recommended.

[1146] Traffic decrease

[1147] Traffic jams occur again

[1148] This will enable drivers and passengers to quickly grasp important traffic-related information and make appropriate decisions while driving autonomous vehicles, contributing to improved operational efficiency and safety.

[1149] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1150] Step 1:

[1151] The user presses the traffic digest button displayed on the in-vehicle display of the autonomous vehicle. The input is the user's operation, and the output is the terminal detecting the operation. Specifically, the button press is recorded by touching the in-vehicle display.

[1152] Step 2:

[1153] The device detects user actions and retrieves traffic-related message data from the past 24 hours. The input is the user pressing a button, and the output is retrieving traffic-related message data. The device collects the latest traffic-related messages from its internal memory or a cloud service.

[1154] Step 3:

[1155] The acquired message data is sent to the server. The input is the traffic-related message data collected by the device, and the output is sending the data to the server. The device sends the message data as a POST request to the server's API endpoint via an Internet connection.

[1156] Step 4:

[1157] The server processes the received message data and supplies it to the generative AI model. The input is traffic-related message data received from the device, and the output is data to be analyzed and passed to the generative AI model. The server preprocesses the received data, converts it into an appropriate format, and sends it to the model.

[1158] Step 5:

[1159] The generative AI model analyzes the message data and extracts key information. The input is preprocessed traffic-related message data, and the output is the extracted key information. Specifically, the generative AI model analyzes the context of traffic messages and identifies important events and information.

[1160] Step 6:

[1161] The generative AI model generates a concise summary based on the extracted information. The input is the extracted key information, and the output is the generated summary. The model converts the key information into a summary format and creates a summary in an easy-to-understand style.

[1162] Step 7:

[1163] The server sends the generated summary to the communication terminal. The input is the generated summary, and the output is the summary sent to the communication terminal. The server again sends the generated summary to the terminal using the API endpoint.

[1164] Step 8:

[1165] The terminal receives the summary and displays it on the in-vehicle display. The input is the summary sent from the server, and the output is the summary displayed on the screen. The terminal receives the summary and displays it on the display so that the user can easily read it.

[1166] This trend will lead to the realization of a system that summarizes important traffic information in real time within autonomous vehicles and provides it to users.

[1167] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1168] The following specific example is provided as a mode for carrying out the present invention: This system allows a user to use a communication terminal and efficiently summarizes message data in a conversation talk room via a server, and further incorporates an emotion engine that recognizes the user's emotions.

[1169] System Overview

[1170] Using conversation digests and emotion recognition

[1171] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[1172] When a user taps this "Conversation Digest" button, a summary of the messages in the conversation chat room will begin.

[1173] Message capture and sentiment data collection

[1174] The device detects user actions and retrieves the latest message data in the conversation chat room. This data typically includes messages from the past 24 hours or a customizable range.

[1175] The device also simultaneously collects and temporarily stores the user's emotional data.

[1176] Sending message data and emotion data

[1177] The acquired message data and emotion data are temporarily stored on the device and then sent to a server for analysis.

[1178] Message and sentiment analysis

[1179] The server processes the received message data and emotion data and supplies it to the generative AI model.

[1180] Generative AI models analyze message data to understand context and extract key information.

[1181] The emotion engine analyzes the user's emotional data and adjusts the summary content according to the user's emotional state. For example, if the user is feeling stressed, the system generates a more concise summary that focuses on the important information.

[1182] Sending and displaying abstracts

[1183] The server sends the generated summary to the terminal.

[1184] The terminal receives this summary and displays it in the conversation talk room.

[1185] Users can check the summary and quickly grasp important information.

[1186] Specific examples

[1187] Use in stressful project management group chats

[1188] 1. A user opens a group chat for project management.

[1189] 2. Group chats are often filled with messages, and important information can get buried, causing stress for users.

[1190] 3. The user taps the "Conversation Digest" button.

[1191] 4. The device retrieves message data from the past 24 hours, collects emotion data, and sends it to the server.

[1192] 5. The server supplies the received message data and emotion data to the generative AI model and begins analysis.

[1193] 6. The generative AI model summarizes the conversation, including the following message:

[1194] "Person A: Please give me a progress report on the project."

[1195] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[1196] "Person C: The design proposal is complete. Please review it."

[1197] "D: What time is the meeting today?"

[1198] 7. The emotion engine takes into account the user's stress level and generates a concise summary that focuses on the important information.

[1199] 8. The generative AI model generates the following summary:

[1200] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[1201] 9. The server sends the summary to the terminal.

[1202] 10. The device receives the summary and displays it in the conversation room.

[1203] 11. Users can quickly review the summary and grasp the important information.

[1204] In this way, the system of the present invention allows users to efficiently extract important information from a large number of messages and make quick decisions. Furthermore, by taking into account the user's emotional state, the system provides a more appropriate summary, contributing to stress reduction.

[1205] The processing flow will be explained below.

[1206] Step 1:

[1207] A user uses a communication terminal to start a chat application and access a talk room.

[1208] The device retrieves the chat room data and displays it on the screen.

[1209] Step 2:

[1210] The device will display a "Conversation Digest" button on the interface within the chat room.

[1211] Step 3:

[1212] The user taps the "Conversation Digest" button.

[1213] The device detects this tap event and starts the following process.

[1214] Step 4:

[1215] The device retrieves the latest message data in the chat room for a certain range (e.g., the past 24 hours).

[1216] At the same time, the device also collects the user's emotional data and temporarily stores it in storage.

[1217] Step 5:

[1218] The message data and emotion data stored on the device are transmitted to the server.

[1219] The message data includes information such as the sender, the time of sending, and the content.

[1220] Step 6:

[1221] The server processes the received message data and emotion data.

[1222] This includes pre-processing the data to ensure accurate analysis of the entire data set.

[1223] Step 7:

[1224] The server uses the generative AI model to analyze the message data.

[1225] Generative AI models understand the context of messages and extract key information.

[1226] Step 8:

[1227] An emotion engine analyzes the user's emotion data to identify the user's emotional state.

[1228] The emotion engine determines whether the user is stressed or relaxed.

[1229] Step 9:

[1230] A concise summary is generated based on the key information extracted by the generative AI model.

[1231] At the same time, the content of the summary is adjusted based on information from the emotion engine.

[1232] For example, if the user is feeling stressed, a more concise and to-the-point summary will be generated.

[1233] Step 10:

[1234] The server transmits the generated summary to the terminal.

[1235] Confirm that the summary has been sent.

[1236] Step 11:

[1237] The summary received by the device is displayed in the chat room.

[1238] Users can check the summary and quickly grasp important information.

[1239] Step 12:

[1240] Collect user feedback and fine-tune the generative AI model and emotion engine algorithms as needed.

[1241] Example 2

[1242] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1243] In today's information society, users are increasingly using communication platforms where a large number of messages are exchanged. However, it is difficult to efficiently extract and understand important information from these numerous messages. The stress users experience while reading messages is also a major problem. Conventional systems simply display messages, without summarizing or adjusting them based on emotions, which results in information being scattered and hinders efficient decision-making. Therefore, a method is needed to enable users to efficiently grasp important information and reduce stress.

[1244] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring message data and user emotion data, means for analyzing the message data using a generative AI model to understand the context and extract important information, and means for analyzing the user emotion data using an emotion engine and adjusting the summary sentence according to the user's emotional state. This allows the user to quickly grasp automatically summarized information from a large number of messages, and furthermore, by adjusting the information according to the user's emotional state, less stressful communication is realized.

[1245] A "communication terminal" is a device that allows a user to connect to the Internet and use various services, and includes smartphones, PCs, tablets, etc.

[1246] "Conversation talk room" refers to a chat room or messaging platform where multiple users can exchange messages.

[1247] "User" refers to an individual who uses a communication terminal to participate in a conversation talk room and send, receive, and view messages.

[1248] "Conversation Digest Button" refers to a button on the interface that activates a function that summarizes messages in a conversation chat room.

[1249] "Message Data" refers to text and media information sent and received by a user within a conversation chat room.

[1250] "Emotion data" refers to information about the user's emotional state obtained from facial expressions, voice, self-input, etc.

[1251] "Server" refers to a central control system for receiving, analyzing, and generating message data and emotion data.

[1252] "Generative AI Model" means an artificial intelligence model used by the Server that includes algorithms that analyze message data and extract key information.

[1253] An "emotion engine" refers to a system that analyzes a user's emotional data and adjusts summary generation according to the user's emotional state.

[1254] This invention relates to a system that allows users to use their communication terminals to efficiently summarize message data in a conversation talk room via a server. It also incorporates an emotion engine that recognizes the user's emotions and adjusts the content of the summary.

[1255] System Overview

[1256] Hardware and software used

[1257] The present invention uses the following hardware and software.

[1258] Communication device: A device that allows users to send and receive messages and use summary functions. This includes smartphones, PCs, tablets, etc.

[1259] Server: A central control system that receives message data and emotion data, analyzes and processes them using generative AI models and emotion engines.

[1260] Generative AI models: Algorithms that analyze message data, understand context, and extract key information.

[1261] Emotion engine: A system that analyzes the user's emotional data and adjusts summary generation according to the user's emotional state.

[1262] Data Flow and Processing

[1263] 1. User Action:

[1264] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[1265] When the user taps this "Conversation Digest" button, the summary process begins.

[1266] 2. Data Acquisition:

[1267] The device detects the user's operations and acquires message data and user emotion data in the conversation talk room.

[1268] This data typically includes messages from the past 24 hours, or a range you configure.

[1269] 3. Data transmission and analysis:

[1270] The message data and emotion data acquired by the terminal are transmitted to a server via the Internet.

[1271] The server formats the received data and feeds it to the generative AI model.

[1272] A generative AI model analyzes message data and extracts key information.

[1273] 4. Use of Emotion Engine:

[1274] The server uses an emotion engine to analyze the user's emotion data and adjusts the summary according to the user's emotional state.

[1275] For example, if a user is feeling stressed, a summary will be generated that is more concise and focuses on the important information.

[1276] 5. Send and display abstracts:

[1277] The server sends the generated summary to the terminal.

[1278] The device receives the summary and displays it in the conversation chat room.

[1279] Users can check the summary and quickly grasp important information.

[1280] Specific examples

[1281] Use in stressful project management group chats

[1282] 1. A user opens a group chat for project management.

[1283] 2. Group chats can be filled with messages, and important information can get lost in the shuffle, causing frustration for users.

[1284] 3. The user taps the "Conversation Digest" button.

[1285] 4. The device retrieves message data from the past 24 hours, collects emotion data, and sends it to the server.

[1286] 5. The server passes the following prompt to the generative AI model:

[1287] "Project status report, task completion, next phase, design proposal submission, review request, meeting time proposal"

[1288] 6. The server adjusts the emotion engine and generates the following summary:

[1289] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[1290] 7. The server sends the summary to the terminal.

[1291] 8. The device receives the summary and displays it in the conversation chat room.

[1292] 9. Users can quickly review the summary and grasp the important information.

[1293] This allows users to efficiently extract important information from a large number of messages and make quick decisions.The emotion engine also provides summaries based on the user's emotional state, contributing to stress reduction.

[1294] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1295] Step 1:

[1296] A user accesses a conversation talk room using a communication terminal.

[1297] Users log in to a chat room using a dedicated application or web browser on a device such as a smartphone or PC. The input is the user's access operation, and the output is the display of the chat room interface, allowing the user to send and receive messages.

[1298] Step 2:

[1299] The user taps the "Conversation Digest" button.

[1300] The user taps the "Conversation Digest" button displayed in the chat room to start the process. The input is the button tap event, and the output is that this event is triggered to proceed to the next step.

[1301] Step 3:

[1302] The device acquires message data and user emotion data from within the chat room.

[1303] The device retrieves message data from the server for the past 24 hours or any specified range, and simultaneously acquires user emotion data from the camera and microphone. Input is button tap events and data read from the interface, and output is temporary storage of the retrieved message data and emotion data.

[1304] Step 4:

[1305] The message data and emotion data acquired by the terminal are transmitted to the server.

[1306] The device sends the temporarily stored data to a server via the Internet. The input is the acquired message data and emotion data, and the output is sending this data to the server. It is preferable to use encryption technology such as SSL / TLS for transmission.

[1307] Step 5:

[1308] The server analyzes the received data and feeds it into a generative AI model.

[1309] The server formats the received message data and emotion data into an appropriate format. The input is the data sent from the device, and the output is a set of formatted data. These data are fed into the generative AI model.

[1310] Step 6:

[1311] A generative AI model analyzes message data and extracts key information.

[1312] The generative AI model analyzes the provided message data, understands the context, and extracts key information. The input is the formatted message data, and the output is the extracted key information. Prompt sentences can be used to improve the accuracy of the model.

[1313] Step 7:

[1314] The emotion engine analyzes the user's emotion data and adjusts the summary text.

[1315] The emotion engine analyzes the user's emotional data and adjusts the summary based on the user's emotional state. For example, if the user is feeling stressed, the summary will be concise and focus on the important information. The input is the emotional data and extracted important information, and the output is a summary adjusted according to the user's emotion.

[1316] Step 8:

[1317] The server sends the generated summary to the terminal.

[1318] The server sends the summary adjusted by the emotion engine to the terminal. The input is the adjusted summary, and the output is the summary reaching the terminal. HTTP or WebSocket is generally used as the transmission protocol.

[1319] Step 9:

[1320] The device receives the summary and displays it in the conversation chat room.

[1321] The terminal displays the summary received from the server on the user interface. The input is the summary from the server, and the output is the user checking the summary. This allows the user to grasp important information in a short time.

[1322] This series of processing steps allows users to efficiently extract important information from a large number of messages, realizing stress-free communication.

[1323] (Application example 2)

[1324] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1325] In logistics centers, business communications are routinely carried out via large volumes of messages and voice input, but important information often gets buried, resulting in problems of reduced work efficiency and increased stress.In addition, there is the issue of the quality of communication not improving because information is provided uniformly without considering the emotional state of staff.

[1326] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing message data using a generative AI model and an emotion engine, extracting important information, and adjusting the content of the summary based on the user's emotional state, means for transmitting the summary generated by the server to the communication terminal, means for the communication terminal to display the summary, and means for summarizing and displaying business communications conducted by conversation and voice input. This makes it possible to efficiently extract important information from business communications at a logistics center and provide it in a form that takes into consideration the user's emotional state.

[1327] "User" refers to the staff working at the logistics center.

[1328] "Communication terminal" refers to a device that can connect to the Internet, including wearable devices such as smart glasses or smartphones.

[1329] "Conversation talk room" refers to a chat room or voice conversation space where business communications take place at a logistics center.

[1330] A "conversation digest button" refers to an interface that a user can tap to start a conversation summary.

[1331] "Message data" refers to text and audio data in a conversation talk room obtained from a communication terminal.

[1332] "Emotion data" refers to data that indicates the user's emotional state.

[1333] "Server" refers to the computer system that is equipped with the generative AI model and emotion engine and performs data analysis.

[1334] "Generative AI model" refers to an artificial intelligence model that analyzes acquired message data and extracts important information.

[1335] "Emotion engine" refers to a system for analyzing a user's emotional data and adjusting the content of the summary.

[1336] "Summary" refers to a sentence that summarizes the main points of a message, generated by a generative AI model and emotion engine.

[1337] The present invention is embodied in a system for summarizing business communications in a logistics center, using wearable devices such as smart glasses or smartphones as communication terminals.

[1338] System Configuration and Operation

[1339] 1. A user uses a communication device to access a conversation talk room.

[1340] Smart glasses (e.g., Google Glass, Microsoft HoloLens) are used as communication devices.

[1341] Users can communicate through text chat or voice input within the conversation room.

[1342] 2. Click the conversation digest button.

[1343] When a user taps the conversation digest button, the system retrieves message and emotion data from the past 24 hours. This button appears on the HUD (Head-Up Display).

[1344] 3. Data acquisition and transmission.

[1345] The communication terminal acquires the message data and emotion data and temporarily stores them.

[1346] The data is sent to a server where it is analyzed using a generative AI model (e.g., OpenAI GPT-3) and an emotion engine (e.g., Affectiva Emotion SDK).

[1347] 4. Data analysis and key points extraction.

[1348] The server uses a generative AI model to analyze the message data and extract key information.

[1349] The emotion engine analyzes the user's emotion data and adjusts the content of the summary according to the user's stress level.

[1350] 5. Send and display the summary.

[1351] The server transmits the generated summary to the communication terminal.

[1352] The communication terminal receives the summary and displays it on the glasses' HUD.

[1353] Specific examples

[1354] Usage example 1

[1355] Prompt statement:

[1356] Logistics Center Staff A: "What's the delivery schedule for this week?"

[1357] Logistics Center Staff B: "We have 10 cases to be delivered on Monday and 20 cases to be delivered on Tuesday."

[1358] summary:

[1359] "This week's delivery schedule: 10 cases on Monday, 20 cases on Tuesday."

[1360] Usage example 2

[1361] Prompt statement:

[1362] Logistics Center Staff C: "Which shelf should I place the next item on?"

[1363] Logistics Center Staff D: "Please place item X on shelf A and item Y on shelf B."

[1364] summary:

[1365] "Item X is on shelf A and item Y is on shelf B"

[1366] In this way, by implementing the invention, it is possible to efficiently extract important information from business communications at a logistics center and provide the information while taking into consideration the emotional state of the user, which is expected to improve work efficiency and reduce stress.

[1367] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1368] Step 1:

[1369] A user uses a communication terminal to access a conversation talk room.

[1370] Specific behavior:

[1371] The communication device (smart glasses or smartphone) connects to the server via the Internet and displays a screen of text chat and voice input data within the conversation room.

[1372] Step 2:

[1373] The user taps the conversation digest button.

[1374] Specific behavior:

[1375] The user taps the "Conversation Digest" button displayed on the communication device's HUD. This action triggers the acquisition of message and emotion data from the past 24 hours.

[1376] Input: User taps

[1377] Output: Start of data acquisition process

[1378] Step 3:

[1379] The device acquires the latest message data and emotion data in the conversation chat room and temporarily stores them.

[1380] Specific behavior:

[1381] Software built into the communication device retrieves the latest message data from the server via an API, and uses the camera and microphone to collect emotional data from the user's facial expressions and tone of voice, which is then temporarily stored in local storage.

[1382] Input: Message data in the conversation chat room, user emotion data

[1383] Output: Temporarily save the acquired data

[1384] Step 4:

[1385] The terminal transmits the acquired message data and emotion data to the server.

[1386] Specific behavior:

[1387] The communication device sends the acquired message data and emotion data to the server via an HTTP request. The sent data is encoded in JSON format.

[1388] Input: Temporarily saved message data and emotion data

[1389] Output: Send data to the server

[1390] Step 5:

[1391] The server uses a generative AI model and an emotion engine to analyze message data, extract key information, and adjust the content of the summary based on the user's emotional state.

[1392] Specific behavior:

[1393] The server inputs the received data into a generative AI model (e.g., OpenAI GPT-3), which analyzes the message data and extracts context and important information. At the same time, an emotion engine (e.g., Affectiva Emotion SDK) analyzes the user's emotional data and makes adjustments, such as making the summary more concise, if the user is feeling stressed.

[1394] Input: Message data and emotion data sent to the server

[1395] Output: Adjusted summary

[1396] Step 6:

[1397] The server transmits the generated summary to the communication terminal.

[1398] Specific behavior:

[1399] The server sends the generated summary to the communication terminal via an HTTP response. The response data is encoded in JSON format.

[1400] Input: Adjusted summary

[1401] Output: Send summary to communication terminal

[1402] Step 7:

[1403] The communication terminal receives the summary and displays it in the conversation talk room.

[1404] Specific behavior:

[1405] The communication terminal displays the received summary on a HUD (Head-Up Display) and provides it to the user, allowing the user to quickly grasp important information.

[1406] Input: Abstract sent from the server

[1407] Output: Summary displayed on HUD

[1408] Through the above steps, important information can be efficiently extracted from business communications at a logistics center, and a summary can be provided that takes into consideration the user's emotional state.

[1409] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1410] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1411] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1412] [Fourth embodiment]

[1413] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1414] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1415] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1416] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1417] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1419] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1420] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1421] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1422] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1423] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1424] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1425] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1426] The following specific example is given as an embodiment for carrying out the present invention: In this system, a user uses a communication terminal to efficiently summarize message data in a conversation talk room via a server.

[1427] System Overview

[1428] Using the Conversation Digest Feature

[1429] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[1430] When a user taps this "Conversation Digest" button, a summary of the messages in the conversation chat room will begin.

[1431] Getting and sending messages

[1432] The device detects user actions and retrieves the latest message data in the conversation chat room. This data typically includes messages from the past 24 hours or a customizable range.

[1433] The acquired message data is temporarily stored on the device and then sent to a server for analysis.

[1434] Message analysis and summarization

[1435] The server processes the received message data and supplies it to the generative AI model.

[1436] The generative AI model analyzes message data, understands the context, and extracts important information. Based on this information, the generative AI model generates a concise summary.

[1437] Sending and displaying abstracts

[1438] The server sends the generated summary to the terminal.

[1439] The terminal receives this summary and displays it in the conversation talk room.

[1440] Users can check the summary and quickly grasp important information.

[1441] Specific examples

[1442] Use in project management group chats

[1443] 1. A user opens a group chat for project management.

[1444] 2. Group chats have a lot of messages exchanged, and important information can get buried.

[1445] 3. The user taps the "Conversation Digest" button.

[1446] 4. The device retrieves message data from the past 24 hours and sends it to the server.

[1447] 5. The server supplies the received message data to the generative AI model and begins analysis.

[1448] 6. The generative AI model summarizes the conversation, including the following message:

[1449] "Person A: Please give me a progress report on the project."

[1450] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[1451] "Person C: The design proposal is complete. Please review it."

[1452] "D: What time is the meeting today?"

[1453] 7. A generative AI model summarizes these messages and generates the following summary:

[1454] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[1455] 8. The server sends the summary to the terminal.

[1456] 9. The device receives the summary and displays it in the conversation chat room.

[1457] 10. Users can quickly review the summary and grasp the important information.

[1458] In this way, by using the system of the present invention, users can efficiently extract important information from a large number of messages and make quick decisions, thereby improving the efficiency of business communications and preventing work delays.

[1459] The processing flow will be explained below.

[1460] Step 1:

[1461] A user uses a communication terminal to start a chat application and access a talk room.

[1462] The device retrieves the chat room data and displays it on the screen.

[1463] Step 2:

[1464] The device will display a "Conversation Digest" button on the interface within the chat room.

[1465] Step 3:

[1466] The user taps the "Conversation Digest" button.

[1467] The device detects this tap event and starts the following process.

[1468] Step 4:

[1469] The device retrieves the latest message data in the chat room for a certain range (e.g., the past 24 hours).

[1470] The acquired message data is temporarily stored in storage.

[1471] Step 5:

[1472] The terminal transmits the stored message data to the server.

[1473] The message data includes information such as the sender, the time of sending, and the content.

[1474] Step 6:

[1475] The server starts processing to analyze the received message data.

[1476] Feed the data into a generative AI model.

[1477] Step 7:

[1478] Generative AI models analyze message data, understand context and extract key information.

[1479] A concise summary is generated based on the extracted information.

[1480] Step 8:

[1481] The server transmits the generated summary to the terminal.

[1482] Confirm that the summary has been sent.

[1483] Step 9:

[1484] The summary received by the device is displayed in the chat room.

[1485] Users can check the summary and quickly grasp important information.

[1486] Example 1

[1487] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1488] In conventional conversation chat rooms, it has been difficult for users to quickly grasp important information amid the large number of messages exchanged. Furthermore, there is no way to concisely summarize the contents of messages, which reduces the efficiency of communication. In response to this problem, the present invention aims to enable users to easily grasp important information in conversation chat rooms.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1490] In this invention, the server includes means for analyzing message data using a generative AI model, understanding the context, and extracting important information, means for understanding the specific context from the analyzed message data, identifying key points, and providing a prompt sentence to the generative AI model, and means for transmitting the generated summary sentence to the communication terminal, thereby enabling a user to efficiently extract important information from a large number of messages and grasp it in a short time.

[1491] "User" means an individual or entity that accesses and operates the System.

[1492] A "communication terminal" is a device used by a user, and includes smartphones, tablets, personal computers, etc.

[1493] A "conversation talk room" is a virtual environment for users to exchange messages.

[1494] A "conversation digest button" is a button on the interface that a user taps to initiate the conversation summary function.

[1495] "Message data" is a collection of text information exchanged within a conversation talk room.

[1496] A "server" is a computer system that receives message data sent from a communication terminal and performs analysis processing.

[1497] A "generative AI model" is an artificial intelligence program that analyzes and summarizes text data based on machine learning algorithms.

[1498] A "summary" is a concise sentence that contains key information from the original message data, extracted and generated by a generative AI model.

[1499] A "prompt sentence" is an instruction sentence input into a generative AI model and is used during summarization and analysis.

[1500] "Context" refers to the situation or background information that allows for understanding the meaning and relationships within the message data.

[1501] "Important information" refers to the main content or points in the message data that are deemed necessary for the user to know.

[1502] This invention is a system that allows users to use their communication terminals to efficiently summarize message data in a conversation talk room via a server. This system uses a generative AI model to analyze the message data, extract important information, and generate a summary.

[1503] Hardware Configuration

[1504] The communication device used by the user is a smartphone, tablet, or PC. These communication devices display a "Conversation Digest" button as part of their interface.

[1505] The server is a computer system that receives message data sent from the communication device, supplies the data to the generative AI model, and performs analysis and summarization. The server has the processing power to run Python programs and a network connection to access the generative AI model (e.g., OpenAI's GPT-3).

[1506] Software Configuration

[1507] The software of the communication terminal is an application that controls the user interface. The application has a function to display a conversation digest button on the interface and retrieve message data after the user taps the button.

[1508] The server software receives message data, preprocesses it, feeds prompts to the generative AI model, receives the analysis results, generates summaries, and sends the summaries, including Python scripts for data cleansing and metadata extraction.

[1509] Data processing and calculation

[1510] When a device retrieves message data, it typically retrieves the past 24 hours. This data is converted to JSON format and sent to the server.

[1511] The server preprocesses the received data, using Python scripts to cleanse the data and extract metadata, and then passes the preprocessed data to the generative AI model.

[1512] The generative AI model analyzes the message data based on the prompt, extracts important information, and generates a summary, which is then converted back to JSON format and sent from the server to the device.

[1513] Specific examples

[1514] For example, consider the following messages exchanged in a project management group chat:

[1515] "Person A: Please give me a progress report on the project."

[1516] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[1517] "Person C: The design proposal is complete. Please review it."

[1518] "D: What time is the meeting today?"

[1519] When the user taps the "Conversation Digest" button, the device retrieves these messages and sends them to the server, which uses a generative AI model to extract key information and generate the following summary:

[1520] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[1521] The generated summary is sent to the terminal and displayed to the user in the conversation chat room, allowing the user to quickly grasp important information.

[1522] In the above-described form, the present invention provides a means for a user to efficiently extract important information from a large number of messages and make quick decisions.

[1523] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1524] Step 1:

[1525] The user taps the "Conversation Digest" button.

[1526] Input: The user taps the conversation digest button.

[1527] Output: Trigger to start the conversation digest function.

[1528] Specific operation: The conversation digest button is tapped on the communication device interface, which uses a user interface written in JavaScript or a similar language.

[1529] Step 2:

[1530] The terminal retrieves the message data.

[1531] Input: Trigger to start the conversation digest function.

[1532] Output: Message data for the past 24 hours.

[1533] Specific operation: The device accesses a database or local storage and retrieves message data for the past 24 hours using an SQL query, etc.

[1534] Step 3:

[1535] The terminal transmits the message data to the server.

[1536] Input: Message data from the past 24 hours.

[1537] Output: Message data converted to JSON format.

[1538] Specific operation: The acquired message data is converted into JSON format and sent to the server using the HTTPS protocol, using a Python library or HTTP client.

[1539] Step 4:

[1540] The server pre-processes the message data.

[1541] Input: Message data in JSON format.

[1542] Output: Preprocessed data and metadata.

[1543] Specific operation: A Python script parses the received data, performs data cleansing (removing missing values, removing text noise, etc.), and extracts metadata (e.g., speaker, timestamp).

[1544] Step 5:

[1545] The server feeds the preprocessed data to the generative AI model.

[1546] Input: Preprocessed data and metadata.

[1547] Output: The prompt sentence that is fed to the generative AI model.

[1548] Specific operation: A prompt sentence is generated based on the preprocessed data and fed to a generative AI model (e.g., GPT-3). This prompt sentence is structured in text format as follows:

[1549] "Please summarize the following message: 'Message data content'"

[1550] Step 6:

[1551] A generative AI model generates a summary.

[1552] Input: The prompt sentence fed to the generative AI model.

[1553] Output: The generated summary.

[1554] How it works: The generative AI model analyzes message data based on the prompt, extracts key information, and generates a summary, a process that takes anywhere from a few seconds to a few minutes.

[1555] Step 7:

[1556] The server transmits the generated summary to the terminal.

[1557] Input: The generated summary sentence.

[1558] Output: Summary text converted to JSON format.

[1559] Specific operation: The generated summary is converted back into JSON format and sent to the terminal via the HTTPS protocol.

[1560] Step 8:

[1561] The terminal displays the summary.

[1562] Input: Abstract in JSON format.

[1563] Output: A summary statement that is displayed in the user interface.

[1564] Specific operation: The device parses the received summary text and displays it on the interface, using a JavaScript framework (e.g., React or Vue.js).

[1565] Step 9:

[1566] The user checks the summary.

[1567] Input: The summary text displayed in the user interface.

[1568] Output: Important information that the user will understand in a short time.

[1569] Specific operation: The user can check the displayed summary text and quickly grasp important information.

[1570] (Application example 1)

[1571] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1572] While autonomous vehicles are operating, it is difficult for drivers and passengers to quickly and efficiently grasp important traffic-related information, such as traffic conditions and accident information, in real time. Furthermore, important information can get lost in the vast amount of traffic information collected, potentially reducing operational efficiency and safety. There is a need for a system that can solve these issues and support quick decision-making while driving.

[1573] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1574] In this invention, the server includes: means for a user to access a conversation talk room using a communication terminal; means for the user to tap a conversation digest button; means for detecting the user's operation and acquiring message data in the conversation talk room; means for transmitting the acquired message data to the server; means for the server to analyze the message data using a generative AI model and extract important information; means for the server to transmit the generated summary to the communication terminal; means for the communication terminal to display the summary; means for acquiring traffic-related messages while the autonomous vehicle is in operation and generating a summary; and means for displaying the generated summary on an in-vehicle display, thereby enabling drivers and passengers to quickly grasp important traffic-related information and make appropriate decisions even while the autonomous vehicle is in operation.

[1575] A "communication terminal" is an electronic device that a user uses for communication.

[1576] A "conversation talk room" is a virtual space where users can exchange messages.

[1577] The "conversation digest button" is an operation button for summarizing messages in a chat room.

[1578] "Sensing an operation" means that the communication terminal detects an input action performed by the user.

[1579] "Message data" refers to text information exchanged within a chat room.

[1580] "Server" means a computer system that provides data analysis and storage functions.

[1581] A "generative AI model" is an algorithm that uses artificial intelligence to extract important information from data and generate a summary.

[1582] A "summary" is a sentence that shortens a long sentence or message and summarizes the main points.

[1583] An "autonomous vehicle" is a vehicle that operates autonomously without the need for a human driver.

[1584] A "traffic-related message" is a message that includes information about traffic conditions or traffic.

[1585] An "in-vehicle display" is a screen device that is installed inside a vehicle and displays information.

[1586] The following specific example is provided as an embodiment of the present invention. This system efficiently summarizes traffic-related message data while a user is driving an autonomous vehicle using a communication terminal. The details of this system are described below.

[1587] System Overview

[1588] Using the Traffic Digest Feature

[1589] The user presses the traffic digest button on the in-vehicle display of the autonomous vehicle, which causes the system to retrieve and summarize traffic-related messages from the past 24 hours.

[1590] Getting and sending messages

[1591] The terminal detects user operation and detects that the traffic digest button has been pressed, which causes the communication terminal to obtain traffic-related message data for the past 24 hours from the communication terminal of the autonomous vehicle.

[1592] The acquired message data is temporarily stored on the device and then sent to a server for analysis.

[1593] Message analysis and summarization

[1594] The server processes the received message data and supplies it to a generative AI model, such as OpenAI's GPT-3.

[1595] The generative AI model analyzes message data, understands the context, and extracts important information. Based on this information, the generative AI model generates a concise summary.

[1596] Sending and displaying abstracts

[1597] The server transmits the generated summary to the communication terminal.

[1598] The terminal receives the summary and displays it on the in-vehicle display, allowing the user to quickly understand important information by checking the summary.

[1599] Specific examples

[1600] For example, consider the following traffic-related messages stored in a vehicle's communications terminal:

[1601] "08:00 Traffic congestion occurs, traffic volume increases"

[1602] "10:00 Accident occurred, detour recommended"

[1603] "15:00 Traffic decrease"

[1604] "18:00 Traffic jam occurs again"

[1605] When the user presses the traffic digest button on the in-car display, the system sends these messages to the server and generates a summary like this:

[1606] Traffic information: Traffic increased around 08:00, causing congestion. An accident occurred around 10:00, and a detour was recommended. Traffic decreased around 15:00, and congestion reoccurred around 18:00.

[1607] Example prompt sentence:

[1608] Congestion and increased traffic volume

[1609] Accident occurred. Detour recommended.

[1610] Traffic decrease

[1611] Traffic jams occur again

[1612] This will enable drivers and passengers to quickly grasp important traffic-related information and make appropriate decisions while driving autonomous vehicles, contributing to improved operational efficiency and safety.

[1613] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1614] Step 1:

[1615] The user presses the traffic digest button displayed on the in-vehicle display of the autonomous vehicle. The input is the user's operation, and the output is the terminal detecting the operation. Specifically, the button press is recorded by touching the in-vehicle display.

[1616] Step 2:

[1617] The device detects user actions and retrieves traffic-related message data from the past 24 hours. The input is the user pressing a button, and the output is retrieving traffic-related message data. The device collects the latest traffic-related messages from its internal memory or a cloud service.

[1618] Step 3:

[1619] The acquired message data is sent to the server. The input is the traffic-related message data collected by the device, and the output is sending the data to the server. The device sends the message data as a POST request to the server's API endpoint via an Internet connection.

[1620] Step 4:

[1621] The server processes the received message data and supplies it to the generative AI model. The input is traffic-related message data received from the device, and the output is data to be analyzed and passed to the generative AI model. The server preprocesses the received data, converts it into an appropriate format, and sends it to the model.

[1622] Step 5:

[1623] The generative AI model analyzes the message data and extracts key information. The input is preprocessed traffic-related message data, and the output is the extracted key information. Specifically, the generative AI model analyzes the context of traffic messages and identifies important events and information.

[1624] Step 6:

[1625] The generative AI model generates a concise summary based on the extracted information. The input is the extracted key information, and the output is the generated summary. The model converts the key information into a summary format and creates a summary in an easy-to-understand style.

[1626] Step 7:

[1627] The server sends the generated summary to the communication terminal. The input is the generated summary, and the output is the summary sent to the communication terminal. The server again sends the generated summary to the terminal using the API endpoint.

[1628] Step 8:

[1629] The terminal receives the summary and displays it on the in-vehicle display. The input is the summary sent from the server, and the output is the summary displayed on the screen. The terminal receives the summary and displays it on the display so that the user can easily read it.

[1630] This trend will lead to the realization of a system that summarizes important traffic information in real time within autonomous vehicles and provides it to users.

[1631] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1632] The following specific example is provided as a mode for carrying out the present invention: This system allows a user to use a communication terminal and efficiently summarizes message data in a conversation talk room via a server, and further incorporates an emotion engine that recognizes the user's emotions.

[1633] System Overview

[1634] Using conversation digests and emotion recognition

[1635] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[1636] When a user taps this "Conversation Digest" button, a summary of the messages in the conversation chat room will begin.

[1637] Message capture and sentiment data collection

[1638] The device detects user actions and retrieves the latest message data in the conversation chat room. This data typically includes messages from the past 24 hours or a customizable range.

[1639] The device also simultaneously collects and temporarily stores the user's emotional data.

[1640] Sending message data and emotion data

[1641] The acquired message data and emotion data are temporarily stored on the device and then sent to a server for analysis.

[1642] Message and sentiment analysis

[1643] The server processes the received message data and emotion data and supplies it to the generative AI model.

[1644] Generative AI models analyze message data to understand context and extract key information.

[1645] The emotion engine analyzes the user's emotional data and adjusts the summary content according to the user's emotional state. For example, if the user is feeling stressed, the system generates a more concise summary that focuses on the important information.

[1646] Sending and displaying abstracts

[1647] The server sends the generated summary to the terminal.

[1648] The terminal receives this summary and displays it in the conversation talk room.

[1649] Users can check the summary and quickly grasp important information.

[1650] Specific examples

[1651] Use in stressful project management group chats

[1652] 1. A user opens a group chat for project management.

[1653] 2. Group chats are often filled with messages, and important information can get buried, causing stress for users.

[1654] 3. The user taps the "Conversation Digest" button.

[1655] 4. The device retrieves message data from the past 24 hours, collects emotion data, and sends it to the server.

[1656] 5. The server supplies the received message data and emotion data to the generative AI model and begins analysis.

[1657] 6. The generative AI model summarizes the conversation, including the following message:

[1658] "Person A: Please give me a progress report on the project."

[1659] "Person B: I've completed all of last week's tasks. I'm ready to move on to the next phase."

[1660] "Person C: The design proposal is complete. Please review it."

[1661] "D: What time is the meeting today?"

[1662] 7. The emotion engine takes into account the user's stress level and generates a concise summary that focuses on the important information.

[1663] 8. The generative AI model generates the following summary:

[1664] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[1665] 9. The server sends the summary to the terminal.

[1666] 10. The device receives the summary and displays it in the conversation room.

[1667] 11. Users can quickly review the summary and grasp the important information.

[1668] In this way, the system of the present invention allows users to efficiently extract important information from a large number of messages and make quick decisions. Furthermore, by taking into account the user's emotional state, the system provides a more appropriate summary, contributing to stress reduction.

[1669] The processing flow will be explained below.

[1670] Step 1:

[1671] A user uses a communication terminal to start a chat application and access a talk room.

[1672] The device retrieves the chat room data and displays it on the screen.

[1673] Step 2:

[1674] The device will display a "Conversation Digest" button on the interface within the chat room.

[1675] Step 3:

[1676] The user taps the "Conversation Digest" button.

[1677] The device detects this tap event and starts the following process.

[1678] Step 4:

[1679] The device retrieves the latest message data in the chat room for a certain range (e.g., the past 24 hours).

[1680] At the same time, the device also collects the user's emotional data and temporarily stores it in storage.

[1681] Step 5:

[1682] The message data and emotion data stored on the device are transmitted to the server.

[1683] The message data includes information such as the sender, the time of sending, and the content.

[1684] Step 6:

[1685] The server processes the received message data and emotion data.

[1686] This includes pre-processing the data to ensure accurate analysis of the entire data set.

[1687] Step 7:

[1688] The server uses the generative AI model to analyze the message data.

[1689] Generative AI models understand the context of messages and extract key information.

[1690] Step 8:

[1691] An emotion engine analyzes the user's emotion data to identify the user's emotional state.

[1692] The emotion engine determines whether the user is stressed or relaxed.

[1693] Step 9:

[1694] A concise summary is generated based on the key information extracted by the generative AI model.

[1695] At the same time, the content of the summary is adjusted based on information from the emotion engine.

[1696] For example, if the user is feeling stressed, a more concise and to-the-point summary will be generated.

[1697] Step 10:

[1698] The server transmits the generated summary to the terminal.

[1699] Confirm that the summary has been sent.

[1700] Step 11:

[1701] The summary received by the device is displayed in the chat room.

[1702] Users can check the summary and quickly grasp important information.

[1703] Step 12:

[1704] Collect user feedback and fine-tune the generative AI model and emotion engine algorithms as needed.

[1705] Example 2

[1706] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1707] In today's information society, users are increasingly using communication platforms where a large number of messages are exchanged. However, it is difficult to efficiently extract and understand important information from these numerous messages. The stress users experience while reading messages is also a major problem. Conventional systems simply display messages, without summarizing or adjusting them based on emotions, which results in information being scattered and hinders efficient decision-making. Therefore, a method is needed to enable users to efficiently grasp important information and reduce stress.

[1708] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring message data and user emotion data, means for analyzing the message data using a generative AI model to understand the context and extract important information, and means for analyzing the user emotion data using an emotion engine and adjusting the summary sentence according to the user's emotional state. This allows the user to quickly grasp automatically summarized information from a large number of messages, and furthermore, by adjusting the information according to the user's emotional state, less stressful communication is realized.

[1709] A "communication terminal" is a device that allows a user to connect to the Internet and use various services, and includes smartphones, PCs, tablets, etc.

[1710] "Conversation talk room" refers to a chat room or messaging platform where multiple users can exchange messages.

[1711] "User" refers to an individual who uses a communication terminal to participate in a conversation talk room and send, receive, and view messages.

[1712] "Conversation Digest Button" refers to a button on the interface that activates a function that summarizes messages in a conversation chat room.

[1713] "Message Data" refers to text and media information sent and received by a user within a conversation chat room.

[1714] "Emotion data" refers to information about the user's emotional state obtained from facial expressions, voice, self-input, etc.

[1715] "Server" refers to a central control system for receiving, analyzing, and generating message data and emotion data.

[1716] "Generative AI Model" means an artificial intelligence model used by the Server that includes algorithms that analyze message data and extract key information.

[1717] An "emotion engine" refers to a system that analyzes a user's emotional data and adjusts summary generation according to the user's emotional state.

[1718] This invention relates to a system that allows users to use their communication terminals to efficiently summarize message data in a conversation talk room via a server. It also incorporates an emotion engine that recognizes the user's emotions and adjusts the content of the summary.

[1719] System Overview

[1720] Hardware and software used

[1721] The present invention uses the following hardware and software.

[1722] Communication device: A device that allows users to send and receive messages and use summary functions. This includes smartphones, PCs, tablets, etc.

[1723] Server: A central control system that receives message data and emotion data, analyzes and processes them using generative AI models and emotion engines.

[1724] Generative AI models: Algorithms that analyze message data, understand context, and extract key information.

[1725] Emotion engine: A system that analyzes the user's emotional data and adjusts summary generation according to the user's emotional state.

[1726] Data Flow and Processing

[1727] 1. User Action:

[1728] A user accesses a conversation talk room using a communication terminal. The interface within the talk room displays a "Conversation Digest" button.

[1729] When the user taps this "Conversation Digest" button, the summary process begins.

[1730] 2. Data Acquisition:

[1731] The device detects the user's operations and acquires message data and user emotion data in the conversation talk room.

[1732] This data typically includes messages from the past 24 hours, or a range you configure.

[1733] 3. Data transmission and analysis:

[1734] The message data and emotion data acquired by the terminal are transmitted to a server via the Internet.

[1735] The server formats the received data and feeds it to the generative AI model.

[1736] A generative AI model analyzes message data and extracts key information.

[1737] 4. Use of Emotion Engine:

[1738] The server uses an emotion engine to analyze the user's emotion data and adjusts the summary according to the user's emotional state.

[1739] For example, if a user is feeling stressed, a summary will be generated that is more concise and focuses on the important information.

[1740] 5. Send and display abstracts:

[1741] The server sends the generated summary to the terminal.

[1742] The device receives the summary and displays it in the conversation chat room.

[1743] Users can check the summary and quickly grasp important information.

[1744] Specific examples

[1745] Use in stressful project management group chats

[1746] 1. A user opens a group chat for project management.

[1747] 2. Group chats can be filled with messages, and important information can get lost in the shuffle, causing frustration for users.

[1748] 3. The user taps the "Conversation Digest" button.

[1749] 4. The device retrieves message data from the past 24 hours, collects emotion data, and sends it to the server.

[1750] 5. The server passes the following prompt to the generative AI model:

[1751] "Project status report, task completion, next phase, design proposal submission, review request, meeting time proposal"

[1752] 6. The server adjusts the emotion engine and generates the following summary:

[1753] "Person B reports that last week's tasks have been completed and that they are ready to move on to the next phase. Person C has submitted a design proposal and requested a review. Person D has suggested a time for today's meeting."

[1754] 7. The server sends the summary to the terminal.

[1755] 8. The device receives the summary and displays it in the conversation chat room.

[1756] 9. Users can quickly review the summary and grasp the important information.

[1757] This allows users to efficiently extract important information from a large number of messages and make quick decisions.The emotion engine also provides summaries based on the user's emotional state, contributing to stress reduction.

[1758] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1759] Step 1:

[1760] A user accesses a conversation talk room using a communication terminal.

[1761] Users log in to a chat room using a dedicated application or web browser on a device such as a smartphone or PC. The input is the user's access operation, and the output is the display of the chat room interface, allowing the user to send and receive messages.

[1762] Step 2:

[1763] The user taps the "Conversation Digest" button.

[1764] The user taps the "Conversation Digest" button displayed in the chat room to start the process. The input is the button tap event, and the output is that this event is triggered to proceed to the next step.

[1765] Step 3:

[1766] The device acquires message data and user emotion data from within the chat room.

[1767] The device retrieves message data from the server for the past 24 hours or any specified range, and simultaneously acquires user emotion data from the camera and microphone. Input is button tap events and data read from the interface, and output is temporary storage of the retrieved message data and emotion data.

[1768] Step 4:

[1769] The message data and emotion data acquired by the terminal are transmitted to the server.

[1770] The device sends the temporarily stored data to a server via the Internet. The input is the acquired message data and emotion data, and the output is sending this data to the server. It is preferable to use encryption technology such as SSL / TLS for transmission.

[1771] Step 5:

[1772] The server analyzes the received data and feeds it into a generative AI model.

[1773] The server formats the received message data and emotion data into an appropriate format. The input is the data sent from the device, and the output is a set of formatted data. These data are fed into the generative AI model.

[1774] Step 6:

[1775] A generative AI model analyzes message data and extracts key information.

[1776] The generative AI model analyzes the provided message data, understands the context, and extracts key information. The input is the formatted message data, and the output is the extracted key information. Prompt sentences can be used to improve the accuracy of the model.

[1777] Step 7:

[1778] The emotion engine analyzes the user's emotion data and adjusts the summary text.

[1779] The emotion engine analyzes the user's emotional data and adjusts the summary based on the user's emotional state. For example, if the user is feeling stressed, the summary will be concise and focus on the important information. The input is the emotional data and extracted important information, and the output is a summary adjusted according to the user's emotion.

[1780] Step 8:

[1781] The server sends the generated summary to the terminal.

[1782] The server sends the summary adjusted by the emotion engine to the terminal. The input is the adjusted summary, and the output is the summary reaching the terminal. HTTP or WebSocket is generally used as the transmission protocol.

[1783] Step 9:

[1784] The device receives the summary and displays it in the conversation chat room.

[1785] The terminal displays the summary received from the server on the user interface. The input is the summary from the server, and the output is the user checking the summary. This allows the user to grasp important information in a short time.

[1786] This series of processing steps allows users to efficiently extract important information from a large number of messages, realizing stress-free communication.

[1787] (Application example 2)

[1788] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1789] In logistics centers, business communications are routinely carried out via large volumes of messages and voice input, but important information often gets buried, resulting in problems of reduced work efficiency and increased stress.In addition, there is the issue of the quality of communication not improving because information is provided uniformly without considering the emotional state of staff.

[1790] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing message data using a generative AI model and an emotion engine, extracting important information, and adjusting the content of the summary based on the user's emotional state, means for transmitting the summary generated by the server to the communication terminal, means for the communication terminal to display the summary, and means for summarizing and displaying business communications conducted by conversation and voice input. This makes it possible to efficiently extract important information from business communications at a logistics center and provide it in a form that takes into consideration the user's emotional state.

[1791] "User" refers to the staff working at the logistics center.

[1792] "Communication terminal" refers to a device that can connect to the Internet, including wearable devices such as smart glasses or smartphones.

[1793] "Conversation talk room" refers to a chat room or voice conversation space where business communications take place at a logistics center.

[1794] A "conversation digest button" refers to an interface that a user can tap to start a conversation summary.

[1795] "Message data" refers to text and audio data in a conversation talk room obtained from a communication terminal.

[1796] "Emotion data" refers to data that indicates the user's emotional state.

[1797] "Server" refers to the computer system that is equipped with the generative AI model and emotion engine and performs data analysis.

[1798] "Generative AI model" refers to an artificial intelligence model that analyzes acquired message data and extracts important information.

[1799] "Emotion engine" refers to a system for analyzing a user's emotional data and adjusting the content of the summary.

[1800] "Summary" refers to a sentence that summarizes the main points of a message, generated by a generative AI model and emotion engine.

[1801] The present invention is embodied in a system for summarizing business communications in a logistics center, using wearable devices such as smart glasses or smartphones as communication terminals.

[1802] System Configuration and Operation

[1803] 1. A user uses a communication device to access a conversation talk room.

[1804] Smart glasses (e.g., Google Glass, Microsoft HoloLens) are used as communication devices.

[1805] Users can communicate through text chat or voice input within the conversation room.

[1806] 2. Click the conversation digest button.

[1807] When a user taps the conversation digest button, the system retrieves message and emotion data from the past 24 hours. This button appears on the HUD (Head-Up Display).

[1808] 3. Data acquisition and transmission.

[1809] The communication terminal acquires the message data and emotion data and temporarily stores them.

[1810] The data is sent to a server where it is analyzed using a generative AI model (e.g., OpenAI GPT-3) and an emotion engine (e.g., Affectiva Emotion SDK).

[1811] 4. Data analysis and key points extraction.

[1812] The server uses a generative AI model to analyze the message data and extract key information.

[1813] The emotion engine analyzes the user's emotion data and adjusts the content of the summary according to the user's stress level.

[1814] 5. Send and display the summary.

[1815] The server transmits the generated summary to the communication terminal.

[1816] The communication terminal receives the summary and displays it on the glasses' HUD.

[1817] Specific examples

[1818] Usage example 1

[1819] Prompt statement:

[1820] Logistics Center Staff A: "What's the delivery schedule for this week?"

[1821] Logistics Center Staff B: "We have 10 cases to be delivered on Monday and 20 cases to be delivered on Tuesday."

[1822] summary:

[1823] "This week's delivery schedule: 10 cases on Monday, 20 cases on Tuesday."

[1824] Usage example 2

[1825] Prompt statement:

[1826] Logistics Center Staff C: "Which shelf should I place the next item on?"

[1827] Logistics Center Staff D: "Please place item X on shelf A and item Y on shelf B."

[1828] summary:

[1829] "Item X is on shelf A and item Y is on shelf B"

[1830] In this way, by implementing the invention, it is possible to efficiently extract important information from business communications at a logistics center and provide the information while taking into consideration the emotional state of the user, which is expected to improve work efficiency and reduce stress.

[1831] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1832] Step 1:

[1833] A user uses a communication terminal to access a conversation talk room.

[1834] Specific behavior:

[1835] The communication device (smart glasses or smartphone) connects to the server via the Internet and displays a screen of text chat and voice input data within the conversation room.

[1836] Step 2:

[1837] The user taps the conversation digest button.

[1838] Specific behavior:

[1839] The user taps the "Conversation Digest" button displayed on the communication device's HUD. This action triggers the acquisition of message and emotion data from the past 24 hours.

[1840] Input: User taps

[1841] Output: Start of data acquisition process

[1842] Step 3:

[1843] The device acquires the latest message data and emotion data in the conversation chat room and temporarily stores them.

[1844] Specific behavior:

[1845] Software built into the communication device retrieves the latest message data from the server via an API, and uses the camera and microphone to collect emotional data from the user's facial expressions and tone of voice, which is then temporarily stored in local storage.

[1846] Input: Message data in the conversation chat room, user emotion data

[1847] Output: Temporarily save the acquired data

[1848] Step 4:

[1849] The terminal transmits the acquired message data and emotion data to the server.

[1850] Specific behavior:

[1851] The communication device sends the acquired message data and emotion data to the server via an HTTP request. The sent data is encoded in JSON format.

[1852] Input: Temporarily saved message data and emotion data

[1853] Output: Send data to the server

[1854] Step 5:

[1855] The server uses a generative AI model and an emotion engine to analyze message data, extract key information, and adjust the content of the summary based on the user's emotional state.

[1856] Specific behavior:

[1857] The server inputs the received data into a generative AI model (e.g., OpenAI GPT-3), which analyzes the message data and extracts context and important information. At the same time, an emotion engine (e.g., Affectiva Emotion SDK) analyzes the user's emotional data and makes adjustments, such as making the summary more concise, if the user is feeling stressed.

[1858] Input: Message data and emotion data sent to the server

[1859] Output: Adjusted summary

[1860] Step 6:

[1861] The server transmits the generated summary to the communication terminal.

[1862] Specific behavior:

[1863] The server sends the generated summary to the communication terminal via an HTTP response. The response data is encoded in JSON format.

[1864] Input: Adjusted summary

[1865] Output: Send summary to communication terminal

[1866] Step 7:

[1867] The communication terminal receives the summary and displays it in the conversation talk room.

[1868] Specific behavior:

[1869] The communication terminal displays the received summary on a HUD (Head-Up Display) and provides it to the user, allowing the user to quickly grasp important information.

[1870] Input: Abstract sent from the server

[1871] Output: Summary displayed on HUD

[1872] Through the above steps, important information can be efficiently extracted from business communications at a logistics center, and a summary can be provided that takes into consideration the user's emotional state.

[1873] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1874] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1875] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1876] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1877] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1878] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1879] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1880] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1881] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1882] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1883] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1884] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1885] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1886] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1887] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1888] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1889] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1890] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1891] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1892] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1893] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1894] The following is further disclosed regarding the above embodiment.

[1895] (Claim 1)

[1896] A means for a user to access a conversation talk room using a communication terminal;

[1897] a means for a user to tap a conversation digest button;

[1898] means for detecting an operation of the user and acquiring message data in the conversation talk room;

[1899] means for transmitting the acquired message data to a server;

[1900] a means for the server to use the generative AI model to analyze the message data and extract key information;

[1901] means for transmitting the abstract generated by the server to the communication terminal;

[1902] A system including a means for displaying a summary sentence in a communication terminal.

[1903] (Claim 2)

[1904] 10. The system of claim 1, wherein the server understands specific context and identifies key points from the parsed message data.

[1905] (Claim 3)

[1906] 10. The system of claim 1, wherein the user can use the conversation digest button in both individual chats and group chats.

[1907] "Example 1"

[1908] (Claim 1)

[1909] A means for a user to access a conversation talk room using a communication terminal;

[1910] a means for a user to tap a conversation digest button;

[1911] means for detecting an operation of the user and acquiring message data in the conversation talk room;

[1912] means for transmitting the acquired message data to a server;

[1913] A means for the server to use a generative AI model to analyze message data, understand the context, and extract key information;

[1914] means for transmitting the abstract generated by the server to the communication terminal;

[1915] The system includes a means for displaying a summary sentence on a communication terminal and for a user to confirm the summary sentence.

[1916] (Claim 2)

[1917] 2. The system of claim 1, wherein the server understands specific context from the parsed message data, identifies key points, and provides prompt sentences to the generative AI model.

[1918] (Claim 3)

[1919] 10. The system of claim 1, wherein the user can use the conversation digest button in both individual chats and group chats.

[1920] "Application Example 1"

[1921] (Claim 1)

[1922] A means for a user to access a conversation talk room using a communication terminal;

[1923] a means for a user to tap a conversation digest button;

[1924] means for detecting an operation of the user and acquiring message data in the conversation talk room;

[1925] means for transmitting the acquired message data to a server;

[1926] a means for the server to use the generative AI model to analyze the message data and extract key information;

[1927] means for transmitting the abstract generated by the server to the communication terminal;

[1928] a means for displaying a summary sentence in the communication terminal;

[1929] A means for acquiring traffic-related messages while the autonomous vehicle is operating and generating a summary sentence;

[1930] The system includes a means for displaying the generated summary on an in-vehicle display.

[1931] (Claim 2)

[1932] 10. The system of claim 1, wherein the server understands specific context and identifies key points from the parsed message data.

[1933] (Claim 3)

[1934] 10. The system of claim 1, wherein the user can use the conversation digest button in both individual chats and group chats.

[1935] "Example 2: Combining Emotion Engines"

[1936] (Claim 1)

[1937] A means for a user to access a conversation talk room using a communication terminal;

[1938] a means for a user to tap a conversation digest button;

[1939] means for detecting the user's operation and acquiring message data and user's emotion data in the conversation talk room;

[1940] means for transmitting the acquired message data and user emotion data to a server;

[1941] A means for the server to use a generative AI model to analyze message data, understand the context, and extract key information;

[1942] a means for the server to analyze the user's emotion data using an emotion engine and adjust the summary sentence according to the user's emotion state;

[1943] means for transmitting the abstract generated by the server to the communication terminal;

[1944] A system including a means for displaying a summary sentence in a communication terminal.

[1945] (Claim 2)

[1946] 2. The system of claim 1, wherein the server comprises means for generating a summary by understanding specific contexts and extracting important information from the analyzed message data.

[1947] (Claim 3)

[1948] 10. The system of claim 1, wherein the user can use the conversation digest button in both individual chats and group chats.

[1949] "Application example 2 when combining emotion engines"

[1950] (Claim 1)

[1951] A means for a user to access a conversation talk room using a communication terminal;

[1952] a means for a user to tap a conversation digest button;

[1953] means for sensing the user's operation and acquiring message data and emotion data in the conversation talk room;

[1954] means for transmitting the acquired message data and emotion data to a server;

[1955] a means for the server to analyze the message data using the generative AI model and the emotion engine, extract important information, and adjust the content of the summary sentence based on the user's emotional state;

[1956] means for transmitting the abstract generated by the server to the communication terminal;

[1957] a means for displaying a summary sentence in the communication terminal;

[1958] means for summarizing and displaying spoken and voice-activated business communications;

[1959] A system including:

[1960] (Claim 2)

[1961] 2. The system of claim 1, wherein the server understands specific contexts and key points from the analyzed message data and sentiment data, and identifies key points.

[1962] (Claim 3)

[1963] The system according to claim 1, wherein the user can use the conversation digest button in both individual chat and group chat, and the system is applicable to business communications in a logistics center. [Explanation of symbols]

[1964] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to access a conversation talk room using a communication terminal; a means for a user to tap a conversation digest button; means for detecting an operation of the user and acquiring message data in the conversation talk room; means for transmitting the acquired message data to a server; a means for the server to use the generative AI model to analyze the message data and extract key information; means for transmitting the abstract generated by the server to the communication terminal; A system including a means for displaying a summary sentence in a communication terminal.

2. 10. The system of claim 1, wherein the server understands specific context and identifies key points from the parsed message data.

3. The system of claim 1 , wherein the user can use the conversation digest button in both individual chats and group chats.

Images