System

Abstract

A system is provided.SOLUTION: A system, comprising: means for a user to send a message; means for a device to obtain and send a most recent interaction to a server; means for the server to generate an optimal sticker using context analysis AI; means for providing the generated sticker to the user; and means for the user to select and send a sticker.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern communication applications, users are typically required to respond quickly and appropriately through messages. However, selecting an appropriate sticker or emoji from the available options is time-consuming, which can disrupt the flow of communication. This problem is particularly acute in communities such as students, where context-sensitive responses are required. Therefore, there is a need for a system that can instantly generate and automatically suggest appropriate stickers based on recent interactions. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system including: a means for a user to send a message; a means for a terminal to acquire the most recent exchange and transmit it to a server; a means for the server to generate optimal stamps using context analysis AI; a means for providing the generated stamps to the user; and a means for the user to select and send stamps. This allows the user to quickly acquire appropriate stamps that fit the context of the most recent exchange and send them while maintaining the flow of communication. Furthermore, the generated stamps can be text messages, image messages, or video messages, making it possible to accommodate a variety of response formats.

[0006] "User" refers to the entity that uses this system to send messages or interact with the system by creating and sending stamps.

[0007] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[0008] "Server" refers to a central computer system that receives and processes data sent from terminals.

[0009] A "message" refers to a unit of communication such as text data, image data, or video data sent by a user.

[0010] "Recent interactions" refers to a collection of data such as multiple messages, images, and videos that a user has had in the past.

[0011] "Contextual analysis AI" refers to artificial intelligence technology that analyzes a user's most recent interactions and generates appropriate stamps.

[0012] "Stamps" refer to visual means of expression such as images, emojis, and videos that users use when communicating.

[0013] "Generative AI" refers to artificial intelligence technology that automatically generates optimal stamps based on recent interactions.

[0014] "Means for providing" refers to a mechanism for displaying the generated stamps on the user's terminal and making them selectable.

[0015] "Means for selecting and sending" refers to a mechanism by which a user selects from the provided stamps and sends them to other users. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] ---

[0038] The present invention relates to a system that automatically generates and provides stamps that are optimal for the context of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[0039] Overall system overview

[0040] This system mainly consists of a user's device, a server, and a generation AI. When a user sends a message, the message and data on the most recent exchange are sent from the device to the server. The server analyzes the received data and generates the most appropriate stamp using context analysis AI. The generated stamp is then sent back from the server to the user's device, where the user can select the appropriate one to send.

[0041] System operation explanation

[0042] 1. A user sends a message

[0043] A user types a message into a text input field in a chat app and presses the send button. For example, if a user sends a message like "What's my homework today?", the message is saved on the device.

[0044] 2. The device retrieves the most recent interaction and sends it to the server

[0045] The device retrieves the most recent messages, images, videos, etc. from the chat history it stores internally. For example, it retrieves the last 10 messages and sends them to the server, including the message that was just sent.

[0046] 3. The server uses contextual analysis AI to generate the optimal stamp

[0047] The server analyzes the most recent interactions received from the device and uses contextual analysis AI to generate stamps that best fit the context. For example, a stamp of "an expression of someone struggling with homework" might be generated.

[0048] 4. Provide the generated stamp to the user

[0049] The server sends the stamps generated by the AI ​​to the device and displays them to the user, who can then choose the best one from the multiple stamps generated.

[0050] 5. The user selects a stamp and sends it.

[0051] Next, the user selects one of the displayed stamps and presses the send button. For example, the user selects the stamp with the expression "worried about homework" and sends it.

[0052] Specific examples

[0053] A specific example of this system is shown below.

[0054] 1. User A sends a message

[0055] User A types "What's today's homework?" into a chat app and sends it.

[0056] 2. The device retrieves the most recent conversation and sends it to the server

[0057] The device retrieves the most recent 10 message history records and sends them to the server, including the message to be sent.

[0058] 3. The server generates stamps using context analysis AI

[0059] Context analysis AI generates a stamp that best fits the context of "What's today's homework?", depicting an "expression of someone struggling with homework."

[0060] 4. Provide the generated stamp to the user

[0061] The server returns the generated stamp to the terminal, which displays it to User A.

[0062] 5. User A selects a stamp and sends it

[0063] User A selects the stamp with the expression "looking worried about homework" and presses the send button.

[0064] This allows user A to quickly select an appropriate stamp and continue communication smoothly.

[0065] ---

[0066] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context of a message sent by a user. This allows users to quickly and easily obtain appropriate stamps, enabling efficient communication.

[0067] The processing flow will be explained below.

[0068] ---

[0069] Step 1:

[0070] The user enters a message and presses the send button.

[0071] A user types a message into a text input field in a chat app, for example, "What's my homework for today?", and presses the send button.

[0072] This message will be saved in the chat history on the device.

[0073] Step 2:

[0074] The device retrieves the most recent interactions and sends them to the server.

[0075] Obtain chat history stored internally on the device. For example, obtain the last 10 messages, images, and videos.

[0076] This data is sent as packets to the server as the "most recent exchange."

[0077] Step 3:

[0078] The server receives the most recent exchange.

[0079] The server receives the data packet sent from the terminal and prepares to analyze it.

[0080] The server temporarily stores this data in memory.

[0081] Step 4:

[0082] The server uses contextual analysis AI to analyze recent interactions.

[0083] The server inputs the received "most recent interaction" into a context analysis AI and analyzes it to generate the most appropriate stamp.

[0084] Contextual analysis AI understands the content of each media, such as text, images, and videos, and generates stamps based on that context.

[0085] Step 5:

[0086] The server uses generation AI to generate the optimal stamp.

[0087] Based on the analysis results, the AI ​​generates the most suitable stamp. For example, it can generate a stamp of someone struggling with their homework.

[0088] The generated stamp is temporarily stored in the server's memory.

[0089] Step 6:

[0090] The server sends the generated stamp to the terminal.

[0091] The server transmits the generated stamp to the terminal as a data packet.

[0092] This data packet may contain multiple stamp candidates.

[0093] Step 7:

[0094] The terminal receives the generated stamp and displays it to the user.

[0095] The terminal receives and analyzes the data packets from the server.

[0096] The interface is updated so that the generated stamp is visually displayed to the user.

[0097] Step 8:

[0098] The user selects and sends a stamp.

[0099] The user selects an appropriate stamp from the displayed ones and presses the send button.

[0100] For example, a user selects and sends a stamp with an expression of "an expression of someone struggling with homework."

[0101] Step 9:

[0102] The terminal transmits the selected stamp to the other chat participants.

[0103] The terminal retransmits the message including the stamp selected by the user and distributes it to the other chat participants.

[0104] The stamps selected in this way are sent in the flow of the conversation.

[0105] ---

[0106] As mentioned above, the processing steps of this system have been explained in detail. By showing the specific operations at each step, the operation of the entire system can be clearly understood.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] In recent years, users often need to select appropriate stamps or graphics when sending messages, but the selection process can be complex and time-consuming. In particular, to find a stamp that fits the context, users must search through a large number of options, which can disrupt communication. This creates a poor user experience and hinders efficient communication.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes means for analyzing a context using a natural language processing model and generating an optimal graphic, means for displaying multiple candidate graphics, and means for providing the generated graphic. This allows a user to quickly and appropriately obtain a graphic that is optimal for the context when sending a message, enabling efficient communication.

[0112] 1. "User" means any entity that operates a communications system to send or receive information.

[0113] 2. "Information" means any message or content expressed in digital form, such as text data, image data, or video data.

[0114] 3. "Terminal" means a device used by a user to input, send, or receive information. Examples include smartphones, tablets, and personal computers.

[0115] 4. "Communication Content" means the history and current messages of information sent and received by a Device.

[0116] 5. "Server" means a computer system that processes and provides information in cooperation with terminals via a communications network.

[0117] 6. A "natural language processing model" is a machine learning model used to analyze text data and understand its meaning and context. Examples include GPT-3 and BERT.

[0118] 7. "Context" refers to the surrounding or background information needed to interpret the meaning of a message.

[0119] 8. "Graphics" is a general term for visual representations such as text, images, and videos, which are used to express the user's emotions and situations.

[0120] 9. A "prompt" is an instruction or question given to a generative AI model to generate an appropriate output.

[0121] 10. "REST API" means an application programming interface for sending and receiving data in the style of Representational State Transfer.

[0122] 11. "HTTP response" refers to data sent from a server to a terminal, and is sent using the HTTP protocol.

[0123] 12. “SQL query” means a statement used in Structured Query Language to retrieve information from a database.

[0124] The above are definitions of important terms included in the rewritten claims.

[0125] MODE FOR CARRYING OUT THE INVENTION

[0126] The present invention relates to a system that automatically generates and provides stamps and graphics that are optimal for the context of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[0127] Overall system overview

[0128] This system is composed of a user's device, a server, and a generative AI model. When a user sends a message, the message and the most recent communication content are sent from the device to the server. The server analyzes the received data and generates the optimal graphic using a natural language processing model. The generated graphic is then provided back to the user's device by the server, and the user can select the appropriate one to send.

[0129] Hardware and software used

[0130] Device: A digital device that a user uses to send and receive data. Examples include smartphones, tablets, and computers.

[0131] Server: A computer system that analyzes the received communications and generates stamps and graphics.

[0132] Natural language processing models: For example, machine learning models such as "GPT-3" and "BERT" are used.

[0133] Generative AI model: A model that works in conjunction with a natural language processing model to generate context-based graphics.

[0134] Communication protocol: REST API and HTTP responses are used to send and receive data.

[0135] System operation explanation

[0136] As a concrete example, we will explain the flow when User A sends a message "What's today's homework?" in a chat app.

[0137] 1. A user sends a message

[0138] User A enters "What's today's homework?" into the text input field of a chat app and presses the send button. This message is temporarily stored on the device.

[0139] 2. The device acquires the most recent communication content and sends it to the server

[0140] The device retrieves the last 10 messages from the internal database using an SQL query, and the retrieved messages, images, and videos are converted to JSON format and sent as a POST request to the server's REST API.

[0141] 3. The server uses contextual analysis AI to generate optimal graphics

[0142] The server passes the received data to a natural language processing model (e.g., GPT-3 or BERT), which analyzes the message "What's today's homework?" and the past 10 interactions as input. Based on the context, an appropriate prompt is input to the generative AI model, which generates a stamp of an "expression of someone struggling with homework."

[0143] 4. Serving the generated graphics to the user's device

[0144] The server generates a URL for the generated graphic and returns it to the user's device as an HTTP response. The device receives this response and dynamically displays the stamp in the chat app's interface.

[0145] 5. The user selects a stamp and sends it.

[0146] User A selects the "face struggling with homework" stamp from the provided stamps and presses the send button again. The URL of this selected stamp is again sent from the device to the server and is finally displayed on the chat partner's screen.

[0147] Prompt Sentence Examples

[0148] Below is an example of a prompt sentence to input to the generative AI model.

[0149] Message Context: What's my homework today?

[0150] Related past messages:

[0151] 1. Homework email from your teacher

[0152] 2. Communicating with classmates about homework

[0153] Generated stamp: An expression of someone struggling with homework

[0154] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context of a message sent by a user. Users can quickly and easily obtain appropriate stamps, enabling efficient communication.

[0155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0156] Explain the program's processing in detail

[0157] Step 1:

[0158] A user sends a message

[0159] explanation:

[0160] A user enters a message into the text input field of a chat app and presses the send button. This message is temporarily stored on the device.

[0161] Specific operation and input / output:

[0162] Input: A message typed by the user (e.g., "What's my homework today?")

[0163] Action: A message is entered into the text field and the send button is pressed.

[0164] Output: Stored in the message queue in the terminal.

[0165] Step 2:

[0166] The device acquires the most recent communication content and sends it to the server.

[0167] explanation:

[0168] The user's device retrieves the most recent message history from an internal database and sends it to the server.

[0169] Specific operation and input / output:

[0170] Input: Messages sent by the user and past communication history (e.g., last 10 messages)

[0171] Operation:

[0172] Retrieve the last 10 messages from the internal database using an SQL query.

[0173] Convert the retrieved data into JSON format.

[0174] Send the JSON data as a POST request to the server's REST API.

[0175] Output: Message history in JSON format is sent to the server.

[0176] Step 3:

[0177] The server uses contextual analysis AI to generate optimal graphics

[0178] explanation:

[0179] The server analyzes the received data, understands the context using natural language processing models, and generates the most appropriate graphics.

[0180] Specific operation and input / output:

[0181] Input: Recent message history (JSON format data)

[0182] Operation:

[0183] Passing JSON data to a natural language processing model (e.g., GPT-3 or BERT).

[0184] Contextual analysis is performed to generate prompts.

[0185] Pass the prompt to a generative AI model to generate the appropriate graphic.

[0186] Output: Generated optimal graphic (e.g., a stamp of a person struggling with homework)

[0187] Step 4:

[0188] Serving the generated graphics to the user's device

[0189] explanation:

[0190] The server generates a URL for the generated graphic and provides it to the user's terminal.

[0191] Specific operation and input / output:

[0192] Input: Generated graphic

[0193] Operation:

[0194] Save the graphic and generate a URL for it.

[0195] The URL is sent to the user's device as an HTTP response.

[0196] Output: The URL of the graphic is displayed on the user's device.

[0197] Step 5:

[0198] The user selects a stamp and sends it.

[0199] explanation:

[0200] The user selects an appropriate stamp from the ones provided and presses the send button.

[0201] Specific operation and input / output:

[0202] Input: A stamp selected by the user from the displayed graphics (e.g., a stamp of a person struggling with homework)

[0203] Operation:

[0204] The user taps or clicks on the displayed stamp.

[0205] The URL of the selected stamp is sent from the terminal to the server by pressing the send button again.

[0206] Output: The URL of the selected stamp will be displayed on the chat partner's screen.

[0207] The above is the specific flow of the program processing for this system, which allows users to quickly select the graphic that best suits the context and achieve smooth communication.

[0208] (Application example 1)

[0209] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0210] In conventional messaging applications, it is difficult for users to quickly select the most suitable stamp when sending a message. This leads to problems such as communication being disrupted and a poor user experience. In addition, selecting stamps must be done manually, which is inefficient.

[0211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0212] In this invention, the server includes a means for a user to send a message, a means for a terminal to acquire the most recent exchange and send it to the server, a means for the server to generate optimal stamps using context analysis AI, a means for providing the generated stamps to the user, a means for the user to select and send stamps, a means for the server to send the results of context analysis to the stamp generation microservice, and a means for providing the URL of the stamp acquired from the microservice to the user terminal.This allows the user to automatically generate and receive stamps that are optimal for the context of the message they are sending, thereby facilitating communication and improving the user experience.

[0213] "User" refers to an individual user of the system.

[0214] "Message" refers to text, image or video information sent by a user to communicate with other users.

[0215] "Terminal" refers to a communication device such as a smartphone or tablet used by a user.

[0216] "Recent interactions" refers to the most recent communication history in the system.

[0217] "Server" refers to a computer system that processes and stores data and operates in conjunction with a terminal.

[0218] "Contextual analysis AI" refers to artificial intelligence technology that uses natural language processing to analyze the meaning and intent of text.

[0219] "Optimal stamp" refers to an icon or illustration that best fits the context of the user's message.

[0220] "Stamp generation means" refers to the process of generating stamps using contextual analysis AI.

[0221] A "stamp generation microservice" is a small, independent component that specializes in a specific function and is dedicated to generating stamps.

[0222] "Stamp URL" refers to the web address where the generated stamp is stored.

[0223] MODE FOR CARRYING OUT THE INVENTION

[0224] System Overview

[0225] This invention is a system that automatically generates and provides stamps that are optimal for the context of a message sent by a user. This system mainly consists of a user device, a server, a context analysis AI, and a stamp generation microservice.

[0226] System configuration

[0227] 1. User's device

[0228] The user's device (such as a smartphone or tablet) is a communication device that sends and receives messages. The device's role is to obtain the most recent exchanges and send them to the server.

[0229] 2. Server

[0230] The server receives messages and chat history sent by users, uses contextual analysis AI to generate optimal stamps, and then sends the generated stamps to a microservice, providing the resulting stamp URL to the user's device.

[0231] 3. Context analysis AI

[0232] Contextual analysis AI uses natural language processing to analyze the context of messages, and based on the results of this analysis, provides information to generate the best stamps for the user's message.

[0233] 4. Stamp Generation Microservice

[0234] The stamp generation microservice is a specialized component that generates the optimal stamp URL based on the results of contextual analysis sent from the server.

[0235] Operation explanation

[0236] 1. A user sends a message

[0237] The user types a message and presses the send button. For example, if the message is "I haven't finished my homework," the message is saved on the device.

[0238] 2. The device retrieves the most recent interaction and sends it to the server

[0239] The device retrieves past conversations, including the most recent message, from its internal chat history and sends that data to the server. Specifically, the last 10 messages are the target.

[0240] 3. The server uses contextual analysis AI to generate the optimal stamp

[0241] The server inputs the sent message and chat history into a context analysis AI to analyze the context. For example, for a message such as "I can't finish my homework," the context analysis AI determines that a stamp with a "worried expression" would be appropriate.

[0242] 4. Send the result to the stamp generation microservice and get the stamp URL.

[0243] The server sends the results of the contextual analysis to the stamp generation microservice, which generates a URL for the most suitable stamp, which is returned to the server.

[0244] 5. Provide the generated stamp to the user

[0245] The server sends the stamp URL to the user's device, and the user can view the stamp using this URL. The user can select and send stamps from the provided stamps.

[0246] Specific examples

[0247] A specific example of the system's operation is shown below.

[0248] 1. A user sends a new message

[0249] A user sends a message saying, "I can't finish my homework."

[0250] 2. Your device will connect your chat history with new messages

[0251] The last 10 chat messages and the new message are concatenated and sent to the server.

[0252] 3. Contextual AI generates stamps

[0253] Context analysis AI generates stamps that best fit the context of "I can't finish my homework."

[0254] 4. The stamp generation microservice generates the stamp URL.

[0255] The server sends the analysis results to the stamp generation microservice and obtains the URL of the optimal stamp.

[0256] 5. User checks and sends the provided stamp

[0257] The user selects a provided stamp and presses the send button.

[0258] Prompt Sentence Examples

[0259] The following is an example of a prompt:

[0260] text

[0261] Below is the chat history between User A and User B. Please generate the stamp that best suits User A's latest message.

[0262] Chat history:

[0263] User A: It's raining today.

[0264] User B: Maybe it'll be sunny tomorrow.

[0265] User A: What should I do about my homework?

[0266] User B: I want to hang out with friends.

[0267] User A: I want to go see a movie.

[0268] Latest Message:

[0269] User A: I can't finish my homework.

[0270] In this way, by using the system of the present invention, a user can quickly and easily obtain a stamp that is optimal for the context of a message, thereby achieving smoother and more efficient communication.

[0271] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0272] Step 1:

[0273] The user enters a message and presses the send button. The input is the message text (e.g., "I haven't finished my homework yet."), and this message is saved on the device. The output is the latest message data.

[0274] Step 2:

[0275] The device retrieves past conversations, including the most recent message, from the chat history it stores internally. The input is the most recent message and chat history (e.g., the last 10 messages), which it then sends to the server. The output is the dataset sent to the server.

[0276] Step 3:

[0277] The server inputs the received data set into the context analysis AI and performs context analysis. The input is all message text (e.g., "I can't finish my homework" and past chat history), and based on this, it generates analysis results. The output is the context analysis results.

[0278] Step 4:

[0279] The server sends the result of the contextual analysis to the stamp generation microservice to obtain the optimal stamp URL. The input is the contextual analysis result, and the appropriate stamp URL is generated based on this. The output is the stamp URL.

[0280] Step 5:

[0281] The server sends the generated stamp URL to the user's device and provides it to the user. The input is the stamp URL, which is returned to the user's device. The output is the stamp URL that is displayed to the user.

[0282] Step 6:

[0283] The user selects the appropriate stamp from the provided ones and presses the submit button. The input is the stamp URL, and the output is the submission of the selected stamp.

[0284] In this way, the input data is processed at each step, and ultimately an appropriate stamp is provided to the user.

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] ---

[0287] The present invention relates to a system for automatically generating and providing stamps that are optimal for the context and emotion of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[0288] Overall system overview

[0289] This system mainly consists of the user's device, a server, a context analysis AI, and an emotion engine. When a user sends a message, that message and data on the most recent exchange are sent from the device to the server. At that time, the emotion engine recognizes the user's emotions, and the context analysis AI analyzes the context. The server generates optimal stamps based on the results of this analysis. The generated stamps are then sent back from the server to the user's device, where the user can select the appropriate one to send.

[0290] System operation explanation

[0291] 1. A user sends a message

[0292] A user types a message into a text input field in a chat app and presses the send button. For example, if a user sends a message like "What's my homework today?", the message is saved on the device.

[0293] 2. The device retrieves the most recent interaction and sends it to the server

[0294] The device retrieves the most recent messages, images, videos, etc. from the chat history it stores internally. For example, it retrieves the last 10 messages and sends them to the server, including the message that was just sent.

[0295] 3. The server receives the most recent exchange

[0296] The server receives the data packets sent from the terminal and prepares them for analysis. The server temporarily stores these data in memory.

[0297] 4. The server uses contextual analysis AI to analyze the most recent conversations

[0298] The server inputs the received "recent interactions" into a context analysis AI, which analyzes them to generate optimal stamps. The context analysis AI understands the content of each media, such as text, images, and videos, and generates stamps based on that context.

[0299] 5. Emotion engine analyzes user emotions

[0300] The emotion engine analyzes the user's emotions based on the messages, images, voice data, etc. received by the server from the device. For example, it can identify emotions such as "worried" or "happy" from the content of the user's message or image.

[0301] 6. The server generates the optimal stamp using generation AI

[0302] Based on the analysis results of the context analysis AI and emotion engine, the generation AI generates the optimal stamp. For example, a stamp of an "expression of someone struggling with homework" may be generated. The generated stamp is temporarily stored in the server's memory.

[0303] 7. The server sends the generated stamp to the device.

[0304] The server transmits the generated stamp to the terminal as a data packet, which may contain multiple stamp candidates.

[0305] 8. The device receives the generated stamp and displays it to the user.

[0306] The device receives and analyzes the data packets from the server and updates the interface so that the generated stamps are visually displayed to the user.

[0307] 9. The user selects a stamp and sends it.

[0308] The user selects an appropriate stamp from the displayed ones and presses the send button. For example, the user selects a stamp with an expression of someone struggling with homework and sends it.

[0309] 10. The device sends the selected stamp to other chat participants.

[0310] The terminal then retransmits the message containing the stamp selected by the user and distributes it to the other chat participants, thus transmitting the selected stamp in the flow of the conversation.

[0311] Specific examples

[0312] A specific example of this system is shown below.

[0313] 1. User A sends a message

[0314] User A types "What's today's homework?" into a chat app and sends it.

[0315] 2. The device retrieves the most recent conversation and sends it to the server

[0316] The device retrieves the most recent 10 message history records and sends them to the server, including the message to be sent.

[0317] 3. The server generates stamps using context analysis AI

[0318] Context analysis AI generates a stamp that best fits the context of "What's today's homework?", depicting an "expression of someone struggling with homework."

[0319] 4. Provide the generated stamp to the user

[0320] The server returns the generated stamp to the terminal, which displays it to User A.

[0321] 5. User A selects a stamp and sends it

[0322] User A selects the stamp with the expression "looking worried about homework" and presses the send button.

[0323] This allows user A to quickly select an appropriate stamp and continue communication smoothly.

[0324] ---

[0325] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context and emotion of a message sent by a user. This allows users to quickly and easily obtain appropriate stamps, enabling efficient communication.

[0326] The processing flow will be explained below.

[0327] ---

[0328] Step 1:

[0329] The user enters a message and presses the send button.

[0330] A user types a message into a text input field in a chat app, for example, "What's my homework for today?", and presses the send button.

[0331] This message will be saved in your device's chat history.

[0332] Step 2:

[0333] The device retrieves the most recent interactions and sends them to the server.

[0334] Data such as the last 10 messages, images, and videos is retrieved from the chat history stored internally on the device.

[0335] These "recent interactions" are packaged into data packets and sent to the server.

[0336] Step 3:

[0337] The server receives the most recent exchange.

[0338] The server receives the data packets sent from the terminal.

[0339] The received data is temporarily stored in memory and prepared for analysis.

[0340] Step 4:

[0341] The server uses contextual analysis AI to analyze recent interactions.

[0342] The server inputs the "most recent interaction" received into a context analysis AI.

[0343] Contextual analysis AI analyzes media content such as text, images, and videos to obtain information to generate stamps based on the context.

[0344] Step 5:

[0345] The server analyzes the emotion of the message using an emotion engine.

[0346] The server uses an emotion engine to analyze the user's messages, images, and voice data.

[0347] The emotion engine recognizes emotions such as "joy," "sadness," and "anger" from the message content. For example, it analyzes emotions such as "confusion" and "impatience" from the message "What's my homework today?"

[0348] Step 6:

[0349] The server integrates the results of contextual analysis and sentiment analysis.

[0350] The server integrates the analysis results from the context analysis AI and the emotion analysis results from the emotion engine.

[0351] This prepares the data to be fed to generative AI, taking into account both context and sentiment.

[0352] Step 7:

[0353] The server uses generation AI to generate the optimal stamp.

[0354] The server inputs the integrated analysis results into the generation AI.

[0355] The AI ​​then uses this data to generate the most suitable stamps, such as a stamp of someone struggling with their homework.

[0356] The generated stamp is temporarily stored in the server's memory.

[0357] Step 8:

[0358] The server sends the generated stamp to the terminal.

[0359] The server transmits a data packet containing the generated stamp to the terminal.

[0360] This data packet may contain multiple stamp candidates.

[0361] Step 9:

[0362] The terminal receives the generated stamp and displays it to the user.

[0363] The terminal receives and analyzes the data packets from the server.

[0364] The generated stamp is updated to be displayed on the user's interface.

[0365] Step 10:

[0366] The user selects and sends a stamp.

[0367] The user selects an appropriate stamp from the displayed stamps.

[0368] The user presses the send button to send the selected stamp.

[0369] Step 11:

[0370] The terminal transmits the selected stamp to the other chat participants.

[0371] The terminal again transmits the message including the stamp selected by the user.

[0372] The sent stamps are distributed to other chat participants and can be used within the flow of the conversation.

[0373] ---

[0374] As mentioned above, the processing of this system has been explained in detail for each step. By showing the specific operation at each step, the operation of the entire system can be clearly understood.

[0375] Example 2

[0376] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0377] In modern communication, the ability for users to quickly select appropriate stickers based on emotions and context is essential for smooth conversations and improved mutual understanding. However, current chat apps require users to manually select appropriate stickers, and it is difficult to find stickers that accurately reflect emotions and context. This not only impairs the user experience but also reduces the effectiveness of communication. To solve this problem, a system is needed that can analyze the user's message content and emotions in real time and automatically provide appropriate stickers.

[0378] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0379] In this invention, the server includes means for analyzing recent interactions using a context analysis engine, means for analyzing user emotions using an emotion analysis engine, and means for generating optimal stamps using a generation engine. As a result, when a user sends a message, a stamp that best suits the context and emotion is automatically generated in real time, allowing the user to quickly obtain an appropriate stamp, facilitating communication and improving the user experience. "User" refers to an entity that sends a message using the system.

[0380] "Terminal" refers to an electronic device that a user uses to send and receive messages.

[0381] "Server" means a central control device for processing, storing, and transmitting data.

[0382] A "contextual analysis engine" refers to a program or algorithm that analyzes the content of messages sent by users and understands their context.

[0383] "Sentiment analysis engine" refers to a program or algorithm that analyzes a user's messages and other data to identify the user's emotions.

[0384] "Generation engine" refers to the program or algorithm for generating optimal stamps based on the results of the context analysis engine and sentiment analysis engine.

[0385] A "stamp" refers to a visual medium such as an image, text, or video that a user attaches to a message and sends it.

[0386] "Message" refers to a means of communication such as text, images, or videos sent by a user.

[0387] "Recent interactions" refers to the history of recent messages and communications sent by a user.

[0388] The present invention relates to a system that automatically generates and provides stamps that are optimal for the context and emotion of a message sent by a user using a chat app. Detailed embodiments of the system are described below.

[0389] Overall system overview

[0390] This system consists of a user's device, a server, a context analysis engine, a sentiment analysis engine, and a generation engine. When a user sends a message, the message and the most recent exchanges are sent from the device to the server. The server analyzes the content of the sent message using the context analysis engine and analyzes the user's sentiment using the sentiment analysis engine. Based on the analysis results, the generation engine generates the most appropriate stamp. The generated stamp is sent from the server to the device, allowing the user to select and send the appropriate stamp.

[0391] Hardware and software used

[0392] 1. Device: Refers to the electronic device used by the user, such as a smartphone or computer, on which the chat app is installed.

[0393] 2. Server: A remote machine that receives, analyzes, generates, and sends data, for example, a cloud-based server.

[0394] 3. Contextual analysis engine: An AI model for analyzing the content of messages sent by users, with GPT-3 being a common example.

[0395] 4. Sentiment analysis engine: Software that analyzes the content of users' messages to identify their emotions. One example is IBM Watson Tone Analyzer.

[0396] 5. Generation engine: An AI model that generates optimal stamps based on the results of the context analysis engine and sentiment analysis engine. Generative AI such as DALL-E is used.

[0397] Data processing and calculation

[0398] 1. Text data analysis: The contextual analysis engine analyzes the messages sent by users and understands the context.

[0399] 2. Sentiment Analysis: The sentiment analysis engine analyzes the message content and identifies the user's emotions.

[0400] 3. Stamp generation: The generation engine generates optimal stamps based on the results of contextual and sentiment analysis.

[0401] Specific examples

[0402] Example 1: User A sends the message "What's my homework today?"

[0403] 1. User A types "What's today's homework?" in the text field of the chat app and presses the send button.

[0404] 2. User A's device retrieves the most recent 10 message history and sends it to the server along with the message to be sent.

[0405] 3. The server inputs the received data into a context analysis engine (GPT-3) and identifies that the message "What's my homework today?" is a homework inquiry.

[0406] 4. The server uses an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to identify the user's emotion as "worried" from the message content.

[0407] 5. The server inputs the prompt into the generation engine (e.g., DALL-E) and generates the best stamp. The following is used as an example prompt:

[0408] A user sends a message saying, "What's my homework today?" Generate a stamp based on this context and sentiment.

[0409] 6. The generated stamp of "an expression of struggling with homework" is sent from the server to the device.

[0410] 7. The device receives the stamp and displays it on the user interface.

[0411] 8. User A selects the appropriate stamp from the displayed ones and presses the send button.

[0412] As described above, the present invention is a system that realizes smoother and faster communication by automatically generating optimal stamps according to the message content and emotions of the user.

[0413] The flow of the identification process in the second embodiment will be described with reference to FIG. 13. Step 1:

[0414] The user sends a message. The user enters a message into the text input field of the chat app and presses the "Send" button. The message text entered is the input for this step, and this text is temporarily saved on the device.

[0415] Step 2:

[0416] The device retrieves the most recent conversations and sends them to the server. The device retrieves the last 10 messages from the chat history stored internally. The retrieved message history is input, and a data packet containing it is sent to the server. The device converts the packet to JSON format and sends it to the server as an HTTP POST request.

[0417] Step 3:

[0418] The server receives the most recent exchange. The server receives the HTTP POST request and temporarily stores the data packet in memory. The received data packet is the input and is prepared for the next analysis step.

[0419] Step 4:

[0420] The server uses a contextual analysis engine to analyze recent interactions. The server inputs data packets into the contextual analysis engine to understand the content of the message. The input is the last 10 messages and the latest message, and the output is the context of the message. The contextual analysis engine (e.g., GPT-3) identifies that the message "What's my homework today?" is a question about homework.

[0421] Step 5:

[0422] The server uses a sentiment analysis engine to analyze the user's emotions. The server inputs the message content into the sentiment analysis engine and performs a process to identify the user's emotions. The input is the user's message content, and the output is emotional information. The sentiment analysis engine (e.g., IBM Watson Tone Analyzer) identifies the user's emotion as "distressed" from the message content.

[0423] Step 6:

[0424] The server uses a generation AI to generate the optimal stamp. Based on the results of the context analysis engine and emotion analysis engine, the server inputs a prompt sentence into the generation AI. The input is contextual information and emotion information, and the output is a stamp image. The generation AI (e.g., DALL-E) generates a stamp of "an expression of someone struggling with homework" based on the prompt sentence. An example of a prompt sentence is as follows:

[0425] A user sends a message saying, "What's my homework today?" Generate a stamp based on this context and sentiment.

[0426] Step 7:

[0427] The server sends the generated stamp to the device. The server saves the generated stamp as an image file and creates a data packet containing it in JSON format. The input is the generated stamp image, and the output is a JSON-formatted data packet. The server sends the data packet to the device as an HTTP POST request.

[0428] Step 8:

[0429] The terminal receives the generated stamp and displays it to the user. The terminal receives the data packet from the server and parses the JSON data. The input is a JSON formatted data packet and the output is a stamp image. The display is updated so that the parsed stamp image is displayed in the user interface.

[0430] Step 9:

[0431] The user selects a stamp and sends it. The user selects the appropriate stamp from the displayed stamps and presses the "Send" button. The selected stamp image is the input, and the operation to send it is performed.

[0432] Step 10:

[0433] The terminal sends the selected stamp to other chat participants. The terminal converts the message containing the selected stamp into a packet and sends it to other chat participants as an HTTP POST request. The input is the selected stamp image, and the output is the packetized message. Other participants receive and display this as well.

[0434] (Application example 2)

[0435] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0436] Conventional chat applications require users to take the time to select appropriate stamps for their messages, and there is no system that can analyze customer emotions in real time and support customer service. This makes it difficult to communicate quickly and appropriately. Furthermore, in customer service, it is difficult for staff to intuitively grasp customer emotions and respond accordingly, making it difficult to improve customer satisfaction.

[0437] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0438] In this invention, the server includes a customer service analysis device that analyzes a customer's facial expression and generates emotion data, a means that analyzes the emotion data and generates an optimal response method, and a means that provides the generated response method to the customer. This makes it possible to analyze a customer's emotions in real time and provide optimal stamps and response methods. This allows users to quickly select and send appropriate stamps, and allows customer service staff to respond according to the customer's emotions, thereby facilitating communication and improving customer satisfaction.

[0439] A "terminal" is a device that allows a user to send messages and retrieve recent interactions and send them to a server.

[0440] The "server" is a device that receives data sent from a terminal, generates optimal stamps and responses using context analysis AI and an emotion engine, and provides them back to the terminal.

[0441] "Contextual AI" is AI that has the ability to analyze a user's messages and recent interactions and generate appropriate stamps and responses based on the context.

[0442] "Emotion data" is information that represents the emotions of a customer, generated by the customer service analysis device through analysis of the customer's facial expressions and comments.

[0443] A "customer service analysis device" is a device that captures and analyzes customers' facial expressions and behavior in real time to generate emotional data.

[0444] The "optimal response method" is generated based on the emotion data stored in the server and is a method that allows the staff to respond appropriately to the customer.

[0445] "Stamps" are visual icons or images that are added to users' messages and are automatically generated based on context and emotion.

[0446] "Smart glasses" are wearable devices that have the ability to capture customers' facial expressions and behavior and display the analysis results in real time.

[0447] The present invention relates to a system that analyzes the context and emotions of messages sent by users and automatically generates and provides optimal stamps or responses. This system is primarily composed of a user terminal, a server, context analysis AI, an emotion engine, generation AI, and a customer service analysis device. A detailed description of how this system can be implemented is provided below.

[0448] Overall system overview

[0449] In this system, when a user sends a message, that message and the most recent exchange are sent from the device to a server. At the same time, a customer service analysis device analyzes the customer's facial expressions and comments in real time and sends emotional data to the server. The server then uses context analysis AI and an emotion engine to generate optimal stamps and responses. Based on this, the generation AI creates optimal output, which is then sent back from the server to the user's device and the customer service staff's smart glasses.

[0450] Hardware and Software Configuration

[0451] The main hardware and software used to realize this system are as follows:

[0452] Hardware

[0453] Device: User's device (smartphone, tablet, etc.)

[0454] Server: Server for data processing and analysis

[0455] Customer service analysis devices: Wearable devices such as smart glasses

[0456] Camera: A device for capturing customer facial expressions

[0457] software

[0458] Context AI: AI that analyzes the context of messages

[0459] Emotion Engine: An engine that analyzes emotions from customers' facial expressions and comments

[0460] Stamp Generator: Artificial intelligence that generates optimal stamps and responses

[0461] System operation and data processing

[0462] Sending messages and retrieving data

[0463] When a user sends a message on a chat app, the device retrieves the most recent message from its built-in chat history and sends it to the server. The server temporarily stores it for analysis. At the same time, the customer service analysis device captures the customer's facial expressions and comments in real time and sends the data to the server.

[0464] Contextual and sentiment analysis

[0465] On the server, the context analysis AI analyzes the context of the received message. The emotion engine analyzes the customer's emotions based on the data obtained from the customer service analysis device. Based on the results of this analysis, the generation AI generates the optimal stamp and response method.

[0466] Generate optimal stamps and responses

[0467] The generation AI generates multiple sticker candidates and response methods based on the results of context analysis AI and the emotion engine. The generated stickers are displayed as text messages, image messages, or video messages.

[0468] Output and Serving

[0469] The generated stamps and responses are provided from the server to the device or smart glasses. The user can select and send the appropriate stamp. The customer service staff responds to the customer based on the optimal response displayed on the smart glasses.

[0470] Examples of concrete examples and prompts

[0471] Example 1: If a customer says, "What product do you recommend today?" and their facial expression shows interest, the system will generate a stamp of an "interested expression" and display it on the smart glasses of the customer service staff. The customer service staff will then use this information to explain the product.

[0472] Example 2: If a user sends a message in chat saying, "What should I do today?" and the context analysis AI identifies the emotion of "confused," it will generate and provide a stamp of a "confused face." The user can select this stamp and resend it, enabling smooth communication.

[0473] Example prompt sentence:

[0474] Customer sentiment: Intrigued

[0475] Customer Says: What products would you recommend today?

[0476] Content of the stamp to generate: Generate stamps that show expressions that will interest your customers.

[0477] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0478] Step 1:

[0479] A user sends a message in a chat app. For example, the user types "What should I do today?" and presses the send button. This message is saved on the device. At the same time, this message is temporarily saved along with the most recent chat history.

[0480] Step 2:

[0481] The device retrieves the most recent conversations and sends them to the server. Specifically, the device retrieves the last 10 messages from the chat history and sends them to the server as a data packet, including the most recent message sent by the user.

[0482] Step 3:

[0483] The customer service analysis device captures the customer's facial expressions in real time and sends the data to a server. For example, smart glasses could capture a customer's face with a camera and send the video data to the analysis device, which then processes the data as emotion data.

[0484] Step 4:

[0485] The server receives the data and prepares it for analysis. The server temporarily stores the message history sent from the device and the facial expression data sent from the customer service analysis device in its memory. This data is used in the subsequent analysis process.

[0486] Step 5:

[0487] The server uses context analysis AI to analyze the context of the sent message. Specifically, the server inputs the message history into the context analysis AI and analyzes it to understand the content and context of the message. The analysis result is generated as output and handed over to the next process.

[0488] Step 6:

[0489] The server uses an emotion engine to analyze the facial expression data and generate emotion data. For example, the emotion engine identifies the customer's emotion, such as "curious" or "troubled," based on the video data sent from the customer service analysis device. This emotion data is used in the next process.

[0490] Step 7:

[0491] Based on the analysis results of the context analysis AI and emotion engine, the generation AI generates optimal stickers and responses. Specifically, the generation AI receives context and emotion data as input, and creates appropriate stickers and responses based on that. For example, it might generate a "sticker with an interesting expression" or a "specific product description." This output is temporarily stored in the server's memory.

[0492] Step 8:

[0493] The server transmits the generated stamps and responses to the terminal or smart glasses. Specifically, the server transmits the generated output as a data packet to the terminal or customer service analysis device. This data packet may contain multiple stamp candidates and responses.

[0494] Step 9:

[0495] The terminal receives the generated stamps and displays them to the user. Specifically, the terminal receives the data packets from the server, analyzes them, and displays them on the user interface. The user can select an appropriate stamp from the displayed stamps.

[0496] Step 10:

[0497] The user selects a stamp and sends it. The user selects the appropriate stamp from the displayed stamps and presses the send button again. For example, by selecting and sending the "confused face" stamp, the message will be sent to the other party.

[0498] Step 11:

[0499] The customer service staff will respond appropriately to the customer based on the optimal response method displayed on the smart glasses. For example, the smart glasses will display "specific product descriptions that will catch the customer's attention" and provide detailed explanations to the customer based on that information. This response will improve customer satisfaction.

[0500] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0501] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0502] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0503] [Second embodiment]

[0504] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0505] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0506] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0507] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0508] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0509] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0510] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0511] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0512] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0513] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0514] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0515] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0516] ---

[0517] The present invention relates to a system that automatically generates and provides stamps that are optimal for the context of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[0518] Overall system overview

[0519] This system mainly consists of a user's device, a server, and a generation AI. When a user sends a message, the message and data on the most recent exchange are sent from the device to the server. The server analyzes the received data and generates the most appropriate stamp using context analysis AI. The generated stamp is then sent back from the server to the user's device, where the user can select the appropriate one to send.

[0520] System operation explanation

[0521] 1. A user sends a message

[0522] A user types a message into a text input field in a chat app and presses the send button. For example, if a user sends a message like "What's my homework today?", the message is saved on the device.

[0523] 2. The device retrieves the most recent interaction and sends it to the server

[0524] The device retrieves the most recent messages, images, videos, etc. from the chat history it stores internally. For example, it retrieves the last 10 messages and sends them to the server, including the message that was just sent.

[0525] 3. The server uses contextual analysis AI to generate the optimal stamp

[0526] The server analyzes the most recent interactions received from the device and uses contextual analysis AI to generate stamps that best fit the context. For example, a stamp of "an expression of someone struggling with homework" might be generated.

[0527] 4. Provide the generated stamp to the user

[0528] The server sends the stamps generated by the AI ​​to the device and displays them to the user, who can then choose the best one from the multiple stamps generated.

[0529] 5. The user selects a stamp and sends it.

[0530] Next, the user selects one of the displayed stamps and presses the send button. For example, the user selects the stamp with the expression "worried about homework" and sends it.

[0531] Specific examples

[0532] A specific example of this system is shown below.

[0533] 1. User A sends a message

[0534] User A types "What's today's homework?" into a chat app and sends it.

[0535] 2. The device retrieves the most recent conversation and sends it to the server

[0536] The device retrieves the most recent 10 message history records and sends them to the server, including the message to be sent.

[0537] 3. The server generates stamps using context analysis AI

[0538] Context analysis AI generates a stamp that best fits the context of "What's today's homework?", depicting an "expression of someone struggling with homework."

[0539] 4. Provide the generated stamp to the user

[0540] The server returns the generated stamp to the terminal, which displays it to User A.

[0541] 5. User A selects a stamp and sends it

[0542] User A selects the stamp with the expression "looking worried about homework" and presses the send button.

[0543] This allows user A to quickly select an appropriate stamp and continue communication smoothly.

[0544] ---

[0545] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context of a message sent by a user. This allows users to quickly and easily obtain appropriate stamps, enabling efficient communication.

[0546] The processing flow will be explained below.

[0547] ---

[0548] Step 1:

[0549] The user enters a message and presses the send button.

[0550] A user types a message into a text input field in a chat app, for example, "What's my homework for today?", and presses the send button.

[0551] This message will be saved in the chat history on the device.

[0552] Step 2:

[0553] The device retrieves the most recent interactions and sends them to the server.

[0554] Obtain chat history stored internally on the device. For example, obtain the last 10 messages, images, and videos.

[0555] This data is sent as packets to the server as the "most recent exchange."

[0556] Step 3:

[0557] The server receives the most recent exchange.

[0558] The server receives the data packet sent from the terminal and prepares to analyze it.

[0559] The server temporarily stores this data in memory.

[0560] Step 4:

[0561] The server uses contextual analysis AI to analyze recent interactions.

[0562] The server inputs the received "most recent interaction" into a context analysis AI and analyzes it to generate the most appropriate stamp.

[0563] Contextual analysis AI understands the content of each media, such as text, images, and videos, and generates stamps based on that context.

[0564] Step 5:

[0565] The server uses generation AI to generate the optimal stamp.

[0566] Based on the analysis results, the AI ​​generates the most suitable stamp. For example, it can generate a stamp of someone struggling with their homework.

[0567] The generated stamp is temporarily stored in the server's memory.

[0568] Step 6:

[0569] The server sends the generated stamp to the terminal.

[0570] The server transmits the generated stamp to the terminal as a data packet.

[0571] This data packet may contain multiple stamp candidates.

[0572] Step 7:

[0573] The terminal receives the generated stamp and displays it to the user.

[0574] The terminal receives and analyzes the data packets from the server.

[0575] The interface is updated so that the generated stamp is visually displayed to the user.

[0576] Step 8:

[0577] The user selects and sends a stamp.

[0578] The user selects an appropriate stamp from the displayed ones and presses the send button.

[0579] For example, a user selects and sends a stamp with an expression of "an expression of someone struggling with homework."

[0580] Step 9:

[0581] The terminal transmits the selected stamp to the other chat participants.

[0582] The terminal retransmits the message including the stamp selected by the user and distributes it to the other chat participants.

[0583] The stamps selected in this way are sent in the flow of the conversation.

[0584] ---

[0585] As mentioned above, the processing steps of this system have been explained in detail. By showing the specific operations at each step, the operation of the entire system can be clearly understood.

[0586] Example 1

[0587] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0588] In recent years, users often need to select appropriate stamps or graphics when sending messages, but the selection process can be complex and time-consuming. In particular, to find a stamp that fits the context, users must search through a large number of options, which can disrupt communication. This creates a poor user experience and hinders efficient communication.

[0589] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0590] In this invention, the server includes means for analyzing a context using a natural language processing model and generating an optimal graphic, means for displaying multiple candidate graphics, and means for providing the generated graphic. This allows a user to quickly and appropriately obtain a graphic that is optimal for the context when sending a message, enabling efficient communication.

[0591] 1. "User" means any entity that operates a communications system to send or receive information.

[0592] 2. "Information" means any message or content expressed in digital form, such as text data, image data, or video data.

[0593] 3. "Terminal" means a device used by a user to input, send, or receive information. Examples include smartphones, tablets, and personal computers.

[0594] 4. "Communication Content" means the history and current messages of information sent and received by a Device.

[0595] 5. "Server" means a computer system that processes and provides information in cooperation with terminals via a communications network.

[0596] 6. A "natural language processing model" is a machine learning model used to analyze text data and understand its meaning and context. Examples include GPT-3 and BERT.

[0597] 7. "Context" refers to the surrounding or background information needed to interpret the meaning of a message.

[0598] 8. "Graphics" is a general term for visual representations such as text, images, and videos, which are used to express the user's emotions and situations.

[0599] 9. A "prompt" is an instruction or question given to a generative AI model to generate an appropriate output.

[0600] 10. "REST API" means an application programming interface for sending and receiving data in the style of Representational State Transfer.

[0601] 11. "HTTP response" refers to data sent from a server to a terminal, and is sent using the HTTP protocol.

[0602] 12. “SQL query” means a statement used in Structured Query Language to retrieve information from a database.

[0603] The above are definitions of important terms included in the rewritten claims.

[0604] MODE FOR CARRYING OUT THE INVENTION

[0605] The present invention relates to a system that automatically generates and provides stamps and graphics that are optimal for the context of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[0606] Overall system overview

[0607] This system is composed of a user's device, a server, and a generative AI model. When a user sends a message, the message and the most recent communication content are sent from the device to the server. The server analyzes the received data and generates the optimal graphic using a natural language processing model. The generated graphic is then provided back to the user's device by the server, and the user can select the appropriate one to send.

[0608] Hardware and software used

[0609] Device: A digital device that a user uses to send and receive data. Examples include smartphones, tablets, and computers.

[0610] Server: A computer system that analyzes the received communications and generates stamps and graphics.

[0611] Natural language processing models: For example, machine learning models such as "GPT-3" and "BERT" are used.

[0612] Generative AI model: A model that works in conjunction with a natural language processing model to generate context-based graphics.

[0613] Communication protocol: REST API and HTTP responses are used to send and receive data.

[0614] System operation explanation

[0615] As a concrete example, we will explain the flow when User A sends a message "What's today's homework?" in a chat app.

[0616] 1. A user sends a message

[0617] User A enters "What's today's homework?" into the text input field of a chat app and presses the send button. This message is temporarily stored on the device.

[0618] 2. The device acquires the most recent communication content and sends it to the server

[0619] The device retrieves the last 10 messages from the internal database using an SQL query, and the retrieved messages, images, and videos are converted to JSON format and sent as a POST request to the server's REST API.

[0620] 3. The server uses contextual analysis AI to generate optimal graphics

[0621] The server passes the received data to a natural language processing model (e.g., GPT-3 or BERT), which analyzes the message "What's today's homework?" and the past 10 interactions as input. Based on the context, an appropriate prompt is input to the generative AI model, which generates a stamp of an "expression of someone struggling with homework."

[0622] 4. Serving the generated graphics to the user's device

[0623] The server generates a URL for the generated graphic and returns it to the user's device as an HTTP response. The device receives this response and dynamically displays the stamp in the chat app's interface.

[0624] 5. The user selects a stamp and sends it.

[0625] User A selects the "face struggling with homework" stamp from the provided stamps and presses the send button again. The URL of this selected stamp is again sent from the device to the server and is finally displayed on the chat partner's screen.

[0626] Prompt Sentence Examples

[0627] Below is an example of a prompt sentence to input to the generative AI model.

[0628] Message Context: What's my homework today?

[0629] Related past messages:

[0630] 1. Homework email from your teacher

[0631] 2. Communicating with classmates about homework

[0632] Generated stamp: An expression of someone struggling with homework

[0633] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context of a message sent by a user. Users can quickly and easily obtain appropriate stamps, enabling efficient communication.

[0634] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0635] Explain the program's processing in detail

[0636] Step 1:

[0637] A user sends a message

[0638] explanation:

[0639] A user enters a message into the text input field of a chat app and presses the send button. This message is temporarily stored on the device.

[0640] Specific operation and input / output:

[0641] Input: A message typed by the user (e.g., "What's my homework today?")

[0642] Action: A message is entered into the text field and the send button is pressed.

[0643] Output: Stored in the message queue in the terminal.

[0644] Step 2:

[0645] The device acquires the most recent communication content and sends it to the server.

[0646] explanation:

[0647] The user's device retrieves the most recent message history from an internal database and sends it to the server.

[0648] Specific operation and input / output:

[0649] Input: Messages sent by the user and past communication history (e.g., last 10 messages)

[0650] Operation:

[0651] Retrieve the last 10 messages from the internal database using an SQL query.

[0652] Convert the retrieved data into JSON format.

[0653] Send the JSON data as a POST request to the server's REST API.

[0654] Output: Message history in JSON format is sent to the server.

[0655] Step 3:

[0656] The server uses contextual analysis AI to generate optimal graphics

[0657] explanation:

[0658] The server analyzes the received data, understands the context using natural language processing models, and generates the most appropriate graphics.

[0659] Specific operation and input / output:

[0660] Input: Recent message history (JSON format data)

[0661] Operation:

[0662] Passing JSON data to a natural language processing model (e.g., GPT-3 or BERT).

[0663] Contextual analysis is performed to generate prompts.

[0664] Pass the prompt to a generative AI model to generate the appropriate graphic.

[0665] Output: Generated optimal graphic (e.g., a stamp of a person struggling with homework)

[0666] Step 4:

[0667] Serving the generated graphics to the user's device

[0668] explanation:

[0669] The server generates a URL for the generated graphic and provides it to the user's terminal.

[0670] Specific operation and input / output:

[0671] Input: Generated graphic

[0672] Operation:

[0673] Save the graphic and generate a URL for it.

[0674] The URL is sent to the user's device as an HTTP response.

[0675] Output: The URL of the graphic is displayed on the user's device.

[0676] Step 5:

[0677] The user selects a stamp and sends it.

[0678] explanation:

[0679] The user selects an appropriate stamp from the ones provided and presses the send button.

[0680] Specific operation and input / output:

[0681] Input: A stamp selected by the user from the displayed graphics (e.g., a stamp of a person struggling with homework)

[0682] Operation:

[0683] The user taps or clicks on the displayed stamp.

[0684] The URL of the selected stamp is sent from the terminal to the server by pressing the send button again.

[0685] Output: The URL of the selected stamp will be displayed on the chat partner's screen.

[0686] The above is the specific flow of the program processing for this system, which allows users to quickly select the graphic that best suits the context and achieve smooth communication.

[0687] (Application example 1)

[0688] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0689] In conventional messaging applications, it is difficult for users to quickly select the most suitable stamp when sending a message. This leads to problems such as communication being disrupted and a poor user experience. In addition, selecting stamps must be done manually, which is inefficient.

[0690] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0691] In this invention, the server includes a means for a user to send a message, a means for a terminal to acquire the most recent exchange and send it to the server, a means for the server to generate optimal stamps using context analysis AI, a means for providing the generated stamps to the user, a means for the user to select and send stamps, a means for the server to send the results of context analysis to the stamp generation microservice, and a means for providing the URL of the stamp acquired from the microservice to the user terminal.This allows the user to automatically generate and receive stamps that are optimal for the context of the message they are sending, thereby facilitating communication and improving the user experience.

[0692] "User" refers to an individual user of the system.

[0693] "Message" refers to text, image or video information sent by a user to communicate with other users.

[0694] "Terminal" refers to a communication device such as a smartphone or tablet used by a user.

[0695] "Recent interactions" refers to the most recent communication history in the system.

[0696] "Server" refers to a computer system that processes and stores data and operates in conjunction with a terminal.

[0697] "Contextual analysis AI" refers to artificial intelligence technology that uses natural language processing to analyze the meaning and intent of text.

[0698] "Optimal stamp" refers to an icon or illustration that best fits the context of the user's message.

[0699] "Stamp generation means" refers to the process of generating stamps using contextual analysis AI.

[0700] A "stamp generation microservice" is a small, independent component that specializes in a specific function and is dedicated to generating stamps.

[0701] "Stamp URL" refers to the web address where the generated stamp is stored.

[0702] MODE FOR CARRYING OUT THE INVENTION

[0703] System Overview

[0704] This invention is a system that automatically generates and provides stamps that are optimal for the context of a message sent by a user. This system mainly consists of a user device, a server, a context analysis AI, and a stamp generation microservice.

[0705] System configuration

[0706] 1. User's device

[0707] The user's device (such as a smartphone or tablet) is a communication device that sends and receives messages. The device's role is to obtain the most recent exchanges and send them to the server.

[0708] 2. Server

[0709] The server receives messages and chat history sent by users, uses contextual analysis AI to generate optimal stamps, and then sends the generated stamps to a microservice, providing the resulting stamp URL to the user's device.

[0710] 3. Context analysis AI

[0711] Contextual analysis AI uses natural language processing to analyze the context of messages, and based on the results of this analysis, provides information to generate the best stamps for the user's message.

[0712] 4. Stamp Generation Microservice

[0713] The stamp generation microservice is a specialized component that generates the optimal stamp URL based on the results of contextual analysis sent from the server.

[0714] Operation explanation

[0715] 1. A user sends a message

[0716] The user types a message and presses the send button. For example, if the message is "I haven't finished my homework," the message is saved on the device.

[0717] 2. The device retrieves the most recent interaction and sends it to the server

[0718] The device retrieves past conversations, including the most recent message, from its internal chat history and sends that data to the server. Specifically, the last 10 messages are the target.

[0719] 3. The server uses contextual analysis AI to generate the optimal stamp

[0720] The server inputs the sent message and chat history into a context analysis AI to analyze the context. For example, for a message such as "I can't finish my homework," the context analysis AI determines that a stamp with a "worried expression" would be appropriate.

[0721] 4. Send the result to the stamp generation microservice and get the stamp URL.

[0722] The server sends the results of the contextual analysis to the stamp generation microservice, which generates a URL for the most suitable stamp, which is returned to the server.

[0723] 5. Provide the generated stamp to the user

[0724] The server sends the stamp URL to the user's device, and the user can view the stamp using this URL. The user can select and send stamps from the provided stamps.

[0725] Specific examples

[0726] A specific example of the system's operation is shown below.

[0727] 1. A user sends a new message

[0728] A user sends a message saying, "I can't finish my homework."

[0729] 2. Your device will connect your chat history with new messages

[0730] The last 10 chat messages and the new message are concatenated and sent to the server.

[0731] 3. Contextual AI generates stamps

[0732] Context analysis AI generates stamps that best fit the context of "I can't finish my homework."

[0733] 4. The stamp generation microservice generates the stamp URL.

[0734] The server sends the analysis results to the stamp generation microservice and obtains the URL of the optimal stamp.

[0735] 5. User checks and sends the provided stamp

[0736] The user selects a provided stamp and presses the send button.

[0737] Prompt Sentence Examples

[0738] The following is an example of a prompt:

[0739] text

[0740] Below is the chat history between User A and User B. Please generate the stamp that best suits User A's latest message.

[0741] Chat history:

[0742] User A: It's raining today.

[0743] User B: Maybe it'll be sunny tomorrow.

[0744] User A: What should I do about my homework?

[0745] User B: I want to hang out with friends.

[0746] User A: I want to go see a movie.

[0747] Latest Message:

[0748] User A: I can't finish my homework.

[0749] In this way, by using the system of the present invention, a user can quickly and easily obtain a stamp that is optimal for the context of a message, thereby achieving smoother and more efficient communication.

[0750] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0751] Step 1:

[0752] The user enters a message and presses the send button. The input is the message text (e.g., "I haven't finished my homework yet."), and this message is saved on the device. The output is the latest message data.

[0753] Step 2:

[0754] The device retrieves past conversations, including the most recent message, from the chat history it stores internally. The input is the most recent message and chat history (e.g., the last 10 messages), which it then sends to the server. The output is the dataset sent to the server.

[0755] Step 3:

[0756] The server inputs the received data set into the context analysis AI and performs context analysis. The input is all message text (e.g., "I can't finish my homework" and past chat history), and based on this, it generates analysis results. The output is the context analysis results.

[0757] Step 4:

[0758] The server sends the result of the contextual analysis to the stamp generation microservice to obtain the optimal stamp URL. The input is the contextual analysis result, and the appropriate stamp URL is generated based on this. The output is the stamp URL.

[0759] Step 5:

[0760] The server sends the generated stamp URL to the user's device and provides it to the user. The input is the stamp URL, which is returned to the user's device. The output is the stamp URL that is displayed to the user.

[0761] Step 6:

[0762] The user selects the appropriate stamp from the provided ones and presses the submit button. The input is the stamp URL, and the output is the submission of the selected stamp.

[0763] In this way, the input data is processed at each step, and ultimately an appropriate stamp is provided to the user.

[0764] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0765] ---

[0766] The present invention relates to a system for automatically generating and providing stamps that are optimal for the context and emotion of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[0767] Overall system overview

[0768] This system mainly consists of the user's device, a server, a context analysis AI, and an emotion engine. When a user sends a message, that message and data on the most recent exchange are sent from the device to the server. At that time, the emotion engine recognizes the user's emotions, and the context analysis AI analyzes the context. The server generates optimal stamps based on the results of this analysis. The generated stamps are then sent back from the server to the user's device, where the user can select the appropriate one to send.

[0769] System operation explanation

[0770] 1. A user sends a message

[0771] A user types a message into a text input field in a chat app and presses the send button. For example, if a user sends a message like "What's my homework today?", the message is saved on the device.

[0772] 2. The device retrieves the most recent interaction and sends it to the server

[0773] The device retrieves the most recent messages, images, videos, etc. from the chat history it stores internally. For example, it retrieves the last 10 messages and sends them to the server, including the message that was just sent.

[0774] 3. The server receives the most recent exchange

[0775] The server receives the data packets sent from the terminal and prepares them for analysis. The server temporarily stores these data in memory.

[0776] 4. The server uses contextual analysis AI to analyze the most recent conversations

[0777] The server inputs the received "recent interactions" into a context analysis AI, which analyzes them to generate optimal stamps. The context analysis AI understands the content of each media, such as text, images, and videos, and generates stamps based on that context.

[0778] 5. Emotion engine analyzes user emotions

[0779] The emotion engine analyzes the user's emotions based on the messages, images, voice data, etc. received by the server from the device. For example, it can identify emotions such as "worried" or "happy" from the content of the user's message or image.

[0780] 6. The server generates the optimal stamp using generation AI

[0781] Based on the analysis results of the context analysis AI and emotion engine, the generation AI generates the optimal stamp. For example, a stamp of an "expression of someone struggling with homework" may be generated. The generated stamp is temporarily stored in the server's memory.

[0782] 7. The server sends the generated stamp to the device.

[0783] The server transmits the generated stamp to the terminal as a data packet, which may contain multiple stamp candidates.

[0784] 8. The device receives the generated stamp and displays it to the user.

[0785] The device receives and analyzes the data packets from the server and updates the interface so that the generated stamps are visually displayed to the user.

[0786] 9. The user selects a stamp and sends it.

[0787] The user selects an appropriate stamp from the displayed ones and presses the send button. For example, the user selects a stamp with an expression of someone struggling with homework and sends it.

[0788] 10. The device sends the selected stamp to other chat participants.

[0789] The terminal then retransmits the message containing the stamp selected by the user and distributes it to the other chat participants, thus transmitting the selected stamp in the flow of the conversation.

[0790] Specific examples

[0791] A specific example of this system is shown below.

[0792] 1. User A sends a message

[0793] User A types "What's today's homework?" into a chat app and sends it.

[0794] 2. The device retrieves the most recent conversation and sends it to the server

[0795] The device retrieves the most recent 10 message history records and sends them to the server, including the message to be sent.

[0796] 3. The server generates stamps using context analysis AI

[0797] Context analysis AI generates a stamp that best fits the context of "What's today's homework?", depicting an "expression of someone struggling with homework."

[0798] 4. Provide the generated stamp to the user

[0799] The server returns the generated stamp to the terminal, which displays it to User A.

[0800] 5. User A selects a stamp and sends it

[0801] User A selects the stamp with the expression "looking worried about homework" and presses the send button.

[0802] This allows user A to quickly select an appropriate stamp and continue communication smoothly.

[0803] ---

[0804] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context and emotion of a message sent by a user. This allows users to quickly and easily obtain appropriate stamps, enabling efficient communication.

[0805] The processing flow will be explained below.

[0806] ---

[0807] Step 1:

[0808] The user enters a message and presses the send button.

[0809] A user types a message into a text input field in a chat app, for example, "What's my homework for today?", and presses the send button.

[0810] This message will be saved in your device's chat history.

[0811] Step 2:

[0812] The device retrieves the most recent interactions and sends them to the server.

[0813] Data such as the last 10 messages, images, and videos is retrieved from the chat history stored internally on the device.

[0814] These "recent interactions" are packaged into data packets and sent to the server.

[0815] Step 3:

[0816] The server receives the most recent exchange.

[0817] The server receives the data packets sent from the terminal.

[0818] The received data is temporarily stored in memory and prepared for analysis.

[0819] Step 4:

[0820] The server uses contextual analysis AI to analyze recent interactions.

[0821] The server inputs the "most recent interaction" received into a context analysis AI.

[0822] Contextual analysis AI analyzes media content such as text, images, and videos to obtain information to generate stamps based on the context.

[0823] Step 5:

[0824] The server analyzes the emotion of the message using an emotion engine.

[0825] The server uses an emotion engine to analyze the user's messages, images, and voice data.

[0826] The emotion engine recognizes emotions such as "joy," "sadness," and "anger" from the message content. For example, it analyzes emotions such as "confusion" and "impatience" from the message "What's my homework today?"

[0827] Step 6:

[0828] The server integrates the results of contextual analysis and sentiment analysis.

[0829] The server integrates the analysis results from the context analysis AI and the emotion analysis results from the emotion engine.

[0830] This prepares the data to be fed to generative AI, taking into account both context and sentiment.

[0831] Step 7:

[0832] The server uses generation AI to generate the optimal stamp.

[0833] The server inputs the integrated analysis results into the generation AI.

[0834] The AI ​​then uses this data to generate the most suitable stamps, such as a stamp of someone struggling with their homework.

[0835] The generated stamp is temporarily stored in the server's memory.

[0836] Step 8:

[0837] The server sends the generated stamp to the terminal.

[0838] The server transmits a data packet containing the generated stamp to the terminal.

[0839] This data packet may contain multiple stamp candidates.

[0840] Step 9:

[0841] The terminal receives the generated stamp and displays it to the user.

[0842] The terminal receives and analyzes the data packets from the server.

[0843] The generated stamp is updated to be displayed on the user's interface.

[0844] Step 10:

[0845] The user selects and sends a stamp.

[0846] The user selects an appropriate stamp from the displayed stamps.

[0847] The user presses the send button to send the selected stamp.

[0848] Step 11:

[0849] The terminal transmits the selected stamp to the other chat participants.

[0850] The terminal again transmits the message including the stamp selected by the user.

[0851] The sent stamps are distributed to other chat participants and can be used within the flow of the conversation.

[0852] ---

[0853] As mentioned above, the processing of this system has been explained in detail for each step. By showing the specific operation at each step, the operation of the entire system can be clearly understood.

[0854] Example 2

[0855] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0856] In modern communication, the ability for users to quickly select appropriate stickers based on emotions and context is essential for smooth conversations and improved mutual understanding. However, current chat apps require users to manually select appropriate stickers, and it is difficult to find stickers that accurately reflect emotions and context. This not only impairs the user experience but also reduces the effectiveness of communication. To solve this problem, a system is needed that can analyze the user's message content and emotions in real time and automatically provide appropriate stickers.

[0857] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0858] In this invention, the server includes means for analyzing recent interactions using a context analysis engine, means for analyzing user emotions using an emotion analysis engine, and means for generating optimal stamps using a generation engine. As a result, when a user sends a message, a stamp that best suits the context and emotion is automatically generated in real time, allowing the user to quickly obtain an appropriate stamp, facilitating communication and improving the user experience. "User" refers to an entity that sends a message using the system.

[0859] "Terminal" refers to an electronic device that a user uses to send and receive messages.

[0860] "Server" means a central control device for processing, storing, and transmitting data.

[0861] A "contextual analysis engine" refers to a program or algorithm that analyzes the content of messages sent by users and understands their context.

[0862] "Sentiment analysis engine" refers to a program or algorithm that analyzes a user's messages and other data to identify the user's emotions.

[0863] "Generation engine" refers to the program or algorithm for generating optimal stamps based on the results of the context analysis engine and sentiment analysis engine.

[0864] A "stamp" refers to a visual medium such as an image, text, or video that a user attaches to a message and sends it.

[0865] "Message" refers to a means of communication such as text, images, or videos sent by a user.

[0866] "Recent interactions" refers to the history of recent messages and communications sent by a user.

[0867] The present invention relates to a system that automatically generates and provides stamps that are optimal for the context and emotion of a message sent by a user using a chat app. Detailed embodiments of the system are described below.

[0868] Overall system overview

[0869] This system consists of a user's device, a server, a context analysis engine, a sentiment analysis engine, and a generation engine. When a user sends a message, the message and the most recent exchanges are sent from the device to the server. The server analyzes the content of the sent message using the context analysis engine and analyzes the user's sentiment using the sentiment analysis engine. Based on the analysis results, the generation engine generates the most appropriate stamp. The generated stamp is sent from the server to the device, allowing the user to select and send the appropriate stamp.

[0870] Hardware and software used

[0871] 1. Device: Refers to the electronic device used by the user, such as a smartphone or computer, on which the chat app is installed.

[0872] 2. Server: A remote machine that receives, analyzes, generates, and sends data, for example, a cloud-based server.

[0873] 3. Contextual analysis engine: An AI model for analyzing the content of messages sent by users, with GPT-3 being a common example.

[0874] 4. Sentiment analysis engine: Software that analyzes the content of users' messages to identify their emotions. One example is IBM Watson Tone Analyzer.

[0875] 5. Generation engine: An AI model that generates optimal stamps based on the results of the context analysis engine and sentiment analysis engine. Generative AI such as DALL-E is used.

[0876] Data processing and calculation

[0877] 1. Text data analysis: The contextual analysis engine analyzes the messages sent by users and understands the context.

[0878] 2. Sentiment Analysis: The sentiment analysis engine analyzes the message content and identifies the user's emotions.

[0879] 3. Stamp generation: The generation engine generates optimal stamps based on the results of contextual and sentiment analysis.

[0880] Specific examples

[0881] Example 1: User A sends the message "What's my homework today?"

[0882] 1. User A types "What's today's homework?" in the text field of the chat app and presses the send button.

[0883] 2. User A's device retrieves the most recent 10 message history and sends it to the server along with the message to be sent.

[0884] 3. The server inputs the received data into a context analysis engine (GPT-3) and identifies that the message "What's my homework today?" is a homework inquiry.

[0885] 4. The server uses an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to identify the user's emotion as "worried" from the message content.

[0886] 5. The server inputs the prompt into the generation engine (e.g., DALL-E) and generates the best stamp. The following is used as an example prompt:

[0887] A user sends a message saying, "What's my homework today?" Generate a stamp based on this context and sentiment.

[0888] 6. The generated stamp of "an expression of struggling with homework" is sent from the server to the device.

[0889] 7. The device receives the stamp and displays it on the user interface.

[0890] 8. User A selects the appropriate stamp from the displayed ones and presses the send button.

[0891] As described above, the present invention is a system that realizes smoother and faster communication by automatically generating optimal stamps according to the message content and emotions of the user.

[0892] The flow of the identification process in the second embodiment will be described with reference to FIG. 13. Step 1:

[0893] The user sends a message. The user enters a message into the text input field of the chat app and presses the "Send" button. The message text entered is the input for this step, and this text is temporarily saved on the device.

[0894] Step 2:

[0895] The device retrieves the most recent conversations and sends them to the server. The device retrieves the last 10 messages from the chat history stored internally. The retrieved message history is input, and a data packet containing it is sent to the server. The device converts the packet to JSON format and sends it to the server as an HTTP POST request.

[0896] Step 3:

[0897] The server receives the most recent exchange. The server receives the HTTP POST request and temporarily stores the data packet in memory. The received data packet is the input and is prepared for the next analysis step.

[0898] Step 4:

[0899] The server uses a contextual analysis engine to analyze recent interactions. The server inputs data packets into the contextual analysis engine to understand the content of the message. The input is the last 10 messages and the latest message, and the output is the context of the message. The contextual analysis engine (e.g., GPT-3) identifies that the message "What's my homework today?" is a question about homework.

[0900] Step 5:

[0901] The server uses a sentiment analysis engine to analyze the user's emotions. The server inputs the message content into the sentiment analysis engine and performs a process to identify the user's emotions. The input is the user's message content, and the output is emotional information. The sentiment analysis engine (e.g., IBM Watson Tone Analyzer) identifies the user's emotion as "distressed" from the message content.

[0902] Step 6:

[0903] The server uses a generation AI to generate the optimal stamp. Based on the results of the context analysis engine and emotion analysis engine, the server inputs a prompt sentence into the generation AI. The input is contextual information and emotion information, and the output is a stamp image. The generation AI (e.g., DALL-E) generates a stamp of "an expression of someone struggling with homework" based on the prompt sentence. An example of a prompt sentence is as follows:

[0904] A user sends a message saying, "What's my homework today?" Generate a stamp based on this context and sentiment.

[0905] Step 7:

[0906] The server sends the generated stamp to the device. The server saves the generated stamp as an image file and creates a data packet containing it in JSON format. The input is the generated stamp image, and the output is a JSON-formatted data packet. The server sends the data packet to the device as an HTTP POST request.

[0907] Step 8:

[0908] The terminal receives the generated stamp and displays it to the user. The terminal receives the data packet from the server and parses the JSON data. The input is a JSON formatted data packet and the output is a stamp image. The display is updated so that the parsed stamp image is displayed in the user interface.

[0909] Step 9:

[0910] The user selects a stamp and sends it. The user selects the appropriate stamp from the displayed stamps and presses the "Send" button. The selected stamp image is the input, and the operation to send it is performed.

[0911] Step 10:

[0912] The terminal sends the selected stamp to other chat participants. The terminal converts the message containing the selected stamp into a packet and sends it to other chat participants as an HTTP POST request. The input is the selected stamp image, and the output is the packetized message. Other participants receive and display this as well.

[0913] (Application example 2)

[0914] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0915] Conventional chat applications require users to take the time to select appropriate stamps for their messages, and there is no system that can analyze customer emotions in real time and support customer service. This makes it difficult to communicate quickly and appropriately. Furthermore, in customer service, it is difficult for staff to intuitively grasp customer emotions and respond accordingly, making it difficult to improve customer satisfaction.

[0916] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0917] In this invention, the server includes a customer service analysis device that analyzes a customer's facial expression and generates emotion data, a means that analyzes the emotion data and generates an optimal response method, and a means that provides the generated response method to the customer. This makes it possible to analyze a customer's emotions in real time and provide optimal stamps and response methods. This allows users to quickly select and send appropriate stamps, and allows customer service staff to respond according to the customer's emotions, thereby facilitating communication and improving customer satisfaction.

[0918] A "terminal" is a device that allows a user to send messages and retrieve recent interactions and send them to a server.

[0919] The "server" is a device that receives data sent from a terminal, generates optimal stamps and responses using context analysis AI and an emotion engine, and provides them back to the terminal.

[0920] "Contextual AI" is AI that has the ability to analyze a user's messages and recent interactions and generate appropriate stamps and responses based on the context.

[0921] "Emotion data" is information that represents the emotions of a customer, generated by the customer service analysis device through analysis of the customer's facial expressions and comments.

[0922] A "customer service analysis device" is a device that captures and analyzes customers' facial expressions and behavior in real time to generate emotional data.

[0923] The "optimal response method" is generated based on the emotion data stored in the server and is a method that allows the staff to respond appropriately to the customer.

[0924] "Stamps" are visual icons or images that are added to users' messages and are automatically generated based on context and emotion.

[0925] "Smart glasses" are wearable devices that have the ability to capture customers' facial expressions and behavior and display the analysis results in real time.

[0926] The present invention relates to a system that analyzes the context and emotions of messages sent by users and automatically generates and provides optimal stamps or responses. This system is primarily composed of a user terminal, a server, context analysis AI, an emotion engine, generation AI, and a customer service analysis device. A detailed description of how this system can be implemented is provided below.

[0927] Overall system overview

[0928] In this system, when a user sends a message, that message and the most recent exchange are sent from the device to a server. At the same time, a customer service analysis device analyzes the customer's facial expressions and comments in real time and sends emotional data to the server. The server then uses context analysis AI and an emotion engine to generate optimal stamps and responses. Based on this, the generation AI creates optimal output, which is then sent back from the server to the user's device and the customer service staff's smart glasses.

[0929] Hardware and Software Configuration

[0930] The main hardware and software used to realize this system are as follows:

[0931] Hardware

[0932] Device: User's device (smartphone, tablet, etc.)

[0933] Server: Server for data processing and analysis

[0934] Customer service analysis devices: Wearable devices such as smart glasses

[0935] Camera: A device for capturing customer facial expressions

[0936] software

[0937] Context AI: AI that analyzes the context of messages

[0938] Emotion Engine: An engine that analyzes emotions from customers' facial expressions and comments

[0939] Stamp Generator: Artificial intelligence that generates optimal stamps and responses

[0940] System operation and data processing

[0941] Sending messages and retrieving data

[0942] When a user sends a message on a chat app, the device retrieves the most recent message from its built-in chat history and sends it to the server. The server temporarily stores it for analysis. At the same time, the customer service analysis device captures the customer's facial expressions and comments in real time and sends the data to the server.

[0943] Contextual and sentiment analysis

[0944] On the server, the context analysis AI analyzes the context of the received message. The emotion engine analyzes the customer's emotions based on the data obtained from the customer service analysis device. Based on the results of this analysis, the generation AI generates the optimal stamp and response method.

[0945] Generate optimal stamps and responses

[0946] The generation AI generates multiple sticker candidates and response methods based on the results of context analysis AI and the emotion engine. The generated stickers are displayed as text messages, image messages, or video messages.

[0947] Output and Serving

[0948] The generated stamps and responses are provided from the server to the device or smart glasses. The user can select and send the appropriate stamp. The customer service staff responds to the customer based on the optimal response displayed on the smart glasses.

[0949] Examples of concrete examples and prompts

[0950] Example 1: If a customer says, "What product do you recommend today?" and their facial expression shows interest, the system will generate a stamp of an "interested expression" and display it on the smart glasses of the customer service staff. The customer service staff will then use this information to explain the product.

[0951] Example 2: If a user sends a message in chat saying, "What should I do today?" and the context analysis AI identifies the emotion of "confused," it will generate and provide a stamp of a "confused face." The user can select this stamp and resend it, enabling smooth communication.

[0952] Example prompt sentence:

[0953] Customer sentiment: Intrigued

[0954] Customer Says: What products would you recommend today?

[0955] Content of the stamp to generate: Generate stamps that show expressions that will interest your customers.

[0956] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0957] Step 1:

[0958] A user sends a message in a chat app. For example, the user types "What should I do today?" and presses the send button. This message is saved on the device. At the same time, this message is temporarily saved along with the most recent chat history.

[0959] Step 2:

[0960] The device retrieves the most recent conversations and sends them to the server. Specifically, the device retrieves the last 10 messages from the chat history and sends them to the server as a data packet, including the most recent message sent by the user.

[0961] Step 3:

[0962] The customer service analysis device captures the customer's facial expressions in real time and sends the data to a server. For example, smart glasses could capture a customer's face with a camera and send the video data to the analysis device, which then processes the data as emotion data.

[0963] Step 4:

[0964] The server receives the data and prepares it for analysis. The server temporarily stores the message history sent from the device and the facial expression data sent from the customer service analysis device in its memory. This data is used in the subsequent analysis process.

[0965] Step 5:

[0966] The server uses context analysis AI to analyze the context of the sent message. Specifically, the server inputs the message history into the context analysis AI and analyzes it to understand the content and context of the message. The analysis result is generated as output and handed over to the next process.

[0967] Step 6:

[0968] The server uses an emotion engine to analyze the facial expression data and generate emotion data. For example, the emotion engine identifies the customer's emotion, such as "curious" or "troubled," based on the video data sent from the customer service analysis device. This emotion data is used in the next process.

[0969] Step 7:

[0970] Based on the analysis results of the context analysis AI and emotion engine, the generation AI generates optimal stickers and responses. Specifically, the generation AI receives context and emotion data as input, and creates appropriate stickers and responses based on that. For example, it might generate a "sticker with an interesting expression" or a "specific product description." This output is temporarily stored in the server's memory.

[0971] Step 8:

[0972] The server transmits the generated stamps and responses to the terminal or smart glasses. Specifically, the server transmits the generated output as a data packet to the terminal or customer service analysis device. This data packet may contain multiple stamp candidates and responses.

[0973] Step 9:

[0974] The terminal receives the generated stamps and displays them to the user. Specifically, the terminal receives the data packets from the server, analyzes them, and displays them on the user interface. The user can select an appropriate stamp from the displayed stamps.

[0975] Step 10:

[0976] The user selects a stamp and sends it. The user selects the appropriate stamp from the displayed stamps and presses the send button again. For example, by selecting and sending the "confused face" stamp, the message will be sent to the other party.

[0977] Step 11:

[0978] The customer service staff will respond appropriately to the customer based on the optimal response method displayed on the smart glasses. For example, the smart glasses will display "specific product descriptions that will catch the customer's attention" and provide detailed explanations to the customer based on that information. This response will improve customer satisfaction.

[0979] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0980] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0981] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0982] [Third embodiment]

[0983] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0984] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0985] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0986] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0987] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0988] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0989] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0990] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0991] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0992] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0993] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0994] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0995] ---

[0996] The present invention relates to a system that automatically generates and provides stamps that are optimal for the context of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[0997] Overall system overview

[0998] This system mainly consists of a user's device, a server, and a generation AI. When a user sends a message, the message and data on the most recent exchange are sent from the device to the server. The server analyzes the received data and generates the most appropriate stamp using context analysis AI. The generated stamp is then sent back from the server to the user's device, where the user can select the appropriate one to send.

[0999] System operation explanation

[1000] 1. A user sends a message

[1001] A user types a message into a text input field in a chat app and presses the send button. For example, if a user sends a message like "What's my homework today?", the message is saved on the device.

[1002] 2. The device retrieves the most recent interaction and sends it to the server

[1003] The device retrieves the most recent messages, images, videos, etc. from the chat history it stores internally. For example, it retrieves the last 10 messages and sends them to the server, including the message that was just sent.

[1004] 3. The server uses contextual analysis AI to generate the optimal stamp

[1005] The server analyzes the most recent interactions received from the device and uses contextual analysis AI to generate stamps that best fit the context. For example, a stamp of "an expression of someone struggling with homework" might be generated.

[1006] 4. Provide the generated stamp to the user

[1007] The server sends the stamps generated by the AI ​​to the device and displays them to the user, who can then choose the best one from the multiple stamps generated.

[1008] 5. The user selects a stamp and sends it.

[1009] Next, the user selects one of the displayed stamps and presses the send button. For example, the user selects the stamp with the expression "worried about homework" and sends it.

[1010] Specific examples

[1011] A specific example of this system is shown below.

[1012] 1. User A sends a message

[1013] User A types "What's today's homework?" into a chat app and sends it.

[1014] 2. The device retrieves the most recent conversation and sends it to the server

[1015] The device retrieves the most recent 10 message history records and sends them to the server, including the message to be sent.

[1016] 3. The server generates stamps using context analysis AI

[1017] Context analysis AI generates a stamp that best fits the context of "What's today's homework?", depicting an "expression of someone struggling with homework."

[1018] 4. Provide the generated stamp to the user

[1019] The server returns the generated stamp to the terminal, which displays it to User A.

[1020] 5. User A selects a stamp and sends it

[1021] User A selects the stamp with the expression "looking worried about homework" and presses the send button.

[1022] This allows user A to quickly select an appropriate stamp and continue communication smoothly.

[1023] ---

[1024] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context of a message sent by a user. This allows users to quickly and easily obtain appropriate stamps, enabling efficient communication.

[1025] The processing flow will be explained below.

[1026] ---

[1027] Step 1:

[1028] The user enters a message and presses the send button.

[1029] A user types a message into a text input field in a chat app, for example, "What's my homework for today?", and presses the send button.

[1030] This message will be saved in the chat history on the device.

[1031] Step 2:

[1032] The device retrieves the most recent interactions and sends them to the server.

[1033] Obtain chat history stored internally on the device. For example, obtain the last 10 messages, images, and videos.

[1034] This data is sent as packets to the server as the "most recent exchange."

[1035] Step 3:

[1036] The server receives the most recent exchange.

[1037] The server receives the data packet sent from the terminal and prepares to analyze it.

[1038] The server temporarily stores this data in memory.

[1039] Step 4:

[1040] The server uses contextual analysis AI to analyze recent interactions.

[1041] The server inputs the received "most recent interaction" into a context analysis AI and analyzes it to generate the most appropriate stamp.

[1042] Contextual analysis AI understands the content of each media, such as text, images, and videos, and generates stamps based on that context.

[1043] Step 5:

[1044] The server uses generation AI to generate the optimal stamp.

[1045] Based on the analysis results, the AI ​​generates the most suitable stamp. For example, it can generate a stamp of someone struggling with their homework.

[1046] The generated stamp is temporarily stored in the server's memory.

[1047] Step 6:

[1048] The server sends the generated stamp to the terminal.

[1049] The server transmits the generated stamp to the terminal as a data packet.

[1050] This data packet may contain multiple stamp candidates.

[1051] Step 7:

[1052] The terminal receives the generated stamp and displays it to the user.

[1053] The terminal receives and analyzes the data packets from the server.

[1054] The interface is updated so that the generated stamp is visually displayed to the user.

[1055] Step 8:

[1056] The user selects and sends a stamp.

[1057] The user selects an appropriate stamp from the displayed ones and presses the send button.

[1058] For example, a user selects and sends a stamp with an expression of "an expression of someone struggling with homework."

[1059] Step 9:

[1060] The terminal transmits the selected stamp to the other chat participants.

[1061] The terminal retransmits the message including the stamp selected by the user and distributes it to the other chat participants.

[1062] The stamps selected in this way are sent in the flow of the conversation.

[1063] ---

[1064] As mentioned above, the processing steps of this system have been explained in detail. By showing the specific operations at each step, the operation of the entire system can be clearly understood.

[1065] Example 1

[1066] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1067] In recent years, users often need to select appropriate stamps or graphics when sending messages, but the selection process can be complex and time-consuming. In particular, to find a stamp that fits the context, users must search through a large number of options, which can disrupt communication. This creates a poor user experience and hinders efficient communication.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1069] In this invention, the server includes means for analyzing a context using a natural language processing model and generating an optimal graphic, means for displaying multiple candidate graphics, and means for providing the generated graphic. This allows a user to quickly and appropriately obtain a graphic that is optimal for the context when sending a message, enabling efficient communication.

[1070] 1. "User" means any entity that operates a communications system to send or receive information.

[1071] 2. "Information" means any message or content expressed in digital form, such as text data, image data, or video data.

[1072] 3. "Terminal" means a device used by a user to input, send, or receive information. Examples include smartphones, tablets, and personal computers.

[1073] 4. "Communication Content" means the history and current messages of information sent and received by a Device.

[1074] 5. "Server" means a computer system that processes and provides information in cooperation with terminals via a communications network.

[1075] 6. A "natural language processing model" is a machine learning model used to analyze text data and understand its meaning and context. Examples include GPT-3 and BERT.

[1076] 7. "Context" refers to the surrounding or background information needed to interpret the meaning of a message.

[1077] 8. "Graphics" is a general term for visual representations such as text, images, and videos, which are used to express the user's emotions and situations.

[1078] 9. A "prompt" is an instruction or question given to a generative AI model to generate an appropriate output.

[1079] 10. "REST API" means an application programming interface for sending and receiving data in the style of Representational State Transfer.

[1080] 11. "HTTP response" refers to data sent from a server to a terminal, and is sent using the HTTP protocol.

[1081] 12. “SQL query” means a statement used in Structured Query Language to retrieve information from a database.

[1082] The above are definitions of important terms included in the rewritten claims.

[1083] MODE FOR CARRYING OUT THE INVENTION

[1084] The present invention relates to a system that automatically generates and provides stamps and graphics that are optimal for the context of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[1085] Overall system overview

[1086] This system is composed of a user's device, a server, and a generative AI model. When a user sends a message, the message and the most recent communication content are sent from the device to the server. The server analyzes the received data and generates the optimal graphic using a natural language processing model. The generated graphic is then provided back to the user's device by the server, and the user can select the appropriate one to send.

[1087] Hardware and software used

[1088] Device: A digital device that a user uses to send and receive data. Examples include smartphones, tablets, and computers.

[1089] Server: A computer system that analyzes the received communications and generates stamps and graphics.

[1090] Natural language processing models: For example, machine learning models such as "GPT-3" and "BERT" are used.

[1091] Generative AI model: A model that works in conjunction with a natural language processing model to generate context-based graphics.

[1092] Communication protocol: REST API and HTTP responses are used to send and receive data.

[1093] System operation explanation

[1094] As a concrete example, we will explain the flow when User A sends a message "What's today's homework?" in a chat app.

[1095] 1. A user sends a message

[1096] User A enters "What's today's homework?" into the text input field of a chat app and presses the send button. This message is temporarily stored on the device.

[1097] 2. The device acquires the most recent communication content and sends it to the server

[1098] The device retrieves the last 10 messages from the internal database using an SQL query, and the retrieved messages, images, and videos are converted to JSON format and sent as a POST request to the server's REST API.

[1099] 3. The server uses contextual analysis AI to generate optimal graphics

[1100] The server passes the received data to a natural language processing model (e.g., GPT-3 or BERT), which analyzes the message "What's today's homework?" and the past 10 interactions as input. Based on the context, an appropriate prompt is input to the generative AI model, which generates a stamp of an "expression of someone struggling with homework."

[1101] 4. Serving the generated graphics to the user's device

[1102] The server generates a URL for the generated graphic and returns it to the user's device as an HTTP response. The device receives this response and dynamically displays the stamp in the chat app's interface.

[1103] 5. The user selects a stamp and sends it.

[1104] User A selects the "face struggling with homework" stamp from the provided stamps and presses the send button again. The URL of this selected stamp is again sent from the device to the server and is finally displayed on the chat partner's screen.

[1105] Prompt Sentence Examples

[1106] Below is an example of a prompt sentence to input to the generative AI model.

[1107] Message Context: What's my homework today?

[1108] Related past messages:

[1109] 1. Homework email from your teacher

[1110] 2. Communicating with classmates about homework

[1111] Generated stamp: An expression of someone struggling with homework

[1112] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context of a message sent by a user. Users can quickly and easily obtain appropriate stamps, enabling efficient communication.

[1113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1114] Explain the program's processing in detail

[1115] Step 1:

[1116] A user sends a message

[1117] explanation:

[1118] A user enters a message into the text input field of a chat app and presses the send button. This message is temporarily stored on the device.

[1119] Specific operation and input / output:

[1120] Input: A message typed by the user (e.g., "What's my homework today?")

[1121] Action: A message is entered into the text field and the send button is pressed.

[1122] Output: Stored in the message queue in the terminal.

[1123] Step 2:

[1124] The device acquires the most recent communication content and sends it to the server.

[1125] explanation:

[1126] The user's device retrieves the most recent message history from an internal database and sends it to the server.

[1127] Specific operation and input / output:

[1128] Input: Messages sent by the user and past communication history (e.g., last 10 messages)

[1129] Operation:

[1130] Retrieve the last 10 messages from the internal database using an SQL query.

[1131] Convert the retrieved data into JSON format.

[1132] Send the JSON data as a POST request to the server's REST API.

[1133] Output: Message history in JSON format is sent to the server.

[1134] Step 3:

[1135] The server uses contextual analysis AI to generate optimal graphics

[1136] explanation:

[1137] The server analyzes the received data, understands the context using natural language processing models, and generates the most appropriate graphics.

[1138] Specific operation and input / output:

[1139] Input: Recent message history (JSON format data)

[1140] Operation:

[1141] Passing JSON data to a natural language processing model (e.g., GPT-3 or BERT).

[1142] Contextual analysis is performed to generate prompts.

[1143] Pass the prompt to a generative AI model to generate the appropriate graphic.

[1144] Output: Generated optimal graphic (e.g., a stamp of a person struggling with homework)

[1145] Step 4:

[1146] Serving the generated graphics to the user's device

[1147] explanation:

[1148] The server generates a URL for the generated graphic and provides it to the user's terminal.

[1149] Specific operation and input / output:

[1150] Input: Generated graphic

[1151] Operation:

[1152] Save the graphic and generate a URL for it.

[1153] The URL is sent to the user's device as an HTTP response.

[1154] Output: The URL of the graphic is displayed on the user's device.

[1155] Step 5:

[1156] The user selects a stamp and sends it.

[1157] explanation:

[1158] The user selects an appropriate stamp from the ones provided and presses the send button.

[1159] Specific operation and input / output:

[1160] Input: A stamp selected by the user from the displayed graphics (e.g., a stamp of a person struggling with homework)

[1161] Operation:

[1162] The user taps or clicks on the displayed stamp.

[1163] The URL of the selected stamp is sent from the terminal to the server by pressing the send button again.

[1164] Output: The URL of the selected stamp will be displayed on the chat partner's screen.

[1165] The above is the specific flow of the program processing for this system, which allows users to quickly select the graphic that best suits the context and achieve smooth communication.

[1166] (Application example 1)

[1167] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1168] In conventional messaging applications, it is difficult for users to quickly select the most suitable stamp when sending a message. This leads to problems such as communication being disrupted and a poor user experience. In addition, selecting stamps must be done manually, which is inefficient.

[1169] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1170] In this invention, the server includes a means for a user to send a message, a means for a terminal to acquire the most recent exchange and send it to the server, a means for the server to generate optimal stamps using context analysis AI, a means for providing the generated stamps to the user, a means for the user to select and send stamps, a means for the server to send the results of context analysis to the stamp generation microservice, and a means for providing the URL of the stamp acquired from the microservice to the user terminal.This allows the user to automatically generate and receive stamps that are optimal for the context of the message they are sending, thereby facilitating communication and improving the user experience.

[1171] "User" refers to an individual user of the system.

[1172] "Message" refers to text, image or video information sent by a user to communicate with other users.

[1173] "Terminal" refers to a communication device such as a smartphone or tablet used by a user.

[1174] "Recent interactions" refers to the most recent communication history in the system.

[1175] "Server" refers to a computer system that processes and stores data and operates in conjunction with a terminal.

[1176] "Contextual analysis AI" refers to artificial intelligence technology that uses natural language processing to analyze the meaning and intent of text.

[1177] "Optimal stamp" refers to an icon or illustration that best fits the context of the user's message.

[1178] "Stamp generation means" refers to the process of generating stamps using contextual analysis AI.

[1179] A "stamp generation microservice" is a small, independent component that specializes in a specific function and is dedicated to generating stamps.

[1180] "Stamp URL" refers to the web address where the generated stamp is stored.

[1181] MODE FOR CARRYING OUT THE INVENTION

[1182] System Overview

[1183] This invention is a system that automatically generates and provides stamps that are optimal for the context of a message sent by a user. This system mainly consists of a user device, a server, a context analysis AI, and a stamp generation microservice.

[1184] System configuration

[1185] 1. User's device

[1186] The user's device (such as a smartphone or tablet) is a communication device that sends and receives messages. The device's role is to obtain the most recent exchanges and send them to the server.

[1187] 2. Server

[1188] The server receives messages and chat history sent by users, uses contextual analysis AI to generate optimal stamps, and then sends the generated stamps to a microservice, providing the resulting stamp URL to the user's device.

[1189] 3. Context analysis AI

[1190] Contextual analysis AI uses natural language processing to analyze the context of messages, and based on the results of this analysis, provides information to generate the best stamps for the user's message.

[1191] 4. Stamp Generation Microservice

[1192] The stamp generation microservice is a specialized component that generates the optimal stamp URL based on the results of contextual analysis sent from the server.

[1193] Operation explanation

[1194] 1. A user sends a message

[1195] The user types a message and presses the send button. For example, if the message is "I haven't finished my homework," the message is saved on the device.

[1196] 2. The device retrieves the most recent interaction and sends it to the server

[1197] The device retrieves past conversations, including the most recent message, from its internal chat history and sends that data to the server. Specifically, the last 10 messages are the target.

[1198] 3. The server uses contextual analysis AI to generate the optimal stamp

[1199] The server inputs the sent message and chat history into a context analysis AI to analyze the context. For example, for a message such as "I can't finish my homework," the context analysis AI determines that a stamp with a "worried expression" would be appropriate.

[1200] 4. Send the result to the stamp generation microservice and get the stamp URL.

[1201] The server sends the results of the contextual analysis to the stamp generation microservice, which generates a URL for the most suitable stamp, which is returned to the server.

[1202] 5. Provide the generated stamp to the user

[1203] The server sends the stamp URL to the user's device, and the user can view the stamp using this URL. The user can select and send stamps from the provided stamps.

[1204] Specific examples

[1205] A specific example of the system's operation is shown below.

[1206] 1. A user sends a new message

[1207] A user sends a message saying, "I can't finish my homework."

[1208] 2. Your device will connect your chat history with new messages

[1209] The last 10 chat messages and the new message are concatenated and sent to the server.

[1210] 3. Contextual AI generates stamps

[1211] Context analysis AI generates stamps that best fit the context of "I can't finish my homework."

[1212] 4. The stamp generation microservice generates the stamp URL.

[1213] The server sends the analysis results to the stamp generation microservice and obtains the URL of the optimal stamp.

[1214] 5. User checks and sends the provided stamp

[1215] The user selects a provided stamp and presses the send button.

[1216] Prompt Sentence Examples

[1217] The following is an example of a prompt:

[1218] text

[1219] Below is the chat history between User A and User B. Please generate the stamp that best suits User A's latest message.

[1220] Chat history:

[1221] User A: It's raining today.

[1222] User B: Maybe it'll be sunny tomorrow.

[1223] User A: What should I do about my homework?

[1224] User B: I want to hang out with friends.

[1225] User A: I want to go see a movie.

[1226] Latest Message:

[1227] User A: I can't finish my homework.

[1228] In this way, by using the system of the present invention, a user can quickly and easily obtain a stamp that is optimal for the context of a message, thereby achieving smoother and more efficient communication.

[1229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1230] Step 1:

[1231] The user enters a message and presses the send button. The input is the message text (e.g., "I haven't finished my homework yet."), and this message is saved on the device. The output is the latest message data.

[1232] Step 2:

[1233] The device retrieves past conversations, including the most recent message, from the chat history it stores internally. The input is the most recent message and chat history (e.g., the last 10 messages), which it then sends to the server. The output is the dataset sent to the server.

[1234] Step 3:

[1235] The server inputs the received data set into the context analysis AI and performs context analysis. The input is all message text (e.g., "I can't finish my homework" and past chat history), and based on this, it generates analysis results. The output is the context analysis results.

[1236] Step 4:

[1237] The server sends the result of the contextual analysis to the stamp generation microservice to obtain the optimal stamp URL. The input is the contextual analysis result, and the appropriate stamp URL is generated based on this. The output is the stamp URL.

[1238] Step 5:

[1239] The server sends the generated stamp URL to the user's device and provides it to the user. The input is the stamp URL, which is returned to the user's device. The output is the stamp URL that is displayed to the user.

[1240] Step 6:

[1241] The user selects the appropriate stamp from the provided ones and presses the submit button. The input is the stamp URL, and the output is the submission of the selected stamp.

[1242] In this way, the input data is processed at each step, and ultimately an appropriate stamp is provided to the user.

[1243] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1244] ---

[1245] The present invention relates to a system for automatically generating and providing stamps that are optimal for the context and emotion of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[1246] Overall system overview

[1247] This system mainly consists of the user's device, a server, a context analysis AI, and an emotion engine. When a user sends a message, that message and data on the most recent exchange are sent from the device to the server. At that time, the emotion engine recognizes the user's emotions, and the context analysis AI analyzes the context. The server generates optimal stamps based on the results of this analysis. The generated stamps are then sent back from the server to the user's device, where the user can select the appropriate one to send.

[1248] System operation explanation

[1249] 1. A user sends a message

[1250] A user types a message into a text input field in a chat app and presses the send button. For example, if a user sends a message like "What's my homework today?", the message is saved on the device.

[1251] 2. The device retrieves the most recent interaction and sends it to the server

[1252] The device retrieves the most recent messages, images, videos, etc. from the chat history it stores internally. For example, it retrieves the last 10 messages and sends them to the server, including the message that was just sent.

[1253] 3. The server receives the most recent exchange

[1254] The server receives the data packets sent from the terminal and prepares them for analysis. The server temporarily stores these data in memory.

[1255] 4. The server uses contextual analysis AI to analyze the most recent conversations

[1256] The server inputs the received "recent interactions" into a context analysis AI, which analyzes them to generate optimal stamps. The context analysis AI understands the content of each media, such as text, images, and videos, and generates stamps based on that context.

[1257] 5. Emotion engine analyzes user emotions

[1258] The emotion engine analyzes the user's emotions based on the messages, images, voice data, etc. received by the server from the device. For example, it can identify emotions such as "worried" or "happy" from the content of the user's message or image.

[1259] 6. The server generates the optimal stamp using generation AI

[1260] Based on the analysis results of the context analysis AI and emotion engine, the generation AI generates the optimal stamp. For example, a stamp of an "expression of someone struggling with homework" may be generated. The generated stamp is temporarily stored in the server's memory.

[1261] 7. The server sends the generated stamp to the device.

[1262] The server transmits the generated stamp to the terminal as a data packet, which may contain multiple stamp candidates.

[1263] 8. The device receives the generated stamp and displays it to the user.

[1264] The device receives and analyzes the data packets from the server and updates the interface so that the generated stamps are visually displayed to the user.

[1265] 9. The user selects a stamp and sends it.

[1266] The user selects an appropriate stamp from the displayed ones and presses the send button. For example, the user selects a stamp with an expression of someone struggling with homework and sends it.

[1267] 10. The device sends the selected stamp to other chat participants.

[1268] The terminal then retransmits the message containing the stamp selected by the user and distributes it to the other chat participants, thus transmitting the selected stamp in the flow of the conversation.

[1269] Specific examples

[1270] A specific example of this system is shown below.

[1271] 1. User A sends a message

[1272] User A types "What's today's homework?" into a chat app and sends it.

[1273] 2. The device retrieves the most recent conversation and sends it to the server

[1274] The device retrieves the most recent 10 message history records and sends them to the server, including the message to be sent.

[1275] 3. The server generates stamps using context analysis AI

[1276] Context analysis AI generates a stamp that best fits the context of "What's today's homework?", depicting an "expression of someone struggling with homework."

[1277] 4. Provide the generated stamp to the user

[1278] The server returns the generated stamp to the terminal, which displays it to User A.

[1279] 5. User A selects a stamp and sends it

[1280] User A selects the stamp with the expression "looking worried about homework" and presses the send button.

[1281] This allows user A to quickly select an appropriate stamp and continue communication smoothly.

[1282] ---

[1283] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context and emotion of a message sent by a user. This allows users to quickly and easily obtain appropriate stamps, enabling efficient communication.

[1284] The processing flow will be explained below.

[1285] ---

[1286] Step 1:

[1287] The user enters a message and presses the send button.

[1288] A user types a message into a text input field in a chat app, for example, "What's my homework for today?", and presses the send button.

[1289] This message will be saved in your device's chat history.

[1290] Step 2:

[1291] The device retrieves the most recent interactions and sends them to the server.

[1292] Data such as the last 10 messages, images, and videos is retrieved from the chat history stored internally on the device.

[1293] These "recent interactions" are packaged into data packets and sent to the server.

[1294] Step 3:

[1295] The server receives the most recent exchange.

[1296] The server receives the data packets sent from the terminal.

[1297] The received data is temporarily stored in memory and prepared for analysis.

[1298] Step 4:

[1299] The server uses contextual analysis AI to analyze recent interactions.

[1300] The server inputs the "most recent interaction" received into a context analysis AI.

[1301] Contextual analysis AI analyzes media content such as text, images, and videos to obtain information to generate stamps based on the context.

[1302] Step 5:

[1303] The server analyzes the emotion of the message using an emotion engine.

[1304] The server uses an emotion engine to analyze the user's messages, images, and voice data.

[1305] The emotion engine recognizes emotions such as "joy," "sadness," and "anger" from the message content. For example, it analyzes emotions such as "confusion" and "impatience" from the message "What's my homework today?"

[1306] Step 6:

[1307] The server integrates the results of contextual analysis and sentiment analysis.

[1308] The server integrates the analysis results from the context analysis AI and the emotion analysis results from the emotion engine.

[1309] This prepares the data to be fed to generative AI, taking into account both context and sentiment.

[1310] Step 7:

[1311] The server uses generation AI to generate the optimal stamp.

[1312] The server inputs the integrated analysis results into the generation AI.

[1313] The AI ​​then uses this data to generate the most suitable stamps, such as a stamp of someone struggling with their homework.

[1314] The generated stamp is temporarily stored in the server's memory.

[1315] Step 8:

[1316] The server sends the generated stamp to the terminal.

[1317] The server transmits a data packet containing the generated stamp to the terminal.

[1318] This data packet may contain multiple stamp candidates.

[1319] Step 9:

[1320] The terminal receives the generated stamp and displays it to the user.

[1321] The terminal receives and analyzes the data packets from the server.

[1322] The generated stamp is updated to be displayed on the user's interface.

[1323] Step 10:

[1324] The user selects and sends a stamp.

[1325] The user selects an appropriate stamp from the displayed stamps.

[1326] The user presses the send button to send the selected stamp.

[1327] Step 11:

[1328] The terminal transmits the selected stamp to the other chat participants.

[1329] The terminal again transmits the message including the stamp selected by the user.

[1330] The sent stamps are distributed to other chat participants and can be used within the flow of the conversation.

[1331] ---

[1332] As mentioned above, the processing of this system has been explained in detail for each step. By showing the specific operation at each step, the operation of the entire system can be clearly understood.

[1333] Example 2

[1334] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1335] In modern communication, the ability for users to quickly select appropriate stickers based on emotions and context is essential for smooth conversations and improved mutual understanding. However, current chat apps require users to manually select appropriate stickers, and it is difficult to find stickers that accurately reflect emotions and context. This not only impairs the user experience but also reduces the effectiveness of communication. To solve this problem, a system is needed that can analyze the user's message content and emotions in real time and automatically provide appropriate stickers.

[1336] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1337] In this invention, the server includes means for analyzing recent interactions using a context analysis engine, means for analyzing user emotions using an emotion analysis engine, and means for generating optimal stamps using a generation engine. As a result, when a user sends a message, a stamp that best suits the context and emotion is automatically generated in real time, allowing the user to quickly obtain an appropriate stamp, facilitating communication and improving the user experience. "User" refers to an entity that sends a message using the system.

[1338] "Terminal" refers to an electronic device that a user uses to send and receive messages.

[1339] "Server" means a central control device for processing, storing, and transmitting data.

[1340] A "contextual analysis engine" refers to a program or algorithm that analyzes the content of messages sent by users and understands their context.

[1341] "Sentiment analysis engine" refers to a program or algorithm that analyzes a user's messages and other data to identify the user's emotions.

[1342] "Generation engine" refers to the program or algorithm for generating optimal stamps based on the results of the context analysis engine and sentiment analysis engine.

[1343] A "stamp" refers to a visual medium such as an image, text, or video that a user attaches to a message and sends it.

[1344] "Message" refers to a means of communication such as text, images, or videos sent by a user.

[1345] "Recent interactions" refers to the history of recent messages and communications sent by a user.

[1346] The present invention relates to a system that automatically generates and provides stamps that are optimal for the context and emotion of a message sent by a user using a chat app. Detailed embodiments of the system are described below.

[1347] Overall system overview

[1348] This system consists of a user's device, a server, a context analysis engine, a sentiment analysis engine, and a generation engine. When a user sends a message, the message and the most recent exchanges are sent from the device to the server. The server analyzes the content of the sent message using the context analysis engine and analyzes the user's sentiment using the sentiment analysis engine. Based on the analysis results, the generation engine generates the most appropriate stamp. The generated stamp is sent from the server to the device, allowing the user to select and send the appropriate stamp.

[1349] Hardware and software used

[1350] 1. Device: Refers to the electronic device used by the user, such as a smartphone or computer, on which the chat app is installed.

[1351] 2. Server: A remote machine that receives, analyzes, generates, and sends data, for example, a cloud-based server.

[1352] 3. Contextual analysis engine: An AI model for analyzing the content of messages sent by users, with GPT-3 being a common example.

[1353] 4. Sentiment analysis engine: Software that analyzes the content of users' messages to identify their emotions. One example is IBM Watson Tone Analyzer.

[1354] 5. Generation engine: An AI model that generates optimal stamps based on the results of the context analysis engine and sentiment analysis engine. Generative AI such as DALL-E is used.

[1355] Data processing and calculation

[1356] 1. Text data analysis: The contextual analysis engine analyzes the messages sent by users and understands the context.

[1357] 2. Sentiment Analysis: The sentiment analysis engine analyzes the message content and identifies the user's emotions.

[1358] 3. Stamp generation: The generation engine generates optimal stamps based on the results of contextual and sentiment analysis.

[1359] Specific examples

[1360] Example 1: User A sends the message "What's my homework today?"

[1361] 1. User A types "What's today's homework?" in the text field of the chat app and presses the send button.

[1362] 2. User A's device retrieves the most recent 10 message history and sends it to the server along with the message to be sent.

[1363] 3. The server inputs the received data into a context analysis engine (GPT-3) and identifies that the message "What's my homework today?" is a homework inquiry.

[1364] 4. The server uses an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to identify the user's emotion as "worried" from the message content.

[1365] 5. The server inputs the prompt into the generation engine (e.g., DALL-E) and generates the best stamp. The following is used as an example prompt:

[1366] A user sends a message saying, "What's my homework today?" Generate a stamp based on this context and sentiment.

[1367] 6. The generated stamp of "an expression of struggling with homework" is sent from the server to the device.

[1368] 7. The device receives the stamp and displays it on the user interface.

[1369] 8. User A selects the appropriate stamp from the displayed ones and presses the send button.

[1370] As described above, the present invention is a system that realizes smoother and faster communication by automatically generating optimal stamps according to the message content and emotions of the user.

[1371] The flow of the identification process in the second embodiment will be described with reference to FIG. 13. Step 1:

[1372] The user sends a message. The user enters a message into the text input field of the chat app and presses the "Send" button. The message text entered is the input for this step, and this text is temporarily saved on the device.

[1373] Step 2:

[1374] The device retrieves the most recent conversations and sends them to the server. The device retrieves the last 10 messages from the chat history stored internally. The retrieved message history is input, and a data packet containing it is sent to the server. The device converts the packet to JSON format and sends it to the server as an HTTP POST request.

[1375] Step 3:

[1376] The server receives the most recent exchange. The server receives the HTTP POST request and temporarily stores the data packet in memory. The received data packet is the input and is prepared for the next analysis step.

[1377] Step 4:

[1378] The server uses a contextual analysis engine to analyze recent interactions. The server inputs data packets into the contextual analysis engine to understand the content of the message. The input is the last 10 messages and the latest message, and the output is the context of the message. The contextual analysis engine (e.g., GPT-3) identifies that the message "What's my homework today?" is a question about homework.

[1379] Step 5:

[1380] The server uses a sentiment analysis engine to analyze the user's emotions. The server inputs the message content into the sentiment analysis engine and performs a process to identify the user's emotions. The input is the user's message content, and the output is emotional information. The sentiment analysis engine (e.g., IBM Watson Tone Analyzer) identifies the user's emotion as "distressed" from the message content.

[1381] Step 6:

[1382] The server uses a generation AI to generate the optimal stamp. Based on the results of the context analysis engine and emotion analysis engine, the server inputs a prompt sentence into the generation AI. The input is contextual information and emotion information, and the output is a stamp image. The generation AI (e.g., DALL-E) generates a stamp of "an expression of someone struggling with homework" based on the prompt sentence. An example of a prompt sentence is as follows:

[1383] A user sends a message saying, "What's my homework today?" Generate a stamp based on this context and sentiment.

[1384] Step 7:

[1385] The server sends the generated stamp to the device. The server saves the generated stamp as an image file and creates a data packet containing it in JSON format. The input is the generated stamp image, and the output is a JSON-formatted data packet. The server sends the data packet to the device as an HTTP POST request.

[1386] Step 8:

[1387] The terminal receives the generated stamp and displays it to the user. The terminal receives the data packet from the server and parses the JSON data. The input is a JSON formatted data packet and the output is a stamp image. The display is updated so that the parsed stamp image is displayed in the user interface.

[1388] Step 9:

[1389] The user selects a stamp and sends it. The user selects the appropriate stamp from the displayed stamps and presses the "Send" button. The selected stamp image is the input, and the operation to send it is performed.

[1390] Step 10:

[1391] The terminal sends the selected stamp to other chat participants. The terminal converts the message containing the selected stamp into a packet and sends it to other chat participants as an HTTP POST request. The input is the selected stamp image, and the output is the packetized message. Other participants receive and display this as well.

[1392] (Application example 2)

[1393] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1394] Conventional chat applications require users to take the time to select appropriate stamps for their messages, and there is no system that can analyze customer emotions in real time and support customer service. This makes it difficult to communicate quickly and appropriately. Furthermore, in customer service, it is difficult for staff to intuitively grasp customer emotions and respond accordingly, making it difficult to improve customer satisfaction.

[1395] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1396] In this invention, the server includes a customer service analysis device that analyzes a customer's facial expression and generates emotion data, a means that analyzes the emotion data and generates an optimal response method, and a means that provides the generated response method to the customer. This makes it possible to analyze a customer's emotions in real time and provide optimal stamps and response methods. This allows users to quickly select and send appropriate stamps, and allows customer service staff to respond according to the customer's emotions, thereby facilitating communication and improving customer satisfaction.

[1397] A "terminal" is a device that allows a user to send messages and retrieve recent interactions and send them to a server.

[1398] The "server" is a device that receives data sent from a terminal, generates optimal stamps and responses using context analysis AI and an emotion engine, and provides them back to the terminal.

[1399] "Contextual AI" is AI that has the ability to analyze a user's messages and recent interactions and generate appropriate stamps and responses based on the context.

[1400] "Emotion data" is information that represents the emotions of a customer, generated by the customer service analysis device through analysis of the customer's facial expressions and comments.

[1401] A "customer service analysis device" is a device that captures and analyzes customers' facial expressions and behavior in real time to generate emotional data.

[1402] The "optimal response method" is generated based on the emotion data stored in the server and is a method that allows the staff to respond appropriately to the customer.

[1403] "Stamps" are visual icons or images that are added to users' messages and are automatically generated based on context and emotion.

[1404] "Smart glasses" are wearable devices that have the ability to capture customers' facial expressions and behavior and display the analysis results in real time.

[1405] The present invention relates to a system that analyzes the context and emotions of messages sent by users and automatically generates and provides optimal stamps or responses. This system is primarily composed of a user terminal, a server, context analysis AI, an emotion engine, generation AI, and a customer service analysis device. A detailed description of how this system can be implemented is provided below.

[1406] Overall system overview

[1407] In this system, when a user sends a message, that message and the most recent exchange are sent from the device to a server. At the same time, a customer service analysis device analyzes the customer's facial expressions and comments in real time and sends emotional data to the server. The server then uses context analysis AI and an emotion engine to generate optimal stamps and responses. Based on this, the generation AI creates optimal output, which is then sent back from the server to the user's device and the customer service staff's smart glasses.

[1408] Hardware and Software Configuration

[1409] The main hardware and software used to realize this system are as follows:

[1410] Hardware

[1411] Device: User's device (smartphone, tablet, etc.)

[1412] Server: Server for data processing and analysis

[1413] Customer service analysis devices: Wearable devices such as smart glasses

[1414] Camera: A device for capturing customer facial expressions

[1415] software

[1416] Context AI: AI that analyzes the context of messages

[1417] Emotion Engine: An engine that analyzes emotions from customers' facial expressions and comments

[1418] Stamp Generator: Artificial intelligence that generates optimal stamps and responses

[1419] System operation and data processing

[1420] Sending messages and retrieving data

[1421] When a user sends a message on a chat app, the device retrieves the most recent message from its built-in chat history and sends it to the server. The server temporarily stores it for analysis. At the same time, the customer service analysis device captures the customer's facial expressions and comments in real time and sends the data to the server.

[1422] Contextual and sentiment analysis

[1423] On the server, the context analysis AI analyzes the context of the received message. The emotion engine analyzes the customer's emotions based on the data obtained from the customer service analysis device. Based on the results of this analysis, the generation AI generates the optimal stamp and response method.

[1424] Generate optimal stamps and responses

[1425] The generation AI generates multiple sticker candidates and response methods based on the results of context analysis AI and the emotion engine. The generated stickers are displayed as text messages, image messages, or video messages.

[1426] Output and Serving

[1427] The generated stamps and responses are provided from the server to the device or smart glasses. The user can select and send the appropriate stamp. The customer service staff responds to the customer based on the optimal response displayed on the smart glasses.

[1428] Examples of concrete examples and prompts

[1429] Example 1: If a customer says, "What product do you recommend today?" and their facial expression shows interest, the system will generate a stamp of an "interested expression" and display it on the smart glasses of the customer service staff. The customer service staff will then use this information to explain the product.

[1430] Example 2: If a user sends a message in chat saying, "What should I do today?" and the context analysis AI identifies the emotion of "confused," it will generate and provide a stamp of a "confused face." The user can select this stamp and resend it, enabling smooth communication.

[1431] Example prompt sentence:

[1432] Customer sentiment: Intrigued

[1433] Customer Says: What products would you recommend today?

[1434] Content of the stamp to generate: Generate stamps that show expressions that will interest your customers.

[1435] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1436] Step 1:

[1437] A user sends a message in a chat app. For example, the user types "What should I do today?" and presses the send button. This message is saved on the device. At the same time, this message is temporarily saved along with the most recent chat history.

[1438] Step 2:

[1439] The device retrieves the most recent conversations and sends them to the server. Specifically, the device retrieves the last 10 messages from the chat history and sends them to the server as a data packet, including the most recent message sent by the user.

[1440] Step 3:

[1441] The customer service analysis device captures the customer's facial expressions in real time and sends the data to a server. For example, smart glasses could capture a customer's face with a camera and send the video data to the analysis device, which then processes the data as emotion data.

[1442] Step 4:

[1443] The server receives the data and prepares it for analysis. The server temporarily stores the message history sent from the device and the facial expression data sent from the customer service analysis device in its memory. This data is used in the subsequent analysis process.

[1444] Step 5:

[1445] The server uses context analysis AI to analyze the context of the sent message. Specifically, the server inputs the message history into the context analysis AI and analyzes it to understand the content and context of the message. The analysis result is generated as output and handed over to the next process.

[1446] Step 6:

[1447] The server uses an emotion engine to analyze the facial expression data and generate emotion data. For example, the emotion engine identifies the customer's emotion, such as "curious" or "troubled," based on the video data sent from the customer service analysis device. This emotion data is used in the next process.

[1448] Step 7:

[1449] Based on the analysis results of the context analysis AI and emotion engine, the generation AI generates optimal stickers and responses. Specifically, the generation AI receives context and emotion data as input, and creates appropriate stickers and responses based on that. For example, it might generate a "sticker with an interesting expression" or a "specific product description." This output is temporarily stored in the server's memory.

[1450] Step 8:

[1451] The server transmits the generated stamps and responses to the terminal or smart glasses. Specifically, the server transmits the generated output as a data packet to the terminal or customer service analysis device. This data packet may contain multiple stamp candidates and responses.

[1452] Step 9:

[1453] The terminal receives the generated stamps and displays them to the user. Specifically, the terminal receives the data packets from the server, analyzes them, and displays them on the user interface. The user can select an appropriate stamp from the displayed stamps.

[1454] Step 10:

[1455] The user selects a stamp and sends it. The user selects the appropriate stamp from the displayed stamps and presses the send button again. For example, by selecting and sending the "confused face" stamp, the message will be sent to the other party.

[1456] Step 11:

[1457] The customer service staff will respond appropriately to the customer based on the optimal response method displayed on the smart glasses. For example, the smart glasses will display "specific product descriptions that will catch the customer's attention" and provide detailed explanations to the customer based on that information. This response will improve customer satisfaction.

[1458] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1459] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1460] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1461] [Fourth embodiment]

[1462] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1463] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1464] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1465] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1466] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1467] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1468] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1469] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1470] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1471] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1472] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1473] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1474] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1475] ---

[1476] The present invention relates to a system that automatically generates and provides stamps that are optimal for the context of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[1477] Overall system overview

[1478] This system mainly consists of a user's device, a server, and a generation AI. When a user sends a message, the message and data on the most recent exchange are sent from the device to the server. The server analyzes the received data and generates the most appropriate stamp using context analysis AI. The generated stamp is then sent back from the server to the user's device, where the user can select the appropriate one to send.

[1479] System operation explanation

[1480] 1. A user sends a message

[1481] A user types a message into a text input field in a chat app and presses the send button. For example, if a user sends a message like "What's my homework today?", the message is saved on the device.

[1482] 2. The device retrieves the most recent interaction and sends it to the server

[1483] The device retrieves the most recent messages, images, videos, etc. from the chat history it stores internally. For example, it retrieves the last 10 messages and sends them to the server, including the message that was just sent.

[1484] 3. The server uses contextual analysis AI to generate the optimal stamp

[1485] The server analyzes the most recent interactions received from the device and uses contextual analysis AI to generate stamps that best fit the context. For example, a stamp of "an expression of someone struggling with homework" might be generated.

[1486] 4. Provide the generated stamp to the user

[1487] The server sends the stamps generated by the AI ​​to the device and displays them to the user, who can then choose the best one from the multiple stamps generated.

[1488] 5. The user selects a stamp and sends it.

[1489] Next, the user selects one of the displayed stamps and presses the send button. For example, the user selects the stamp with the expression "worried about homework" and sends it.

[1490] Specific examples

[1491] A specific example of this system is shown below.

[1492] 1. User A sends a message

[1493] User A types "What's today's homework?" into a chat app and sends it.

[1494] 2. The device retrieves the most recent conversation and sends it to the server

[1495] The device retrieves the most recent 10 message history records and sends them to the server, including the message to be sent.

[1496] 3. The server generates stamps using context analysis AI

[1497] Context analysis AI generates a stamp that best fits the context of "What's today's homework?", depicting an "expression of someone struggling with homework."

[1498] 4. Provide the generated stamp to the user

[1499] The server returns the generated stamp to the terminal, which displays it to User A.

[1500] 5. User A selects a stamp and sends it

[1501] User A selects the stamp with the expression "looking worried about homework" and presses the send button.

[1502] This allows user A to quickly select an appropriate stamp and continue communication smoothly.

[1503] ---

[1504] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context of a message sent by a user. This allows users to quickly and easily obtain appropriate stamps, enabling efficient communication.

[1505] The processing flow will be explained below.

[1506] ---

[1507] Step 1:

[1508] The user enters a message and presses the send button.

[1509] A user types a message into a text input field in a chat app, for example, "What's my homework for today?", and presses the send button.

[1510] This message will be saved in the chat history on the device.

[1511] Step 2:

[1512] The device retrieves the most recent interactions and sends them to the server.

[1513] Obtain chat history stored internally on the device. For example, obtain the last 10 messages, images, and videos.

[1514] This data is sent as packets to the server as the "most recent exchange."

[1515] Step 3:

[1516] The server receives the most recent exchange.

[1517] The server receives the data packet sent from the terminal and prepares to analyze it.

[1518] The server temporarily stores this data in memory.

[1519] Step 4:

[1520] The server uses contextual analysis AI to analyze recent interactions.

[1521] The server inputs the received "most recent interaction" into a context analysis AI and analyzes it to generate the most appropriate stamp.

[1522] Contextual analysis AI understands the content of each media, such as text, images, and videos, and generates stamps based on that context.

[1523] Step 5:

[1524] The server uses generation AI to generate the optimal stamp.

[1525] Based on the analysis results, the AI ​​generates the most suitable stamp. For example, it can generate a stamp of someone struggling with their homework.

[1526] The generated stamp is temporarily stored in the server's memory.

[1527] Step 6:

[1528] The server sends the generated stamp to the terminal.

[1529] The server transmits the generated stamp to the terminal as a data packet.

[1530] This data packet may contain multiple stamp candidates.

[1531] Step 7:

[1532] The terminal receives the generated stamp and displays it to the user.

[1533] The terminal receives and analyzes the data packets from the server.

[1534] The interface is updated so that the generated stamp is visually displayed to the user.

[1535] Step 8:

[1536] The user selects and sends a stamp.

[1537] The user selects an appropriate stamp from the displayed ones and presses the send button.

[1538] For example, a user selects and sends a stamp with an expression of "an expression of someone struggling with homework."

[1539] Step 9:

[1540] The terminal transmits the selected stamp to the other chat participants.

[1541] The terminal retransmits the message including the stamp selected by the user and distributes it to the other chat participants.

[1542] The stamps selected in this way are sent in the flow of the conversation.

[1543] ---

[1544] As mentioned above, the processing steps of this system have been explained in detail. By showing the specific operations at each step, the operation of the entire system can be clearly understood.

[1545] Example 1

[1546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1547] In recent years, users often need to select appropriate stamps or graphics when sending messages, but the selection process can be complex and time-consuming. In particular, to find a stamp that fits the context, users must search through a large number of options, which can disrupt communication. This creates a poor user experience and hinders efficient communication.

[1548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1549] In this invention, the server includes means for analyzing a context using a natural language processing model and generating an optimal graphic, means for displaying multiple candidate graphics, and means for providing the generated graphic. This allows a user to quickly and appropriately obtain a graphic that is optimal for the context when sending a message, enabling efficient communication.

[1550] 1. "User" means any entity that operates a communications system to send or receive information.

[1551] 2. "Information" means any message or content expressed in digital form, such as text data, image data, or video data.

[1552] 3. "Terminal" means a device used by a user to input, send, or receive information. Examples include smartphones, tablets, and personal computers.

[1553] 4. "Communication Content" means the history and current messages of information sent and received by a Device.

[1554] 5. "Server" means a computer system that processes and provides information in cooperation with terminals via a communications network.

[1555] 6. A "natural language processing model" is a machine learning model used to analyze text data and understand its meaning and context. Examples include GPT-3 and BERT.

[1556] 7. "Context" refers to the surrounding or background information needed to interpret the meaning of a message.

[1557] 8. "Graphics" is a general term for visual representations such as text, images, and videos, which are used to express the user's emotions and situations.

[1558] 9. A "prompt" is an instruction or question given to a generative AI model to generate an appropriate output.

[1559] 10. "REST API" means an application programming interface for sending and receiving data in the style of Representational State Transfer.

[1560] 11. "HTTP response" refers to data sent from a server to a terminal, and is sent using the HTTP protocol.

[1561] 12. “SQL query” means a statement used in Structured Query Language to retrieve information from a database.

[1562] The above are definitions of important terms included in the rewritten claims.

[1563] MODE FOR CARRYING OUT THE INVENTION

[1564] The present invention relates to a system that automatically generates and provides stamps and graphics that are optimal for the context of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[1565] Overall system overview

[1566] This system is composed of a user's device, a server, and a generative AI model. When a user sends a message, the message and the most recent communication content are sent from the device to the server. The server analyzes the received data and generates the optimal graphic using a natural language processing model. The generated graphic is then provided back to the user's device by the server, and the user can select the appropriate one to send.

[1567] Hardware and software used

[1568] Device: A digital device that a user uses to send and receive data. Examples include smartphones, tablets, and computers.

[1569] Server: A computer system that analyzes the received communications and generates stamps and graphics.

[1570] Natural language processing models: For example, machine learning models such as "GPT-3" and "BERT" are used.

[1571] Generative AI model: A model that works in conjunction with a natural language processing model to generate context-based graphics.

[1572] Communication protocol: REST API and HTTP responses are used to send and receive data.

[1573] System operation explanation

[1574] As a concrete example, we will explain the flow when User A sends a message "What's today's homework?" in a chat app.

[1575] 1. A user sends a message

[1576] User A enters "What's today's homework?" into the text input field of a chat app and presses the send button. This message is temporarily stored on the device.

[1577] 2. The device acquires the most recent communication content and sends it to the server

[1578] The device retrieves the last 10 messages from the internal database using an SQL query, and the retrieved messages, images, and videos are converted to JSON format and sent as a POST request to the server's REST API.

[1579] 3. The server uses contextual analysis AI to generate optimal graphics

[1580] The server passes the received data to a natural language processing model (e.g., GPT-3 or BERT), which analyzes the message "What's today's homework?" and the past 10 interactions as input. Based on the context, an appropriate prompt is input to the generative AI model, which generates a stamp of an "expression of someone struggling with homework."

[1581] 4. Serving the generated graphics to the user's device

[1582] The server generates a URL for the generated graphic and returns it to the user's device as an HTTP response. The device receives this response and dynamically displays the stamp in the chat app's interface.

[1583] 5. The user selects a stamp and sends it.

[1584] User A selects the "face struggling with homework" stamp from the provided stamps and presses the send button again. The URL of this selected stamp is again sent from the device to the server and is finally displayed on the chat partner's screen.

[1585] Prompt Sentence Examples

[1586] Below is an example of a prompt sentence to input to the generative AI model.

[1587] Message Context: What's my homework today?

[1588] Related past messages:

[1589] 1. Homework email from your teacher

[1590] 2. Communicating with classmates about homework

[1591] Generated stamp: An expression of someone struggling with homework

[1592] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context of a message sent by a user. Users can quickly and easily obtain appropriate stamps, enabling efficient communication.

[1593] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1594] Explain the program's processing in detail

[1595] Step 1:

[1596] A user sends a message

[1597] explanation:

[1598] A user enters a message into the text input field of a chat app and presses the send button. This message is temporarily stored on the device.

[1599] Specific operation and input / output:

[1600] Input: A message typed by the user (e.g., "What's my homework today?")

[1601] Action: A message is entered into the text field and the send button is pressed.

[1602] Output: Stored in the message queue in the terminal.

[1603] Step 2:

[1604] The device acquires the most recent communication content and sends it to the server.

[1605] explanation:

[1606] The user's device retrieves the most recent message history from an internal database and sends it to the server.

[1607] Specific operation and input / output:

[1608] Input: Messages sent by the user and past communication history (e.g., last 10 messages)

[1609] Operation:

[1610] Retrieve the last 10 messages from the internal database using an SQL query.

[1611] Convert the retrieved data into JSON format.

[1612] Send the JSON data as a POST request to the server's REST API.

[1613] Output: Message history in JSON format is sent to the server.

[1614] Step 3:

[1615] The server uses contextual analysis AI to generate optimal graphics

[1616] explanation:

[1617] The server analyzes the received data, understands the context using natural language processing models, and generates the most appropriate graphics.

[1618] Specific operation and input / output:

[1619] Input: Recent message history (JSON format data)

[1620] Operation:

[1621] Passing JSON data to a natural language processing model (e.g., GPT-3 or BERT).

[1622] Contextual analysis is performed to generate prompts.

[1623] Pass the prompt to a generative AI model to generate the appropriate graphic.

[1624] Output: Generated optimal graphic (e.g., a stamp of a person struggling with homework)

[1625] Step 4:

[1626] Serving the generated graphics to the user's device

[1627] explanation:

[1628] The server generates a URL for the generated graphic and provides it to the user's terminal.

[1629] Specific operation and input / output:

[1630] Input: Generated graphic

[1631] Operation:

[1632] Save the graphic and generate a URL for it.

[1633] The URL is sent to the user's device as an HTTP response.

[1634] Output: The URL of the graphic is displayed on the user's device.

[1635] Step 5:

[1636] The user selects a stamp and sends it.

[1637] explanation:

[1638] The user selects an appropriate stamp from the ones provided and presses the send button.

[1639] Specific operation and input / output:

[1640] Input: A stamp selected by the user from the displayed graphics (e.g., a stamp of a person struggling with homework)

[1641] Operation:

[1642] The user taps or clicks on the displayed stamp.

[1643] The URL of the selected stamp is sent from the terminal to the server by pressing the send button again.

[1644] Output: The URL of the selected stamp will be displayed on the chat partner's screen.

[1645] The above is the specific flow of the program processing for this system, which allows users to quickly select the graphic that best suits the context and achieve smooth communication.

[1646] (Application example 1)

[1647] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1648] In conventional messaging applications, it is difficult for users to quickly select the most suitable stamp when sending a message. This leads to problems such as communication being disrupted and a poor user experience. In addition, selecting stamps must be done manually, which is inefficient.

[1649] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1650] In this invention, the server includes a means for a user to send a message, a means for a terminal to acquire the most recent exchange and send it to the server, a means for the server to generate optimal stamps using context analysis AI, a means for providing the generated stamps to the user, a means for the user to select and send stamps, a means for the server to send the results of context analysis to the stamp generation microservice, and a means for providing the URL of the stamp acquired from the microservice to the user terminal.This allows the user to automatically generate and receive stamps that are optimal for the context of the message they are sending, thereby facilitating communication and improving the user experience.

[1651] "User" refers to an individual user of the system.

[1652] "Message" refers to text, image or video information sent by a user to communicate with other users.

[1653] "Terminal" refers to a communication device such as a smartphone or tablet used by a user.

[1654] "Recent interactions" refers to the most recent communication history in the system.

[1655] "Server" refers to a computer system that processes and stores data and operates in conjunction with a terminal.

[1656] "Contextual analysis AI" refers to artificial intelligence technology that uses natural language processing to analyze the meaning and intent of text.

[1657] "Optimal stamp" refers to an icon or illustration that best fits the context of the user's message.

[1658] "Stamp generation means" refers to the process of generating stamps using contextual analysis AI.

[1659] A "stamp generation microservice" is a small, independent component that specializes in a specific function and is dedicated to generating stamps.

[1660] "Stamp URL" refers to the web address where the generated stamp is stored.

[1661] MODE FOR CARRYING OUT THE INVENTION

[1662] System Overview

[1663] This invention is a system that automatically generates and provides stamps that are optimal for the context of a message sent by a user. This system mainly consists of a user device, a server, a context analysis AI, and a stamp generation microservice.

[1664] System configuration

[1665] 1. User's device

[1666] The user's device (such as a smartphone or tablet) is a communication device that sends and receives messages. The device's role is to obtain the most recent exchanges and send them to the server.

[1667] 2. Server

[1668] The server receives messages and chat history sent by users, uses contextual analysis AI to generate optimal stamps, and then sends the generated stamps to a microservice, providing the resulting stamp URL to the user's device.

[1669] 3. Context analysis AI

[1670] Contextual analysis AI uses natural language processing to analyze the context of messages, and based on the results of this analysis, provides information to generate the best stamps for the user's message.

[1671] 4. Stamp Generation Microservice

[1672] The stamp generation microservice is a specialized component that generates the optimal stamp URL based on the results of contextual analysis sent from the server.

[1673] Operation explanation

[1674] 1. A user sends a message

[1675] The user types a message and presses the send button. For example, if the message is "I haven't finished my homework," the message is saved on the device.

[1676] 2. The device retrieves the most recent interaction and sends it to the server

[1677] The device retrieves past conversations, including the most recent message, from its internal chat history and sends that data to the server. Specifically, the last 10 messages are the target.

[1678] 3. The server uses contextual analysis AI to generate the optimal stamp

[1679] The server inputs the sent message and chat history into a context analysis AI to analyze the context. For example, for a message such as "I can't finish my homework," the context analysis AI determines that a stamp with a "worried expression" would be appropriate.

[1680] 4. Send the result to the stamp generation microservice and get the stamp URL.

[1681] The server sends the results of the contextual analysis to the stamp generation microservice, which generates a URL for the most suitable stamp, which is returned to the server.

[1682] 5. Provide the generated stamp to the user

[1683] The server sends the stamp URL to the user's device, and the user can view the stamp using this URL. The user can select and send stamps from the provided stamps.

[1684] Specific examples

[1685] A specific example of the system's operation is shown below.

[1686] 1. A user sends a new message

[1687] A user sends a message saying, "I can't finish my homework."

[1688] 2. Your device will connect your chat history with new messages

[1689] The last 10 chat messages and the new message are concatenated and sent to the server.

[1690] 3. Contextual AI generates stamps

[1691] Context analysis AI generates stamps that best fit the context of "I can't finish my homework."

[1692] 4. The stamp generation microservice generates the stamp URL.

[1693] The server sends the analysis results to the stamp generation microservice and obtains the URL of the optimal stamp.

[1694] 5. User checks and sends the provided stamp

[1695] The user selects a provided stamp and presses the send button.

[1696] Prompt Sentence Examples

[1697] The following is an example of a prompt:

[1698] text

[1699] Below is the chat history between User A and User B. Please generate the stamp that best suits User A's latest message.

[1700] Chat history:

[1701] User A: It's raining today.

[1702] User B: Maybe it'll be sunny tomorrow.

[1703] User A: What should I do about my homework?

[1704] User B: I want to hang out with friends.

[1705] User A: I want to go see a movie.

[1706] Latest Message:

[1707] User A: I can't finish my homework.

[1708] In this way, by using the system of the present invention, a user can quickly and easily obtain a stamp that is optimal for the context of a message, thereby achieving smoother and more efficient communication.

[1709] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1710] Step 1:

[1711] The user enters a message and presses the send button. The input is the message text (e.g., "I haven't finished my homework yet."), and this message is saved on the device. The output is the latest message data.

[1712] Step 2:

[1713] The device retrieves past conversations, including the most recent message, from the chat history it stores internally. The input is the most recent message and chat history (e.g., the last 10 messages), which it then sends to the server. The output is the dataset sent to the server.

[1714] Step 3:

[1715] The server inputs the received data set into the context analysis AI and performs context analysis. The input is all message text (e.g., "I can't finish my homework" and past chat history), and based on this, it generates analysis results. The output is the context analysis results.

[1716] Step 4:

[1717] The server sends the result of the contextual analysis to the stamp generation microservice to obtain the optimal stamp URL. The input is the contextual analysis result, and the appropriate stamp URL is generated based on this. The output is the stamp URL.

[1718] Step 5:

[1719] The server sends the generated stamp URL to the user's device and provides it to the user. The input is the stamp URL, which is returned to the user's device. The output is the stamp URL that is displayed to the user.

[1720] Step 6:

[1721] The user selects the appropriate stamp from the provided ones and presses the submit button. The input is the stamp URL, and the output is the submission of the selected stamp.

[1722] In this way, the input data is processed at each step, and ultimately an appropriate stamp is provided to the user.

[1723] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1724] ---

[1725] The present invention relates to a system for automatically generating and providing stamps that are optimal for the context and emotion of a message sent by a user. Detailed embodiments for implementing this system will be described below.

[1726] Overall system overview

[1727] This system mainly consists of the user's device, a server, a context analysis AI, and an emotion engine. When a user sends a message, that message and data on the most recent exchange are sent from the device to the server. At that time, the emotion engine recognizes the user's emotions, and the context analysis AI analyzes the context. The server generates optimal stamps based on the results of this analysis. The generated stamps are then sent back from the server to the user's device, where the user can select the appropriate one to send.

[1728] System operation explanation

[1729] 1. A user sends a message

[1730] A user types a message into a text input field in a chat app and presses the send button. For example, if a user sends a message like "What's my homework today?", the message is saved on the device.

[1731] 2. The device retrieves the most recent interaction and sends it to the server

[1732] The device retrieves the most recent messages, images, videos, etc. from the chat history it stores internally. For example, it retrieves the last 10 messages and sends them to the server, including the message that was just sent.

[1733] 3. The server receives the most recent exchange

[1734] The server receives the data packets sent from the terminal and prepares them for analysis. The server temporarily stores these data in memory.

[1735] 4. The server uses contextual analysis AI to analyze the most recent conversations

[1736] The server inputs the received "recent interactions" into a context analysis AI, which analyzes them to generate optimal stamps. The context analysis AI understands the content of each media, such as text, images, and videos, and generates stamps based on that context.

[1737] 5. Emotion engine analyzes user emotions

[1738] The emotion engine analyzes the user's emotions based on the messages, images, voice data, etc. received by the server from the device. For example, it can identify emotions such as "worried" or "happy" from the content of the user's message or image.

[1739] 6. The server generates the optimal stamp using generation AI

[1740] Based on the analysis results of the context analysis AI and emotion engine, the generation AI generates the optimal stamp. For example, a stamp of an "expression of someone struggling with homework" may be generated. The generated stamp is temporarily stored in the server's memory.

[1741] 7. The server sends the generated stamp to the device.

[1742] The server transmits the generated stamp to the terminal as a data packet, which may contain multiple stamp candidates.

[1743] 8. The device receives the generated stamp and displays it to the user.

[1744] The device receives and analyzes the data packets from the server and updates the interface so that the generated stamps are visually displayed to the user.

[1745] 9. The user selects a stamp and sends it.

[1746] The user selects an appropriate stamp from the displayed ones and presses the send button. For example, the user selects a stamp with an expression of someone struggling with homework and sends it.

[1747] 10. The device sends the selected stamp to other chat participants.

[1748] The terminal then retransmits the message containing the stamp selected by the user and distributes it to the other chat participants, thus transmitting the selected stamp in the flow of the conversation.

[1749] Specific examples

[1750] A specific example of this system is shown below.

[1751] 1. User A sends a message

[1752] User A types "What's today's homework?" into a chat app and sends it.

[1753] 2. The device retrieves the most recent conversation and sends it to the server

[1754] The device retrieves the most recent 10 message history records and sends them to the server, including the message to be sent.

[1755] 3. The server generates stamps using context analysis AI

[1756] Context analysis AI generates a stamp that best fits the context of "What's today's homework?", depicting an "expression of someone struggling with homework."

[1757] 4. Provide the generated stamp to the user

[1758] The server returns the generated stamp to the terminal, which displays it to User A.

[1759] 5. User A selects a stamp and sends it

[1760] User A selects the stamp with the expression "looking worried about homework" and presses the send button.

[1761] This allows user A to quickly select an appropriate stamp and continue communication smoothly.

[1762] ---

[1763] As described above, the present invention aims to facilitate smooth communication by automatically generating and providing stamps that are optimal for the context and emotion of a message sent by a user. This allows users to quickly and easily obtain appropriate stamps, enabling efficient communication.

[1764] The processing flow will be explained below.

[1765] ---

[1766] Step 1:

[1767] The user enters a message and presses the send button.

[1768] A user types a message into a text input field in a chat app, for example, "What's my homework for today?", and presses the send button.

[1769] This message will be saved in your device's chat history.

[1770] Step 2:

[1771] The device retrieves the most recent interactions and sends them to the server.

[1772] Data such as the last 10 messages, images, and videos is retrieved from the chat history stored internally on the device.

[1773] These "recent interactions" are packaged into data packets and sent to the server.

[1774] Step 3:

[1775] The server receives the most recent exchange.

[1776] The server receives the data packets sent from the terminal.

[1777] The received data is temporarily stored in memory and prepared for analysis.

[1778] Step 4:

[1779] The server uses contextual analysis AI to analyze recent interactions.

[1780] The server inputs the "most recent interaction" received into a context analysis AI.

[1781] Contextual analysis AI analyzes media content such as text, images, and videos to obtain information to generate stamps based on the context.

[1782] Step 5:

[1783] The server analyzes the emotion of the message using an emotion engine.

[1784] The server uses an emotion engine to analyze the user's messages, images, and voice data.

[1785] The emotion engine recognizes emotions such as "joy," "sadness," and "anger" from the message content. For example, it analyzes emotions such as "confusion" and "impatience" from the message "What's my homework today?"

[1786] Step 6:

[1787] The server integrates the results of contextual analysis and sentiment analysis.

[1788] The server integrates the analysis results from the context analysis AI and the emotion analysis results from the emotion engine.

[1789] This prepares the data to be fed to generative AI, taking into account both context and sentiment.

[1790] Step 7:

[1791] The server uses generation AI to generate the optimal stamp.

[1792] The server inputs the integrated analysis results into the generation AI.

[1793] The AI ​​then uses this data to generate the most suitable stamps, such as a stamp of someone struggling with their homework.

[1794] The generated stamp is temporarily stored in the server's memory.

[1795] Step 8:

[1796] The server sends the generated stamp to the terminal.

[1797] The server transmits a data packet containing the generated stamp to the terminal.

[1798] This data packet may contain multiple stamp candidates.

[1799] Step 9:

[1800] The terminal receives the generated stamp and displays it to the user.

[1801] The terminal receives and analyzes the data packets from the server.

[1802] The generated stamp is updated to be displayed on the user's interface.

[1803] Step 10:

[1804] The user selects and sends a stamp.

[1805] The user selects an appropriate stamp from the displayed stamps.

[1806] The user presses the send button to send the selected stamp.

[1807] Step 11:

[1808] The terminal transmits the selected stamp to the other chat participants.

[1809] The terminal again transmits the message including the stamp selected by the user.

[1810] The sent stamps are distributed to other chat participants and can be used within the flow of the conversation.

[1811] ---

[1812] As mentioned above, the processing of this system has been explained in detail for each step. By showing the specific operation at each step, the operation of the entire system can be clearly understood.

[1813] Example 2

[1814] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1815] In modern communication, the ability for users to quickly select appropriate stickers based on emotions and context is essential for smooth conversations and improved mutual understanding. However, current chat apps require users to manually select appropriate stickers, and it is difficult to find stickers that accurately reflect emotions and context. This not only impairs the user experience but also reduces the effectiveness of communication. To solve this problem, a system is needed that can analyze the user's message content and emotions in real time and automatically provide appropriate stickers.

[1816] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1817] In this invention, the server includes means for analyzing recent interactions using a context analysis engine, means for analyzing user emotions using an emotion analysis engine, and means for generating optimal stamps using a generation engine. As a result, when a user sends a message, a stamp that best suits the context and emotion is automatically generated in real time, allowing the user to quickly obtain an appropriate stamp, facilitating communication and improving the user experience. "User" refers to an entity that sends a message using the system.

[1818] "Terminal" refers to an electronic device that a user uses to send and receive messages.

[1819] "Server" means a central control device for processing, storing, and transmitting data.

[1820] A "contextual analysis engine" refers to a program or algorithm that analyzes the content of messages sent by users and understands their context.

[1821] "Sentiment analysis engine" refers to a program or algorithm that analyzes a user's messages and other data to identify the user's emotions.

[1822] "Generation engine" refers to the program or algorithm for generating optimal stamps based on the results of the context analysis engine and sentiment analysis engine.

[1823] A "stamp" refers to a visual medium such as an image, text, or video that a user attaches to a message and sends it.

[1824] "Message" refers to a means of communication such as text, images, or videos sent by a user.

[1825] "Recent interactions" refers to the history of recent messages and communications sent by a user.

[1826] The present invention relates to a system that automatically generates and provides stamps that are optimal for the context and emotion of a message sent by a user using a chat app. Detailed embodiments of the system are described below.

[1827] Overall system overview

[1828] This system consists of a user's device, a server, a context analysis engine, a sentiment analysis engine, and a generation engine. When a user sends a message, the message and the most recent exchanges are sent from the device to the server. The server analyzes the content of the sent message using the context analysis engine and analyzes the user's sentiment using the sentiment analysis engine. Based on the analysis results, the generation engine generates the most appropriate stamp. The generated stamp is sent from the server to the device, allowing the user to select and send the appropriate stamp.

[1829] Hardware and software used

[1830] 1. Device: Refers to the electronic device used by the user, such as a smartphone or computer, on which the chat app is installed.

[1831] 2. Server: A remote machine that receives, analyzes, generates, and sends data, for example, a cloud-based server.

[1832] 3. Contextual analysis engine: An AI model for analyzing the content of messages sent by users, with GPT-3 being a common example.

[1833] 4. Sentiment analysis engine: Software that analyzes the content of users' messages to identify their emotions. One example is IBM Watson Tone Analyzer.

[1834] 5. Generation engine: An AI model that generates optimal stamps based on the results of the context analysis engine and sentiment analysis engine. Generative AI such as DALL-E is used.

[1835] Data processing and calculation

[1836] 1. Text data analysis: The contextual analysis engine analyzes the messages sent by users and understands the context.

[1837] 2. Sentiment Analysis: The sentiment analysis engine analyzes the message content and identifies the user's emotions.

[1838] 3. Stamp generation: The generation engine generates optimal stamps based on the results of contextual and sentiment analysis.

[1839] Specific examples

[1840] Example 1: User A sends the message "What's my homework today?"

[1841] 1. User A types "What's today's homework?" in the text field of the chat app and presses the send button.

[1842] 2. User A's device retrieves the most recent 10 message history and sends it to the server along with the message to be sent.

[1843] 3. The server inputs the received data into a context analysis engine (GPT-3) and identifies that the message "What's my homework today?" is a homework inquiry.

[1844] 4. The server uses an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to identify the user's emotion as "worried" from the message content.

[1845] 5. The server inputs the prompt into the generation engine (e.g., DALL-E) and generates the best stamp. The following is used as an example prompt:

[1846] A user sends a message saying, "What's my homework today?" Generate a stamp based on this context and sentiment.

[1847] 6. The generated stamp of "an expression of struggling with homework" is sent from the server to the device.

[1848] 7. The device receives the stamp and displays it on the user interface.

[1849] 8. User A selects the appropriate stamp from the displayed ones and presses the send button.

[1850] As described above, the present invention is a system that realizes smoother and faster communication by automatically generating optimal stamps according to the message content and emotions of the user.

[1851] The flow of the identification process in the second embodiment will be described with reference to FIG. 13. Step 1:

[1852] The user sends a message. The user enters a message into the text input field of the chat app and presses the "Send" button. The message text entered is the input for this step, and this text is temporarily saved on the device.

[1853] Step 2:

[1854] The device retrieves the most recent conversations and sends them to the server. The device retrieves the last 10 messages from the chat history stored internally. The retrieved message history is input, and a data packet containing it is sent to the server. The device converts the packet to JSON format and sends it to the server as an HTTP POST request.

[1855] Step 3:

[1856] The server receives the most recent exchange. The server receives the HTTP POST request and stores the data packet in memory. The received data packet is the input and is prepared for the next analysis step.

[1857] Step 4:

[1858] The server uses a contextual analysis engine to analyze recent interactions. The server inputs data packets into the contextual analysis engine to understand the content of the message. The input is the last 10 messages and the latest message, and the output is the context of the message. The contextual analysis engine (e.g., GPT-3) identifies that the message "What's my homework today?" is a question about homework.

[1859] Step 5:

[1860] The server uses a sentiment analysis engine to analyze the user's emotions. The server inputs the message content into the sentiment analysis engine and performs a process to identify the user's emotions. The input is the user's message content, and the output is emotional information. The sentiment analysis engine (e.g., IBM Watson Tone Analyzer) identifies the user's emotion as "distressed" from the message content.

[1861] Step 6:

[1862] The server uses a generation AI to generate the optimal stamp. Based on the results of the context analysis engine and emotion analysis engine, the server inputs a prompt sentence into the generation AI. The input is contextual information and emotion information, and the output is a stamp image. The generation AI (e.g., DALL-E) generates a stamp of "an expression of someone struggling with homework" based on the prompt sentence. An example of a prompt sentence is as follows:

[1863] A user sends a message saying, "What's my homework today?" Generate a stamp based on this context and sentiment.

[1864] Step 7:

[1865] The server sends the generated stamp to the device. The server saves the generated stamp as an image file and creates a data packet containing it in JSON format. The input is the generated stamp image, and the output is a JSON-formatted data packet. The server sends the data packet to the device as an HTTP POST request.

[1866] Step 8:

[1867] The terminal receives the generated stamp and displays it to the user. The terminal receives the data packet from the server and parses the JSON data. The input is a JSON formatted data packet and the output is a stamp image. The display is updated so that the parsed stamp image is displayed in the user interface.

[1868] Step 9:

[1869] The user selects a stamp and sends it. The user selects the appropriate stamp from the displayed stamps and presses the "Send" button. The selected stamp image is the input, and the operation to send it is performed.

[1870] Step 10:

[1871] The terminal sends the selected stamp to other chat participants. The terminal converts the message containing the selected stamp into a packet and sends it to other chat participants as an HTTP POST request. The input is the selected stamp image, and the output is the packetized message. Other participants receive and display this as well.

[1872] (Application example 2)

[1873] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1874] Conventional chat applications require users to take the time to select appropriate stamps for their messages, and there is no system that can analyze customer emotions in real time and support customer service. This makes it difficult to communicate quickly and appropriately. Furthermore, in customer service, it is difficult for staff to intuitively grasp customer emotions and respond accordingly, making it difficult to improve customer satisfaction.

[1875] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1876] In this invention, the server includes a customer service analysis device that analyzes a customer's facial expression and generates emotion data, a means that analyzes the emotion data and generates an optimal response method, and a means that provides the generated response method to the customer. This makes it possible to analyze a customer's emotions in real time and provide optimal stamps and response methods. This allows users to quickly select and send appropriate stamps, and allows customer service staff to respond according to the customer's emotions, thereby facilitating communication and improving customer satisfaction.

[1877] A "terminal" is a device that allows a user to send messages and retrieve recent interactions and send them to a server.

[1878] The "server" is a device that receives data sent from a terminal, generates optimal stamps and responses using context analysis AI and an emotion engine, and provides them back to the terminal.

[1879] "Contextual AI" is AI that has the ability to analyze a user's messages and recent interactions and generate appropriate stamps and responses based on the context.

[1880] "Emotion data" is information that represents the emotions of a customer, generated by the customer service analysis device through analysis of the customer's facial expressions and comments.

[1881] A "customer service analysis device" is a device that captures and analyzes customers' facial expressions and behavior in real time to generate emotional data.

[1882] The "optimal response method" is generated based on the emotion data stored in the server and is a method that allows the staff to respond appropriately to the customer.

[1883] "Stamps" are visual icons or images that are added to users' messages and are automatically generated based on context and emotion.

[1884] "Smart glasses" are wearable devices that have the ability to capture customers' facial expressions and behavior and display the analysis results in real time.

[1885] The present invention relates to a system that analyzes the context and emotions of messages sent by users and automatically generates and provides optimal stamps or responses. This system is primarily composed of a user terminal, a server, context analysis AI, an emotion engine, generation AI, and a customer service analysis device. A detailed description of how this system can be implemented is provided below.

[1886] Overall system overview

[1887] In this system, when a user sends a message, that message and the most recent exchange are sent from the device to a server. At the same time, a customer service analysis device analyzes the customer's facial expressions and comments in real time and sends emotional data to the server. The server then uses context analysis AI and an emotion engine to generate optimal stamps and responses. Based on this, the generation AI creates optimal output, which is then sent back from the server to the user's device and the customer service staff's smart glasses.

[1888] Hardware and Software Configuration

[1889] The main hardware and software used to realize this system are as follows:

[1890] Hardware

[1891] Device: User's device (smartphone, tablet, etc.)

[1892] Server: Server for data processing and analysis

[1893] Customer service analysis devices: Wearable devices such as smart glasses

[1894] Camera: A device for capturing customer facial expressions

[1895] software

[1896] Context AI: AI that analyzes the context of messages

[1897] Emotion Engine: An engine that analyzes emotions from customers' facial expressions and comments

[1898] Stamp Generator: Artificial intelligence that generates optimal stamps and responses

[1899] System operation and data processing

[1900] Sending messages and retrieving data

[1901] When a user sends a message on a chat app, the device retrieves the most recent message from its built-in chat history and sends it to the server. The server temporarily stores it for analysis. At the same time, the customer service analysis device captures the customer's facial expressions and comments in real time and sends the data to the server.

[1902] Contextual and sentiment analysis

[1903] On the server, the context analysis AI analyzes the context of the received message. The emotion engine analyzes the customer's emotions based on the data obtained from the customer service analysis device. Based on the results of this analysis, the generation AI generates the optimal stamp and response method.

[1904] Generate optimal stamps and responses

[1905] The generation AI generates multiple sticker candidates and response methods based on the results of context analysis AI and the emotion engine. The generated stickers are displayed as text messages, image messages, or video messages.

[1906] Output and Serving

[1907] The generated stamps and responses are provided from the server to the device or smart glasses. The user can select and send the appropriate stamp. The customer service staff responds to the customer based on the optimal response displayed on the smart glasses.

[1908] Examples of concrete examples and prompts

[1909] Example 1: If a customer says, "What product do you recommend today?" and their facial expression shows interest, the system will generate a stamp of an "interested expression" and display it on the smart glasses of the customer service staff. The customer service staff will then use this information to explain the product.

[1910] Example 2: If a user sends a message in chat saying, "What should I do today?" and the context analysis AI identifies the emotion of "confused," it will generate and provide a stamp of a "confused face." The user can select this stamp and resend it, enabling smooth communication.

[1911] Example prompt sentence:

[1912] Customer sentiment: Intrigued

[1913] Customer Says: What products would you recommend today?

[1914] Content of the stamp to generate: Generate stamps that show expressions that will interest your customers.

[1915] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1916] Step 1:

[1917] A user sends a message in a chat app. For example, the user types "What should I do today?" and presses the send button. This message is saved on the device. At the same time, this message is temporarily saved along with the most recent chat history.

[1918] Step 2:

[1919] The device retrieves the most recent conversations and sends them to the server. Specifically, the device retrieves the last 10 messages from the chat history and sends them to the server as a data packet, including the most recent message sent by the user.

[1920] Step 3:

[1921] The customer service analysis device captures the customer's facial expressions in real time and sends the data to a server. For example, smart glasses could capture a customer's face with a camera and send the video data to the analysis device, which then processes the data as emotion data.

[1922] Step 4:

[1923] The server receives the data and prepares it for analysis. The server temporarily stores the message history sent from the device and the facial expression data sent from the customer service analysis device in its memory. This data is used in the subsequent analysis process.

[1924] Step 5:

[1925] The server uses context analysis AI to analyze the context of the sent message. Specifically, the server inputs the message history into the context analysis AI and analyzes it to understand the content and context of the message. The analysis result is generated as output and handed over to the next process.

[1926] Step 6:

[1927] The server uses an emotion engine to analyze the facial expression data and generate emotion data. For example, the emotion engine identifies the customer's emotion, such as "curious" or "troubled," based on the video data sent from the customer service analysis device. This emotion data is used in the next process.

[1928] Step 7:

[1929] Based on the analysis results of the context analysis AI and emotion engine, the generation AI generates optimal stickers and responses. Specifically, the generation AI receives context and emotion data as input, and creates appropriate stickers and responses based on that. For example, it might generate a "sticker with an interesting expression" or a "specific product description." This output is temporarily stored in the server's memory.

[1930] Step 8:

[1931] The server transmits the generated stamps and responses to the terminal or smart glasses. Specifically, the server transmits the generated output as a data packet to the terminal or customer service analysis device. This data packet may contain multiple stamp candidates and responses.

[1932] Step 9:

[1933] The terminal receives the generated stamps and displays them to the user. Specifically, the terminal receives the data packets from the server, analyzes them, and displays them on the user interface. The user can select an appropriate stamp from the displayed stamps.

[1934] Step 10:

[1935] The user selects a stamp and sends it. The user selects the appropriate stamp from the displayed stamps and presses the send button again. For example, by selecting and sending the "confused face" stamp, the message will be sent to the other party.

[1936] Step 11:

[1937] The customer service staff will respond appropriately to the customer based on the optimal response method displayed on the smart glasses. For example, the smart glasses will display "specific product descriptions that will catch the customer's attention" and provide detailed explanations to the customer based on that information. This response will improve customer satisfaction.

[1938] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1939] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1940] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1941] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1942] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1943] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1944] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1945] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1946] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1947] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1948] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1949] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1950] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1951] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1952] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1953] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1954] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1955] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1956] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1957] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1958] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1959] The following is further disclosed regarding the above embodiment.

[1960] (Claim 1)

[1961] a means for a user to send a message;

[1962] A means for the device to acquire and transmit the most recent interactions to a server;

[1963] A means for the server to generate optimal stamps using context analysis AI;

[1964] means for providing the generated stamp to a user;

[1965] A means for a user to select and send stamps;

[1966] A system including:

[1967] (Claim 2)

[1968] The system according to claim 1, wherein the server displays multiple stamp candidates generated by context analysis AI.

[1969] (Claim 3)

[1970] 10. The system of claim 1, wherein the generated stamp is one of a text message, an image message, or a video message.

[1971] "Example 1"

[1972] (Claim 1)

[1973] a means for a user to transmit information;

[1974] A means for the terminal to acquire the most recent communication content and transmit it to the server;

[1975] A means for the server to analyze the context using a natural language processing model and generate an optimal graphic;

[1976] means for providing the generated graphic to a user;

[1977] means for a user to select and transmit a graphic;

[1978] A system including:

[1979] (Claim 2)

[1980] 2. The system according to claim 1, wherein the server displays a plurality of graphic candidates generated by a natural language processing model.

[1981] (Claim 3)

[1982] 2. The system of claim 1, wherein the generated graphic is any one of text data, image data, and video data.

[1983] "Application Example 1"

[1984] (Claim 1)

[1985] a means for a user to send a message;

[1986] A means for the device to acquire and transmit the most recent interactions to a server;

[1987] A means for the server to generate optimal stamps using context analysis AI;

[1988] means for providing the generated stamp to a user;

[1989] A means for a user to select and send stamps;

[1990] means for the server to send the results of the contextual analysis to the stamp generation microservice;

[1991] A means for providing the stamp URL obtained from the microservice to the user's device;

[1992] A system including:

[1993] (Claim 2)

[1994] The system according to claim 1, wherein the server displays multiple stamp candidates generated by context analysis AI.

[1995] (Claim 3)

[1996] 10. The system of claim 1, wherein the generated stamp is one of a text message, an image message, or a video message.

[1997] "Example 2 when combining emotion engines" (Claim 1)

[1998] a means for a user to send a message;

[1999] A means for the device to acquire and transmit the most recent interactions to a server;

[2000] a means for the server to analyze the most recent exchange using a contextual analysis engine;

[2001] A means for the server to analyze the user's emotions using an emotion analysis engine;

[2002] A means for the server to generate an optimal stamp using a generation engine;

[2003] means for providing the generated stamp to a user;

[2004] A means for a user to select and send stamps;

[2005] A system including:

[2006] (Claim 2)

[2007] The system according to claim 1, wherein the server displays a plurality of stamp candidates based on the results of the context analysis engine and the sentiment analysis engine.

[2008] (Claim 3)

[2009] 10. The system of claim 1, wherein the generated stamp is one of text, an image, or a video.

[2010] That's all.

[2011] "Application example 2 when combining emotion engines"

[2012] (Claim 1)

[2013] a means for a user to send a message;

[2014] A means for the device to acquire and transmit the most recent interactions to a server;

[2015] A means for the server to generate optimal stamps using context analysis artificial intelligence;

[2016] means for providing the generated stamp to a user;

[2017] A means for a user to select and send stamps;

[2018] A means for the customer service analysis device to analyze the facial expressions of customers and generate emotion data;

[2019] A means for analyzing emotion data and generating an optimal response method;

[2020] A means for providing the generated response method to the customer;

[2021] A system including:

[2022] (Claim 2)

[2023] The system according to claim 1, wherein the server displays a plurality of stamp candidates generated by the context analysis artificial intelligence.

[2024] (Claim 3)

[2025] 10. The system of claim 1, wherein the generated stamp is one of a text message, an image message, or a video message. [Explanation of symbols]

[2026] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to send a message; A means for the device to acquire and transmit the most recent interactions to a server; A means for the server to generate optimal stamps using context analysis AI; means for providing the generated stamp to a user; A means for a user to select and send stamps; A system including:

2. The system according to claim 1, wherein the server displays a plurality of stamp candidates generated by the context analysis AI.

3. 10. The system of claim 1, wherein the generated stamp is one of a text message, an image message, or a video message.

Images