System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving conversation data, a means for preprocessing the received conversation data, a means for analyzing a context and a tone on the basis of the preprocessed conversation data, a means for generating appropriate information on the basis of an analysis result, and a means for returning the generated information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Messaging platforms, a popular means of modern communication, pose a challenge in that it is difficult for users to quickly and accurately obtain the information they are looking for during conversations. In particular, search results that ignore context and tone may hinder user efficiency by failing to provide the necessary information effectively. Furthermore, it is nearly impossible with conventional technology to provide information that takes into account complex conditions such as physical condition and mood. There is a need for technology that can solve these issues and provide users with the information they are looking for accurately and quickly. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for receiving conversation data, a means for preprocessing the received conversation data, a means for analyzing the context and tone based on the preprocessed conversation data, a means for generating appropriate information based on the analysis results, and a means for replying with the generated information. In the preprocessing stage, key keywords are extracted from the conversation data and the tone is analyzed using sentiment analysis. This makes it possible to filter irrelevant messages and generate highly accurate information based on the context. Furthermore, the system takes into account complex conditions such as the user's physical condition and mood, and obtains and provides the necessary information using an external API. This allows users to obtain the most appropriate information in real time, enabling efficient decision-making.

[0006] "Conversation data" refers to digital information such as text messages, images, stamps, etc. that are exchanged as part of communication between users.

[0007] "Means for receiving" refers to the functions and processes for obtaining data from an external platform.

[0008] "Preprocessing means" refers to the initial processing or manipulation of received data to convert it into a format that is easier to analyze.

[0009] "Context and tone analysis methods" are algorithms and methods for identifying and assessing the content of conversational data and the emotions and attitudes contained within it.

[0010] "Means for generating appropriate information" refers to processes or functions that automatically create information that meets the user's needs based on the analysis results.

[0011] A "response means" is a method or system for sending a message to convey the generated information to the user.

[0012] "Filtering irrelevant messages" is the process of filtering out information that is not relevant or irrelevant to the content of a conversation.

[0013] "Sentiment analysis" is a technique for identifying and evaluating the emotional state (e.g., positive, negative, neutral) of text data.

[0014] "Obtaining information using external APIs" refers to a method of obtaining the necessary data by linking with external information sources or services.

[0015] "User's physical condition and mood" refers to the user's current physical health and mental state. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This system analyzes conversations between users in LINE groups in real time and provides the information users are looking for quickly and accurately. Below is a specific example of the system and a natural language explanation of its program.

[0038] Server-side processing

[0039] 1. Receiving Phase

[0040] The server receives new messages from the LINE Platform periodically or in real time, using notification mechanisms such as Webhooks.

[0041] The received data includes the message content, sender ID, group ID, and timestamp.

[0042] The server performs data integrity verification to filter out malformed data and spam messages.

[0043] 2. Preprocessing Phase

[0044] The server passes the received message to a text analysis engine to perform natural language processing (NLP).

[0045] The NLP engine breaks the message into tokens and extracts key keywords and phrases.

[0046] Morphological analysis is used to add part-of-speech tags to understand the meaning of the sentence.

[0047] The server uses a sentiment analysis engine to identify the emotional state of the message (positive, negative, neutral).

[0048] The server filters irrelevant and spam messages and passes only the minimum necessary data to the next processing phase.

[0049] 3. Generative AI Processing Phase

[0050] The server inputs the preprocessed data into the generative AI model.

[0051] The generative AI model takes into account context, tone, and past message history to generate the optimal response.

[0052] Use external APIs (e.g., Google Maps API, medical information API, etc.) to obtain the necessary information.

[0053] The server packages the generated response content with the metadata and prepares a reply message.

[0054] 4. Result sending phase

[0055] The server sends the generated reply message to the user's group chat via the official LINE API.

[0056] The results of the submission are logged and used for future response generation and analysis.

[0057] Terminal side processing

[0058] 1. User Input Phase

[0059] Users use the LINE app on their devices to type messages into group chats.

[0060] When the user presses the send button, the message is sent via the LINE server.

[0061] 2. Receiving Phase

[0062] The device will wait for messages from the LINE server and will receive new messages as they arrive.

[0063] Received messages are displayed on the chat screen for the user to view.

[0064] Specific examples

[0065] Example 1: Restaurant selection

[0066] User A sends a message in a group chat saying, "I want to know about some good Italian restaurants in Shinjuku."

[0067] 1. Server (receiving phase)

[0068] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0069] 2. Pre-processing phase (server)

[0070] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[0071] 3. Generative AI processing phase (server)

[0072] Based on the extracted keywords, the server uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[0073] The server generates an optimal restaurant list based on the search results.

[0074] 4. Result sending phase (server)

[0075] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0076] 5. Receiving Phase (Terminal)

[0077] The device will receive this message and display it in the group chat.

[0078] The user can review the displayed information and decide on the next action to take.

[0079] This system makes conversations within LINE groups smoother, allowing users to efficiently obtain the information they need. It also provides highly accurate information that takes into account context and tone, supporting user decision-making.

[0080] The processing flow will be explained below.

[0081] Server-side processing

[0082] Receiving Phase

[0083] Step 1:

[0084] The server receives new messages from the LINE Platform.

[0085] The received data is in JSON format and includes the message content, sender ID, group ID, and timestamp.

[0086] Step 2:

[0087] The server checks the integrity of the received message data and filters out messages that are malformed or considered spam.

[0088] Preprocessing Phase

[0089] Step 3:

[0090] The server passes the received message to a text analysis engine for natural language processing (NLP).

[0091] Step 4:

[0092] The server performs a morphological analysis and divides the message into tokens.

[0093] Morphological analysis identifies the part of speech and meaning of words.

[0094] Step 5:

[0095] The server extracts key keywords and phrases from the extracted tokens.

[0096] Step 6:

[0097] The server uses a sentiment analysis engine to evaluate the emotional state of the message (positive, negative, neutral).

[0098] Step 7:

[0099] The server filters irrelevant messages and messages identified as spam.

[0100] Generative AI processing phase

[0101] Step 8:

[0102] The server inputs the preprocessed data into the generative AI model.

[0103] Step 9:

[0104] The generative AI model takes into account context and tone to generate the most appropriate response.

[0105] Step 10:

[0106] The server calls an external API (for example, Google Maps API or medical information API) to obtain the necessary information.

[0107] Step 11:

[0108] The server packages the generated response information and prepares a reply message along with the metadata.

[0109] Result sending phase

[0110] Step 12:

[0111] The server uses the official LINE API to send the generated reply message to the user's group chat.

[0112] Step 13:

[0113] The results of the submission are logged and used for future response generation and analysis.

[0114] Terminal side processing

[0115] User Input Phase

[0116] Step 1:

[0117] The user opens the LINE app and types a message into the group chat.

[0118] Step 2:

[0119] The user presses the send button to send the message to the LINE server.

[0120] Receiving Phase

[0121] Step 3:

[0122] The device receives the message from the LINE server.

[0123] Step 4:

[0124] The device displays the received message on the chat screen.

[0125] Processing flow of specific example

[0126] Example 1: Restaurant selection

[0127] 1. Receiving Phase (Server)

[0128] Step 1:

[0129] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0130] Step 2:

[0131] The server checks the data for integrity and filters out malformed and spam.

[0132] 2. Pre-processing phase (server)

[0133] Step 3:

[0134] The server passes the received message to a text analysis engine and performs morphological analysis.

[0135] Step 4:

[0136] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant."

[0137] Step 5:

[0138] The server uses a sentiment analysis engine to evaluate the emotional state of the message as positive.

[0139] Step 6:

[0140] The server filters out irrelevant messages.

[0141] 3. Generative AI processing phase (server)

[0142] Step 7:

[0143] The server inputs the preprocessed data into the generative AI model.

[0144] Step 8:

[0145] Based on the analyzed keywords, the generated AI model uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[0146] Step 9:

[0147] The server generates a list of restaurants based on the search results.

[0148] 4. Result sending phase (server)

[0149] Step 10:

[0150] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0151] Step 11:

[0152] The results of the transmission are logged and used to generate future responses.

[0153] 5. Receiving Phase (Terminal)

[0154] Step 12:

[0155] The device receives a reply message from the LINE server.

[0156] Step 13:

[0157] The device will display this message on the chat screen.

[0158] Example 1

[0159] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] Conventional conversation analysis systems have struggled to provide users with the information they need quickly and accurately in real time. They also struggled to provide accurate information that takes into account the context and tone of the conversation, making it difficult to adequately support user decision-making.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0162] In this invention, the server includes means for receiving conversation data, means for verifying the integrity of the received conversation data, means for dividing the received conversation data into tokens and extracting key keywords and phrases, means for performing morphological analysis of the conversation data and adding part-of-speech tags, means for identifying the emotional state of the conversation data, means for inputting prompt sentences to a generative AI model based on the preprocessed conversation data, means for generating appropriate information using the generative AI model, and means for returning the generated information. This enables real-time analysis of conversation data and rapid and accurate provision of information desired by the user. Furthermore, highly accurate information provision taking into account context and tone is realized, enabling effective support for user decision-making.

[0163] "Conversation Data" is information including text messages, voice messages, and associated metadata (e.g., sender ID, group ID, timestamps, etc.) sent over a communications system.

[0164] "Integrity verification" is the process of verifying that the data received is accurate and meets the required format and fields.

[0165] "Tokenizing" refers to breaking down a text message into units of words or phrases, and is a preprocessing step in natural language processing.

[0166] "Key keywords and phrase extraction" is the process of identifying semantically significant words and expressions from text data.

[0167] "Morphological analysis" is the process of dividing text into its constituent morphemes (smallest semantic units) and adding part-of-speech tags to each of them.

[0168] "Emotional state identification" is the process of determining whether the content of analyzed text data is positive, negative, or neutral.

[0169] A "generative AI model" is an artificial intelligence algorithm that uses natural language processing technology to generate new text based on input text data.

[0170] A "prompt" is text containing context and instructions that is given as input to a generative AI model.

[0171] "Generating appropriate information" means generating answers or information that match the user's request based on the input text data and prompt sentence.

[0172] "Metadata" is information about the data itself, and in this case includes sender ID, group ID, timestamp, etc.

[0173] This system analyzes conversations between users on a messaging platform in real time and provides the information users are looking for quickly and accurately. This system is implemented using the following hardware and software.

[0174] Server-side processing

[0175] The server receives the conversation data, analyzes it, generates appropriate information, and sends it back to the user. The specific technologies used for processing include a text analysis engine, a sentiment analysis engine, and a generative AI model.

[0176] 1. Receiving and verifying conversation data

[0177] The server receives messages in real time from a messaging platform (e.g., a messaging application API) using the Webhook function. The received data includes the message content, sender ID, group ID, and timestamp.

[0178] 2. Data Preprocessing

[0179] The server verifies the integrity of the received data (e.g., JSON format validation) and filters out any invalid data or spam. It then uses Google NLP libraries to split the text message into tokens and extract key keywords and phrases. It also performs morphological analysis to add part-of-speech tags and identifies emotional states using IBM Watson's sentiment analysis API.

[0180] 3. Generative AI processing

[0181] The server inputs a prompt sentence into a generative AI model (e.g., OpenAI's GPT-3) based on the preprocessed data. For example, the prompt sentence for "I want to know about good Italian restaurants in Shinjuku" would be "Good Italian restaurants in Shinjuku." The generative AI model uses this prompt sentence to generate an appropriate response and, if necessary, retrieves related information from external sources (e.g., map information APIs).

[0182] 4. Sending a Message

[0183] The server formats the generated response content into a LINE message and packages it with metadata (sender ID, group ID, timestamp, etc.). The server then sends the generated message to the user via the official LINE API. The transmission results are recorded in a database and used for future response generation and analysis.

[0184] Terminal side processing

[0185] Users type and receive messages through messaging apps on their devices.

[0186] 1. Enter and send a message

[0187] Users can send messages by typing them into a group chat using a messaging app on their device and pressing the send button, which then passes through the messaging platform's servers.

[0188] 2. Receiving and viewing messages

[0189] The device receives new messages from the messaging platform and displays them in real time on the chat screen, allowing the user to review the received information and decide on the next action.

[0190] Specific examples

[0191] For example, if a user sends a message to a group chat saying, "I want to know about a good Italian restaurant in Shinjuku," the server processes it as follows:

[0192] 1. Receiving conversation data

[0193] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0194] 2. Data Preprocessing

[0195] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[0196] 3. Generative AI processing

[0197] The server generates a prompt sentence, "Delicious Italian restaurants in Shinjuku," and inputs it into the generative AI model. It then uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku and creates a list of the best restaurants.

[0198] 4. Sending a Message

[0199] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0200] 5. Receiving Messages

[0201] The device will receive this message and display it in the group chat, allowing the user to review the displayed information and decide on their next action.

[0202] This makes conversations within LINE groups smoother, allowing users to efficiently obtain the information they need. It also makes it possible to provide highly accurate information that takes into account context and tone, supporting user decision-making.

[0203] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0204] Step 1:

[0205] The server receives conversation data from the messaging platform in real time. Specifically, it uses a webhook to receive data such as the message, sender ID, group ID, and timestamp. The input is raw data from the messaging platform, and the output is the raw data received by the server. The server securely receives the data using SSL / TLS.

[0206] Step 2:

[0207] The server checks the integrity of the data it receives. It validates the JSON format and checks that required fields are included. Malformed data and spam messages are filtered out at this stage. The input is the raw data received in step 1, and the output is the valid, consistent data.

[0208] Step 3:

[0209] The server divides the consistent data into tokens and extracts key keywords and phrases. It performs text analysis using the Google NLP library. For example, from a message such as "I want to know about good Italian restaurants in Shinjuku," it extracts keywords such as "Shinjuku," "delicious," and "Italian restaurant." The input is the data whose consistency was confirmed in step 2, and the output is the tokenized text data and extracted keywords.

[0210] Step 4:

[0211] The server performs morphological analysis and tags the tokens with part-of-speech tags, using the Google NLP library or a similar morphological analysis engine. This generates the basis for understanding the meaning of the sentence. The input is the tokenized data from step 3, and the output is the morphologically analyzed data with part-of-speech tags.

[0212] Step 5:

[0213] The server uses the acquired keywords and the morphological analysis results to identify the emotional state. It uses IBM Watson's sentiment analysis API to determine whether the text data corresponds to positive, negative, or neutral. The input is the data morphologically analyzed in step 4, and the output is the data with the emotional state added.

[0214] Step 6:

[0215] The server inputs a prompt sentence to the generative AI model based on the preprocessed data. The prompt sentence is generated from the preprocessed data and takes the form of, for example, "Delicious Italian restaurants in Shinjuku." The generative AI model uses OpenAI's GPT-3. The input is the preprocessed data and the generated prompt sentence, and the output is the response text generated by the generative AI model.

[0216] Step 7:

[0217] The server integrates the generated response text with additional information obtained from external sources as needed. For example, it may use the Google Maps API to obtain a list of highly rated Italian restaurants near Shinjuku and add it to the generated response. The input is the response text generated in step 6 and the information obtained from the external API, and the output is the integrated final response message.

[0218] Step 8:

[0219] The server sends the generated response message to the user via the official LINE API. The message contains metadata such as the sender ID, group ID, and timestamp. The sending result is recorded as a log in the database and used for future response generation and analysis. The input is the final response message generated in step 7, and the output is the message sent to the user and its log.

[0220] Step 9:

[0221] The device receives new messages from the LINE server. Received messages are displayed on the chat screen in real time so that the user can check them. The input is the message delivered from the LINE server, and the output is the message displayed on the chat screen.

[0222] Step 10:

[0223] The user checks the displayed information and decides on the next action. For example, they may select a specific restaurant from a list of suggested restaurants and send a new message. The input is the information displayed on the terminal, and the output is a new message as the user's next action.

[0224] (Application example 1)

[0225] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0226] The automated response systems on existing chat platforms faced the challenge of being unable to provide accurate information based on users' specific needs. They also lacked a mechanism for recommending entertainment content and specialized information desired by users in real time. This made it difficult for users to obtain high-quality information and content without any effort.

[0227] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0228] In this invention, the server includes means for receiving conversation data, means for preprocessing the received conversation data, means for analyzing the context and tone based on the preprocessed conversation data, means for generating appropriate information based on the analysis results, means for returning the generated information, means for recommending content based on the preprocessed conversation data, and means for accessing an external database to obtain recommended content according to the received conversation data. This enables the server to quickly and accurately provide information desired by the user and to recommend related entertainment content in real time.

[0229] "Conversation Data" is a collection of text messages exchanged in communications between users.

[0230] The "receiving means" is a mechanism by which the server receives conversation data from the outside.

[0231] "Preprocessing means" refers to the process of preparing received conversation data in a format suitable for analysis and processing.

[0232] "Means of analyzing context" refers to technology for understanding the linguistic structure and meaning within conversation data.

[0233] "Tone analysis" refers to techniques for determining the emotion and intent of conversational data.

[0234] "Means of generating appropriate information" is the process of creating answers and information that best suit the user based on the analysis results.

[0235] A "return mechanism" is a mechanism for sending generated information back to the user.

[0236] "Content recommendation tools" are mechanisms that suggest relevant information and entertainment based on the user's needs and the content of the conversation.

[0237] "Means for accessing external databases" refers to methods for obtaining required data from external sources.

[0238] A "generative AI model" is an artificial intelligence model used to analyze conversation data and generate appropriate replies and recommended content.

[0239] A "prompt" is text containing a question or instruction that is input to a generative AI model.

[0240] In this invention, a system is provided in which a server, a terminal, and a user cooperate to generate and provide information. A specific embodiment of the system is described below.

[0241] Server-side processing

[0242] 1. Receiving Phase

[0243] The server receives conversation data from an external source. To do this, it uses the communication platform's API to obtain metadata such as message content, sender ID, group ID, and timestamp. The specific software used in this step is the official LINE API.

[0244] 2. Preprocessing Phase

[0245] The server preprocesses the received conversation data using the NLPProcessor library, splitting the conversation data into tokens and extracting key keywords and phrases. It also performs sentiment analysis to determine the tone of the message.

[0246] 3. Generative AI Processing Phase

[0247] The server generates appropriate information using a generative AI model based on the preprocessed data, and then makes an API request to access an external database to obtain the necessary recommendation information, using the AIRecommender library or an external service API (e.g., a movie database API).

[0248] 4. Result sending phase

[0249] The generated information and recommended content will be sent back to the user via the official LINE API, where the user can receive and view it on their device.

[0250] Terminal side processing

[0251] 1. User Input Phase

[0252] A user can type and send a message through a chat app on their device, which is then sent to the server and processed as described above.

[0253] 2. Receiving Phase

[0254] The device receives the information sent from the server and displays it on the chat screen. The user can then take the next action based on the displayed information.

[0255] Specific examples

[0256] Example 1: Movie Recommendations

[0257] If a user sends a message to a group chat saying "Tell me some good movies," the flow is as follows:

[0258] 1. The server receives a message saying, "Tell me some good movies."

[0259] 2. In the preprocessing phase, the keywords "interesting" and "movie" are extracted and sentiment analysis is performed.

[0260] 3. In the generation AI processing phase, we use the AIRecommender library to retrieve a list of highly rated movies from an external API.

[0261] 4. In the result sending phase, send the movie list along with the message "Here are some recommended movies!"

[0262] Prompt Sentence Examples

[0263] "Tell me some interesting movies you've seen recently."

[0264] In this way, the system can provide appropriate information and content that meets the user's needs.

[0265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0266] Step 1:

[0267] The server receives conversation data from an external chat platform. Specifically, it uses a communication API such as the official LINE API to obtain metadata such as message content, sender ID, group ID, and timestamp. The input data is the new message itself, and the output data is the raw message data to be passed to the preprocessing phase.

[0268] Step 2:

[0269] The server preprocesses the received conversation data using the NLPProcessor library. Specifically, it divides the conversation data into tokens, performs morphological analysis to extract key keywords and phrases, and performs sentiment analysis to determine the tone of the message. The input data is the raw message data, and the output data is the analyzed keywords and tone information.

[0270] Step 3:

[0271] The server inputs the preprocessed data into the generative AI model to generate appropriate information. It uses the AIRecommender library to obtain recommended content. If necessary, it accesses external databases to collect additional information (e.g., movie or music lists). The input data is the analyzed keywords and tone information, and the output data is the generated response message.

[0272] Step 4:

[0273] The server formats the generated information and prepares it as a reply message. Specifically, it converts it into a format that is easy for the user to understand and includes meta information such as images and links if necessary. The input data is the generated response message, and the output data is the complete message package to be sent via the official LINE API.

[0274] Step 5:

[0275] The server uses the official LINE API to send the generated message to the user. The sending result is recorded in a log and used for future analysis and response improvement. The input data is the reply message package, and the output data is log information indicating that the message was sent to the user.

[0276] Step 6:

[0277] The user's device receives new messages through the LINE app and displays them on the chat screen. The user can check the displayed message and take further action. The input data is the reply message from the server, and the output data is the text displayed on the chat screen and (if necessary) meta information such as images and links.

[0278] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0279] This system analyzes conversations between users in LINE groups in real time, accurately grasps the user's emotions and context, and then provides the information the user is looking for quickly and appropriately. Below is a specific implementation of the system and a natural language explanation of its program.

[0280] Server-side processing

[0281] 1. Receiving Phase

[0282] The server receives new messages from the LINE Platform in real time using notification mechanisms such as Webhooks.

[0283] The received data includes the message content, sender ID, group ID, and timestamp.

[0284] The server checks the integrity of the data and filters out messages that are malformed or considered spam.

[0285] 2. Preprocessing Phase

[0286] The server passes the received message to a text analysis engine for natural language processing (NLP).

[0287] The NLP engine breaks the message into tokens and extracts key keywords and phrases.

[0288] Morphological analysis is used to add part-of-speech tags to understand the meaning of the sentence.

[0289] The server uses an emotion engine to identify the emotional state of the message (positive, negative, neutral).

[0290] The server filters irrelevant messages and messages identified as spam, passing only the minimum necessary data to the next processing phase.

[0291] 3. Generative AI Processing Phase

[0292] The server inputs the preprocessed data into the generative AI model.

[0293] The generative AI model takes into account context, tone, past message history, and the results of the emotion engine analysis to generate the optimal response.

[0294] Use external APIs (e.g., Google Maps API or medical information API) to obtain the necessary information.

[0295] The server packages the generated response content and prepares a reply message along with the metadata.

[0296] 4. Result sending phase

[0297] The server sends the generated reply message to the user's group chat via the official LINE API.

[0298] The results of the submission are logged and used for future response generation and analysis.

[0299] Terminal side processing

[0300] 1. User Input Phase

[0301] Users use the LINE app on their devices to type messages into group chats.

[0302] When the user presses the send button, the message is sent via the LINE server.

[0303] 2. Receiving Phase

[0304] The device receives the message from the LINE server.

[0305] Received messages are displayed on the chat screen for the user to view.

[0306] Specific examples

[0307] Example 1: Restaurant selection

[0308] User A sends a message in a group chat saying, "I want to know about some good Italian restaurants in Shinjuku."

[0309] 1. Server (receiving phase)

[0310] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0311] 2. Pre-processing phase (server)

[0312] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[0313] 3. Generative AI processing phase (server)

[0314] Based on the extracted keywords, the server uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[0315] The server generates an optimal restaurant list based on the search results and adjusts the reply content to reflect the positive sentiment of the message.

[0316] 4. Result sending phase (server)

[0317] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0318] 5. Receiving Phase (Terminal)

[0319] The device will receive this message and display it in the group chat.

[0320] The user can review the displayed information and decide on the next action to take.

[0321] Example 2: Health advice

[0322] User B posts in a group chat, "I have a headache, what should I do?"

[0323] 1. Server (receiving phase)

[0324] The server receives the message "I have a headache, what should I do?"

[0325] 2. Pre-processing phase (server)

[0326] The server extracts the keyword "headache" and determines the emotional state of the message as negative.

[0327] 3. Generative AI processing phase (server)

[0328] Based on the extracted keywords, the server uses a medical information API to obtain the most appropriate advice.

[0329] The server takes into account the negative emotional state and generates a response such as, "If your headache persists, I recommend you see a doctor as soon as possible."

[0330] 4. Result sending phase (server)

[0331] The server replies with the message, "If your headache persists, we recommend that you see a doctor as soon as possible. First, make sure you drink plenty of fluids and try lying down for a while."

[0332] 5. Receiving Phase (Terminal)

[0333] The device will receive this message and display it in the group chat.

[0334] The user can review the displayed information and decide on an action, such as visiting a medical institution.

[0335] This system makes conversations within LINE groups smoother, allowing users to quickly obtain accurate and relevant information that takes into account context and emotional state.

[0336] The processing flow will be explained below.

[0337] Server-side processing

[0338] Receiving Phase

[0339] Step 1:

[0340] The server receives new messages from the LINE Platform.

[0341] Step 2:

[0342] The server analyzes data such as the content, sender ID, group ID, and timestamp of the received message.

[0343] Step 3:

[0344] The server checks the data for integrity and filters out malformed and spam messages.

[0345] Preprocessing Phase

[0346] Step 4:

[0347] The server passes the received message to a text analysis engine for natural language processing (NLP).

[0348] Step 5:

[0349] The server performs morphological analysis and breaks the message into tokens (e.g., words).

[0350] Step 6:

[0351] The server extracts key keywords and phrases from the split tokens.

[0352] Step 7:

[0353] The server uses an emotion engine to analyze the emotional state of the message (positive, negative, neutral).

[0354] Step 8:

[0355] The server filters out irrelevant and spam messages.

[0356] Generative AI processing phase

[0357] Step 9:

[0358] The server inputs the preprocessed data into the generative AI model.

[0359] Step 10:

[0360] The generative AI model takes into account the context, tone, and analysis results of the emotion engine to generate an appropriate response.

[0361] Step 11:

[0362] The server uses external APIs (e.g., Google Maps API or medical information API) to obtain the necessary information.

[0363] Step 12:

[0364] The server packages the generated response content with the metadata and prepares a reply message.

[0365] Result sending phase

[0366] Step 13:

[0367] The server uses the official LINE API to send the generated reply message to the user's group chat.

[0368] Step 14:

[0369] The server logs the results of the transmission and uses them for future response generation and analysis.

[0370] Terminal side processing

[0371] User Input Phase

[0372] Step 1:

[0373] The user opens the LINE app and types a message into the group chat.

[0374] Step 2:

[0375] The user presses the send button to send the message to the LINE server.

[0376] Receiving Phase

[0377] Step 3:

[0378] The device receives a reply message from the LINE server.

[0379] Step 4:

[0380] The device displays the received message on the chat screen.

[0381] Processing flow of specific example

[0382] Example 1: Restaurant selection

[0383] Step 1: Server (receiving phase)

[0384] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0385] Step 2: Server (Verify Data Integrity)

[0386] The server checks the data for integrity and filters out malformed and spam.

[0387] Step 3: Preprocessing Phase (Server)

[0388] The server passes the received message to a text analysis engine.

[0389] Step 4: Perform morphological analysis

[0390] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant."

[0391] Step 5: Analyze your emotional state

[0392] The server uses an emotion engine to evaluate the positive emotional state of the message.

[0393] Step 6: Filtering

[0394] The server filters out irrelevant messages.

[0395] Step 7: Generative AI processing phase (input of preprocessed data)

[0396] The server inputs the preprocessed data into the generative AI model.

[0397] Step 8: Generative AI model generates appropriate responses

[0398] Based on the analyzed keywords, the generated AI model uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[0399] Step 9: Generate response content based on search results

[0400] The server generates an optimal restaurant list based on the search results.

[0401] Step 10: Result sending phase (preparing the reply message)

[0402] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0403] Step 11: Logging the transmission results

[0404] The server logs the results of the transmission.

[0405] Example 2: Health advice

[0406] Step 1: Server (receiving phase)

[0407] The server receives the message "I have a headache, what should I do?"

[0408] Step 2: Server (Verify Data Integrity)

[0409] The server checks the data for integrity and filters out malformed and spam.

[0410] Step 3: Preprocessing Phase (Server)

[0411] The server passes the received message to a text analysis engine.

[0412] Step 4: Perform morphological analysis

[0413] The server extracts the keyword "headache."

[0414] Step 5: Analyze your emotional state

[0415] The server uses an emotion engine to evaluate the negative emotional state of the message.

[0416] Step 6: Filtering

[0417] The server filters out irrelevant messages.

[0418] Step 7: Generative AI processing phase (input of preprocessed data)

[0419] The server inputs the preprocessed data into the generative AI model.

[0420] Step 8: Generative AI model generates appropriate responses

[0421] Based on the analyzed keywords, the generative AI model uses a medical information API to obtain optimal advice.

[0422] Step 9: Generate response content

[0423] The server takes into account the negative emotional state and generates a response such as, "If your headache persists, I recommend you see a doctor as soon as possible."

[0424] Step 10: Result sending phase (preparing the reply message)

[0425] The server replies with the message, "If your headache persists, we recommend that you see a doctor as soon as possible. First, make sure you drink plenty of fluids and try lying down for a while."

[0426] Step 11: Logging the transmission results

[0427] The server logs the results of the transmission.

[0428] Example 2

[0429] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0430] Conversation analysis systems in conventional communication platforms have difficulty accurately grasping the context and emotional state of users' conversations, and are therefore unable to provide users with the information they require quickly and appropriately. For this reason, there is a need to achieve more accurate natural language processing and emotion analysis in order to improve user satisfaction.

[0431] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving conversation data from a communication platform in real time, means for verifying the integrity of the received conversation data and filtering out invalid and spam data, means for dividing the received conversation data into tokens using a natural language processing engine and extracting key keywords and phrases, means for analyzing the context and emotional state of the conversation data using morphological analysis and an emotion engine, means for generating appropriate prompt sentences based on the analysis results and generating information using a generative AI model, means for acquiring supplementary information using an external information acquisition API as needed, means for packaging the generated information and preparing a reply message together with metadata, and means for sending the generated reply message via the API of the communication platform. This makes it possible to accurately grasp the context and emotional state of the conversation between users and quickly and appropriately provide the desired information.

[0432] "Conversation Data" means text messages exchanged between Users via a communications platform.

[0433] "Communications Platform" means an online service that enables users to send and receive text messages.

[0434] "Real-time" refers to processing or responding to an event almost as soon as it occurs.

[0435] "Integrity" refers to data that is accurate, consistent, and free from malformed or malformed content.

[0436] "Spam data" refers to a large volume of messages or information that is inappropriate or irrelevant to the user.

[0437] "Natural language processing engine" refers to a computer program or system for analyzing and understanding human language.

[0438] A "token" refers to a unit (such as a word or phrase) that is divided into text for analysis in natural language processing.

[0439] "Keywords" refer to words or phrases that have important meanings in conversation data.

[0440] "Morphological analysis" refers to a method of analyzing the structure of a sentence by breaking it down into parts of speech and assigning part-of-speech tags to each word.

[0441] An "emotion engine" refers to a program or system that analyzes and identifies a user's emotional state from text data.

[0442] A "prompt" refers to an instruction or question that is input into a generative AI model.

[0443] A "generative AI model" refers to an artificial intelligence model that generates responses or information in natural language based on a given prompt.

[0444] "External Information Acquisition API" refers to an application program interface for acquiring data from external information sources.

[0445] "Metadata" refers to data that describes the attributes, structure, meaning, etc. of data.

[0446] "Reply Message" refers to a response message to a user that includes generated information.

[0447] This invention provides a system that analyzes conversations between users on a communication platform, understands the user's emotions and context, and then provides prompt and appropriate information. The main components of the system include a server and a terminal.

[0448] Server-side processing

[0449] The server includes hardware and software for realizing the following main functions:

[0450] Interface with communication platforms: Use Webhooks to receive messages in real time from communication platforms (e.g., LINE, WhatsApp, etc.).

[0451] Data integrity check: Checks the integrity of received messages, including formatting errors and filtering out spam messages.

[0452] Natural language processing engine: Breaks incoming messages into tokens and extracts key keywords and phrases. Specifically, it uses an NLP library (e.g., spaCy or NLTK).

[0453] Morphological analysis and sentiment engine: Morphological analysis is used to analyze the structure of the sentence, and a sentiment engine (e.g., a sentiment analysis library) is used to identify the emotional state of the message (positive, negative, neutral).

[0454] Generative AI model: Generate a prompt sentence based on the analysis results and input it into a generative AI model (e.g., GPT-3 or BERT) to generate an appropriate response.

[0455] External information acquisition: If necessary, external APIs (e.g., map information APIs or medical information APIs) are used to acquire supplementary information.

[0456] Generate and send a reply message: The generated response content is packaged with metadata and the reply message is sent using the API of the communication platform.

[0457] Terminal side processing

[0458] The terminal (user's device) is used by the user to enter and send messages and to receive and display new messages through the communication platform application.

[0459] User input: A user opens a communication platform app on their device, types a text message, and sends it to the group chat.

[0460] Receiving and displaying messages: The terminal receives messages from the communication platform server and displays them on the chat screen.

[0461] Specific examples

[0462] Example 1: Restaurant selection

[0463] When User A sends a message saying, "I want to know about a good Italian restaurant in Shinjuku"

[0464] 1. The server receives this message from the communication platform and checks the data integrity.

[0465] 2. The server uses a natural language processing engine to extract the keywords "Shinjuku," "delicious," and "Italian restaurant," and determines that the message has a positive sentiment.

[0466] 3. Based on the extracted keywords, the server generates a prompt sentence such as, "Please tell me about some highly rated Italian restaurants around Shinjuku," and inputs it into the generative AI model.

[0467] 4. The generative AI model obtains information about Italian restaurants in Shinjuku from an external information acquisition API (e.g., map information API) and generates the optimal restaurant list.

[0468] 5. The server generates and sends a reply message saying, "Here are some delicious Italian restaurants in Shinjuku!"

[0469] 6. The terminal receives this message and displays it to User A.

[0470] Example 2: Health advice

[0471] If User B sends a message saying, "I have a headache. What should I do?"

[0472] 1. The server receives this message and verifies the data integrity.

[0473] 2. The server extracts the keyword "headache" and determines that the message has a negative emotion.

[0474] 3. The server generates a prompt sentence, "Please tell me what to do if my headache persists," and inputs it into the generative AI model.

[0475] 4. The generative AI model obtains appropriate advice from an external information acquisition API (e.g., medical information API) and generates a response message such as, "If your headache persists, we recommend that you see a doctor as soon as possible."

[0476] 5. The server sends this response message to User B.

[0477] 6. The device receives this message and displays it to User B.

[0478] The system allows users to quickly obtain accurate and relevant information that takes into account the context and emotional state of conversations on communication platforms.

[0479] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0480] Step 1:

[0481] The server receives new messages from the communication platform in real time. Specifically, notifications are sent using Webhooks. The input is the message data sent from the communication platform, and the output is saving this as pre-stage data for consistency check. Specifically, when a user sends a message, the message is forwarded to the server.

[0482] Step 2:

[0483] The server checks the integrity of the received message. The input is the message data received in step 1, and it detects and filters out any malformed or spam messages. The output is the clean message data after the integrity check. Specifically, it checks whether the message format is correct and whether the content is spam.

[0484] Step 3:

[0485] The server passes the message after integrity check to a natural language processing engine, which splits it into tokens and extracts key keywords and phrases. The input is clean message data, and the output is tokenized data and extracted keywords and phrases. Specifically, it uses an NLP library to tokenize the text and extract keywords.

[0486] Step 4:

[0487] The server passes the extracted tokens and keywords to a morphological analysis engine and a sentiment engine to analyze the context and emotional state. The input is the output data from step 3, and the output is the analyzed context information and emotional state (positive, negative, neutral). Specifically, the analysis is performed using a morphological analysis library and a sentiment analysis library.

[0488] Step 5:

[0489] The server generates an appropriate prompt sentence based on the analysis results and inputs it into the generative AI model. The input is contextual information and emotional state, and the output is response data from the generative AI model. Specifically, the server uses the analysis results to construct a prompt sentence and inputs it into the generative AI model. Example: "Please tell me about some highly rated Italian restaurants around Shinjuku."

[0490] Step 6:

[0491] The server obtains supplementary information as needed using an external information acquisition API (e.g., map information API, medical information API) based on the output data of the generative AI model. The input is the response data of the generative AI model, and the output is the complete response data including the supplementary information. Specifically, the server makes an API call and obtains the required information.

[0492] Step 7:

[0493] The server packages the response data, including the complementary information, with metadata to generate a reply message. The input is the generative AI model's response data, including the complementary information, and the output is the final reply message. Specifically, the server formats the response content and adds metadata.

[0494] Step 8:

[0495] The server sends the generated reply message through the API of the communication platform. The input is the final reply message and the output is the sending result. The specific operation is to send the message using the API of the communication platform and record the sending result in a log.

[0496] Step 9:

[0497] The terminal receives a reply message from the communication platform and displays it in the group chat. The input is the reply message from the communication platform, and the output is display data in a format that can be viewed by the user. The specific operation is to display the received message on the chat screen.

[0498] Step 10:

[0499] The user checks the reply message displayed on the device and decides on the next action. The input is the message displayed on the chat screen, and the output is the user's next action. In concrete terms, the user decides on an action based on the information. Examples: checking restaurant information and making a reservation, following health advice.

[0500] (Application example 2)

[0501] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0502] Currently, many food delivery services require users to access apps or websites to search for food, making it difficult for them to provide appropriate suggestions that reflect the user's sentiment and context. As a result, it can take a long time for users to decide what to order. Furthermore, the lack of automatic and appropriate menu suggestions based on conversations on chat platforms such as LINE does not improve the food delivery user experience. A solution to these issues is needed.

[0503] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving conversation data, means for preprocessing the received conversation data, means for analyzing context and tone based on the preprocessed conversation data, means for generating appropriate information based on the analysis results, means for returning the generated information, and means for suggesting food delivery menus. This enables the user to quickly and appropriately receive food delivery menu suggestions based on conversations on chat platforms such as LINE.

[0504] "Conversation Data" refers to the text and statements sent and received by Users on the chat platform.

[0505] "Means for receiving" refers to a function or interface that allows the server to obtain data from the outside.

[0506] "Preprocessing means" refers to the processes and algorithms that format acquired conversation data and extract necessary items to make it easier to analyze.

[0507] "Means for analyzing context and tone" refers to methods for understanding the meaning and sentiment of conversational data using natural language processing and sentiment analysis techniques.

[0508] "Generating means" refers to the processes and algorithms used to generate the necessary information and recommendations based on the analysis results.

[0509] "Means for replying" refers to the interface and pathway for sending generated information and suggestions to the user.

[0510] "Means for suggesting food delivery menus" refers to systems and processes for suggesting appropriate food and beverage ordering menus based on keywords and sentiments extracted from conversation data.

[0511] This invention is a system that suggests food delivery menus in real time based on conversations between users on chat platforms such as LINE. This system operates on both the server and the device side. The server side analyzes the conversation data and performs a series of processes to generate and return appropriate information. On the device side, messages are sent and received through a user interface.

[0512] Server-side processing

[0513] The server first receives new messages from the LINE Platform in real time using a notification mechanism such as a webhook. The received data includes the message content, sender ID, group ID, and timestamp. Next, the server preprocesses the received messages and performs natural language processing (NLP). This preprocessing phase involves tokenization, keyword extraction, and morphological analysis. It also uses a sentiment analysis engine to identify the emotional state of the message and filter out irrelevant messages and spam.

[0514] Once preprocessed, the data is input into a generative AI model. The generative AI model generates a response to suggest the optimal food delivery menu, taking into account the context, past message history, and sentiment analysis results. During this process, it may obtain necessary information using external APIs (e.g., food delivery APIs). The generated response is sent to the user's group chat via the official LINE API. This series of processes allows the system to suggest appropriate food delivery menus in real time based on the user's conversations on the chat platform.

[0515] Terminal side processing

[0516] A user uses the LINE app on their device to enter a message into a group chat. When the user presses the send button, the message is sent via the LINE server. The received message is displayed on the chat screen so that the user can view it. Reply messages from the server are also received in the same way and displayed on the chat screen.

[0517] Specific examples

[0518] 1. Lunch consultation

[0519] User A: "What are you having for lunch?"

[0520] User B: "I like Italian food."

[0521] Server: "This pizza place is great! (Rating: 4.5)"

[0522] To achieve this process, the server uses the following software and hardware: First, a web framework such as Flask is used to receive and process webhook notifications, then TextBlob and other natural language processing libraries are used for sentiment analysis and keyword extraction, and the official LINE API is used to send the final reply message.

[0523] Prompt Sentence Examples

[0524] "A user says, 'I'd like Italian food.' Please suggest delivery menu items based on this message."

[0525] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0526] Step 1:

[0527] The server receives new messages from the LINE Platform in real time. The received data includes the message content, sender ID, group ID, and timestamp. It uses the Webhook function to receive notifications from the LINE server. The input is new message data, and the output is the received message information.

[0528] Step 2:

[0529] The server preprocesses the received message. Specifically, it performs natural language processing (NLP) using the TextBlob library to split the message into tokens and extract key keywords and phrases. At the same time, it performs sentiment analysis to identify positive, negative, and neutral emotional states. The input is the received message information, and the output is the analysis results, extracted keywords, and emotional states.

[0530] Step 3:

[0531] The server inputs the preprocessed data into a generative AI model, which takes into account the context, tone, past message history, and sentiment analysis results to generate the optimal response. This process also involves using a food delivery API to obtain relevant menu information. The inputs are the analysis results and extracted keywords, and the output is the generated response message.

[0532] Step 4:

[0533] The server packages the generated response message and sends it to the user's group chat via the official LINE API. The input is the generated response message, and the output is the sent reply message. This allows users to receive appropriate food delivery menu suggestions in real time in the group chat.

[0534] Step 5:

[0535] The user checks the received message using the LINE app on their device. The device displays the message received from the LINE server on the chat screen. The input is the message received from the LINE server, and the output is the message displayed on the user's device. This allows the user to decide on their next action and order food delivery.

[0536] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0537] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0538] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0539] [Second embodiment]

[0540] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0541] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0542] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0543] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0544] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0545] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0546] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0547] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0548] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0549] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0550] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0551] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0552] This system analyzes conversations between users in LINE groups in real time and provides the information users are looking for quickly and accurately. Below is a specific example of the system and a natural language explanation of its program.

[0553] Server-side processing

[0554] 1. Receiving Phase

[0555] The server receives new messages from the LINE Platform periodically or in real time, using notification mechanisms such as Webhooks.

[0556] The received data includes the message content, sender ID, group ID, and timestamp.

[0557] The server performs data integrity verification to filter out malformed data and spam messages.

[0558] 2. Preprocessing Phase

[0559] The server passes the received message to a text analysis engine to perform natural language processing (NLP).

[0560] The NLP engine breaks the message into tokens and extracts key keywords and phrases.

[0561] Morphological analysis is used to add part-of-speech tags to understand the meaning of the sentence.

[0562] The server uses a sentiment analysis engine to identify the emotional state of the message (positive, negative, neutral).

[0563] The server filters irrelevant and spam messages and passes only the minimum necessary data to the next processing phase.

[0564] 3. Generative AI Processing Phase

[0565] The server inputs the preprocessed data into the generative AI model.

[0566] The generative AI model takes into account context, tone, and past message history to generate the optimal response.

[0567] Use external APIs (e.g., Google Maps API, medical information API, etc.) to obtain the necessary information.

[0568] The server packages the generated response content with the metadata and prepares a reply message.

[0569] 4. Result sending phase

[0570] The server sends the generated reply message to the user's group chat via the official LINE API.

[0571] The results of the submission are logged and used for future response generation and analysis.

[0572] Terminal side processing

[0573] 1. User Input Phase

[0574] Users use the LINE app on their devices to type messages into group chats.

[0575] When the user presses the send button, the message is sent via the LINE server.

[0576] 2. Receiving Phase

[0577] The device will wait for messages from the LINE server and will receive new messages as they arrive.

[0578] Received messages are displayed on the chat screen for the user to view.

[0579] Specific examples

[0580] Example 1: Restaurant selection

[0581] User A sends a message in a group chat saying, "I want to know about some good Italian restaurants in Shinjuku."

[0582] 1. Server (receiving phase)

[0583] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0584] 2. Pre-processing phase (server)

[0585] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[0586] 3. Generative AI processing phase (server)

[0587] Based on the extracted keywords, the server uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[0588] The server generates an optimal restaurant list based on the search results.

[0589] 4. Result sending phase (server)

[0590] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0591] 5. Receiving Phase (Terminal)

[0592] The device will receive this message and display it in the group chat.

[0593] The user can review the displayed information and decide on the next action to take.

[0594] This system makes conversations within LINE groups smoother, allowing users to efficiently obtain the information they need. It also provides highly accurate information that takes into account context and tone, supporting user decision-making.

[0595] The processing flow will be explained below.

[0596] Server-side processing

[0597] Receiving Phase

[0598] Step 1:

[0599] The server receives new messages from the LINE Platform.

[0600] The received data is in JSON format and includes the message content, sender ID, group ID, and timestamp.

[0601] Step 2:

[0602] The server checks the integrity of the received message data and filters out messages that are malformed or considered spam.

[0603] Preprocessing Phase

[0604] Step 3:

[0605] The server passes the received message to a text analysis engine for natural language processing (NLP).

[0606] Step 4:

[0607] The server performs a morphological analysis and divides the message into tokens.

[0608] Morphological analysis identifies the part of speech and meaning of words.

[0609] Step 5:

[0610] The server extracts key keywords and phrases from the extracted tokens.

[0611] Step 6:

[0612] The server uses a sentiment analysis engine to evaluate the emotional state of the message (positive, negative, neutral).

[0613] Step 7:

[0614] The server filters irrelevant messages and messages identified as spam.

[0615] Generative AI processing phase

[0616] Step 8:

[0617] The server inputs the preprocessed data into the generative AI model.

[0618] Step 9:

[0619] The generative AI model takes into account context and tone to generate the most appropriate response.

[0620] Step 10:

[0621] The server calls an external API (for example, Google Maps API or medical information API) to obtain the necessary information.

[0622] Step 11:

[0623] The server packages the generated response information and prepares a reply message along with the metadata.

[0624] Result sending phase

[0625] Step 12:

[0626] The server uses the official LINE API to send the generated reply message to the user's group chat.

[0627] Step 13:

[0628] The results of the submission are logged and used for future response generation and analysis.

[0629] Terminal side processing

[0630] User Input Phase

[0631] Step 1:

[0632] The user opens the LINE app and types a message into the group chat.

[0633] Step 2:

[0634] The user presses the send button to send the message to the LINE server.

[0635] Receiving Phase

[0636] Step 3:

[0637] The device receives the message from the LINE server.

[0638] Step 4:

[0639] The device displays the received message on the chat screen.

[0640] Processing flow of specific example

[0641] Example 1: Restaurant selection

[0642] 1. Receiving Phase (Server)

[0643] Step 1:

[0644] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0645] Step 2:

[0646] The server checks the data for integrity and filters out malformed and spam.

[0647] 2. Pre-processing phase (server)

[0648] Step 3:

[0649] The server passes the received message to a text analysis engine and performs morphological analysis.

[0650] Step 4:

[0651] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant."

[0652] Step 5:

[0653] The server uses a sentiment analysis engine to evaluate the emotional state of the message as positive.

[0654] Step 6:

[0655] The server filters out irrelevant messages.

[0656] 3. Generative AI processing phase (server)

[0657] Step 7:

[0658] The server inputs the preprocessed data into the generative AI model.

[0659] Step 8:

[0660] Based on the analyzed keywords, the generated AI model uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[0661] Step 9:

[0662] The server generates a list of restaurants based on the search results.

[0663] 4. Result sending phase (server)

[0664] Step 10:

[0665] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0666] Step 11:

[0667] The results of the transmission are logged and used to generate future responses.

[0668] 5. Receiving Phase (Terminal)

[0669] Step 12:

[0670] The device receives a reply message from the LINE server.

[0671] Step 13:

[0672] The device will display this message on the chat screen.

[0673] Example 1

[0674] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] Conventional conversation analysis systems have struggled to provide users with the information they need quickly and accurately in real time. They also struggled to provide accurate information that takes into account the context and tone of the conversation, making it difficult to adequately support user decision-making.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0677] In this invention, the server includes means for receiving conversation data, means for verifying the integrity of the received conversation data, means for dividing the received conversation data into tokens and extracting key keywords and phrases, means for performing morphological analysis of the conversation data and adding part-of-speech tags, means for identifying the emotional state of the conversation data, means for inputting prompt sentences to a generative AI model based on the preprocessed conversation data, means for generating appropriate information using the generative AI model, and means for returning the generated information. This enables real-time analysis of conversation data and rapid and accurate provision of information desired by the user. Furthermore, highly accurate information provision taking into account context and tone is realized, enabling effective support for user decision-making.

[0678] "Conversation Data" is information including text messages, voice messages, and associated metadata (e.g., sender ID, group ID, timestamps, etc.) sent over a communications system.

[0679] "Integrity verification" is the process of verifying that the data received is accurate and meets the required format and fields.

[0680] "Tokenizing" refers to breaking down a text message into units of words or phrases, and is a preprocessing step in natural language processing.

[0681] "Key keywords and phrase extraction" is the process of identifying semantically significant words and expressions from text data.

[0682] "Morphological analysis" is the process of dividing text into its constituent morphemes (smallest semantic units) and adding part-of-speech tags to each of them.

[0683] "Emotional state identification" is the process of determining whether the content of analyzed text data is positive, negative, or neutral.

[0684] A "generative AI model" is an artificial intelligence algorithm that uses natural language processing technology to generate new text based on input text data.

[0685] A "prompt" is text containing context and instructions that is given as input to a generative AI model.

[0686] "Generating appropriate information" means generating answers or information that match the user's request based on the input text data and prompt sentence.

[0687] "Metadata" is information about the data itself, and in this case includes sender ID, group ID, timestamp, etc.

[0688] This system analyzes conversations between users on a messaging platform in real time and provides the information users are looking for quickly and accurately. This system is implemented using the following hardware and software.

[0689] Server-side processing

[0690] The server receives the conversation data, analyzes it, generates appropriate information, and sends it back to the user. The specific technologies used for processing include a text analysis engine, a sentiment analysis engine, and a generative AI model.

[0691] 1. Receiving and verifying conversation data

[0692] The server receives messages in real time from a messaging platform (e.g., a messaging application API) using the Webhook function. The received data includes the message content, sender ID, group ID, and timestamp.

[0693] 2. Data Preprocessing

[0694] The server verifies the integrity of the received data (e.g., JSON format validation) and filters out any invalid data or spam. It then uses Google NLP libraries to split the text message into tokens and extract key keywords and phrases. It also performs morphological analysis to add part-of-speech tags and identifies emotional states using IBM Watson's sentiment analysis API.

[0695] 3. Generative AI processing

[0696] The server inputs a prompt sentence into a generative AI model (e.g., OpenAI's GPT-3) based on the preprocessed data. For example, the prompt sentence for "I want to know about good Italian restaurants in Shinjuku" would be "Good Italian restaurants in Shinjuku." The generative AI model uses this prompt sentence to generate an appropriate response and, if necessary, retrieves related information from external sources (e.g., map information APIs).

[0697] 4. Sending a Message

[0698] The server formats the generated response content into a LINE message and packages it with metadata (sender ID, group ID, timestamp, etc.). The server then sends the generated message to the user via the official LINE API. The transmission results are recorded in a database and used for future response generation and analysis.

[0699] Terminal side processing

[0700] Users type and receive messages through messaging apps on their devices.

[0701] 1. Enter and send a message

[0702] Users can send messages by typing them into a group chat using a messaging app on their device and pressing the send button, which then passes through the messaging platform's servers.

[0703] 2. Receiving and viewing messages

[0704] The device receives new messages from the messaging platform and displays them in real time on the chat screen, allowing the user to review the received information and decide on the next action.

[0705] Specific examples

[0706] For example, if a user sends a message to a group chat saying, "I want to know about a good Italian restaurant in Shinjuku," the server processes it as follows:

[0707] 1. Receiving conversation data

[0708] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0709] 2. Data Preprocessing

[0710] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[0711] 3. Generative AI processing

[0712] The server generates a prompt sentence, "Delicious Italian restaurants in Shinjuku," and inputs it into the generative AI model. It then uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku and creates a list of the best restaurants.

[0713] 4. Sending a Message

[0714] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0715] 5. Receiving Messages

[0716] The device will receive this message and display it in the group chat, allowing the user to review the displayed information and decide on their next action.

[0717] This makes conversations within LINE groups smoother, allowing users to efficiently obtain the information they need. It also makes it possible to provide highly accurate information that takes into account context and tone, supporting user decision-making.

[0718] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0719] Step 1:

[0720] The server receives conversation data from the messaging platform in real time. Specifically, it uses a webhook to receive data such as the message, sender ID, group ID, and timestamp. The input is raw data from the messaging platform, and the output is the raw data received by the server. The server securely receives the data using SSL / TLS.

[0721] Step 2:

[0722] The server checks the integrity of the data it receives. It validates the JSON format and checks that required fields are included. Malformed data and spam messages are filtered out at this stage. The input is the raw data received in step 1, and the output is the valid, consistent data.

[0723] Step 3:

[0724] The server divides the consistent data into tokens and extracts key keywords and phrases. It performs text analysis using the Google NLP library. For example, from a message such as "I want to know about good Italian restaurants in Shinjuku," it extracts keywords such as "Shinjuku," "delicious," and "Italian restaurant." The input is the data whose consistency was confirmed in step 2, and the output is the tokenized text data and extracted keywords.

[0725] Step 4:

[0726] The server performs morphological analysis and tags the tokens with part-of-speech tags, using the Google NLP library or a similar morphological analysis engine. This generates the basis for understanding the meaning of the sentence. The input is the tokenized data from step 3, and the output is the morphologically analyzed data with part-of-speech tags.

[0727] Step 5:

[0728] The server uses the acquired keywords and the morphological analysis results to identify the emotional state. It uses IBM Watson's sentiment analysis API to determine whether the text data corresponds to positive, negative, or neutral. The input is the data morphologically analyzed in step 4, and the output is the data with the emotional state added.

[0729] Step 6:

[0730] The server inputs a prompt sentence to the generative AI model based on the preprocessed data. The prompt sentence is generated from the preprocessed data and takes the form of, for example, "Delicious Italian restaurants in Shinjuku." The generative AI model uses OpenAI's GPT-3. The input is the preprocessed data and the generated prompt sentence, and the output is the response text generated by the generative AI model.

[0731] Step 7:

[0732] The server integrates the generated response text with additional information obtained from external sources as needed. For example, it may use the Google Maps API to obtain a list of highly rated Italian restaurants near Shinjuku and add it to the generated response. The input is the response text generated in step 6 and the information obtained from the external API, and the output is the integrated final response message.

[0733] Step 8:

[0734] The server sends the generated response message to the user via the official LINE API. The message contains metadata such as the sender ID, group ID, and timestamp. The sending result is recorded as a log in the database and used for future response generation and analysis. The input is the final response message generated in step 7, and the output is the message sent to the user and its log.

[0735] Step 9:

[0736] The device receives new messages from the LINE server. Received messages are displayed on the chat screen in real time so that the user can check them. The input is the message delivered from the LINE server, and the output is the message displayed on the chat screen.

[0737] Step 10:

[0738] The user checks the displayed information and decides on the next action. For example, they may select a specific restaurant from a list of suggested restaurants and send a new message. The input is the information displayed on the terminal, and the output is a new message as the user's next action.

[0739] (Application example 1)

[0740] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0741] The automated response systems on existing chat platforms faced the challenge of being unable to provide accurate information based on users' specific needs. They also lacked a mechanism for recommending entertainment content and specialized information desired by users in real time. This made it difficult for users to obtain high-quality information and content without any effort.

[0742] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0743] In this invention, the server includes means for receiving conversation data, means for preprocessing the received conversation data, means for analyzing the context and tone based on the preprocessed conversation data, means for generating appropriate information based on the analysis results, means for returning the generated information, means for recommending content based on the preprocessed conversation data, and means for accessing an external database to obtain recommended content according to the received conversation data. This enables the server to quickly and accurately provide information desired by the user and to recommend related entertainment content in real time.

[0744] "Conversation Data" is a collection of text messages exchanged in communications between users.

[0745] The "receiving means" is a mechanism by which the server receives conversation data from the outside.

[0746] "Preprocessing means" refers to the process of preparing received conversation data in a format suitable for analysis and processing.

[0747] "Means of analyzing context" refers to technology for understanding the linguistic structure and meaning within conversation data.

[0748] "Tone analysis" refers to techniques for determining the emotion and intent of conversational data.

[0749] "Means of generating appropriate information" is the process of creating answers and information that best suit the user based on the analysis results.

[0750] A "return mechanism" is a mechanism for sending generated information back to the user.

[0751] "Content recommendation tools" are mechanisms that suggest relevant information and entertainment based on the user's needs and the content of the conversation.

[0752] "Means for accessing external databases" refers to methods for obtaining required data from external sources.

[0753] A "generative AI model" is an artificial intelligence model used to analyze conversation data and generate appropriate replies and recommended content.

[0754] A "prompt" is text containing a question or instruction that is input to a generative AI model.

[0755] In this invention, a system is provided in which a server, a terminal, and a user cooperate to generate and provide information. A specific embodiment of the system is described below.

[0756] Server-side processing

[0757] 1. Receiving Phase

[0758] The server receives conversation data from an external source. To do this, it uses the communication platform's API to obtain metadata such as message content, sender ID, group ID, and timestamp. The specific software used in this step is the official LINE API.

[0759] 2. Preprocessing Phase

[0760] The server preprocesses the received conversation data using the NLPProcessor library, splitting the conversation data into tokens and extracting key keywords and phrases. It also performs sentiment analysis to determine the tone of the message.

[0761] 3. Generative AI Processing Phase

[0762] The server generates appropriate information using a generative AI model based on the preprocessed data, and then makes an API request to access an external database to obtain the necessary recommendation information, using the AIRecommender library or an external service API (e.g., a movie database API).

[0763] 4. Result sending phase

[0764] The generated information and recommended content will be sent back to the user via the official LINE API, where the user can receive and view it on their device.

[0765] Terminal side processing

[0766] 1. User Input Phase

[0767] A user can type and send a message through a chat app on their device, which is then sent to the server and processed as described above.

[0768] 2. Receiving Phase

[0769] The device receives the information sent from the server and displays it on the chat screen. The user can then take the next action based on the displayed information.

[0770] Specific examples

[0771] Example 1: Movie Recommendations

[0772] If a user sends a message to a group chat saying "Tell me some good movies," the flow is as follows:

[0773] 1. The server receives a message saying, "Tell me some good movies."

[0774] 2. In the preprocessing phase, the keywords "interesting" and "movie" are extracted and sentiment analysis is performed.

[0775] 3. In the generation AI processing phase, we use the AIRecommender library to retrieve a list of highly rated movies from an external API.

[0776] 4. In the result sending phase, send the movie list along with the message "Here are some recommended movies!"

[0777] Prompt Sentence Examples

[0778] "Tell me some interesting movies you've seen recently."

[0779] In this way, the system can provide appropriate information and content that meets the user's needs.

[0780] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0781] Step 1:

[0782] The server receives conversation data from an external chat platform. Specifically, it uses a communication API such as the official LINE API to obtain metadata such as message content, sender ID, group ID, and timestamp. The input data is the new message itself, and the output data is the raw message data to be passed to the preprocessing phase.

[0783] Step 2:

[0784] The server preprocesses the received conversation data using the NLPProcessor library. Specifically, it divides the conversation data into tokens, performs morphological analysis to extract key keywords and phrases, and performs sentiment analysis to determine the tone of the message. The input data is the raw message data, and the output data is the analyzed keywords and tone information.

[0785] Step 3:

[0786] The server inputs the preprocessed data into the generative AI model to generate appropriate information. It uses the AIRecommender library to obtain recommended content. If necessary, it accesses external databases to collect additional information (e.g., movie or music lists). The input data is the analyzed keywords and tone information, and the output data is the generated response message.

[0787] Step 4:

[0788] The server formats the generated information and prepares it as a reply message. Specifically, it converts it into a format that is easy for the user to understand and includes meta information such as images and links if necessary. The input data is the generated response message, and the output data is the complete message package to be sent via the official LINE API.

[0789] Step 5:

[0790] The server uses the official LINE API to send the generated message to the user. The sending result is recorded in a log and used for future analysis and response improvement. The input data is the reply message package, and the output data is log information indicating that the message was sent to the user.

[0791] Step 6:

[0792] The user's device receives new messages through the LINE app and displays them on the chat screen. The user can check the displayed message and take further action. The input data is the reply message from the server, and the output data is the text displayed on the chat screen and (if necessary) meta information such as images and links.

[0793] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0794] This system analyzes conversations between users in LINE groups in real time, accurately grasps the user's emotions and context, and then provides the information the user is looking for quickly and appropriately. Below is a specific implementation of the system and a natural language explanation of its program.

[0795] Server-side processing

[0796] 1. Receiving Phase

[0797] The server receives new messages from the LINE Platform in real time using notification mechanisms such as Webhooks.

[0798] The received data includes the message content, sender ID, group ID, and timestamp.

[0799] The server checks the integrity of the data and filters out messages that are malformed or considered spam.

[0800] 2. Preprocessing Phase

[0801] The server passes the received message to a text analysis engine for natural language processing (NLP).

[0802] The NLP engine breaks the message into tokens and extracts key keywords and phrases.

[0803] Morphological analysis is used to add part-of-speech tags to understand the meaning of the sentence.

[0804] The server uses an emotion engine to identify the emotional state of the message (positive, negative, neutral).

[0805] The server filters irrelevant messages and messages identified as spam, passing only the minimum necessary data to the next processing phase.

[0806] 3. Generative AI Processing Phase

[0807] The server inputs the preprocessed data into the generative AI model.

[0808] The generative AI model takes into account context, tone, past message history, and the results of the emotion engine analysis to generate the optimal response.

[0809] Use external APIs (e.g., Google Maps API or medical information API) to obtain the necessary information.

[0810] The server packages the generated response content and prepares a reply message along with the metadata.

[0811] 4. Result sending phase

[0812] The server sends the generated reply message to the user's group chat via the official LINE API.

[0813] The results of the submission are logged and used for future response generation and analysis.

[0814] Terminal side processing

[0815] 1. User Input Phase

[0816] Users use the LINE app on their devices to type messages into group chats.

[0817] When the user presses the send button, the message is sent via the LINE server.

[0818] 2. Receiving Phase

[0819] The device receives the message from the LINE server.

[0820] Received messages are displayed on the chat screen for the user to view.

[0821] Specific examples

[0822] Example 1: Restaurant selection

[0823] User A sends a message in a group chat saying, "I want to know about some good Italian restaurants in Shinjuku."

[0824] 1. Server (receiving phase)

[0825] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0826] 2. Pre-processing phase (server)

[0827] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[0828] 3. Generative AI processing phase (server)

[0829] Based on the extracted keywords, the server uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[0830] The server generates an optimal restaurant list based on the search results and adjusts the reply content to reflect the positive sentiment of the message.

[0831] 4. Result sending phase (server)

[0832] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0833] 5. Receiving Phase (Terminal)

[0834] The device will receive this message and display it in the group chat.

[0835] The user can review the displayed information and decide on the next action to take.

[0836] Example 2: Health advice

[0837] User B posts in a group chat, "I have a headache, what should I do?"

[0838] 1. Server (receiving phase)

[0839] The server receives the message "I have a headache, what should I do?"

[0840] 2. Pre-processing phase (server)

[0841] The server extracts the keyword "headache" and determines the emotional state of the message as negative.

[0842] 3. Generative AI processing phase (server)

[0843] Based on the extracted keywords, the server uses a medical information API to obtain the most appropriate advice.

[0844] The server takes into account the negative emotional state and generates a response such as, "If your headache persists, I recommend you see a doctor as soon as possible."

[0845] 4. Result sending phase (server)

[0846] The server replies with the message, "If your headache persists, we recommend that you see a doctor as soon as possible. First, make sure you drink plenty of fluids and try lying down for a while."

[0847] 5. Receiving Phase (Terminal)

[0848] The device will receive this message and display it in the group chat.

[0849] The user can review the displayed information and decide on an action, such as visiting a medical institution.

[0850] This system makes conversations within LINE groups smoother, allowing users to quickly obtain accurate and relevant information that takes into account context and emotional state.

[0851] The processing flow will be explained below.

[0852] Server-side processing

[0853] Receiving Phase

[0854] Step 1:

[0855] The server receives new messages from the LINE Platform.

[0856] Step 2:

[0857] The server analyzes data such as the content, sender ID, group ID, and timestamp of the received message.

[0858] Step 3:

[0859] The server checks the data for integrity and filters out malformed and spam messages.

[0860] Preprocessing Phase

[0861] Step 4:

[0862] The server passes the received message to a text analysis engine for natural language processing (NLP).

[0863] Step 5:

[0864] The server performs morphological analysis and breaks the message into tokens (e.g., words).

[0865] Step 6:

[0866] The server extracts key keywords and phrases from the split tokens.

[0867] Step 7:

[0868] The server uses an emotion engine to analyze the emotional state of the message (positive, negative, neutral).

[0869] Step 8:

[0870] The server filters out irrelevant and spam messages.

[0871] Generative AI processing phase

[0872] Step 9:

[0873] The server inputs the preprocessed data into the generative AI model.

[0874] Step 10:

[0875] The generative AI model takes into account the context, tone, and analysis results of the emotion engine to generate an appropriate response.

[0876] Step 11:

[0877] The server uses external APIs (e.g., Google Maps API or medical information API) to obtain the necessary information.

[0878] Step 12:

[0879] The server packages the generated response content with the metadata and prepares a reply message.

[0880] Result sending phase

[0881] Step 13:

[0882] The server uses the official LINE API to send the generated reply message to the user's group chat.

[0883] Step 14:

[0884] The server logs the results of the transmission and uses them for future response generation and analysis.

[0885] Terminal side processing

[0886] User Input Phase

[0887] Step 1:

[0888] The user opens the LINE app and types a message into the group chat.

[0889] Step 2:

[0890] The user presses the send button to send the message to the LINE server.

[0891] Receiving Phase

[0892] Step 3:

[0893] The device receives a reply message from the LINE server.

[0894] Step 4:

[0895] The device displays the received message on the chat screen.

[0896] Processing flow of specific example

[0897] Example 1: Restaurant selection

[0898] Step 1: Server (receiving phase)

[0899] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[0900] Step 2: Server (Verify Data Integrity)

[0901] The server checks the data for integrity and filters out malformed and spam.

[0902] Step 3: Preprocessing Phase (Server)

[0903] The server passes the received message to a text analysis engine.

[0904] Step 4: Perform morphological analysis

[0905] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant."

[0906] Step 5: Analyze your emotional state

[0907] The server uses an emotion engine to evaluate the positive emotional state of the message.

[0908] Step 6: Filtering

[0909] The server filters out irrelevant messages.

[0910] Step 7: Generative AI processing phase (input of preprocessed data)

[0911] The server inputs the preprocessed data into the generative AI model.

[0912] Step 8: Generative AI model generates appropriate responses

[0913] Based on the analyzed keywords, the generated AI model uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[0914] Step 9: Generate response content based on search results

[0915] The server generates an optimal restaurant list based on the search results.

[0916] Step 10: Result sending phase (preparing the reply message)

[0917] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[0918] Step 11: Logging the transmission results

[0919] The server logs the results of the transmission.

[0920] Example 2: Health advice

[0921] Step 1: Server (receiving phase)

[0922] The server receives the message "I have a headache, what should I do?"

[0923] Step 2: Server (Verify Data Integrity)

[0924] The server checks the data for integrity and filters out malformed and spam.

[0925] Step 3: Preprocessing Phase (Server)

[0926] The server passes the received message to a text analysis engine.

[0927] Step 4: Perform morphological analysis

[0928] The server extracts the keyword "headache."

[0929] Step 5: Analyze your emotional state

[0930] The server uses an emotion engine to evaluate the negative emotional state of the message.

[0931] Step 6: Filtering

[0932] The server filters out irrelevant messages.

[0933] Step 7: Generative AI processing phase (input of preprocessed data)

[0934] The server inputs the preprocessed data into the generative AI model.

[0935] Step 8: Generative AI model generates appropriate responses

[0936] Based on the analyzed keywords, the generative AI model uses a medical information API to obtain optimal advice.

[0937] Step 9: Generate response content

[0938] The server takes into account the negative emotional state and generates a response such as, "If your headache persists, I recommend you see a doctor as soon as possible."

[0939] Step 10: Result sending phase (preparing the reply message)

[0940] The server replies with the message, "If your headache persists, we recommend that you see a doctor as soon as possible. First, make sure you drink plenty of fluids and try lying down for a while."

[0941] Step 11: Logging the transmission results

[0942] The server logs the results of the transmission.

[0943] Example 2

[0944] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0945] Conversation analysis systems in conventional communication platforms have difficulty accurately grasping the context and emotional state of users' conversations, and are therefore unable to provide users with the information they require quickly and appropriately. For this reason, there is a need to achieve more accurate natural language processing and emotion analysis in order to improve user satisfaction.

[0946] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving conversation data from a communication platform in real time, means for verifying the integrity of the received conversation data and filtering out invalid and spam data, means for dividing the received conversation data into tokens using a natural language processing engine and extracting key keywords and phrases, means for analyzing the context and emotional state of the conversation data using morphological analysis and an emotion engine, means for generating appropriate prompt sentences based on the analysis results and generating information using a generative AI model, means for acquiring supplementary information using an external information acquisition API as needed, means for packaging the generated information and preparing a reply message together with metadata, and means for sending the generated reply message via the API of the communication platform. This makes it possible to accurately grasp the context and emotional state of the conversation between users and quickly and appropriately provide the desired information.

[0947] "Conversation Data" means text messages exchanged between Users via a communications platform.

[0948] "Communications Platform" means an online service that enables users to send and receive text messages.

[0949] "Real-time" refers to processing or responding to an event almost as soon as it occurs.

[0950] "Integrity" refers to data that is accurate, consistent, and free from malformed or malformed content.

[0951] "Spam data" refers to a large volume of messages or information that is inappropriate or irrelevant to the user.

[0952] "Natural language processing engine" refers to a computer program or system for analyzing and understanding human language.

[0953] A "token" refers to a unit (such as a word or phrase) that is divided into text for analysis in natural language processing.

[0954] "Keywords" refer to words or phrases that have important meanings in conversation data.

[0955] "Morphological analysis" refers to a method of analyzing the structure of a sentence by breaking it down into parts of speech and assigning part-of-speech tags to each word.

[0956] An "emotion engine" refers to a program or system that analyzes and identifies a user's emotional state from text data.

[0957] A "prompt" refers to an instruction or question that is input into a generative AI model.

[0958] A "generative AI model" refers to an artificial intelligence model that generates responses or information in natural language based on a given prompt.

[0959] "External Information Acquisition API" refers to an application program interface for acquiring data from external information sources.

[0960] "Metadata" refers to data that describes the attributes, structure, meaning, etc. of data.

[0961] "Reply Message" refers to a response message to a user that includes generated information.

[0962] This invention provides a system that analyzes conversations between users on a communication platform, understands the user's emotions and context, and then provides prompt and appropriate information. The main components of the system include a server and a terminal.

[0963] Server-side processing

[0964] The server includes hardware and software for realizing the following main functions:

[0965] Interface with communication platforms: Use Webhooks to receive messages in real time from communication platforms (e.g., LINE, WhatsApp, etc.).

[0966] Data integrity check: Checks the integrity of received messages, including formatting errors and filtering out spam messages.

[0967] Natural language processing engine: Breaks incoming messages into tokens and extracts key keywords and phrases. Specifically, it uses an NLP library (e.g., spaCy or NLTK).

[0968] Morphological analysis and sentiment engine: Morphological analysis is used to analyze the structure of the sentence, and a sentiment engine (e.g., a sentiment analysis library) is used to identify the emotional state of the message (positive, negative, neutral).

[0969] Generative AI model: Generate a prompt sentence based on the analysis results and input it into a generative AI model (e.g., GPT-3 or BERT) to generate an appropriate response.

[0970] External information acquisition: If necessary, external APIs (e.g., map information APIs or medical information APIs) are used to acquire supplementary information.

[0971] Generate and send a reply message: The generated response content is packaged with metadata and the reply message is sent using the API of the communication platform.

[0972] Terminal side processing

[0973] The terminal (user's device) is used by the user to enter and send messages and to receive and display new messages through the communication platform application.

[0974] User input: A user opens a communication platform app on their device, types a text message, and sends it to the group chat.

[0975] Receiving and displaying messages: The terminal receives messages from the communication platform server and displays them on the chat screen.

[0976] Specific examples

[0977] Example 1: Restaurant selection

[0978] When User A sends a message saying, "I want to know about a good Italian restaurant in Shinjuku"

[0979] 1. The server receives this message from the communication platform and checks the data integrity.

[0980] 2. The server uses a natural language processing engine to extract the keywords "Shinjuku," "delicious," and "Italian restaurant," and determines that the message has a positive sentiment.

[0981] 3. Based on the extracted keywords, the server generates a prompt sentence such as, "Please tell me about some highly rated Italian restaurants around Shinjuku," and inputs it into the generative AI model.

[0982] 4. The generative AI model obtains information about Italian restaurants in Shinjuku from an external information acquisition API (e.g., map information API) and generates the optimal restaurant list.

[0983] 5. The server generates and sends a reply message saying, "Here are some delicious Italian restaurants in Shinjuku!"

[0984] 6. The terminal receives this message and displays it to User A.

[0985] Example 2: Health advice

[0986] If User B sends a message saying, "I have a headache. What should I do?"

[0987] 1. The server receives this message and verifies the data integrity.

[0988] 2. The server extracts the keyword "headache" and determines that the message has a negative emotion.

[0989] 3. The server generates a prompt sentence, "Please tell me what to do if my headache persists," and inputs it into the generative AI model.

[0990] 4. The generative AI model obtains appropriate advice from an external information acquisition API (e.g., medical information API) and generates a response message such as, "If your headache persists, we recommend that you see a doctor as soon as possible."

[0991] 5. The server sends this response message to User B.

[0992] 6. The device receives this message and displays it to User B.

[0993] The system allows users to quickly obtain accurate and relevant information that takes into account the context and emotional state of conversations on communication platforms.

[0994] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0995] Step 1:

[0996] The server receives new messages from the communication platform in real time. Specifically, notifications are sent using Webhooks. The input is the message data sent from the communication platform, and the output is saving this as pre-stage data for consistency check. Specifically, when a user sends a message, the message is forwarded to the server.

[0997] Step 2:

[0998] The server checks the integrity of the received message. The input is the message data received in step 1, and it detects and filters out any malformed or spam messages. The output is the clean message data after the integrity check. Specifically, it checks whether the message format is correct and whether the content is spam.

[0999] Step 3:

[1000] The server passes the message after integrity check to a natural language processing engine, which splits it into tokens and extracts key keywords and phrases. The input is clean message data, and the output is tokenized data and extracted keywords and phrases. Specifically, it uses an NLP library to tokenize the text and extract keywords.

[1001] Step 4:

[1002] The server passes the extracted tokens and keywords to a morphological analysis engine and a sentiment engine to analyze the context and emotional state. The input is the output data from step 3, and the output is the analyzed context information and emotional state (positive, negative, neutral). Specifically, the analysis is performed using a morphological analysis library and a sentiment analysis library.

[1003] Step 5:

[1004] The server generates an appropriate prompt sentence based on the analysis results and inputs it into the generative AI model. The input is contextual information and emotional state, and the output is response data from the generative AI model. Specifically, the server uses the analysis results to construct a prompt sentence and inputs it into the generative AI model. Example: "Please tell me about some highly rated Italian restaurants around Shinjuku."

[1005] Step 6:

[1006] The server obtains supplementary information as needed using an external information acquisition API (e.g., map information API, medical information API) based on the output data of the generative AI model. The input is the response data of the generative AI model, and the output is the complete response data including the supplementary information. Specifically, the server makes an API call and obtains the required information.

[1007] Step 7:

[1008] The server packages the response data, including the complementary information, with metadata to generate a reply message. The input is the generative AI model's response data, including the complementary information, and the output is the final reply message. Specifically, the server formats the response content and adds metadata.

[1009] Step 8:

[1010] The server sends the generated reply message through the API of the communication platform. The input is the final reply message and the output is the sending result. The specific operation is to send the message using the API of the communication platform and record the sending result in a log.

[1011] Step 9:

[1012] The terminal receives a reply message from the communication platform and displays it in the group chat. The input is the reply message from the communication platform, and the output is display data in a format that can be viewed by the user. The specific operation is to display the received message on the chat screen.

[1013] Step 10:

[1014] The user checks the reply message displayed on the device and decides on the next action. The input is the message displayed on the chat screen, and the output is the user's next action. In concrete terms, the user decides on an action based on the information. Examples: checking restaurant information and making a reservation, following health advice.

[1015] (Application example 2)

[1016] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1017] Currently, many food delivery services require users to access apps or websites to search for food, making it difficult for them to provide appropriate suggestions that reflect the user's sentiment and context. As a result, it can take a long time for users to decide what to order. Furthermore, the lack of automatic and appropriate menu suggestions based on conversations on chat platforms such as LINE does not improve the food delivery user experience. A solution to these issues is needed.

[1018] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving conversation data, means for preprocessing the received conversation data, means for analyzing context and tone based on the preprocessed conversation data, means for generating appropriate information based on the analysis results, means for returning the generated information, and means for suggesting food delivery menus. This enables the user to quickly and appropriately receive food delivery menu suggestions based on conversations on chat platforms such as LINE.

[1019] "Conversation Data" refers to the text and statements sent and received by Users on the chat platform.

[1020] "Means for receiving" refers to a function or interface that allows the server to obtain data from the outside.

[1021] "Preprocessing means" refers to the processes and algorithms that format acquired conversation data and extract necessary items to make it easier to analyze.

[1022] "Means for analyzing context and tone" refers to methods for understanding the meaning and sentiment of conversational data using natural language processing and sentiment analysis techniques.

[1023] "Generating means" refers to the processes and algorithms used to generate the necessary information and recommendations based on the analysis results.

[1024] "Means for replying" refers to the interface and pathway for sending generated information and suggestions to the user.

[1025] "Means for suggesting food delivery menus" refers to systems and processes for suggesting appropriate food and beverage ordering menus based on keywords and sentiments extracted from conversation data.

[1026] This invention is a system that suggests food delivery menus in real time based on conversations between users on chat platforms such as LINE. This system operates on both the server and the device side. The server side analyzes the conversation data and performs a series of processes to generate and return appropriate information. On the device side, messages are sent and received through a user interface.

[1027] Server-side processing

[1028] The server first receives new messages from the LINE Platform in real time using a notification mechanism such as a webhook. The received data includes the message content, sender ID, group ID, and timestamp. Next, the server preprocesses the received messages and performs natural language processing (NLP). This preprocessing phase involves tokenization, keyword extraction, and morphological analysis. It also uses a sentiment analysis engine to identify the emotional state of the message and filter out irrelevant messages and spam.

[1029] Once preprocessed, the data is input into a generative AI model. The generative AI model generates a response to suggest the optimal food delivery menu, taking into account the context, past message history, and sentiment analysis results. During this process, it may obtain necessary information using external APIs (e.g., food delivery APIs). The generated response is sent to the user's group chat via the official LINE API. This series of processes allows the system to suggest appropriate food delivery menus in real time based on the user's conversations on the chat platform.

[1030] Terminal side processing

[1031] A user uses the LINE app on their device to enter a message into a group chat. When the user presses the send button, the message is sent via the LINE server. The received message is displayed on the chat screen so that the user can view it. Reply messages from the server are also received in the same way and displayed on the chat screen.

[1032] Specific examples

[1033] 1. Lunch consultation

[1034] User A: "What are you having for lunch?"

[1035] User B: "I like Italian food."

[1036] Server: "This pizza place is great! (Rating: 4.5)"

[1037] To achieve this process, the server uses the following software and hardware: First, a web framework such as Flask is used to receive and process webhook notifications, then TextBlob and other natural language processing libraries are used for sentiment analysis and keyword extraction, and the official LINE API is used to send the final reply message.

[1038] Prompt Sentence Examples

[1039] "A user says, 'I'd like Italian food.' Please suggest delivery menu items based on this message."

[1040] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1041] Step 1:

[1042] The server receives new messages from the LINE Platform in real time. The received data includes the message content, sender ID, group ID, and timestamp. It uses the Webhook function to receive notifications from the LINE server. The input is new message data, and the output is the received message information.

[1043] Step 2:

[1044] The server preprocesses the received message. Specifically, it performs natural language processing (NLP) using the TextBlob library to split the message into tokens and extract key keywords and phrases. At the same time, it performs sentiment analysis to identify positive, negative, and neutral emotional states. The input is the received message information, and the output is the analysis results, extracted keywords, and emotional states.

[1045] Step 3:

[1046] The server inputs the preprocessed data into a generative AI model, which takes into account the context, tone, past message history, and sentiment analysis results to generate the optimal response. This process also involves using a food delivery API to obtain relevant menu information. The inputs are the analysis results and extracted keywords, and the output is the generated response message.

[1047] Step 4:

[1048] The server packages the generated response message and sends it to the user's group chat via the official LINE API. The input is the generated response message, and the output is the sent reply message. This allows users to receive appropriate food delivery menu suggestions in real time in the group chat.

[1049] Step 5:

[1050] The user checks the received message using the LINE app on their device. The device displays the message received from the LINE server on the chat screen. The input is the message received from the LINE server, and the output is the message displayed on the user's device. This allows the user to decide on their next action and order food delivery.

[1051] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1052] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1053] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1054] [Third embodiment]

[1055] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1056] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1057] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1058] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1059] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1060] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1061] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1062] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1063] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1064] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1065] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1066] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1067] This system analyzes conversations between users in LINE groups in real time and provides the information users are looking for quickly and accurately. Below is a specific example of the system and a natural language explanation of its program.

[1068] Server-side processing

[1069] 1. Receiving Phase

[1070] The server receives new messages from the LINE Platform periodically or in real time, using notification mechanisms such as Webhooks.

[1071] The received data includes the message content, sender ID, group ID, and timestamp.

[1072] The server performs data integrity verification to filter out malformed data and spam messages.

[1073] 2. Preprocessing Phase

[1074] The server passes the received message to a text analysis engine to perform natural language processing (NLP).

[1075] The NLP engine breaks the message down into tokens and extracts key keywords and phrases.

[1076] Morphological analysis is used to add part-of-speech tags to understand the meaning of the sentence.

[1077] The server uses a sentiment analysis engine to identify the emotional state of the message (positive, negative, neutral).

[1078] The server filters irrelevant and spam messages and passes only the minimum necessary data to the next processing phase.

[1079] 3. Generative AI Processing Phase

[1080] The server inputs the preprocessed data into the generative AI model.

[1081] The generative AI model takes into account context, tone, and past message history to generate the optimal response.

[1082] Use external APIs (e.g., Google Maps API, medical information API, etc.) to obtain the necessary information.

[1083] The server packages the generated response content with the metadata and prepares a reply message.

[1084] 4. Result sending phase

[1085] The server sends the generated reply message to the user's group chat via the official LINE API.

[1086] The results of the submission are logged and used for future response generation and analysis.

[1087] Terminal side processing

[1088] 1. User Input Phase

[1089] Users use the LINE app on their devices to type messages into group chats.

[1090] When the user presses the send button, the message is sent via the LINE server.

[1091] 2. Receiving Phase

[1092] The device will wait for messages from the LINE server and will receive new messages as they arrive.

[1093] Received messages are displayed on the chat screen for the user to view.

[1094] Specific examples

[1095] Example 1: Restaurant selection

[1096] User A sends a message in a group chat saying, "I want to know about some good Italian restaurants in Shinjuku."

[1097] 1. Server (receiving phase)

[1098] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1099] 2. Pre-processing phase (server)

[1100] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[1101] 3. Generative AI processing phase (server)

[1102] Based on the extracted keywords, the server uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[1103] The server generates an optimal restaurant list based on the search results.

[1104] 4. Result sending phase (server)

[1105] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1106] 5. Receiving Phase (Terminal)

[1107] The device will receive this message and display it in the group chat.

[1108] The user can review the displayed information and decide on the next action to take.

[1109] This system makes conversations within LINE groups smoother, allowing users to efficiently obtain the information they need. It also provides highly accurate information that takes into account context and tone, supporting user decision-making.

[1110] The processing flow will be explained below.

[1111] Server-side processing

[1112] Receiving Phase

[1113] Step 1:

[1114] The server receives new messages from the LINE Platform.

[1115] The received data is in JSON format and includes the message content, sender ID, group ID, and timestamp.

[1116] Step 2:

[1117] The server checks the integrity of the received message data and filters out messages that are malformed or considered spam.

[1118] Preprocessing Phase

[1119] Step 3:

[1120] The server passes the received message to a text analysis engine for natural language processing (NLP).

[1121] Step 4:

[1122] The server performs a morphological analysis and divides the message into tokens.

[1123] Morphological analysis identifies the part of speech and meaning of words.

[1124] Step 5:

[1125] The server extracts key keywords and phrases from the extracted tokens.

[1126] Step 6:

[1127] The server uses a sentiment analysis engine to evaluate the emotional state of the message (positive, negative, neutral).

[1128] Step 7:

[1129] The server filters irrelevant messages and messages identified as spam.

[1130] Generative AI processing phase

[1131] Step 8:

[1132] The server inputs the preprocessed data into the generative AI model.

[1133] Step 9:

[1134] The generative AI model takes into account context and tone to generate the most appropriate response.

[1135] Step 10:

[1136] The server calls an external API (for example, Google Maps API or medical information API) to obtain the necessary information.

[1137] Step 11:

[1138] The server packages the generated response information and prepares a reply message along with the metadata.

[1139] Result sending phase

[1140] Step 12:

[1141] The server uses the official LINE API to send the generated reply message to the user's group chat.

[1142] Step 13:

[1143] The results of the submission are logged and used for future response generation and analysis.

[1144] Terminal side processing

[1145] User Input Phase

[1146] Step 1:

[1147] The user opens the LINE app and types a message into the group chat.

[1148] Step 2:

[1149] The user presses the send button to send the message to the LINE server.

[1150] Receiving Phase

[1151] Step 3:

[1152] The device receives the message from the LINE server.

[1153] Step 4:

[1154] The device displays the received message on the chat screen.

[1155] Processing flow of specific example

[1156] Example 1: Restaurant selection

[1157] 1. Receiving Phase (Server)

[1158] Step 1:

[1159] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1160] Step 2:

[1161] The server checks the data for integrity and filters out malformed and spam.

[1162] 2. Pre-processing phase (server)

[1163] Step 3:

[1164] The server passes the received message to a text analysis engine and performs morphological analysis.

[1165] Step 4:

[1166] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant."

[1167] Step 5:

[1168] The server uses a sentiment analysis engine to evaluate the emotional state of the message as positive.

[1169] Step 6:

[1170] The server filters out irrelevant messages.

[1171] 3. Generative AI processing phase (server)

[1172] Step 7:

[1173] The server inputs the preprocessed data into the generative AI model.

[1174] Step 8:

[1175] Based on the analyzed keywords, the generated AI model uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[1176] Step 9:

[1177] The server generates a list of restaurants based on the search results.

[1178] 4. Result sending phase (server)

[1179] Step 10:

[1180] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1181] Step 11:

[1182] The results of the transmission are logged and used to generate future responses.

[1183] 5. Receiving Phase (Terminal)

[1184] Step 12:

[1185] The device receives a reply message from the LINE server.

[1186] Step 13:

[1187] The device will display this message on the chat screen.

[1188] Example 1

[1189] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1190] Conventional conversation analysis systems have struggled to provide users with the information they need quickly and accurately in real time. They also struggled to provide highly accurate information that takes into account the context and tone of the conversation, making it difficult to adequately support user decision-making.

[1191] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1192] In this invention, the server includes means for receiving conversation data, means for verifying the integrity of the received conversation data, means for dividing the received conversation data into tokens and extracting key keywords and phrases, means for performing morphological analysis of the conversation data and adding part-of-speech tags, means for identifying the emotional state of the conversation data, means for inputting prompt sentences to a generative AI model based on the preprocessed conversation data, means for generating appropriate information using the generative AI model, and means for returning the generated information. This enables real-time analysis of conversation data and rapid and accurate provision of information desired by the user. Furthermore, highly accurate information provision taking into account context and tone is realized, enabling effective support for user decision-making.

[1193] "Conversation Data" is information including text messages, voice messages, and associated metadata (e.g., sender ID, group ID, timestamps, etc.) sent over a communications system.

[1194] "Integrity verification" is the process of verifying that the data received is accurate and meets the required format and fields.

[1195] "Tokenizing" refers to breaking a text message into units of words or phrases, and is a preprocessing step in natural language processing.

[1196] "Key keywords and phrase extraction" is the process of identifying semantically significant words and expressions from text data.

[1197] "Morphological analysis" is the process of dividing text into its constituent morphemes (smallest semantic units) and adding part-of-speech tags to each of them.

[1198] "Emotional state identification" is the process of determining whether the content of analyzed text data is positive, negative, or neutral.

[1199] A "generative AI model" is an artificial intelligence algorithm that uses natural language processing technology to generate new text based on input text data.

[1200] A "prompt" is text containing context and instructions that is given as input to a generative AI model.

[1201] "Generating appropriate information" means generating answers or information that match the user's request based on the input text data and prompt sentence.

[1202] "Metadata" is information about the data itself, and in this case includes sender ID, group ID, timestamp, etc.

[1203] This system analyzes conversations between users on a messaging platform in real time and provides the information users are looking for quickly and accurately. This system is implemented using the following hardware and software.

[1204] Server-side processing

[1205] The server receives the conversation data, analyzes it, generates appropriate information, and sends it back to the user. The specific technologies used for processing include a text analysis engine, a sentiment analysis engine, and a generative AI model.

[1206] 1. Receiving and verifying conversation data

[1207] The server receives messages in real time from a messaging platform (e.g., a messaging application API) using the Webhook function. The received data includes the message content, sender ID, group ID, and timestamp.

[1208] 2. Data Preprocessing

[1209] The server verifies the integrity of the received data (e.g., JSON format validation) and filters out any invalid data or spam. It then uses Google NLP libraries to split the text message into tokens and extract key keywords and phrases. It also performs morphological analysis to add part-of-speech tags and identifies emotional states using IBM Watson's sentiment analysis API.

[1210] 3. Generative AI processing

[1211] The server inputs a prompt sentence into a generative AI model (e.g., OpenAI's GPT-3) based on the preprocessed data. For example, the prompt sentence for "I want to know about good Italian restaurants in Shinjuku" would be "Good Italian restaurants in Shinjuku." The generative AI model uses this prompt sentence to generate an appropriate response and, if necessary, retrieves related information from external sources (e.g., map information APIs).

[1212] 4. Sending a Message

[1213] The server formats the generated response content into a LINE message and packages it with metadata (sender ID, group ID, timestamp, etc.). The server then sends the generated message to the user via the official LINE API. The transmission results are recorded in a database and used for future response generation and analysis.

[1214] Terminal side processing

[1215] Users type and receive messages through messaging apps on their devices.

[1216] 1. Enter and send a message

[1217] Users can send messages by typing them into a group chat using a messaging app on their device and pressing the send button, which then passes through the messaging platform's servers.

[1218] 2. Receiving and viewing messages

[1219] The device receives new messages from the messaging platform and displays them in real time on the chat screen, allowing the user to review the received information and decide on the next action.

[1220] Specific examples

[1221] For example, if a user sends a message to a group chat saying, "I want to know about a good Italian restaurant in Shinjuku," the server processes it as follows:

[1222] 1. Receiving conversation data

[1223] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1224] 2. Data Preprocessing

[1225] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[1226] 3. Generative AI processing

[1227] The server generates a prompt sentence, "Delicious Italian restaurants in Shinjuku," and inputs it into the generative AI model. It then uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku and creates a list of the best restaurants.

[1228] 4. Sending a Message

[1229] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1230] 5. Receiving Messages

[1231] The device will receive this message and display it in the group chat, allowing the user to review the displayed information and decide on their next action.

[1232] This makes conversations within LINE groups smoother, allowing users to efficiently obtain the information they need. It also makes it possible to provide highly accurate information that takes into account context and tone, supporting user decision-making.

[1233] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1234] Step 1:

[1235] The server receives conversation data from the messaging platform in real time. Specifically, it uses a webhook to receive data such as the message, sender ID, group ID, and timestamp. The input is raw data from the messaging platform, and the output is the raw data received by the server. The server securely receives the data using SSL / TLS.

[1236] Step 2:

[1237] The server checks the integrity of the data it receives. It validates the JSON format and checks that required fields are included. Malformed data and spam messages are filtered out at this stage. The input is the raw data received in step 1, and the output is the valid, consistent data.

[1238] Step 3:

[1239] The server divides the consistent data into tokens and extracts key keywords and phrases. It performs text analysis using the Google NLP library. For example, from a message such as "I want to know about good Italian restaurants in Shinjuku," it extracts keywords such as "Shinjuku," "delicious," and "Italian restaurant." The input is the data whose consistency was confirmed in step 2, and the output is the tokenized text data and extracted keywords.

[1240] Step 4:

[1241] The server performs morphological analysis and tags the tokens with part-of-speech tags, using the Google NLP library or a similar morphological analysis engine. This generates the basis for understanding the meaning of the sentence. The input is the tokenized data from step 3, and the output is the morphologically analyzed data with part-of-speech tags.

[1242] Step 5:

[1243] The server uses the acquired keywords and the morphological analysis results to identify the emotional state. It uses IBM Watson's sentiment analysis API to determine whether the text data corresponds to positive, negative, or neutral. The input is the data morphologically analyzed in step 4, and the output is the data with the emotional state added.

[1244] Step 6:

[1245] The server inputs a prompt sentence to the generative AI model based on the preprocessed data. The prompt sentence is generated from the preprocessed data and takes the form of, for example, "Delicious Italian restaurants in Shinjuku." The generative AI model uses OpenAI's GPT-3. The input is the preprocessed data and the generated prompt sentence, and the output is the response text generated by the generative AI model.

[1246] Step 7:

[1247] The server integrates the generated response text with additional information obtained from external sources as needed. For example, it may use the Google Maps API to obtain a list of highly rated Italian restaurants near Shinjuku and add it to the generated response. The input is the response text generated in step 6 and the information obtained from the external API, and the output is the integrated final response message.

[1248] Step 8:

[1249] The server sends the generated response message to the user via the official LINE API. The message contains metadata such as the sender ID, group ID, and timestamp. The sending result is recorded as a log in the database and used for future response generation and analysis. The input is the final response message generated in step 7, and the output is the message sent to the user and its log.

[1250] Step 9:

[1251] The device receives new messages from the LINE server. Received messages are displayed on the chat screen in real time so that the user can check them. The input is the message delivered from the LINE server, and the output is the message displayed on the chat screen.

[1252] Step 10:

[1253] The user checks the displayed information and decides on the next action. For example, they may select a specific restaurant from a list of suggested restaurants and send a new message. The input is the information displayed on the terminal, and the output is a new message as the user's next action.

[1254] (Application example 1)

[1255] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1256] The automated response systems on existing chat platforms faced the challenge of being unable to provide accurate information based on users' specific needs. They also lacked a mechanism for recommending entertainment content and specialized information desired by users in real time. This made it difficult for users to obtain high-quality information and content without any effort.

[1257] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1258] In this invention, the server includes means for receiving conversation data, means for preprocessing the received conversation data, means for analyzing the context and tone based on the preprocessed conversation data, means for generating appropriate information based on the analysis results, means for returning the generated information, means for recommending content based on the preprocessed conversation data, and means for accessing an external database to obtain recommended content according to the received conversation data. This enables the server to quickly and accurately provide information desired by the user and to recommend related entertainment content in real time.

[1259] "Conversation Data" is a collection of text messages exchanged in communications between users.

[1260] The "receiving means" is a mechanism by which the server receives conversation data from the outside.

[1261] "Preprocessing means" refers to the process of preparing received conversation data in a format suitable for analysis and processing.

[1262] "Means of analyzing context" refers to technology for understanding the linguistic structure and meaning within conversation data.

[1263] "Tone analysis" refers to techniques for determining the emotion and intent of conversational data.

[1264] "Means of generating appropriate information" is the process of creating answers and information that best suit the user based on the analysis results.

[1265] A "return mechanism" is a mechanism for sending generated information back to the user.

[1266] "Content recommendation tools" are mechanisms that suggest relevant information and entertainment based on the user's needs and the content of the conversation.

[1267] "Means for accessing external databases" refers to methods for obtaining required data from external sources.

[1268] A "generative AI model" is an artificial intelligence model used to analyze conversation data and generate appropriate replies and recommended content.

[1269] A "prompt" is text containing a question or instruction that is input to a generative AI model.

[1270] In this invention, a system is provided in which a server, a terminal, and a user cooperate to generate and provide information. A specific embodiment of the system is described below.

[1271] Server-side processing

[1272] 1. Receiving Phase

[1273] The server receives conversation data from an external source. To do this, it uses the API of the communication platform to obtain metadata such as the message content, sender ID, group ID, and timestamp. The specific software used in this step is the official LINE API.

[1274] 2. Preprocessing Phase

[1275] The server preprocesses the received conversation data using the NLPProcessor library, splitting the conversation data into tokens and extracting key keywords and phrases. It also performs sentiment analysis to determine the tone of the message.

[1276] 3. Generative AI Processing Phase

[1277] The server generates appropriate information using a generative AI model based on the preprocessed data, and then makes an API request to access an external database to obtain the necessary recommendation information, using the AIRecommender library or an external service API (e.g., a movie database API).

[1278] 4. Result sending phase

[1279] The generated information and recommended content will be sent back to the user via the official LINE API, where the user can receive and view it on their device.

[1280] Terminal side processing

[1281] 1. User Input Phase

[1282] A user can type and send a message through a chat app on their device, which is then sent to the server and processed as described above.

[1283] 2. Receiving Phase

[1284] The device receives the information sent from the server and displays it on the chat screen. The user can then take the next action based on the displayed information.

[1285] Specific examples

[1286] Example 1: Movie Recommendations

[1287] If a user sends a message to a group chat saying "Tell me some good movies," the flow is as follows:

[1288] 1. The server receives a message saying, "Tell me some good movies."

[1289] 2. In the preprocessing phase, the keywords "interesting" and "movie" are extracted and sentiment analysis is performed.

[1290] 3. In the generation AI processing phase, we use the AIRecommender library to retrieve a list of highly rated movies from an external API.

[1291] 4. In the result sending phase, send the movie list along with the message "Here are some recommended movies!"

[1292] Prompt Sentence Examples

[1293] "Tell me some interesting movies you've seen recently."

[1294] In this way, the system can provide appropriate information and content that meets the user's needs.

[1295] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1296] Step 1:

[1297] The server receives conversation data from an external chat platform. Specifically, it uses a communication API such as the official LINE API to obtain metadata such as message content, sender ID, group ID, and timestamp. The input data is the new message itself, and the output data is the raw message data to be passed to the preprocessing phase.

[1298] Step 2:

[1299] The server preprocesses the received conversation data using the NLPProcessor library. Specifically, it divides the conversation data into tokens, performs morphological analysis to extract key keywords and phrases, and performs sentiment analysis to determine the tone of the message. The input data is the raw message data, and the output data is the analyzed keywords and tone information.

[1300] Step 3:

[1301] The server inputs the preprocessed data into the generative AI model to generate appropriate information. It uses the AIRecommender library to obtain recommended content. If necessary, it accesses external databases to collect additional information (e.g., movie or music lists). The input data is the analyzed keywords and tone information, and the output data is the generated response message.

[1302] Step 4:

[1303] The server formats the generated information and prepares it as a reply message. Specifically, it converts it into a format that is easy for the user to understand and includes meta information such as images and links if necessary. The input data is the generated response message, and the output data is the complete message package to be sent via the official LINE API.

[1304] Step 5:

[1305] The server uses the official LINE API to send the generated message to the user. The sending result is recorded in a log and used for future analysis and response improvement. The input data is the reply message package, and the output data is log information indicating that the message was sent to the user.

[1306] Step 6:

[1307] The user's device receives new messages through the LINE app and displays them on the chat screen. The user can check the displayed message and take further action. The input data is the reply message from the server, and the output data is the text displayed on the chat screen and (if necessary) meta information such as images and links.

[1308] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1309] This system analyzes conversations between users in LINE groups in real time, accurately grasps the user's emotions and context, and then provides the information the user is looking for quickly and appropriately. Below is a specific implementation of the system and a natural language explanation of its program.

[1310] Server-side processing

[1311] 1. Receiving Phase

[1312] The server receives new messages from the LINE Platform in real time using notification mechanisms such as Webhooks.

[1313] The received data includes the message content, sender ID, group ID, and timestamp.

[1314] The server checks the integrity of the data and filters out messages that are malformed or considered spam.

[1315] 2. Preprocessing Phase

[1316] The server passes the received message to a text analysis engine for natural language processing (NLP).

[1317] The NLP engine breaks the message down into tokens and extracts key keywords and phrases.

[1318] Morphological analysis is used to add part-of-speech tags to understand the meaning of the sentence.

[1319] The server uses an emotion engine to identify the emotional state of the message (positive, negative, neutral).

[1320] The server filters irrelevant messages and messages identified as spam, passing only the minimum necessary data to the next processing phase.

[1321] 3. Generative AI Processing Phase

[1322] The server inputs the preprocessed data into the generative AI model.

[1323] The generative AI model takes into account context, tone, past message history, and the results of the emotion engine analysis to generate the optimal response.

[1324] Use external APIs (e.g., Google Maps API or medical information API) to obtain the necessary information.

[1325] The server packages the generated response content and prepares a reply message along with the metadata.

[1326] 4. Result sending phase

[1327] The server sends the generated reply message to the user's group chat via the official LINE API.

[1328] The results of the submission are logged and used for future response generation and analysis.

[1329] Terminal side processing

[1330] 1. User Input Phase

[1331] Users use the LINE app on their devices to type messages into group chats.

[1332] When the user presses the send button, the message is sent via the LINE server.

[1333] 2. Receiving Phase

[1334] The device receives the message from the LINE server.

[1335] Received messages are displayed on the chat screen for the user to view.

[1336] Specific examples

[1337] Example 1: Restaurant selection

[1338] User A sends a message in a group chat saying, "I want to know about some good Italian restaurants in Shinjuku."

[1339] 1. Server (receiving phase)

[1340] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1341] 2. Pre-processing phase (server)

[1342] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[1343] 3. Generative AI processing phase (server)

[1344] Based on the extracted keywords, the server uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[1345] The server generates an optimal restaurant list based on the search results and adjusts the reply content to reflect the positive sentiment of the message.

[1346] 4. Result sending phase (server)

[1347] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1348] 5. Receiving Phase (Terminal)

[1349] The device will receive this message and display it in the group chat.

[1350] The user can review the displayed information and decide on the next action to take.

[1351] Example 2: Health advice

[1352] User B posts in a group chat, "I have a headache, what should I do?"

[1353] 1. Server (receiving phase)

[1354] The server receives the message "I have a headache, what should I do?"

[1355] 2. Pre-processing phase (server)

[1356] The server extracts the keyword "headache" and determines the emotional state of the message as negative.

[1357] 3. Generative AI processing phase (server)

[1358] Based on the extracted keywords, the server uses a medical information API to obtain the most appropriate advice.

[1359] The server takes into account the negative emotional state and generates a response such as, "If your headache persists, I recommend you see a doctor as soon as possible."

[1360] 4. Result sending phase (server)

[1361] The server replies with the message, "If your headache persists, we recommend that you see a doctor as soon as possible. First, make sure you drink plenty of fluids and try lying down for a while."

[1362] 5. Receiving Phase (Terminal)

[1363] The device will receive this message and display it in the group chat.

[1364] The user can review the displayed information and decide on an action, such as visiting a medical institution.

[1365] This system makes conversations within LINE groups smoother, allowing users to quickly obtain accurate and relevant information that takes into account context and emotional state.

[1366] The processing flow will be explained below.

[1367] Server-side processing

[1368] Receiving Phase

[1369] Step 1:

[1370] The server receives new messages from the LINE Platform.

[1371] Step 2:

[1372] The server analyzes data such as the content, sender ID, group ID, and timestamp of the received message.

[1373] Step 3:

[1374] The server checks the data for integrity and filters out malformed and spam messages.

[1375] Preprocessing Phase

[1376] Step 4:

[1377] The server passes the received message to a text analysis engine for natural language processing (NLP).

[1378] Step 5:

[1379] The server performs morphological analysis and breaks the message into tokens (e.g., words).

[1380] Step 6:

[1381] The server extracts key keywords and phrases from the split tokens.

[1382] Step 7:

[1383] The server uses an emotion engine to analyze the emotional state of the message (positive, negative, neutral).

[1384] Step 8:

[1385] The server filters out irrelevant and spam messages.

[1386] Generative AI processing phase

[1387] Step 9:

[1388] The server inputs the preprocessed data into the generative AI model.

[1389] Step 10:

[1390] The generative AI model takes into account the context, tone, and analysis results of the emotion engine to generate an appropriate response.

[1391] Step 11:

[1392] The server uses external APIs (e.g., Google Maps API or medical information API) to obtain the necessary information.

[1393] Step 12:

[1394] The server packages the generated response content with the metadata and prepares a reply message.

[1395] Result sending phase

[1396] Step 13:

[1397] The server uses the official LINE API to send the generated reply message to the user's group chat.

[1398] Step 14:

[1399] The server logs the results of the transmission and uses them for future response generation and analysis.

[1400] Terminal side processing

[1401] User Input Phase

[1402] Step 1:

[1403] The user opens the LINE app and types a message into the group chat.

[1404] Step 2:

[1405] The user presses the send button to send the message to the LINE server.

[1406] Receiving Phase

[1407] Step 3:

[1408] The device receives a reply message from the LINE server.

[1409] Step 4:

[1410] The device displays the received message on the chat screen.

[1411] Processing flow of specific example

[1412] Example 1: Restaurant selection

[1413] Step 1: Server (receiving phase)

[1414] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1415] Step 2: Server (Verify Data Integrity)

[1416] The server checks the data for integrity and filters out malformed and spam.

[1417] Step 3: Preprocessing Phase (Server)

[1418] The server passes the received message to a text analysis engine.

[1419] Step 4: Perform morphological analysis

[1420] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant."

[1421] Step 5: Analyze your emotional state

[1422] The server uses an emotion engine to evaluate the positive emotional state of the message.

[1423] Step 6: Filtering

[1424] The server filters out irrelevant messages.

[1425] Step 7: Generative AI processing phase (input of preprocessed data)

[1426] The server inputs the preprocessed data into the generative AI model.

[1427] Step 8: Generative AI model generates appropriate responses

[1428] Based on the analyzed keywords, the generated AI model uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[1429] Step 9: Generate response content based on search results

[1430] The server generates an optimal restaurant list based on the search results.

[1431] Step 10: Result sending phase (preparing the reply message)

[1432] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1433] Step 11: Logging the transmission results

[1434] The server logs the results of the transmission.

[1435] Example 2: Health advice

[1436] Step 1: Server (receiving phase)

[1437] The server receives the message "I have a headache, what should I do?"

[1438] Step 2: Server (Verify Data Integrity)

[1439] The server checks the data for integrity and filters out malformed and spam.

[1440] Step 3: Preprocessing Phase (Server)

[1441] The server passes the received message to a text analysis engine.

[1442] Step 4: Perform morphological analysis

[1443] The server extracts the keyword "headache."

[1444] Step 5: Analyze your emotional state

[1445] The server uses an emotion engine to evaluate the negative emotional state of the message.

[1446] Step 6: Filtering

[1447] The server filters out irrelevant messages.

[1448] Step 7: Generative AI processing phase (input of preprocessed data)

[1449] The server inputs the preprocessed data into the generative AI model.

[1450] Step 8: Generative AI model generates appropriate responses

[1451] Based on the analyzed keywords, the generative AI model uses a medical information API to obtain optimal advice.

[1452] Step 9: Generate response content

[1453] The server takes into account the negative emotional state and generates a response such as, "If your headache persists, I recommend you see a doctor as soon as possible."

[1454] Step 10: Result sending phase (preparing the reply message)

[1455] The server replies with the message, "If your headache persists, we recommend that you see a doctor as soon as possible. First, make sure you drink plenty of fluids and try lying down for a while."

[1456] Step 11: Logging the transmission results

[1457] The server logs the results of the transmission.

[1458] Example 2

[1459] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1460] Conversation analysis systems in conventional communication platforms have difficulty accurately grasping the context and emotional state of users' conversations, and are therefore unable to provide users with the information they require quickly and appropriately. For this reason, there is a need to achieve more accurate natural language processing and emotion analysis in order to improve user satisfaction.

[1461] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving conversation data from a communication platform in real time, means for verifying the integrity of the received conversation data and filtering out invalid and spam data, means for dividing the received conversation data into tokens using a natural language processing engine and extracting key keywords and phrases, means for analyzing the context and emotional state of the conversation data using morphological analysis and an emotion engine, means for generating appropriate prompt sentences based on the analysis results and generating information using a generative AI model, means for acquiring supplementary information using an external information acquisition API as needed, means for packaging the generated information and preparing a reply message together with metadata, and means for sending the generated reply message via the API of the communication platform. This makes it possible to accurately grasp the context and emotional state of the conversation between users and quickly and appropriately provide the desired information.

[1462] "Conversation Data" means text messages exchanged between Users via a communications platform.

[1463] "Communications Platform" means an online service that enables users to send and receive text messages.

[1464] "Real-time" refers to processing or responding to an event almost as soon as it occurs.

[1465] "Integrity" refers to data that is accurate, consistent, and free from malformed or malformed content.

[1466] "Spam data" refers to a large volume of messages or information that is inappropriate or irrelevant to the user.

[1467] "Natural language processing engine" refers to a computer program or system for analyzing and understanding human language.

[1468] A "token" refers to a unit (such as a word or phrase) that is divided into text for analysis in natural language processing.

[1469] "Keywords" refer to words or phrases that have important meanings in conversation data.

[1470] "Morphological analysis" refers to a method of analyzing the structure of a sentence by breaking it down into parts of speech and assigning part-of-speech tags to each word.

[1471] An "emotion engine" refers to a program or system that analyzes and identifies a user's emotional state from text data.

[1472] A "prompt" refers to an instruction or question that is input into a generative AI model.

[1473] A "generative AI model" refers to an artificial intelligence model that generates responses or information in natural language based on a given prompt.

[1474] "External Information Acquisition API" refers to an application program interface for acquiring data from external information sources.

[1475] "Metadata" refers to data that describes the attributes, structure, meaning, etc. of data.

[1476] "Reply Message" refers to a response message to a user that includes generated information.

[1477] This invention provides a system that analyzes conversations between users on a communication platform, understands the user's emotions and context, and then provides prompt and appropriate information. The main components of the system include a server and a terminal.

[1478] Server-side processing

[1479] The server includes hardware and software for realizing the following main functions:

[1480] Interface with communication platforms: Use Webhooks to receive messages in real time from communication platforms (e.g., LINE, WhatsApp, etc.).

[1481] Data integrity check: Checks the integrity of received messages, including formatting errors and filtering out spam messages.

[1482] Natural language processing engine: Breaks incoming messages into tokens and extracts key keywords and phrases. Specifically, it uses an NLP library (e.g., spaCy or NLTK).

[1483] Morphological analysis and sentiment engine: Morphological analysis is used to analyze the structure of the sentence, and a sentiment engine (e.g., a sentiment analysis library) is used to identify the emotional state of the message (positive, negative, neutral).

[1484] Generative AI model: Generate a prompt sentence based on the analysis results and input it into a generative AI model (e.g., GPT-3 or BERT) to generate an appropriate response.

[1485] External information acquisition: If necessary, external APIs (e.g., map information APIs or medical information APIs) are used to acquire supplementary information.

[1486] Generate and send a reply message: The generated response content is packaged with metadata and the reply message is sent using the API of the communication platform.

[1487] Terminal side processing

[1488] The terminal (user's device) is used by the user to enter and send messages and to receive and display new messages through the communication platform application.

[1489] User input: The user opens the communication platform app on their device, types a text message, and sends it to the group chat.

[1490] Receiving and displaying messages: The terminal receives messages from the communication platform server and displays them on the chat screen.

[1491] Specific examples

[1492] Example 1: Restaurant selection

[1493] When User A sends a message saying, "I want to know about a good Italian restaurant in Shinjuku"

[1494] 1. The server receives this message from the communication platform and checks the data integrity.

[1495] 2. The server uses a natural language processing engine to extract the keywords "Shinjuku," "delicious," and "Italian restaurant," and determines that the message has a positive sentiment.

[1496] 3. Based on the extracted keywords, the server generates a prompt sentence such as, "Please tell me about some highly rated Italian restaurants around Shinjuku," and inputs it into the generative AI model.

[1497] 4. The generative AI model obtains information about Italian restaurants in Shinjuku from an external information acquisition API (e.g., map information API) and generates the optimal restaurant list.

[1498] 5. The server generates and sends a reply message saying, "Here are some delicious Italian restaurants in Shinjuku!"

[1499] 6. The terminal receives this message and displays it to User A.

[1500] Example 2: Health advice

[1501] If User B sends a message saying, "I have a headache. What should I do?"

[1502] 1. The server receives this message and verifies the data integrity.

[1503] 2. The server extracts the keyword "headache" and determines that the message has a negative emotion.

[1504] 3. The server generates a prompt sentence, "Please tell me what to do if my headache persists," and inputs it into the generative AI model.

[1505] 4. The generative AI model obtains appropriate advice from an external information acquisition API (e.g., medical information API) and generates a response message such as, "If your headache persists, we recommend that you see a doctor as soon as possible."

[1506] 5. The server sends this response message to User B.

[1507] 6. The device receives this message and displays it to User B.

[1508] The system allows users to quickly obtain accurate and relevant information that takes into account the context and emotional state of conversations on communication platforms.

[1509] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1510] Step 1:

[1511] The server receives new messages from the communication platform in real time. Specifically, notifications are sent using Webhooks. The input is the message data sent from the communication platform, and the output is saving this as pre-stage data for consistency check. Specifically, when a user sends a message, the message is forwarded to the server.

[1512] Step 2:

[1513] The server checks the integrity of the received message. The input is the message data received in step 1, and it detects and filters out any malformed or spam messages. The output is the clean message data after the integrity check. Specifically, it checks whether the message format is correct and whether the content is spam.

[1514] Step 3:

[1515] The server passes the message after integrity check to a natural language processing engine, which splits it into tokens and extracts key keywords and phrases. The input is clean message data, and the output is tokenized data and extracted keywords and phrases. Specifically, it uses an NLP library to tokenize the text and extract keywords.

[1516] Step 4:

[1517] The server passes the extracted tokens and keywords to a morphological analysis engine and a sentiment engine to analyze the context and emotional state. The input is the output data from step 3, and the output is the analyzed context information and emotional state (positive, negative, neutral). Specifically, the analysis is performed using a morphological analysis library and a sentiment analysis library.

[1518] Step 5:

[1519] The server generates an appropriate prompt sentence based on the analysis results and inputs it into the generative AI model. The input is contextual information and emotional state, and the output is response data from the generative AI model. Specifically, the server uses the analysis results to construct a prompt sentence and inputs it into the generative AI model. Example: "Please tell me about some highly rated Italian restaurants around Shinjuku."

[1520] Step 6:

[1521] The server obtains supplementary information as needed using an external information acquisition API (e.g., map information API, medical information API) based on the output data of the generative AI model. The input is the response data of the generative AI model, and the output is the complete response data including the supplementary information. Specifically, the server makes an API call and obtains the required information.

[1522] Step 7:

[1523] The server packages the response data, including the complementary information, with metadata to generate a reply message. The input is the generative AI model's response data, including the complementary information, and the output is the final reply message. Specifically, the server formats the response content and adds metadata.

[1524] Step 8:

[1525] The server sends the generated reply message through the API of the communication platform. The input is the final reply message and the output is the transmission result. The specific operation is to send the message using the API of the communication platform and record the transmission result in a log.

[1526] Step 9:

[1527] The terminal receives a reply message from the communication platform and displays it in the group chat. The input is the reply message from the communication platform, and the output is display data in a format that can be viewed by the user. The specific operation is to display the received message on the chat screen.

[1528] Step 10:

[1529] The user checks the reply message displayed on the device and decides on the next action. The input is the message displayed on the chat screen, and the output is the user's next action. In concrete terms, the user decides on an action based on the information. Examples: checking restaurant information and making a reservation, following health advice.

[1530] (Application example 2)

[1531] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1532] Currently, many food delivery services require users to access apps or websites to search for food, making it difficult for them to provide appropriate suggestions that reflect the user's sentiment and context. As a result, it can take a long time for users to decide what to order. Furthermore, the lack of automatic and appropriate menu suggestions based on conversations on chat platforms such as LINE does not improve the food delivery user experience. A solution to these issues is needed.

[1533] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving conversation data, means for preprocessing the received conversation data, means for analyzing context and tone based on the preprocessed conversation data, means for generating appropriate information based on the analysis results, means for returning the generated information, and means for suggesting food delivery menus. This enables the user to quickly and appropriately receive food delivery menu suggestions based on conversations on chat platforms such as LINE.

[1534] "Conversation Data" refers to the text and statements sent and received by Users on the chat platform.

[1535] "Means for receiving" refers to a function or interface that allows the server to obtain data from the outside.

[1536] "Preprocessing means" refers to the processes and algorithms that format acquired conversation data and extract necessary items to make it easier to analyze.

[1537] "Means for analyzing context and tone" refers to methods for understanding the meaning and sentiment of conversational data using natural language processing and sentiment analysis techniques.

[1538] "Generating means" refers to the processes and algorithms used to generate the necessary information and recommendations based on the analysis results.

[1539] "Means for replying" refers to the interface and pathway for sending generated information and suggestions to the user.

[1540] "Means for suggesting food delivery menus" refers to systems and processes for suggesting appropriate food and beverage ordering menus based on keywords and sentiments extracted from conversation data.

[1541] This invention is a system that suggests food delivery menus in real time based on conversations between users on chat platforms such as LINE. This system operates on both the server and the device side. The server side analyzes the conversation data and performs a series of processes to generate and return appropriate information. On the device side, messages are sent and received through a user interface.

[1542] Server-side processing

[1543] The server first receives new messages from the LINE Platform in real time using a notification mechanism such as a webhook. The received data includes the message content, sender ID, group ID, and timestamp. Next, the server preprocesses the received messages and performs natural language processing (NLP). This preprocessing phase involves tokenization, keyword extraction, and morphological analysis. It also uses a sentiment analysis engine to identify the emotional state of the message and filter out irrelevant messages and spam.

[1544] Once preprocessed, the data is input into a generative AI model. The generative AI model generates a response to suggest the optimal food delivery menu, taking into account the context, past message history, and sentiment analysis results. During this process, it may obtain necessary information using external APIs (e.g., food delivery APIs). The generated response is sent to the user's group chat via the official LINE API. This series of processes allows the system to suggest appropriate food delivery menus in real time based on the user's conversations on the chat platform.

[1545] Terminal side processing

[1546] A user uses the LINE app on their device to enter a message into a group chat. When the user presses the send button, the message is sent via the LINE server. The received message is displayed on the chat screen so that the user can view it. Reply messages from the server are also received in the same way and displayed on the chat screen.

[1547] Specific examples

[1548] 1. Lunch consultation

[1549] User A: "What are you having for lunch?"

[1550] User B: "I like Italian food."

[1551] Server: "This pizza place is great! (Rating: 4.5)"

[1552] To achieve this process, the server uses the following software and hardware: First, a web framework such as Flask is used to receive and process webhook notifications, then TextBlob and other natural language processing libraries are used for sentiment analysis and keyword extraction, and the official LINE API is used to send the final reply message.

[1553] Prompt Sentence Examples

[1554] "A user says, 'I'd like Italian food.' Please suggest delivery menu items based on this message."

[1555] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1556] Step 1:

[1557] The server receives new messages from the LINE Platform in real time. The received data includes the message content, sender ID, group ID, and timestamp. It uses the Webhook function to receive notifications from the LINE server. The input is new message data, and the output is the received message information.

[1558] Step 2:

[1559] The server preprocesses the received message. Specifically, it performs natural language processing (NLP) using the TextBlob library to split the message into tokens and extract key keywords and phrases. At the same time, it performs sentiment analysis to identify positive, negative, and neutral emotional states. The input is the received message information, and the output is the analysis results, extracted keywords, and emotional states.

[1560] Step 3:

[1561] The server inputs the preprocessed data into a generative AI model, which takes into account the context, tone, past message history, and sentiment analysis results to generate the optimal response. This process also involves using a food delivery API to obtain relevant menu information. The inputs are the analysis results and extracted keywords, and the output is the generated response message.

[1562] Step 4:

[1563] The server packages the generated response message and sends it to the user's group chat via the official LINE API. The input is the generated response message, and the output is the sent reply message. This allows users to receive appropriate food delivery menu suggestions in real time in the group chat.

[1564] Step 5:

[1565] The user checks the received message using the LINE app on their device. The device displays the message received from the LINE server on the chat screen. The input is the message received from the LINE server, and the output is the message displayed on the user's device. This allows the user to decide on their next action and order food delivery.

[1566] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1567] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1568] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1569] [Fourth embodiment]

[1570] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1571] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1572] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1573] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1574] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1575] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1576] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1577] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1578] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1579] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1580] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1581] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1582] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1583] This system analyzes conversations between users in LINE groups in real time and provides the information users are looking for quickly and accurately. Below is a specific example of the system and a natural language explanation of its program.

[1584] Server-side processing

[1585] 1. Receiving Phase

[1586] The server receives new messages from the LINE Platform periodically or in real time, using notification mechanisms such as Webhooks.

[1587] The received data includes the message content, sender ID, group ID, and timestamp.

[1588] The server performs data integrity verification to filter out malformed data and spam messages.

[1589] 2. Preprocessing Phase

[1590] The server passes the received message to a text analysis engine to perform natural language processing (NLP).

[1591] The NLP engine breaks the message down into tokens and extracts key keywords and phrases.

[1592] Morphological analysis is used to add part-of-speech tags to understand the meaning of the sentence.

[1593] The server uses a sentiment analysis engine to identify the emotional state of the message (positive, negative, neutral).

[1594] The server filters irrelevant and spam messages and passes only the minimum necessary data to the next processing phase.

[1595] 3. Generative AI Processing Phase

[1596] The server inputs the preprocessed data into the generative AI model.

[1597] The generative AI model takes into account context, tone, and past message history to generate the optimal response.

[1598] Use external APIs (e.g., Google Maps API, medical information API, etc.) to obtain the necessary information.

[1599] The server packages the generated response content with the metadata and prepares a reply message.

[1600] 4. Result sending phase

[1601] The server sends the generated reply message to the user's group chat via the official LINE API.

[1602] The results of the submission are logged and used for future response generation and analysis.

[1603] Terminal side processing

[1604] 1. User Input Phase

[1605] Users use the LINE app on their devices to type messages into group chats.

[1606] When the user presses the send button, the message is sent via the LINE server.

[1607] 2. Receiving Phase

[1608] The device will wait for messages from the LINE server and will receive new messages as they arrive.

[1609] Received messages are displayed on the chat screen for the user to view.

[1610] Specific examples

[1611] Example 1: Restaurant selection

[1612] User A sends a message in a group chat saying, "I want to know about some good Italian restaurants in Shinjuku."

[1613] 1. Server (receiving phase)

[1614] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1615] 2. Pre-processing phase (server)

[1616] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[1617] 3. Generative AI processing phase (server)

[1618] Based on the extracted keywords, the server uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[1619] The server generates an optimal restaurant list based on the search results.

[1620] 4. Result sending phase (server)

[1621] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1622] 5. Receiving Phase (Terminal)

[1623] The device will receive this message and display it in the group chat.

[1624] The user can review the displayed information and decide on the next action to take.

[1625] This system makes conversations within LINE groups smoother, allowing users to efficiently obtain the information they need. It also provides highly accurate information that takes into account context and tone, supporting user decision-making.

[1626] The processing flow will be explained below.

[1627] Server-side processing

[1628] Receiving Phase

[1629] Step 1:

[1630] The server receives new messages from the LINE Platform.

[1631] The received data is in JSON format and includes the message content, sender ID, group ID, and timestamp.

[1632] Step 2:

[1633] The server checks the integrity of the received message data and filters out messages that are malformed or considered spam.

[1634] Preprocessing Phase

[1635] Step 3:

[1636] The server passes the received message to a text analysis engine for natural language processing (NLP).

[1637] Step 4:

[1638] The server performs a morphological analysis and divides the message into tokens.

[1639] Morphological analysis identifies the part of speech and meaning of words.

[1640] Step 5:

[1641] The server extracts key keywords and phrases from the extracted tokens.

[1642] Step 6:

[1643] The server uses a sentiment analysis engine to evaluate the emotional state of the message (positive, negative, neutral).

[1644] Step 7:

[1645] The server filters irrelevant messages and messages identified as spam.

[1646] Generative AI processing phase

[1647] Step 8:

[1648] The server inputs the preprocessed data into the generative AI model.

[1649] Step 9:

[1650] The generative AI model takes into account context and tone to generate the most appropriate response.

[1651] Step 10:

[1652] The server calls an external API (for example, Google Maps API or medical information API) to obtain the necessary information.

[1653] Step 11:

[1654] The server packages the generated response information and prepares a reply message along with the metadata.

[1655] Result sending phase

[1656] Step 12:

[1657] The server uses the official LINE API to send the generated reply message to the user's group chat.

[1658] Step 13:

[1659] The results of the submission are logged and used for future response generation and analysis.

[1660] Terminal side processing

[1661] User Input Phase

[1662] Step 1:

[1663] The user opens the LINE app and types a message into the group chat.

[1664] Step 2:

[1665] The user presses the send button to send the message to the LINE server.

[1666] Receiving Phase

[1667] Step 3:

[1668] The device receives the message from the LINE server.

[1669] Step 4:

[1670] The device displays the received message on the chat screen.

[1671] Processing flow of specific example

[1672] Example 1: Restaurant selection

[1673] 1. Receiving Phase (Server)

[1674] Step 1:

[1675] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1676] Step 2:

[1677] The server checks the data for integrity and filters out malformed and spam.

[1678] 2. Pre-processing phase (server)

[1679] Step 3:

[1680] The server passes the received message to a text analysis engine and performs morphological analysis.

[1681] Step 4:

[1682] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant."

[1683] Step 5:

[1684] The server uses a sentiment analysis engine to evaluate the emotional state of the message as positive.

[1685] Step 6:

[1686] The server filters out irrelevant messages.

[1687] 3. Generative AI processing phase (server)

[1688] Step 7:

[1689] The server inputs the preprocessed data into the generative AI model.

[1690] Step 8:

[1691] Based on the analyzed keywords, the generated AI model uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[1692] Step 9:

[1693] The server generates a list of restaurants based on the search results.

[1694] 4. Result sending phase (server)

[1695] Step 10:

[1696] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1697] Step 11:

[1698] The results of the transmission are logged and used to generate future responses.

[1699] 5. Receiving Phase (Terminal)

[1700] Step 12:

[1701] The device receives a reply message from the LINE server.

[1702] Step 13:

[1703] The device will display this message on the chat screen.

[1704] Example 1

[1705] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1706] Conventional conversation analysis systems have struggled to provide users with the information they need quickly and accurately in real time. They also struggled to provide highly accurate information that takes into account the context and tone of the conversation, making it difficult to adequately support user decision-making.

[1707] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1708] In this invention, the server includes means for receiving conversation data, means for verifying the integrity of the received conversation data, means for dividing the received conversation data into tokens and extracting key keywords and phrases, means for performing morphological analysis of the conversation data and adding part-of-speech tags, means for identifying the emotional state of the conversation data, means for inputting prompt sentences to a generative AI model based on the preprocessed conversation data, means for generating appropriate information using the generative AI model, and means for returning the generated information. This enables real-time analysis of conversation data and rapid and accurate provision of information desired by the user. Furthermore, highly accurate information provision taking into account context and tone is realized, enabling effective support for user decision-making.

[1709] "Conversation Data" is information including text messages, voice messages, and associated metadata (e.g., sender ID, group ID, timestamps, etc.) sent over a communications system.

[1710] "Integrity verification" is the process of verifying that the data received is accurate and meets the required format and fields.

[1711] "Tokenizing" refers to breaking a text message into units of words or phrases, and is a preprocessing step in natural language processing.

[1712] "Key keywords and phrase extraction" is the process of identifying semantically significant words and expressions from text data.

[1713] "Morphological analysis" is the process of dividing text into its constituent morphemes (smallest semantic units) and adding part-of-speech tags to each of them.

[1714] "Emotional state identification" is the process of determining whether the content of analyzed text data is positive, negative, or neutral.

[1715] A "generative AI model" is an artificial intelligence algorithm that uses natural language processing technology to generate new text based on input text data.

[1716] A "prompt" is text containing context and instructions that is given as input to a generative AI model.

[1717] "Generating appropriate information" means generating answers or information that match the user's request based on the input text data and prompt sentence.

[1718] "Metadata" is information about the data itself, and in this case includes sender ID, group ID, timestamp, etc.

[1719] This system analyzes conversations between users on a messaging platform in real time and provides the information users are looking for quickly and accurately. This system is implemented using the following hardware and software.

[1720] Server-side processing

[1721] The server receives the conversation data, analyzes it, generates appropriate information, and sends it back to the user. The specific technologies used for processing include a text analysis engine, a sentiment analysis engine, and a generative AI model.

[1722] 1. Receiving and verifying conversation data

[1723] The server receives messages in real time from a messaging platform (e.g., a messaging application API) using the Webhook function. The received data includes the message content, sender ID, group ID, and timestamp.

[1724] 2. Data Preprocessing

[1725] The server verifies the integrity of the received data (e.g., JSON format validation) and filters out any invalid data or spam. It then uses Google NLP libraries to split the text message into tokens and extract key keywords and phrases. It also performs morphological analysis to add part-of-speech tags and identifies emotional states using IBM Watson's sentiment analysis API.

[1726] 3. Generative AI processing

[1727] The server inputs a prompt sentence into a generative AI model (e.g., OpenAI's GPT-3) based on the preprocessed data. For example, the prompt sentence for "I want to know about good Italian restaurants in Shinjuku" would be "Good Italian restaurants in Shinjuku." The generative AI model uses this prompt sentence to generate an appropriate response and, if necessary, retrieves related information from external sources (e.g., map information APIs).

[1728] 4. Sending a Message

[1729] The server formats the generated response content into a LINE message and packages it with metadata (sender ID, group ID, timestamp, etc.). The server then sends the generated message to the user via the official LINE API. The transmission results are recorded in a database and used for future response generation and analysis.

[1730] Terminal side processing

[1731] Users type and receive messages through messaging apps on their devices.

[1732] 1. Enter and send a message

[1733] Users can send messages by typing them into a group chat using a messaging app on their device and pressing the send button, which then passes through the messaging platform's servers.

[1734] 2. Receiving and viewing messages

[1735] The device receives new messages from the messaging platform and displays them in real time on the chat screen, allowing the user to review the received information and decide on the next action.

[1736] Specific examples

[1737] For example, if a user sends a message to a group chat saying, "I want to know about a good Italian restaurant in Shinjuku," the server processes it as follows:

[1738] 1. Receiving conversation data

[1739] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1740] 2. Data Preprocessing

[1741] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[1742] 3. Generative AI processing

[1743] The server generates a prompt sentence, "Delicious Italian restaurants in Shinjuku," and inputs it into the generative AI model. It then uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku and creates a list of the best restaurants.

[1744] 4. Sending a Message

[1745] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1746] 5. Receiving Messages

[1747] The device will receive this message and display it in the group chat, allowing the user to review the displayed information and decide on their next action.

[1748] This makes conversations within LINE groups smoother, allowing users to efficiently obtain the information they need. It also makes it possible to provide highly accurate information that takes into account context and tone, supporting user decision-making.

[1749] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1750] Step 1:

[1751] The server receives conversation data from the messaging platform in real time. Specifically, it uses a webhook to receive data such as the message, sender ID, group ID, and timestamp. The input is raw data from the messaging platform, and the output is the raw data received by the server. The server securely receives the data using SSL / TLS.

[1752] Step 2:

[1753] The server checks the integrity of the data it receives. It validates the JSON format and checks that required fields are included. Malformed data and spam messages are filtered out at this stage. The input is the raw data received in step 1, and the output is the valid, consistent data.

[1754] Step 3:

[1755] The server divides the consistent data into tokens and extracts key keywords and phrases. It performs text analysis using the Google NLP library. For example, from a message such as "I want to know about good Italian restaurants in Shinjuku," it extracts keywords such as "Shinjuku," "delicious," and "Italian restaurant." The input is the data whose consistency was confirmed in step 2, and the output is the tokenized text data and extracted keywords.

[1756] Step 4:

[1757] The server performs morphological analysis and tags the tokens with part-of-speech tags, using the Google NLP library or a similar morphological analysis engine. This generates the basis for understanding the meaning of the sentence. The input is the tokenized data from step 3, and the output is the morphologically analyzed data with part-of-speech tags.

[1758] Step 5:

[1759] The server uses the acquired keywords and the morphological analysis results to identify the emotional state. It uses IBM Watson's sentiment analysis API to determine whether the text data corresponds to positive, negative, or neutral. The input is the data morphologically analyzed in step 4, and the output is the data with the emotional state added.

[1760] Step 6:

[1761] The server inputs a prompt sentence to the generative AI model based on the preprocessed data. The prompt sentence is generated from the preprocessed data and takes the form of, for example, "Delicious Italian restaurants in Shinjuku." The generative AI model uses OpenAI's GPT-3. The input is the preprocessed data and the generated prompt sentence, and the output is the response text generated by the generative AI model.

[1762] Step 7:

[1763] The server integrates the generated response text with additional information obtained from external sources as needed. For example, it may use the Google Maps API to obtain a list of highly rated Italian restaurants near Shinjuku and add it to the generated response. The input is the response text generated in step 6 and the information obtained from the external API, and the output is the integrated final response message.

[1764] Step 8:

[1765] The server sends the generated response message to the user via the official LINE API. The message contains metadata such as the sender ID, group ID, and timestamp. The sending result is recorded as a log in the database and used for future response generation and analysis. The input is the final response message generated in step 7, and the output is the message sent to the user and its log.

[1766] Step 9:

[1767] The device receives new messages from the LINE server. Received messages are displayed on the chat screen in real time so that the user can check them. The input is the message delivered from the LINE server, and the output is the message displayed on the chat screen.

[1768] Step 10:

[1769] The user checks the displayed information and decides on the next action. For example, they may select a specific restaurant from a list of suggested restaurants and send a new message. The input is the information displayed on the terminal, and the output is a new message as the user's next action.

[1770] (Application example 1)

[1771] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1772] The automated response systems on existing chat platforms faced the challenge of being unable to provide accurate information based on users' specific needs. They also lacked a mechanism for recommending entertainment content and specialized information desired by users in real time. This made it difficult for users to obtain high-quality information and content without any effort.

[1773] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1774] In this invention, the server includes means for receiving conversation data, means for preprocessing the received conversation data, means for analyzing the context and tone based on the preprocessed conversation data, means for generating appropriate information based on the analysis results, means for returning the generated information, means for recommending content based on the preprocessed conversation data, and means for accessing an external database to obtain recommended content according to the received conversation data. This enables the server to quickly and accurately provide information desired by the user and to recommend related entertainment content in real time.

[1775] "Conversation Data" is a collection of text messages exchanged in communications between users.

[1776] The "receiving means" is a mechanism by which the server receives conversation data from the outside.

[1777] "Preprocessing means" refers to the process of preparing received conversation data in a format suitable for analysis and processing.

[1778] "Means of analyzing context" refers to technology for understanding the linguistic structure and meaning within conversation data.

[1779] "Tone analysis" refers to techniques for determining the emotion and intent of conversational data.

[1780] "Means of generating appropriate information" is the process of creating answers and information that best suit the user based on the analysis results.

[1781] A "return mechanism" is a mechanism for sending generated information back to the user.

[1782] "Content recommendation tools" are mechanisms that suggest relevant information and entertainment based on the user's needs and the content of the conversation.

[1783] "Means for accessing external databases" refers to methods for obtaining required data from external sources.

[1784] A "generative AI model" is an artificial intelligence model used to analyze conversation data and generate appropriate replies and recommended content.

[1785] A "prompt" is text containing a question or instruction that is input to a generative AI model.

[1786] In this invention, a system is provided in which a server, a terminal, and a user cooperate to generate and provide information. A specific embodiment of the system is described below.

[1787] Server-side processing

[1788] 1. Receiving Phase

[1789] The server receives conversation data from an external source. To do this, it uses the API of the communication platform to obtain metadata such as the message content, sender ID, group ID, and timestamp. The specific software used in this step is the official LINE API.

[1790] 2. Preprocessing Phase

[1791] The server preprocesses the received conversation data using the NLPProcessor library, splitting the conversation data into tokens and extracting key keywords and phrases. It also performs sentiment analysis to determine the tone of the message.

[1792] 3. Generative AI Processing Phase

[1793] The server generates appropriate information using a generative AI model based on the preprocessed data, and then makes an API request to access an external database to obtain the necessary recommendation information, using the AIRecommender library or an external service API (e.g., a movie database API).

[1794] 4. Result sending phase

[1795] The generated information and recommended content will be sent back to the user via the official LINE API, where the user can receive and view it on their device.

[1796] Terminal side processing

[1797] 1. User Input Phase

[1798] A user can type and send a message through a chat app on their device, which is then sent to the server and processed as described above.

[1799] 2. Receiving Phase

[1800] The device receives the information sent from the server and displays it on the chat screen. The user can then take the next action based on the displayed information.

[1801] Specific examples

[1802] Example 1: Movie Recommendations

[1803] If a user sends a message to a group chat saying "Tell me some good movies," the flow is as follows:

[1804] 1. The server receives a message saying, "Tell me some good movies."

[1805] 2. In the preprocessing phase, the keywords "interesting" and "movie" are extracted and sentiment analysis is performed.

[1806] 3. In the generation AI processing phase, we use the AIRecommender library to retrieve a list of highly rated movies from an external API.

[1807] 4. In the result sending phase, send the movie list along with the message "Here are some recommended movies!"

[1808] Prompt Sentence Examples

[1809] "Tell me some interesting movies you've seen recently."

[1810] In this way, the system can provide appropriate information and content that meets the user's needs.

[1811] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1812] Step 1:

[1813] The server receives conversation data from an external chat platform. Specifically, it uses a communication API such as the official LINE API to obtain metadata such as message content, sender ID, group ID, and timestamp. The input data is the new message itself, and the output data is the raw message data to be passed to the preprocessing phase.

[1814] Step 2:

[1815] The server preprocesses the received conversation data using the NLPProcessor library. Specifically, it divides the conversation data into tokens, performs morphological analysis to extract key keywords and phrases, and performs sentiment analysis to determine the tone of the message. The input data is the raw message data, and the output data is the analyzed keywords and tone information.

[1816] Step 3:

[1817] The server inputs the preprocessed data into the generative AI model to generate appropriate information. It uses the AIRecommender library to obtain recommended content. If necessary, it accesses external databases to collect additional information (e.g., movie or music lists). The input data is the analyzed keywords and tone information, and the output data is the generated response message.

[1818] Step 4:

[1819] The server formats the generated information and prepares it as a reply message. Specifically, it converts it into a format that is easy for the user to understand and includes meta information such as images and links if necessary. The input data is the generated response message, and the output data is the complete message package to be sent via the official LINE API.

[1820] Step 5:

[1821] The server uses the official LINE API to send the generated message to the user. The sending result is recorded in a log and used for future analysis and response improvement. The input data is the reply message package, and the output data is log information indicating that the message was sent to the user.

[1822] Step 6:

[1823] The user's device receives new messages through the LINE app and displays them on the chat screen. The user can check the displayed message and take further action. The input data is the reply message from the server, and the output data is the text displayed on the chat screen and (if necessary) meta information such as images and links.

[1824] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1825] This system analyzes conversations between users in LINE groups in real time, accurately grasps the user's emotions and context, and then provides the information the user is looking for quickly and appropriately. Below is a specific implementation of the system and a natural language explanation of its program.

[1826] Server-side processing

[1827] 1. Receiving Phase

[1828] The server receives new messages from the LINE Platform in real time using notification mechanisms such as Webhooks.

[1829] The received data includes the message content, sender ID, group ID, and timestamp.

[1830] The server checks the integrity of the data and filters out messages that are malformed or considered spam.

[1831] 2. Preprocessing Phase

[1832] The server passes the received message to a text analysis engine for natural language processing (NLP).

[1833] The NLP engine breaks the message down into tokens and extracts key keywords and phrases.

[1834] Morphological analysis is used to add part-of-speech tags to understand the meaning of the sentence.

[1835] The server uses an emotion engine to identify the emotional state of the message (positive, negative, neutral).

[1836] The server filters irrelevant messages and messages identified as spam, passing only the minimum necessary data to the next processing phase.

[1837] 3. Generative AI Processing Phase

[1838] The server inputs the preprocessed data into the generative AI model.

[1839] The generative AI model takes into account context, tone, past message history, and the results of the emotion engine analysis to generate the optimal response.

[1840] Use external APIs (e.g., Google Maps API or medical information API) to obtain the necessary information.

[1841] The server packages the generated response content and prepares a reply message along with the metadata.

[1842] 4. Result sending phase

[1843] The server sends the generated reply message to the user's group chat via the official LINE API.

[1844] The results of the submission are logged and used for future response generation and analysis.

[1845] Terminal side processing

[1846] 1. User Input Phase

[1847] Users use the LINE app on their devices to type messages into group chats.

[1848] When the user presses the send button, the message is sent via the LINE server.

[1849] 2. Receiving Phase

[1850] The device receives the message from the LINE server.

[1851] Received messages are displayed on the chat screen for the user to view.

[1852] Specific examples

[1853] Example 1: Restaurant selection

[1854] User A sends a message in a group chat saying, "I want to know about some good Italian restaurants in Shinjuku."

[1855] 1. Server (receiving phase)

[1856] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1857] 2. Pre-processing phase (server)

[1858] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant" and determines the emotional state of the message as positive.

[1859] 3. Generative AI processing phase (server)

[1860] Based on the extracted keywords, the server uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[1861] The server generates an optimal restaurant list based on the search results and adjusts the reply content to reflect the positive sentiment of the message.

[1862] 4. Result sending phase (server)

[1863] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1864] 5. Receiving Phase (Terminal)

[1865] The device will receive this message and display it in the group chat.

[1866] The user can review the displayed information and decide on the next action to take.

[1867] Example 2: Health advice

[1868] User B posts in a group chat, "I have a headache, what should I do?"

[1869] 1. Server (receiving phase)

[1870] The server receives the message "I have a headache, what should I do?"

[1871] 2. Pre-processing phase (server)

[1872] The server extracts the keyword "headache" and determines the emotional state of the message as negative.

[1873] 3. Generative AI processing phase (server)

[1874] Based on the extracted keywords, the server uses a medical information API to obtain the most appropriate advice.

[1875] The server takes into account the negative emotional state and generates a response such as, "If your headache persists, I recommend you see a doctor as soon as possible."

[1876] 4. Result sending phase (server)

[1877] The server replies with the message, "If your headache persists, we recommend that you see a doctor as soon as possible. First, make sure you drink plenty of fluids and try lying down for a while."

[1878] 5. Receiving Phase (Terminal)

[1879] The device will receive this message and display it in the group chat.

[1880] The user can review the displayed information and decide on an action, such as visiting a medical institution.

[1881] This system makes conversations within LINE groups smoother, allowing users to quickly obtain accurate and relevant information that takes into account context and emotional state.

[1882] The processing flow will be explained below.

[1883] Server-side processing

[1884] Receiving Phase

[1885] Step 1:

[1886] The server receives new messages from the LINE Platform.

[1887] Step 2:

[1888] The server analyzes data such as the content, sender ID, group ID, and timestamp of the received message.

[1889] Step 3:

[1890] The server checks the data for integrity and filters out malformed and spam messages.

[1891] Preprocessing Phase

[1892] Step 4:

[1893] The server passes the received message to a text analysis engine for natural language processing (NLP).

[1894] Step 5:

[1895] The server performs morphological analysis and breaks the message into tokens (e.g., words).

[1896] Step 6:

[1897] The server extracts key keywords and phrases from the split tokens.

[1898] Step 7:

[1899] The server uses an emotion engine to analyze the emotional state of the message (positive, negative, neutral).

[1900] Step 8:

[1901] The server filters out irrelevant and spam messages.

[1902] Generative AI processing phase

[1903] Step 9:

[1904] The server inputs the preprocessed data into the generative AI model.

[1905] Step 10:

[1906] The generative AI model takes into account the context, tone, and analysis results of the emotion engine to generate an appropriate response.

[1907] Step 11:

[1908] The server uses external APIs (e.g., Google Maps API or medical information API) to obtain the necessary information.

[1909] Step 12:

[1910] The server packages the generated response content with the metadata and prepares a reply message.

[1911] Result sending phase

[1912] Step 13:

[1913] The server uses the official LINE API to send the generated reply message to the user's group chat.

[1914] Step 14:

[1915] The server logs the results of the transmission and uses them for future response generation and analysis.

[1916] Terminal side processing

[1917] User Input Phase

[1918] Step 1:

[1919] The user opens the LINE app and types a message into the group chat.

[1920] Step 2:

[1921] The user presses the send button to send the message to the LINE server.

[1922] Receiving Phase

[1923] Step 3:

[1924] The device receives a reply message from the LINE server.

[1925] Step 4:

[1926] The device displays the received message on the chat screen.

[1927] Processing flow of specific example

[1928] Example 1: Restaurant selection

[1929] Step 1: Server (receiving phase)

[1930] The server receives a message saying, "I want to know about a good Italian restaurant in Shinjuku."

[1931] Step 2: Server (Verify Data Integrity)

[1932] The server checks the data for integrity and filters out malformed and spam.

[1933] Step 3: Preprocessing Phase (Server)

[1934] The server passes the received message to a text analysis engine.

[1935] Step 4: Perform morphological analysis

[1936] The server extracts the keywords "Shinjuku," "delicious," and "Italian restaurant."

[1937] Step 5: Analyze your emotional state

[1938] The server uses an emotion engine to evaluate the positive emotional state of the message.

[1939] Step 6: Filtering

[1940] The server filters out irrelevant messages.

[1941] Step 7: Generative AI processing phase (input of preprocessed data)

[1942] The server inputs the preprocessed data into the generative AI model.

[1943] Step 8: Generative AI model generates appropriate responses

[1944] Based on the analyzed keywords, the generated AI model uses the Google Maps API to search for highly rated Italian restaurants around Shinjuku.

[1945] Step 9: Generate response content based on search results

[1946] The server generates an optimal restaurant list based on the search results.

[1947] Step 10: Result sending phase (preparing the reply message)

[1948] The server replies with a list of restaurants along with the message, "Here are some delicious Italian restaurants in Shinjuku!"

[1949] Step 11: Logging the transmission results

[1950] The server logs the results of the transmission.

[1951] Example 2: Health advice

[1952] Step 1: Server (receiving phase)

[1953] The server receives the message "I have a headache, what should I do?"

[1954] Step 2: Server (Verify Data Integrity)

[1955] The server checks the data for integrity and filters out malformed and spam.

[1956] Step 3: Preprocessing Phase (Server)

[1957] The server passes the received message to a text analysis engine.

[1958] Step 4: Perform morphological analysis

[1959] The server extracts the keyword "headache."

[1960] Step 5: Analyze your emotional state

[1961] The server uses an emotion engine to evaluate the negative emotional state of the message.

[1962] Step 6: Filtering

[1963] The server filters out irrelevant messages.

[1964] Step 7: Generative AI processing phase (input of preprocessed data)

[1965] The server inputs the preprocessed data into the generative AI model.

[1966] Step 8: Generative AI model generates appropriate responses

[1967] Based on the analyzed keywords, the generative AI model uses a medical information API to obtain optimal advice.

[1968] Step 9: Generate response content

[1969] The server takes into account the negative emotional state and generates a response such as, "If your headache persists, I recommend you see a doctor as soon as possible."

[1970] Step 10: Result sending phase (preparing the reply message)

[1971] The server replies with the message, "If your headache persists, we recommend that you see a doctor as soon as possible. First, make sure you drink plenty of fluids and try lying down for a while."

[1972] Step 11: Logging the transmission results

[1973] The server logs the results of the transmission.

[1974] Example 2

[1975] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1976] Conversation analysis systems in conventional communication platforms have difficulty accurately grasping the context and emotional state of users' conversations, and are therefore unable to provide users with the information they require quickly and appropriately. For this reason, there is a need to achieve more accurate natural language processing and emotion analysis in order to improve user satisfaction.

[1977] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving conversation data from a communication platform in real time, means for verifying the integrity of the received conversation data and filtering out invalid and spam data, means for dividing the received conversation data into tokens using a natural language processing engine and extracting key keywords and phrases, means for analyzing the context and emotional state of the conversation data using morphological analysis and an emotion engine, means for generating appropriate prompt sentences based on the analysis results and generating information using a generative AI model, means for acquiring supplementary information using an external information acquisition API as needed, means for packaging the generated information and preparing a reply message together with metadata, and means for sending the generated reply message via the API of the communication platform. This makes it possible to accurately grasp the context and emotional state of the conversation between users and quickly and appropriately provide the desired information.

[1978] "Conversation Data" means text messages exchanged between Users via a communications platform.

[1979] "Communications Platform" means an online service that enables users to send and receive text messages.

[1980] "Real-time" refers to processing or responding to an event almost as soon as it occurs.

[1981] "Integrity" refers to data that is accurate, consistent, and free from malformed or malformed content.

[1982] "Spam data" refers to a large volume of messages or information that is inappropriate or irrelevant to the user.

[1983] "Natural language processing engine" refers to a computer program or system for analyzing and understanding human language.

[1984] A "token" refers to a unit (such as a word or phrase) that is divided into text for analysis in natural language processing.

[1985] "Keywords" refer to words or phrases that have important meanings in conversation data.

[1986] "Morphological analysis" refers to a method of analyzing the structure of a sentence by breaking it down into parts of speech and assigning part-of-speech tags to each word.

[1987] An "emotion engine" refers to a program or system that analyzes and identifies a user's emotional state from text data.

[1988] A "prompt" refers to an instruction or question that is input into a generative AI model.

[1989] A "generative AI model" refers to an artificial intelligence model that generates responses or information in natural language based on a given prompt.

[1990] "External Information Acquisition API" refers to an application program interface for acquiring data from external information sources.

[1991] "Metadata" refers to data that describes the attributes, structure, meaning, etc. of data.

[1992] "Reply Message" refers to a response message to a user that includes generated information.

[1993] This invention provides a system that analyzes conversations between users on a communication platform, understands the user's emotions and context, and then provides prompt and appropriate information. The main components of the system include a server and a terminal.

[1994] Server-side processing

[1995] The server includes hardware and software for realizing the following main functions:

[1996] Interface with communication platforms: Use Webhooks to receive messages in real time from communication platforms (e.g., LINE, WhatsApp, etc.).

[1997] Data integrity check: Checks the integrity of received messages, including formatting errors and filtering out spam messages.

[1998] Natural language processing engine: Breaks incoming messages into tokens and extracts key keywords and phrases. Specifically, it uses an NLP library (e.g., spaCy or NLTK).

[1999] Morphological analysis and sentiment engine: Morphological analysis is used to analyze the structure of the sentence, and a sentiment engine (e.g., a sentiment analysis library) is used to identify the emotional state of the message (positive, negative, neutral).

[2000] Generative AI model: Generate a prompt sentence based on the analysis results and input it into a generative AI model (e.g., GPT-3 or BERT) to generate an appropriate response.

[2001] External information acquisition: If necessary, external APIs (e.g., map information APIs or medical information APIs) are used to acquire supplementary information.

[2002] Generate and send a reply message: The generated response content is packaged with metadata and the reply message is sent using the API of the communication platform.

[2003] Terminal side processing

[2004] The terminal (user's device) is used by the user to enter and send messages and to receive and display new messages through the communication platform application.

[2005] User input: The user opens the communication platform app on their device, types a text message, and sends it to the group chat.

[2006] Receiving and displaying messages: The terminal receives messages from the communication platform server and displays them on the chat screen.

[2007] Specific examples

[2008] Example 1: Restaurant selection

[2009] When User A sends a message saying, "I want to know about a good Italian restaurant in Shinjuku"

[2010] 1. The server receives this message from the communication platform and checks the data integrity.

[2011] 2. The server uses a natural language processing engine to extract the keywords "Shinjuku," "delicious," and "Italian restaurant," and determines that the message has a positive sentiment.

[2012] 3. Based on the extracted keywords, the server generates a prompt sentence such as, "Please tell me about some highly rated Italian restaurants around Shinjuku," and inputs it into the generative AI model.

[2013] 4. The generative AI model obtains information about Italian restaurants in Shinjuku from an external information acquisition API (e.g., map information API) and generates the optimal restaurant list.

[2014] 5. The server generates and sends a reply message saying, "Here are some delicious Italian restaurants in Shinjuku!"

[2015] 6. The terminal receives this message and displays it to User A.

[2016] Example 2: Health advice

[2017] If User B sends a message saying, "I have a headache. What should I do?"

[2018] 1. The server receives this message and verifies the data integrity.

[2019] 2. The server extracts the keyword "headache" and determines that the message has a negative emotion.

[2020] 3. The server generates a prompt sentence, "Please tell me what to do if my headache persists," and inputs it into the generative AI model.

[2021] 4. The generative AI model obtains appropriate advice from an external information acquisition API (e.g., medical information API) and generates a response message such as, "If your headache persists, we recommend that you see a doctor as soon as possible."

[2022] 5. The server sends this response message to User B.

[2023] 6. The device receives this message and displays it to User B.

[2024] The system allows users to quickly obtain accurate and relevant information that takes into account the context and emotional state of conversations on communication platforms.

[2025] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2026] Step 1:

[2027] The server receives new messages from the communication platform in real time. Specifically, notifications are sent using Webhooks. The input is the message data sent from the communication platform, and the output is saving this as pre-stage data for consistency check. Specifically, when a user sends a message, the message is forwarded to the server.

[2028] Step 2:

[2029] The server checks the integrity of the received message. The input is the message data received in step 1, and it detects and filters out any malformed or spam messages. The output is the clean message data after the integrity check. Specifically, it checks whether the message format is correct and whether the content is spam.

[2030] Step 3:

[2031] The server passes the message after integrity check to a natural language processing engine, which splits it into tokens and extracts key keywords and phrases. The input is clean message data, and the output is tokenized data and extracted keywords and phrases. Specifically, it uses an NLP library to tokenize the text and extract keywords.

[2032] Step 4:

[2033] The server passes the extracted tokens and keywords to a morphological analysis engine and a sentiment engine to analyze the context and emotional state. The input is the output data from step 3, and the output is the analyzed context information and emotional state (positive, negative, neutral). Specifically, the analysis is performed using a morphological analysis library and a sentiment analysis library.

[2034] Step 5:

[2035] The server generates an appropriate prompt sentence based on the analysis results and inputs it into the generative AI model. The input is contextual information and emotional state, and the output is response data from the generative AI model. Specifically, the server uses the analysis results to construct a prompt sentence and inputs it into the generative AI model. Example: "Please tell me about some highly rated Italian restaurants around Shinjuku."

[2036] Step 6:

[2037] The server obtains supplementary information as needed using an external information acquisition API (e.g., map information API, medical information API) based on the output data of the generative AI model. The input is the response data of the generative AI model, and the output is the complete response data including the supplementary information. Specifically, the server makes an API call and obtains the required information.

[2038] Step 7:

[2039] The server packages the response data, including the complementary information, with metadata to generate a reply message. The input is the generative AI model's response data, including the complementary information, and the output is the final reply message. Specifically, the server formats the response content and adds metadata.

[2040] Step 8:

[2041] The server sends the generated reply message through the API of the communication platform. The input is the final reply message and the output is the transmission result. The specific operation is to send the message using the API of the communication platform and record the transmission result in a log.

[2042] Step 9:

[2043] The terminal receives a reply message from the communication platform and displays it in the group chat. The input is the reply message from the communication platform, and the output is display data in a format that can be viewed by the user. The specific operation is to display the received message on the chat screen.

[2044] Step 10:

[2045] The user checks the reply message displayed on the device and decides on the next action. The input is the message displayed on the chat screen, and the output is the user's next action. In concrete terms, the user decides on an action based on the information. Examples: checking restaurant information and making a reservation, following health advice.

[2046] (Application example 2)

[2047] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2048] Currently, many food delivery services require users to access apps or websites to search for food, making it difficult for them to provide appropriate suggestions that reflect the user's sentiment and context. As a result, it can take a long time for users to decide what to order. Furthermore, the lack of automatic and appropriate menu suggestions based on conversations on chat platforms such as LINE does not improve the food delivery user experience. A solution to these issues is needed.

[2049] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving conversation data, means for preprocessing the received conversation data, means for analyzing context and tone based on the preprocessed conversation data, means for generating appropriate information based on the analysis results, means for returning the generated information, and means for suggesting food delivery menus. This enables the user to quickly and appropriately receive food delivery menu suggestions based on conversations on chat platforms such as LINE.

[2050] "Conversation Data" refers to the text and statements sent and received by Users on the chat platform.

[2051] "Means for receiving" refers to a function or interface that allows the server to obtain data from the outside.

[2052] "Preprocessing means" refers to the processes and algorithms that format acquired conversation data and extract necessary items to make it easier to analyze.

[2053] "Means for analyzing context and tone" refers to methods for understanding the meaning and sentiment of conversational data using natural language processing and sentiment analysis techniques.

[2054] "Generating means" refers to the processes and algorithms used to generate the necessary information and recommendations based on the analysis results.

[2055] "Means for replying" refers to the interface and pathway for sending generated information and suggestions to the user.

[2056] "Means for suggesting food delivery menus" refers to systems and processes for suggesting appropriate food and beverage ordering menus based on keywords and sentiments extracted from conversation data.

[2057] This invention is a system that suggests food delivery menus in real time based on conversations between users on chat platforms such as LINE. This system operates on both the server and the device side. The server side analyzes the conversation data and performs a series of processes to generate and return appropriate information. On the device side, messages are sent and received through a user interface.

[2058] Server-side processing

[2059] The server first receives new messages from the LINE Platform in real time using a notification mechanism such as a webhook. The received data includes the message content, sender ID, group ID, and timestamp. Next, the server preprocesses the received messages and performs natural language processing (NLP). This preprocessing phase involves tokenization, keyword extraction, and morphological analysis. It also uses a sentiment analysis engine to identify the emotional state of the message and filter out irrelevant messages and spam.

[2060] Once preprocessed, the data is input into a generative AI model. The generative AI model generates a response to suggest the optimal food delivery menu, taking into account the context, past message history, and sentiment analysis results. During this process, it may obtain necessary information using external APIs (e.g., food delivery APIs). The generated response is sent to the user's group chat via the official LINE API. This series of processes allows the system to suggest appropriate food delivery menus in real time based on the user's conversations on the chat platform.

[2061] Terminal side processing

[2062] A user uses the LINE app on their device to enter a message into a group chat. When the user presses the send button, the message is sent via the LINE server. The received message is displayed on the chat screen so that the user can view it. Reply messages from the server are also received in the same way and displayed on the chat screen.

[2063] Specific examples

[2064] 1. Lunch consultation

[2065] User A: "What are you having for lunch?"

[2066] User B: "I like Italian food."

[2067] Server: "This pizza place is great! (Rating: 4.5)"

[2068] To achieve this process, the server uses the following software and hardware: First, a web framework such as Flask is used to receive and process webhook notifications, then TextBlob and other natural language processing libraries are used for sentiment analysis and keyword extraction, and the official LINE API is used to send the final reply message.

[2069] Prompt Sentence Examples

[2070] "A user says, 'I'd like Italian food.' Please suggest delivery menu items based on this message."

[2071] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2072] Step 1:

[2073] The server receives new messages from the LINE Platform in real time. The received data includes the message content, sender ID, group ID, and timestamp. It uses the Webhook function to receive notifications from the LINE server. The input is new message data, and the output is the received message information.

[2074] Step 2:

[2075] The server preprocesses the received message. Specifically, it performs natural language processing (NLP) using the TextBlob library to split the message into tokens and extract key keywords and phrases. At the same time, it performs sentiment analysis to identify positive, negative, and neutral emotional states. The input is the received message information, and the output is the analysis results, extracted keywords, and emotional states.

[2076] Step 3:

[2077] The server inputs the preprocessed data into a generative AI model, which takes into account the context, tone, past message history, and sentiment analysis results to generate the optimal response. This process also involves using a food delivery API to obtain relevant menu information. The inputs are the analysis results and extracted keywords, and the output is the generated response message.

[2078] Step 4:

[2079] The server packages the generated response message and sends it to the user's group chat via the official LINE API. The input is the generated response message, and the output is the sent reply message. This allows users to receive appropriate food delivery menu suggestions in real time in the group chat.

[2080] Step 5:

[2081] The user checks the received message using the LINE app on their device. The device displays the message received from the LINE server on the chat screen. The input is the message received from the LINE server, and the output is the message displayed on the user's device. This allows the user to decide on their next action and order food delivery.

[2082] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2083] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2084] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2085] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2086] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2087] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2088] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2089] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2090] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2091] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2092] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2093] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2094] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2095] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2096] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2097] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2098] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2099] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2100] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2101] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2102] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2103] The following is further disclosed regarding the above embodiment.

[2104] (Claim 1)

[2105] means for receiving conversation data;

[2106] means for preprocessing received conversation data;

[2107] A means of analyzing context and tone based on preprocessed conversation data;

[2108] means for generating appropriate information based on the analysis results;

[2109] The system includes a means for returning the generated information.

[2110] (Claim 2)

[2111] 10. The system of claim 1, further comprising means for extracting key keywords from the preprocessed conversation data.

[2112] (Claim 3)

[2113] 10. The system of claim 1, further comprising means for analyzing the tone of the speech data.

[2114] (Claim 4)

[2115] 10. The system according to claim 1, further comprising means for generating information based on the user's physical condition and mood.

[2116] (Claim 5)

[2117] 10. The system of claim 1, further comprising means for obtaining information using an external API.

[2118] "Example 1"

[2119] (Claim 1)

[2120] means for receiving conversation data;

[2121] means for verifying the integrity of received conversation data;

[2122] A means of dividing the received conversation data into tokens and extracting key keywords and phrases;

[2123] A means for performing morphological analysis of conversation data and adding part-of-speech tags;

[2124] means for identifying an emotional state of the speech data;

[2125] A means for inputting prompt sentences into the generative AI model based on the preprocessed conversation data;

[2126] A means for generating appropriate information using a generative AI model;

[2127] The system includes a means for returning the generated information.

[2128] (Claim 2)

[2129] 2. The system according to claim 1, further comprising means for acquiring necessary information from an external information source based on the extracted main keywords.

[2130] (Claim 3)

[2131] 10. The system of claim 1, further comprising means for processing metadata including a sender ID, a group ID, and a timestamp of the conversation data.

[2132] "Application Example 1"

[2133] (Claim 1)

[2134] means for receiving conversation data;

[2135] means for preprocessing received conversation data;

[2136] A means of analyzing context and tone based on preprocessed conversation data;

[2137] means for generating appropriate information based on the analysis results;

[2138] means for returning the generated information;

[2139] a means for recommending content based on the preprocessed conversation data;

[2140] The system includes means for accessing an external database to obtain recommended content in response to the received conversation data.

[2141] (Claim 2)

[2142] 10. The system of claim 1, further comprising means for extracting key keywords from the preprocessed conversation data.

[2143] (Claim 3)

[2144] 10. The system of claim 1, further comprising means for analyzing the tone of the speech data.

[2145] "Example 2: Combining Emotion Engines"

[2146] (Claim 1)

[2147] means for receiving conversation data in real time from a communications platform;

[2148] A means to verify the integrity of received conversation data and filter out malformed or spam data;

[2149] A means of dividing the received conversation data into tokens using a natural language processing engine and extracting key keywords and phrases;

[2150] means for analyzing the context and emotional state of the conversation data using morphological analysis and an emotion engine;

[2151] A means for generating an appropriate prompt sentence based on the analysis result and generating information using a generative AI model;

[2152] A means for acquiring supplementary information using an external information acquisition API as needed;

[2153] means for packaging the generated information and preparing a reply message together with the metadata;

[2154] The system includes means for transmitting the generated reply message through an API of a communications platform.

[2155] (Claim 2)

[2156] 10. The system of claim 1, further comprising a natural language processing engine for extracting key keywords and phrases from the received conversation data.

[2157] (Claim 3)

[2158] 10. The system of claim 1, further comprising an emotion engine that identifies an emotional state of the received speech data.

[2159] "Application example 2 when combining emotion engines"

[2160] (Claim 1)

[2161] means for receiving conversation data;

[2162] means for preprocessing received conversation data;

[2163] A means of analyzing context and tone based on preprocessed conversation data;

[2164] means for generating appropriate information based on the analysis results;

[2165] means for returning the generated information;

[2166] A system including a means for suggesting food delivery menus.

[2167] (Claim 2)

[2168] 10. The system of claim 1, further comprising means for extracting key keywords from the preprocessed conversation data.

[2169] (Claim 3)

[2170] 10. The system of claim 1, further comprising means for analyzing the tone of the speech data. [Explanation of symbols]

[2171] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving conversation data; means for preprocessing received conversation data; A means of analyzing context and tone based on preprocessed conversation data; means for generating appropriate information based on the analysis results; The system includes a means for returning the generated information.

2. 10. The system of claim 1, further comprising means for extracting key keywords from the preprocessed conversation data.

3. 10. The system of claim 1, further comprising means for analyzing the tone of the speech data.

4. 2. The system according to claim 1, further comprising means for generating information based on the user's physical condition and mood.

5. The system of claim 1 further comprising means for obtaining information using an external API.

Images