system

Abstract

We provide the system. [Solution] A means for receiving user input, Analysis means including a generative artificial intelligence for analyzing the user's input, A generation means for generating a response based on the analysis results, A transmission means for sending the generated response to the user, A storage means for recording the user's input and the generated response, An analysis means for analyzing the recorded data, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, the shortage of human resources has become serious in retail enterprises operating stores. Also, it is difficult to collect real voices of consumers. Due to this problem, it has become difficult to improve customer satisfaction in stores and to provide products and services that meet market needs. To address this issue, there is a need for means to improve customer satisfaction and business efficiency by responding to customers' questions in real time and collecting and analyzing conversation data.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides the following means: a system including means for receiving user input, analysis means including a generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, and analysis means for analyzing the recorded data. This system enables automatic responses to user questions, as well as the storage and analysis of the conversation data, thereby improving customer satisfaction and operational efficiency.

[0006] "User input" refers to inquiries and requests made by customers using the system.

[0007] "Generative artificial intelligence" refers to algorithms and models that analyze user input data and generate appropriate responses or processes.

[0008] "Analysis means" refers to methods and devices that use generative artificial intelligence to analyze user input and determine actions based on that analysis.

[0009] "Generation means" refers to methods or apparatus for producing an appropriate response based on the analysis results obtained by the analysis means.

[0010] "Transmission means" refers to the method or device for delivering the response generated by the generation means to the user.

[0011] "Storage means" refers to methods and devices for recording responses created by user input or generation means in a database or similar format.

[0012] "Analysis means" refers to methods and devices for analyzing data recorded by storage means and extracting useful information.

[0013] A "system" refers to a collection of processing units and software that have the functions of receiving user input, analyzing it, generating and transmitting responses, and storing and analyzing data. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system designed to solve problems related to labor shortages and the collection of authentic customer feedback. This system receives user input, analyzes the input using generative artificial intelligence, generates an appropriate response and sends it to the user, and then analyzes the data to understand customer needs.

[0036] System Configuration

[0037] A means (terminal) for receiving user input.

[0038] Users send questions and inquiries to the store's official account using messaging apps such as LINE. For example, a user might send a message saying, "Please tell me the availability of the product."

[0039] Analysis means (server)

[0040] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI) that acts as an analysis tool. This AI uses a natural language processing engine to analyze the content of the user's message. For example, it analyzes the input message, "Please tell me the stock status of the product."

[0041] Generation means (server)

[0042] Based on the analysis performed by the analysis means, the generation means generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0043] Transmission method (server and terminal)

[0044] The generated response is sent to the user using the LINE API. The server sends the generated response to the user's device, and the user receives the response in their messaging app. For example, the user's smartphone might display a message saying, "We currently have very little of this product in stock. Please hurry."

[0045] Storage method (server)

[0046] The server records user input and generated responses. This ensures that all conversation data is stored in a database. The stored data includes, for example, a conversation timestamp and user ID.

[0047] Analysis method (server)

[0048] The stored data is analyzed by analytical tools. These tools utilize the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that the product is popular.

[0049] Specific example

[0050] 1. The user asks a question.

[0051] User: "When will the new shoes be in stock?"

[0052] Device (user's smartphone): Send a message to the store's official LINE account.

[0053] 2. Receive and analyze user messages.

[0054] Server: Receives messages via LINE API.

[0055] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0056] 3. Generate and send an appropriate response.

[0057] Server: Based on the analysis results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[0058] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0059] 4. Storage and analysis of conversation data

[0060] Server: Stores the user's questions and generated responses.

[0061] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing.

[0062] This system allows stores to effectively respond to user inquiries, analyze that data to understand customer needs, and thereby improve the operational efficiency of their stores.

[0063] The following describes the processing flow.

[0064] Step 1:

[0065] Users send messages to the store's official account via the LINE app. These messages may contain questions or inquiries.

[0066] Step 2:

[0067] The device (the user's smartphone) sends this message to the LINE server. The LINE server receives the message and forwards it to the store's official account server.

[0068] Step 3:

[0069] The server receives messages from users using the LINE API. It then prepares the received messages for transmission to a generative artificial intelligence system, which is used for analysis.

[0070] Step 4:

[0071] The server sends the user's message to the generative artificial intelligence API. The generative artificial intelligence uses a natural language processing engine to analyze the message content. For example, it might analyze the message, "Please tell me the product's stock status."

[0072] Step 5:

[0073] Generative artificial intelligence generates appropriate responses based on analysis results. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0074] Step 6:

[0075] The generative artificial intelligence generates a response, which is then sent back to the server's API. The server receives this response and prepares it for transmission to the user.

[0076] Step 7:

[0077] The server uses the LINE API to send a response generated by generative artificial intelligence to the user's device. The device (the user's smartphone) receives the message sent from the server and displays the message to the user saying, "There are only a few of that product left in stock. Please hurry."

[0078] Step 8:

[0079] The server records user input and generated responses in a database, which serves as a storage mechanism. The recorded data also includes metadata such as conversation timestamps and user IDs.

[0080] Step 9:

[0081] Data recorded by storage devices is analyzed by analysis devices. The server analyzes the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it indicates that the product is popular.

[0082] Step 10:

[0083] The server provides feedback based on the extracted information to improve business strategies. This enables the development of appropriate inventory management and marketing strategies.

[0084] (Example 1)

[0085] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0086] Traditional customer service systems face challenges such as a shortage of personnel and difficulty in collecting authentic customer feedback. In particular, responding quickly to a large volume of inquiries requires a large workforce, posing problems in terms of cost and efficiency. Furthermore, there is a lack of effective means to analyze collected customer feedback and understand customer needs. As a result, companies often miss opportunities to improve customer satisfaction.

[0087] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0088] In this invention, the server includes means for receiving user input, means for receiving the user input using communication technology, analysis means including a generative artificial intelligence that analyzes the received input using a natural language processing engine, generation means including an artificial intelligence model that generates a response based on the analysis results, transmission means for sending the generated response to the user using communication technology, data management means for recording the user input and the generated response, and data analysis means for analyzing the recorded data. This makes it possible to efficiently respond to customer inquiries and understand customer needs by analyzing the data.

[0089] "User input" refers to information, questions, or requests that a user submits through the system.

[0090] "Communication technology" refers to the technologies and protocols used to send and receive data between digital devices.

[0091] "Means of receiving" refers to a digital system that has the function of capturing input data sent from a user and transferring it to other components within the system for processing.

[0092] A "natural language processing engine" is a general term for the technologies and algorithms that enable computers to understand and analyze human language.

[0093] "Generative artificial intelligence" refers to artificial intelligence systems that can automatically generate text and data similar to those generated by humans.

[0094] "Analysis means" refers to a combination of software and hardware used to analyze received data and understand its contents.

[0095] "Analysis results" refers to the output of user input data analyzed by a natural language processing engine.

[0096] "Generation means" refers to the function of a system that generates appropriate responses or data based on analysis results.

[0097] An "artificial intelligence model" refers to a data processing model that has been trained to perform a specific task using machine learning or deep learning.

[0098] "Transmission means" refers to the function of a communication system that delivers the generated response to the user.

[0099] "Data management means" refers to the system's function for recording and storing user input and generated responses.

[0100] "Data analysis methods" refer to techniques for analyzing recorded data to extract important trends and information.

[0101] A "data platform" refers to the infrastructure and associated software systems used for storing, managing, and processing data.

[0102] This invention is a system designed to address challenges such as labor shortages and the collection of authentic customer feedback. It accepts user input, analyzes the input using generative artificial intelligence, generates an appropriate response and sends it to the user, and then analyzes the data to understand customer needs. An embodiment of this system will be described in detail below.

[0103] System Overview

[0104] This system includes the following components:

[0105] A means (terminal) for receiving user input.

[0106] A means (server) for receiving input data using communication technology.

[0107] Analysis means (server) including a generative artificial intelligence that analyzes input using a natural language processing engine.

[0108] A generation means (server) including an artificial intelligence model that generates a response based on the analysis results.

[0109] Means (server and terminal) for sending the generated response to the user.

[0110] A data management means (server) for recording user input and generated responses.

[0111] Data analysis means (server) for analyzing recorded data

[0112] Detailed explanation

[0113] User input reception

[0114] Users can use smartphones or other devices to send questions and inquiries to the store's official account via messaging apps. For example, they might send a message asking, "Please tell me the availability of this product."

[0115] Message reception and analysis

[0116] The server receives messages from users using the LINE API and other communication technologies. The received messages are passed to a generative artificial intelligence (AI), which is used for analysis by a natural language processing engine. For example, it analyzes a message such as "Please tell me the stock status of the product" and extracts keywords and context.

[0117] Response generation

[0118] Based on the analysis performed by the analysis tool, the artificial intelligence model, which acts as the generation tool, generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0119] Sending a response

[0120] The generated response is then sent back to the user's device using the LINE API or other communication technologies. The server sends the generated response to the user's device, and the user receives and confirms the message in their messaging app. For example, a message such as "We currently have very little of this product in stock. Please hurry." might appear on the smartphone.

[0121] Data storage and analysis

[0122] The server records user input and generated responses. This ensures all conversation data is stored on the data platform. The stored data can then be analyzed by data analytics tools to extract customer needs and trends. For example, it might detect that many users are inquiring about a particular product, revealing its popularity.

[0123] Explanation of specific examples

[0124] The user asks a question.

[0125] User: "When will the new shoes be in stock?"

[0126] Device (user's smartphone): Send a message to the store's official account.

[0127] Receive and analyze user messages.

[0128] Server: Receives messages via LINE API.

[0129] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0130] Generate and send an appropriate response.

[0131] Server: Based on the analysis results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[0132] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0133] Storage and analysis of conversation data

[0134] Server: Stores the user's questions and generated responses.

[0135] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing.

[0136] Examples of prompts for generative AI models

[0137] When a user sends the message "When will the new shoes be in stock?", the following prompt is entered into the generative artificial intelligence:

[0138] A user sent the message, "When will the new shoes be in stock?" Please refer to the inventory database and generate a response.

[0139] This system allows stores to efficiently respond to customer inquiries, analyze the data to understand customer needs, and significantly improve the operational efficiency of their stores.

[0140] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0141] Step 1:

[0142] User input acceptance

[0143] User: Uses a smartphone or computer to send questions and inquiries to the store's official account via a messaging app. The user enters their specific question and presses the send button. Example: "Please tell me the product's stock status."

[0144] Input: Text data sent by the user via a messaging app.

[0145] Output: The message sent by the user arrives at the application server.

[0146] Step 2:

[0147] Message received

[0148] Server: Uses the LINE API to receive messages sent by users in real time. Upon receiving a message, it retrieves relevant data such as the message content, user ID, and timestamp.

[0149] Input: Metadata such as message text from the user, user ID, and timestamp.

[0150] Output: Structured data for passing received message data to the analysis tool.

[0151] Step 3:

[0152] Message parsing

[0153] Server: Passes received message data to a generative artificial intelligence system, which then analyzes it using a natural language processing engine. Specifically, it tokenizes the text and extracts context and keywords.

[0154] Input: Structured message data (user message, user ID, timestamp).

[0155] Output: Data including keywords and intent as analysis results.

[0156] Step 4:

[0157] Response generation

[0158] Server: Based on the analysis results, the generative artificial intelligence generates an appropriate response. Specifically, it refers to the inventory management database and retrieves information corresponding to the user's question. For example, in response to "Please tell me the inventory status of the product," it generates a sentence such as "Currently, there is only a small amount of stock left of that product."

[0159] Input: Analysis results and corresponding data (e.g., inventory status).

[0160] Output: The generated response message.

[0161] Step 5:

[0162] Sending a response

[0163] Server: The generated response is sent to the user's device using the LINE API. Before sending, a process is performed to verify the format and security of the response content.

[0164] Device: The user's smartphone receives the message via a messaging app and notifies the user.

[0165] Input: The generated response message.

[0166] Output: The response message displayed on the user's terminal.

[0167] Step 6:

[0168] Data storage

[0169] Server: Records user input and generated responses using data management means. For example, it stores them in a database along with the conversation timestamp and user ID.

[0170] Input: User input data and generated response data.

[0171] Output: Conversation history stored in the database.

[0172] Step 7:

[0173] Data analysis

[0174] Server: Analyzes stored conversation data using analysis tools. For example, machine learning algorithms are used to extract interest levels and trends for specific products.

[0175] Input: Conversation history stored in the database.

[0176] Output: Reports and data that clearly indicate customer needs and market trends.

[0177] The above steps result in a system that efficiently responds to user inquiries and allows for the understanding of customer needs based on that data.

[0178] (Application Example 1)

[0179] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0180] Traditional customer support systems faced challenges such as staff shortages and difficulty in gathering authentic customer feedback. Furthermore, limited access to product information within stores made prompt and appropriate responses difficult. This hindered improvements in customer satisfaction and accurate understanding of customer needs.

[0181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0182] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, analysis means for analyzing the recorded data, scanning means for the user to identify products in the store, and speech recognition means for receiving and analyzing the user's voice input. This enables customers to easily obtain product information in the store using their smartphones and to receive quick and appropriate responses to their voice and text questions. Furthermore, by analyzing all inquiry data, it is possible to accurately grasp customer needs and trends and improve services.

[0183] "Means for receiving user input" refers to an interface that allows users to input text or voice from a device.

[0184] "Generative artificial intelligence" refers to artificial intelligence that analyzes input text or audio and generates appropriate responses based on that content.

[0185] "Analysis means" refers to a method for analyzing data input by a user using generative artificial intelligence.

[0186] "Generation means" refers to means for generating an appropriate response to the user based on the results analyzed by the analysis means.

[0187] "Transmission means" refers to the means for sending the generated response to the user.

[0188] "Storage means" refers to means for recording and saving user input and generated responses.

[0189] "Analysis methods" refer to means of analyzing recorded data to understand customer needs and trends.

[0190] "Scanning method" refers to a means by which users can obtain product information by scanning barcodes or QR codes (registered trademarks) within a store.

[0191] A "speech recognition method" is a means for converting and analyzing content entered by a user via voice into text.

[0192] This invention is a system for users to obtain product information in a physical store and receive real-time responses to their questions. The following describes a specific configuration for realizing this system.

[0193] First, the user scans the product barcode using a smartphone app. The product barcode data is captured by the scanning device and transmitted to an analysis device, including a generative artificial intelligence system. Similarly, when voice input is used, the voice data is converted into text data via a voice recognition device and passed to the analysis device.

[0194] In the analysis method, a generative artificial intelligence analyzes the input data and uses a natural language processing engine to understand the user's question. For example, if a user asks "What is the price of this product?" by voice, a speech recognition device converts it to text, and that text is passed to the generative AI model.

[0195] Next, the generation mechanism operates and generates an appropriate response based on the analysis results. Response generation involves integration with inventory management systems and product databases. For example, product information is retrieved, and a response such as "The price of this product is 1980 yen" is generated.

[0196] The generated response is sent to the user's smartphone via the LINE API using the designated transmission method. The user can then view this response in real time using the smartphone app.

[0197] Furthermore, user input and generated responses are recorded by storage devices and stored on a data platform. The stored data can be analyzed by analytical devices to extract customer needs and trends. For example, if many users inquired about "new shoes," it could be analyzed that there is a high demand for that product.

[0198] A concrete example of a prompt would be a user asking, "How many days can I use this product?" In response to this input, the generative AI model analyzes the input and generates a response such as, "The usage period is approximately 30 days."

[0199] In this way, users can easily obtain product information and receive real-time answers to their questions, thereby improving customer satisfaction. Furthermore, by utilizing analytical tools, stores can accurately grasp customer demand and trends, enabling them to respond quickly.

[0200] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0201] Step 1:

[0202] The user uses their smartphone to scan product barcodes in the store. The scanned barcode data is captured by the scanning device and sent to a database within the smartphone app. The input is the identification data of the product barcode, and this data is ready to be sent to the server as output.

[0203] Step 2:

[0204] When a user performs voice input, they input their question by voice using their smartphone's microphone. This voice data passes through a speech recognition system, is converted into text data, and sent to an analysis system. The input is voice data, and the converted text data is passed to the generating AI model as output.

[0205] Step 3:

[0206] The server receives messages from users via the LINE API. These messages are received as text data and passed to a generative artificial intelligence (AI) system for analysis. The input is a text message, and the output is data ready for analysis, which is then passed to the generative AI model.

[0207] Step 4:

[0208] In the analysis method, a generative artificial intelligence analyzes the input data (text data or product barcode data). Using a natural language processing engine, it understands the user's question and extracts information to create an appropriate response. The input is either text data or product barcode data, and the analysis results are passed to the generation method as output.

[0209] Step 5:

[0210] The generation mechanism generates an appropriate response based on the analysis results. For example, it might refer to an inventory management system or product database to generate a response such as, "The price of this product is 1980 yen." The input is the analysis results, and the generated response is passed to the transmission mechanism as the output.

[0211] Step 6:

[0212] The transmission method uses the LINE API to send the generated response to the user's smartphone. The user can view this response in real time on the smartphone app. The input is the generated response, and the output is the response message that is displayed on the user's device.

[0213] Step 7:

[0214] The storage mechanism records and stores user input (text messages or voice input) and the generated responses. This data is stored on a data platform for later analysis. Inputs are user inputs and generated responses, and the recorded data is stored as output.

[0215] Step 8:

[0216] The analysis method involves analyzing stored data to understand customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that demand for that product is increasing. The input is stored data, and the output is the analysis results.

[0217] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0218] This invention relates to a system that analyzes user input and generates appropriate responses. In particular, by incorporating an emotion engine, this system can recognize the user's emotions and adjust its responses accordingly. The aim of this system is to provide a more sophisticated conversational experience and improve customer satisfaction.

[0219] System Configuration

[0220] A means (terminal) for receiving user input.

[0221] Users send questions and inquiries to the store's official account using messaging apps such as LINE. For example, a user might send a message saying, "Please tell me the availability of the product."

[0222] Analysis means (server)

[0223] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI) that acts as an analysis tool. This AI uses a natural language processing engine to analyze the content of the user's message. For example, it analyzes the input message, "Please tell me the stock status of the product."

[0224] Emotional Engine

[0225] Based on the message content analyzed by the generative artificial intelligence, the emotion engine recognizes the user's emotions (e.g., joy, anger, sadness). The emotion engine also adjusts the response according to the user's emotions. For example, if the user is expressing dissatisfaction, the response will include a polite apology.

[0226] Generation means (server)

[0227] Based on the analysis results and the emotion engine's findings, the generation mechanism generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "We currently have very little of that item in stock. Please hurry." If the emotion engine determines that the user is dissatisfied, it might generate a response such as, "We are sorry, but we have very little of this item in stock. We apologize for the inconvenience."

[0228] Transmission method (server and terminal)

[0229] The generated response is sent to the user using the LINE API. The server sends the generated response to the user's device, and the user receives the response in their messaging app. For example, the user's smartphone might display a message saying, "We currently have very little of this product in stock. Please hurry."

[0230] Storage method (server)

[0231] The server records user input and generated responses in a database, which serves as a storage mechanism. This ensures that all conversation data is stored in the database. The stored data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results from the sentiment engine.

[0232] Analysis method (server)

[0233] The stored data is analyzed using analytical tools. The server utilizes the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that the product is popular. It is also possible to analyze trends in customer satisfaction using data from the emotion engine.

[0234] Specific example

[0235] 1. The user asks a question.

[0236] User: "When will the new shoes be in stock?"

[0237] Device (user's smartphone): Send a message to the store's official LINE account.

[0238] 2. Receive and analyze user messages.

[0239] Server: Receives messages via LINE API.

[0240] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0241] 3. Emotion recognition by an emotion engine

[0242] Emotion Engine: Recognizes emotions from user messages and determines if the user is excited.

[0243] 4. Generate and send an appropriate response.

[0244] Server: Based on the analysis results and emotion recognition results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[0245] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0246] 5. Storage and analysis of conversation data

[0247] Server: Stores user questions, generated responses, and sentiment recognition results.

[0248] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing or to understand customers' emotional responses.

[0249] This system allows for more personalized responses to user inquiries, adjusting them based on their emotions. Furthermore, analysis of conversational data can reveal customer needs and emotional tendencies, which can then be incorporated into business strategies.

[0250] The following describes the processing flow.

[0251] Step 1:

[0252] Users send messages to the store's official account via the LINE app. These messages may contain questions or inquiries.

[0253] Step 2:

[0254] The device (the user's smartphone) sends this message to the LINE server. The LINE server receives the message and forwards it to the store's official account server.

[0255] Step 3:

[0256] The server receives messages from users using the LINE API. It then prepares the received messages for transmission to a generative artificial intelligence system, which is used for analysis.

[0257] Step 4:

[0258] The server sends the user's message to the generative artificial intelligence API. The generative artificial intelligence uses a natural language processing engine to analyze the message content. For example, it might analyze the message, "Please tell me the product's stock status."

[0259] Step 5:

[0260] The generative artificial intelligence outputs the analysis results and passes them to the emotion engine. The emotion engine recognizes the user's emotions based on the message content. For example, it might determine from the message that the user is feeling dissatisfied.

[0261] Step 6:

[0262] Based on the results of generative artificial intelligence and an emotion engine, the server generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "We currently have very little of that item in stock. Please hurry." If it determines that the user is expressing dissatisfaction, it will generate a response such as, "We are sorry, but we have very little of this item in stock. We apologize for the inconvenience."

[0263] Step 7:

[0264] The server sends the generated response to the user's device using the LINE API. The device (the user's smartphone) receives the message sent from the server and displays a response to the user such as, "We currently have very little of that item in stock. Please hurry," or "We are sorry, but we have very little of that item in stock. We apologize for the inconvenience."

[0265] Step 8:

[0266] The server records user input and generated responses in a database, which serves as a storage mechanism. The recorded data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results from the sentiment engine.

[0267] Step 9:

[0268] The server analyzes stored data using analytical tools. For example, it uses stored conversation data to extract customer needs and trends. It also uses emotion engine data to analyze trends in customer satisfaction.

[0269] Step 10:

[0270] The server provides feedback based on the extracted information to improve business strategies. This enables the development of appropriate inventory management and marketing strategies.

[0271] (Example 2)

[0272] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0273] Traditional user-system interactions suffered from a problem where responses were uniform because they did not take user emotions into account. As a result, users could not receive responses that suited their feelings, leading to decreased satisfaction. Furthermore, it was difficult to gain deep insights through the analysis of dialogue data, making it challenging to grasp customer needs and trends.

[0274] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0275] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing user input, recognition means including an emotion engine for recognizing user emotions based on the message content analyzed by the generative artificial intelligence, generation means for generating a response based on the analysis results and the results of the recognition means, transmission means for sending the generated response to the user, storage means for recording user input and generated responses, and analysis means for analyzing the recorded data. This makes it possible to generate personalized responses that correspond to the user's emotions, thereby improving user satisfaction. Furthermore, analysis based on the stored dialogue data allows for efficient understanding of customer needs and trends, contributing to business strategy.

[0276] "Means of receiving user input" refers to an interface for users to send messages or inquiries to a system, and typically refers to messaging applications or web forms.

[0277] The "analysis means" is a device or system that uses generative artificial intelligence to analyze the content of a message sent by a user and performs processing to understand its intention and content.

[0278] The "generative artificial intelligence" is an artificial intelligence that has the ability to analyze a user's input by a natural language processing engine and understand and interpret its content.

[0279] The "natural language processing engine" is a software system that analyzes text data as human language and understands context and intention.

[0280] The "emotion engine" is a computer program or device that recognizes a user's emotion based on the analyzed message content and adjusts responses.

[0281] The "recognition means" is a function that includes an emotion engine and executes a process of recognizing a user's emotion based on the information obtained from the analysis means.

[0282] The "generation means" is a device or program that generates an appropriate response to a user based on the analysis result and the emotion recognition result.

[0283] The "transmission means" is an interface for transmitting the response generated by the generation means to the user, and usually refers to the API of a messaging application.

[0284] The "storage means" is a database or storage device that records a user's input and the generated response and stores them in a retrievable state.

[0285] The "analysis means" is a process and device that analyzes the data stored in the storage means and extracts customer needs and trends.

[0286] The "data platform" is an infrastructure system or software that manages the stored data and retrieves and analyzes it as needed.

[0287] The present invention relates to a system that analyzes input from a user and generates an appropriate response. In particular, by incorporating an emotion engine, this system can recognize the user's emotion and adjust the response based on it. This system aims to provide a more advanced dialogue experience and improve customer satisfaction.

[0288] System Configuration

[0289] Means for receiving user input

[0290] The user uses a messaging application (e.g., LINE) to send messages or inquiries to the official account of the store. For this purpose, the user's terminal (such as a smartphone or tablet) is utilized.

[0291] Analysis means

[0292] The server receives messages from the user using the LINE API. The received messages are passed to a generative artificial intelligence (AI), and the message content is analyzed using a natural language processing engine (e.g., Google (registered trademark) Cloud Natural Language API).

[0293] Recognition means including an emotion engine

[0294] Based on the analyzed message content, an emotion engine (e.g., IBM Watson (registered trademark) Tone Analyzer) recognizes the user's emotion. For example, when the user sends a message "When will the new shoes be in stock?" and is determined to be excited.

[0295] Generation means

[0296] Based on the analysis results and emotion recognition results, the server generates an appropriate response. In this process, generative artificial intelligence is utilized. For example, by referring to the inventory management database, a response such as "The new shoes are scheduled to arrive next Monday. Please look forward to it!" is generated.

[0297] Transmission means

[0298] The generated response is sent to the user's terminal using the LINE API. The server sends the generated response to the user's smartphone, and the user receives the response in the messaging app.

[0299] Storage means

[0300] The server stores the user's input and the generated response in the database. The stored data also includes metadata such as the conversation timestamp, user ID, and emotion recognition results.

[0301] Analysis means

[0302] The stored data is analyzed by the analysis means. The server utilizes the stored conversation data to extract customer needs and trends. For example, if there are many inquiries about a specific product, it indicates that the product is popular. It is also possible to analyze the trend of customer satisfaction using the data of the emotion engine.

[0303] Specific example

[0304] 1. The user asks a question

[0305] User: "When will the new shoes arrive?"

[0306] Terminal (user's smartphone): Sends a message to the store's LINE official account.

[0307] 2. Receive and analyze the user's message

[0308] Server: Receives messages via LINE API.

[0309] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0310] 3. Emotion recognition by an emotion engine

[0311] Emotion Engine: Recognizes "excitement" from the user's message.

[0312] 4. Generate and send an appropriate response.

[0313] Server: Based on the analysis results and emotion recognition results, it generates the response, "The new shoes will be in stock next Monday. Stay tuned!"

[0314] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0315] 5. Storage and analysis of conversation data

[0316] Server: Stores user questions, generated responses, and sentiment recognition results in a database.

[0317] Server: Based on stored data, it analyzes and understands trends in interest in specific products and customers' emotional responses.

[0318] Example of a prompt

[0319] "Generate a response for when a user inquires about the new shoes. The user is excited."

[0320] "Generate a response for when a user is dissatisfied with the stock situation."

[0321] This system allows for more personalized responses to user inquiries, as the response is adjusted according to the user's emotions. Furthermore, analysis of stored conversation data allows for the understanding of customer needs and emotional tendencies, which can then be reflected in business strategies.

[0322] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0323] Step 1:

[0324] The user sends a message using the LINE messaging app.

[0325] In terms of the specific action, the user uses the LINE app on their smartphone to type and send the text message "Please tell me the product's stock status" to the store's official account. At this point, the input is the text message the user entered into the LINE app. The device then sends this text message.

[0326] Step 2:

[0327] The server uses the LINE API to receive user messages.

[0328] In terms of specific operation, the server receives message data sent by the user via the LINE API. This message data includes the sender's user ID and the message body. The input is the text message sent by the user, and the output is the message data stored on the server.

[0329] Step 3:

[0330] The received message is analyzed by a generative artificial intelligence.

[0331] The server passes the received message data to a generative artificial intelligence (e.g., Google Cloud Natural Language API). Specifically, the generative AI analyzes the text message and extracts the main themes and content. The input is the user's message data, and the output is the analyzed message content (e.g., "Inquiry about product availability").

[0332] Step 4:

[0333] The emotion engine recognizes the user's emotions based on the analysis results.

[0334] Based on the analysis results, the server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to determine the emotions contained in the message. Specifically, the emotion engine analyzes the emotional characteristics of the message text and recognizes emotions such as "dissatisfaction," "excitement," and "joy." The input is the analyzed message content, and the output is the recognized emotion data.

[0335] Step 5:

[0336] The server generates a response based on the analysis results and emotion recognition results.

[0337] Based on the analysis results and sentiment recognition results, the server generates an appropriate response. Specifically, the response generation algorithm refers to the inventory management database and creates an appropriate response statement. For example, if the stock is low, it will generate a response such as, "We currently have very little of that product left in stock. Please hurry." The input is the analysis results and sentiment recognition results, and the output is the generated response.

[0338] Step 6:

[0339] The server sends the generated response to the user.

[0340] The generated response is sent to the user via the LINE API. Specifically, the server uses the LINE API to send the generated text message to the user's smartphone. The input is the generated response data, and the output is the message displayed on the user's device.

[0341] Step 7:

[0342] The server saves messages and responses to the database.

[0343] In practice, the server records all user messages and generated responses in a database. This includes metadata such as message content, timestamp, user ID, and sentiment recognition results. The input is conversation data and metadata, and the output is the record stored in the database.

[0344] Step 8:

[0345] The system analyzes data stored on the server to understand trends and customer needs.

[0346] In practice, the server uses stored data for analysis to extract customer needs and trends. For example, if many users are inquiring about a particular product, it indicates that the product is popular. The input is stored conversation data, and the output is trend information and customer needs information as a result of the analysis.

[0347] (Application Example 2)

[0348] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0349] Conventional user input analysis systems have the problem of not being able to respond while considering the user's emotions, making it difficult to achieve sufficient customer satisfaction. Furthermore, even in customer service by store staff in physical stores, it is difficult for staff to quickly grasp the emotional state of customers and respond appropriately. Especially in busy stores, it is difficult for staff to provide individual attention to each customer, which may lead to a decrease in customer satisfaction. In addition, because it is not possible to efficiently utilize the history of conversations with customers and reflect it in future service interactions, the same questions are asked repeatedly.

[0350] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0351] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results and emotion recognition results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, analysis means for analyzing the recorded data, and store clerk support means including smart glasses that recognize customer emotions in real time and suggest appropriate responses. This enables responses that take user emotions into consideration, allowing store clerks to efficiently handle customers in physical stores and improve customer satisfaction.

[0352] "Means of receiving user input" refers to an interface for receiving information and inquiries from users.

[0353] "Analysis means" refers to methods, including generative artificial intelligence, for interpreting user input and understanding its content.

[0354] "Generation means" refers to means for creating an appropriate response to be provided to the user based on the analysis results and emotion recognition results.

[0355] "Transmission means" refers to a communication mechanism for sending the generated response to the user.

[0356] A "storage mechanism" is a function that saves user input and generated responses so that they can be referenced later.

[0357] "Analysis methods" refer to means of analyzing stored data to understand user behavior patterns and trends.

[0358] "Smart glasses" are devices worn by store employees that display information and provide real-time customer emotion recognition results and response suggestions.

[0359] "Staff support tools" are technologies that assist store staff in physical stores and facilitate smooth interactions with customers.

[0360] "Generative artificial intelligence" is an AI technology that analyzes user input data and generates responses based on that data.

[0361] "Emotion recognition results" refer to information used to analyze user input and identify their emotional state.

[0362] "Response suggestions" are pieces of information that suggest the most appropriate answer or action based on the user's emotional state and the content of their inquiry.

[0363] This invention provides a system that analyzes user input, recognizes emotions, and generates a corresponding response. In particular, this system can perform exceptionally well in customer service at physical stores. The embodiments are described below in detail.

[0364] System Configuration

[0365] The system includes the following main components:

[0366] A means (terminal) for receiving user input.

[0367] This is an interface that allows users to make inquiries via voice input or text messages. Smartphones and tablets are examples of this.

[0368] Analysis means (server)

[0369] The server uses a natural language processing engine, including generative artificial intelligence (e.g., spaCy), to analyze user input. This analysis identifies the content of the user's inquiry.

[0370] Emotional Engine

[0371] Based on the user input data analyzed by the analysis tool, the user's emotions are recognized using an emotion engine (e.g., IBM Watson Tone Analyzer). This identifies the emotions the user is experiencing (joy, anger, sadness, etc.).

[0372] Generation means (server)

[0373] This is a means for generating an appropriate response based on the results obtained by the analysis means and the emotion engine. The generative artificial intelligence model creates a response that matches the user's emotions.

[0374] Transmission method (server and terminal)

[0375] The generated response is provided to the user via a transmission method. For example, the response message is sent to the user's smartphone via the LINE API.

[0376] Storage method (server)

[0377] User input and generated responses are stored in a database, which serves as a storage mechanism. This allows for the management of past conversation data and its use for later analysis.

[0378] Analysis method (server)

[0379] The stored data is analyzed using analytical tools. This allows for an understanding of customer behavior patterns and emotional tendencies.

[0380] Shop assistant support device (smart glasses)

[0381] Smart glasses and other store clerk assistance devices are devices that display emotion recognition results and response suggestions in real time when store clerks interact with customers. This allows store clerks to quickly provide the most appropriate response.

[0382] Program Processing Description

[0383] Hardware: Smartphones, tablets, smart glasses, servers

[0384] Software: Speech recognition software (e.g., Google Cloud Speech-to-Text), natural language processing engine (e.g., spaCy), emotion recognition engine (e.g., IBM Watson Tone Analyzer)

[0385] Data processing and calculations:

[0386] The server converts the user's voice input into text (speech recognition).

[0387] Analyze the content of the text using a natural language processing engine.

[0388] Identifying customer emotions with an emotion recognition engine

[0389] Based on the analysis results and emotions, a generative artificial intelligence model generates a response.

[0390] The generated response is provided to the user via a transmission method.

[0391] Specific example

[0392] For example, if a user asks, "How big is this product?" and simultaneously expresses anxiety, the system re-analyzes the question and displays a response on the salesperson's smart glasses saying, "Don't worry. This product should be the perfect size for you."

[0393] Example of a prompt:

[0394] "Design a system that receives voice input from customers expressing questions or complaints about a product, and generates and displays appropriate responses in real time through smart glasses."

[0395] The system as a whole provides a sophisticated conversational experience that takes user emotions into account, improving customer satisfaction in physical stores. Furthermore, analyzing the stored conversation data can be used to improve future business strategies.

[0396] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0397] Step 1:

[0398] The user makes an inquiry.

[0399] Users can use their smartphones or tablets to make inquiries via voice input or text message. For example, they might ask, "How big is this product?"

[0400] Input: User voice or text inquiry

[0401] Output: User voice data or text data

[0402] Step 2:

[0403] Voice input to text conversion

[0404] The device uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's voice input into text data.

[0405] Input: User's voice data

[0406] Output: Text data

[0407] Step 3:

[0408] Analyze user input

[0409] The server uses a natural language processing engine, including generative artificial intelligence (e.g., spaCy), to analyze the user's text input. This allows the content of the inquiry to be identified.

[0410] Input: User's text data

[0411] Output: Analysis results (inquiry details)

[0412] Step 4:

[0413] Recognizing user emotions

[0414] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotions based on the analysis results. For example, it might identify that the user is feeling anxious.

[0415] Input: Analysis results

[0416] Output: Emotion recognition result (user's emotional state)

[0417] Step 5:

[0418] Generate an appropriate response

[0419] Based on the analysis results and emotion recognition results, the server uses a generative artificial intelligence model to generate an appropriate response. For example, it might create a response such as, "Please rest assured. We believe this product is the perfect size for you."

[0420] Input: Analysis results, emotion recognition results

[0421] Output: Generated response

[0422] Step 6:

[0423] Send a response

[0424] The server sends the generated response to the user's terminal via a transmission method. For example, the response message is sent to the user's smartphone via the LINE API.

[0425] Input: Generated response

[0426] Output: Response message displayed on the user's terminal

[0427] Step 7:

[0428] Saving conversation data

[0429] The server saves user input and generated responses to a database. This records the conversation data for later reference.

[0430] Input: User input data, generated response

[0431] Output: Saved conversation data

[0432] Step 8:

[0433] Analysis of conversation data

[0434] The server uses stored conversation data to analyze customer behavior patterns and emotional tendencies. This analysis helps improve future customer interactions and business strategies.

[0435] Input: Saved conversation data

[0436] Output: Analysis results (customer behavior patterns, emotional tendencies)

[0437] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0438] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0439] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0440] [Second Embodiment]

[0441] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0442] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0443] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0444] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0445] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0446] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0447] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0448] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0449] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0450] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0451] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0452] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0453] This invention is a system designed to solve problems related to labor shortages and the collection of authentic customer feedback. This system receives user input, analyzes the input using generative artificial intelligence, generates an appropriate response and sends it to the user, and then analyzes the data to understand customer needs.

[0454] System Configuration

[0455] A means (terminal) for receiving user input.

[0456] Users send questions and inquiries to the store's official account using messaging apps such as LINE. For example, a user might send a message saying, "Please tell me the availability of the product."

[0457] Analysis means (server)

[0458] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI) that acts as an analysis tool. This AI uses a natural language processing engine to analyze the content of the user's message. For example, it analyzes the input message, "Please tell me the stock status of the product."

[0459] Generation means (server)

[0460] Based on the analysis performed by the analysis means, the generation means generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0461] Transmission method (server and terminal)

[0462] The generated response is sent to the user using the LINE API. The server sends the generated response to the user's device, and the user receives the response in their messaging app. For example, the user's smartphone might display a message saying, "We currently have very little of this product in stock. Please hurry."

[0463] Storage method (server)

[0464] The server records user input and generated responses. This ensures that all conversation data is stored in a database. The stored data includes, for example, a conversation timestamp and user ID.

[0465] Analysis method (server)

[0466] The stored data is analyzed by analytical tools. These tools utilize the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that the product is popular.

[0467] Specific example

[0468] 1. The user asks a question.

[0469] User: "When will the new shoes be in stock?"

[0470] Device (user's smartphone): Send a message to the store's official LINE account.

[0471] 2. Receive and analyze user messages.

[0472] Server: Receives messages via LINE API.

[0473] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0474] 3. Generate and send an appropriate response.

[0475] Server: Based on the analysis results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[0476] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0477] 4. Storage and analysis of conversation data

[0478] Server: Stores the user's questions and generated responses.

[0479] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing.

[0480] This system allows stores to effectively respond to user inquiries, analyze that data to understand customer needs, and thereby improve the operational efficiency of their stores.

[0481] The following describes the processing flow.

[0482] Step 1:

[0483] Users send messages to the store's official account via the LINE app. These messages may contain questions or inquiries.

[0484] Step 2:

[0485] The device (the user's smartphone) sends this message to the LINE server. The LINE server receives the message and forwards it to the store's official account server.

[0486] Step 3:

[0487] The server receives messages from users using the LINE API. It then prepares the received messages for transmission to a generative artificial intelligence system, which is used for analysis.

[0488] Step 4:

[0489] The server sends the user's message to the generative artificial intelligence API. The generative artificial intelligence uses a natural language processing engine to analyze the message content. For example, it might analyze the message, "Please tell me the product's stock status."

[0490] Step 5:

[0491] Generative artificial intelligence generates appropriate responses based on analysis results. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0492] Step 6:

[0493] The generative artificial intelligence generates a response, which is then sent back to the server's API. The server receives this response and prepares it for transmission to the user.

[0494] Step 7:

[0495] The server uses the LINE API to send a response generated by generative artificial intelligence to the user's device. The device (the user's smartphone) receives the message sent from the server and displays the message to the user saying, "There are only a few of that product left in stock. Please hurry."

[0496] Step 8:

[0497] The server records user input and generated responses in a database, which serves as a storage mechanism. The recorded data also includes metadata such as conversation timestamps and user IDs.

[0498] Step 9:

[0499] Data recorded by storage devices is analyzed by analysis devices. The server analyzes the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it indicates that the product is popular.

[0500] Step 10:

[0501] The server provides feedback based on the extracted information to improve business strategies. This enables the development of appropriate inventory management and marketing strategies.

[0502] (Example 1)

[0503] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0504] Traditional customer service systems face challenges such as a shortage of personnel and difficulty in collecting authentic customer feedback. In particular, responding quickly to a large volume of inquiries requires a large workforce, posing problems in terms of cost and efficiency. Furthermore, there is a lack of effective means to analyze collected customer feedback and understand customer needs. As a result, companies often miss opportunities to improve customer satisfaction.

[0505] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0506] In this invention, the server includes means for receiving user input, means for receiving the user input using communication technology, analysis means including a generative artificial intelligence that analyzes the received input using a natural language processing engine, generation means including an artificial intelligence model that generates a response based on the analysis results, transmission means for sending the generated response to the user using communication technology, data management means for recording the user input and the generated response, and data analysis means for analyzing the recorded data. This makes it possible to efficiently respond to customer inquiries and understand customer needs by analyzing the data.

[0507] "User input" refers to information, questions, or requests that a user submits through the system.

[0508] "Communication technology" refers to the technologies and protocols used to send and receive data between digital devices.

[0509] "Means of receiving" refers to a digital system that has the function of capturing input data sent from a user and transferring it to other components within the system for processing.

[0510] A "natural language processing engine" is a general term for the technologies and algorithms that enable computers to understand and analyze human language.

[0511] "Generative artificial intelligence" refers to artificial intelligence systems that can automatically generate text and data similar to those generated by humans.

[0512] "Analysis means" refers to a combination of software and hardware used to analyze received data and understand its contents.

[0513] "Analysis results" refers to the output of user input data analyzed by a natural language processing engine.

[0514] "Generation means" refers to the function of a system that generates appropriate responses or data based on analysis results.

[0515] An "artificial intelligence model" refers to a data processing model that has been trained to perform a specific task using machine learning or deep learning.

[0516] "Transmission means" refers to the function of a communication system that delivers the generated response to the user.

[0517] "Data management means" refers to the system's function for recording and storing user input and generated responses.

[0518] "Data analysis methods" refer to techniques for analyzing recorded data to extract important trends and information.

[0519] A "data platform" refers to the infrastructure and associated software systems used for storing, managing, and processing data.

[0520] This invention is a system designed to address challenges such as labor shortages and the collection of authentic customer feedback. It accepts user input, analyzes the input using generative artificial intelligence, generates an appropriate response and sends it to the user, and then analyzes the data to understand customer needs. An embodiment of this system will be described in detail below.

[0521] System Overview

[0522] This system includes the following components:

[0523] A means (terminal) for receiving user input.

[0524] A means (server) for receiving input data using communication technology.

[0525] Analysis means (server) including a generative artificial intelligence that analyzes input using a natural language processing engine.

[0526] A generation means (server) including an artificial intelligence model that generates a response based on the analysis results.

[0527] Means (server and terminal) for sending the generated response to the user.

[0528] A data management means (server) for recording user input and generated responses.

[0529] Data analysis means (server) for analyzing recorded data

[0530] Detailed explanation

[0531] User input reception

[0532] Users can use smartphones or other devices to send questions and inquiries to the store's official account via messaging apps. For example, they might send a message asking, "Please tell me the availability of this product."

[0533] Message reception and analysis

[0534] The server receives messages from users using the LINE API and other communication technologies. The received messages are passed to a generative artificial intelligence (AI), which is used for analysis by a natural language processing engine. For example, it analyzes a message such as "Please tell me the stock status of the product" and extracts keywords and context.

[0535] Response generation

[0536] Based on the analysis performed by the analysis tool, the artificial intelligence model, which acts as the generation tool, generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0537] Sending a response

[0538] The generated response is then sent back to the user's device using the LINE API or other communication technologies. The server sends the generated response to the user's device, and the user receives and confirms the message in their messaging app. For example, a message such as "We currently have very little of this product in stock. Please hurry." might appear on the smartphone.

[0539] Data storage and analysis

[0540] The server records user input and generated responses. This ensures all conversation data is stored on the data platform. The stored data can then be analyzed by data analytics tools to extract customer needs and trends. For example, it might detect that many users are inquiring about a particular product, revealing its popularity.

[0541] Explanation of specific examples

[0542] The user asks a question.

[0543] User: "When will the new shoes be in stock?"

[0544] Device (user's smartphone): Send a message to the store's official account.

[0545] Receive and analyze user messages.

[0546] Server: Receives messages via LINE API.

[0547] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0548] Generate and send an appropriate response.

[0549] Server: Based on the analysis results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[0550] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0551] Storage and analysis of conversation data

[0552] Server: Stores the user's questions and generated responses.

[0553] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing.

[0554] Examples of prompts for generative AI models

[0555] When a user sends the message "When will the new shoes be in stock?", the following prompt is entered into the generative artificial intelligence:

[0556] A user sent the message, "When will the new shoes be in stock?" Please refer to the inventory database and generate a response.

[0557] This system allows stores to efficiently respond to customer inquiries, analyze the data to understand customer needs, and significantly improve the operational efficiency of their stores.

[0558] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0559] Step 1:

[0560] User input acceptance

[0561] User: Uses a smartphone or computer to send questions and inquiries to the store's official account via a messaging app. The user enters their specific question and presses the send button. Example: "Please tell me the product's stock status."

[0562] Input: Text data sent by the user via a messaging app.

[0563] Output: The message sent by the user arrives at the application server.

[0564] Step 2:

[0565] Message received

[0566] Server: Uses the LINE API to receive messages sent by users in real time. Upon receiving a message, it retrieves relevant data such as the message content, user ID, and timestamp.

[0567] Input: Metadata such as message text from the user, user ID, and timestamp.

[0568] Output: Structured data for passing received message data to the analysis tool.

[0569] Step 3:

[0570] Message parsing

[0571] Server: Passes received message data to a generative artificial intelligence system, which then analyzes it using a natural language processing engine. Specifically, it tokenizes the text and extracts context and keywords.

[0572] Input: Structured message data (user message, user ID, timestamp).

[0573] Output: Data including keywords and intent as analysis results.

[0574] Step 4:

[0575] Response generation

[0576] Server: Based on the analysis results, the generative artificial intelligence generates an appropriate response. Specifically, it refers to the inventory management database and retrieves information corresponding to the user's question. For example, in response to "Please tell me the inventory status of the product," it generates a sentence such as "Currently, there is only a small amount of stock left of that product."

[0577] Input: Analysis results and corresponding data (e.g., inventory status).

[0578] Output: The generated response message.

[0579] Step 5:

[0580] Sending a response

[0581] Server: The generated response is sent to the user's device using the LINE API. Before sending, a process is performed to verify the format and security of the response content.

[0582] Device: The user's smartphone receives the message via a messaging app and notifies the user.

[0583] Input: The generated response message.

[0584] Output: The response message displayed on the user's terminal.

[0585] Step 6:

[0586] Data storage

[0587] Server: Records user input and generated responses using data management means. For example, it stores them in a database along with the conversation timestamp and user ID.

[0588] Input: User input data and generated response data.

[0589] Output: Conversation history stored in the database.

[0590] Step 7:

[0591] Data analysis

[0592] Server: Analyzes stored conversation data using analysis tools. For example, machine learning algorithms are used to extract interest levels and trends for specific products.

[0593] Input: Conversation history stored in the database.

[0594] Output: Reports and data that clearly indicate customer needs and market trends.

[0595] The above steps result in a system that efficiently responds to user inquiries and allows for the understanding of customer needs based on that data.

[0596] (Application Example 1)

[0597] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0598] Traditional customer support systems faced challenges such as staff shortages and difficulty in gathering authentic customer feedback. Furthermore, limited access to product information within stores made prompt and appropriate responses difficult. This hindered improvements in customer satisfaction and accurate understanding of customer needs.

[0599] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0600] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, analysis means for analyzing the recorded data, scanning means for the user to identify products in the store, and speech recognition means for receiving and analyzing the user's voice input. This enables customers to easily obtain product information in the store using their smartphones and to receive quick and appropriate responses to their voice and text questions. Furthermore, by analyzing all inquiry data, it is possible to accurately grasp customer needs and trends and improve services.

[0601] "Means for receiving user input" refers to an interface that allows users to input text or voice from a device.

[0602] "Generative artificial intelligence" refers to artificial intelligence that analyzes input text or audio and generates appropriate responses based on that content.

[0603] "Analysis means" refers to a method for analyzing data input by a user using generative artificial intelligence.

[0604] "Generation means" refers to means for generating an appropriate response to the user based on the results analyzed by the analysis means.

[0605] "Transmission means" refers to the means for sending the generated response to the user.

[0606] "Storage means" refers to means for recording and saving user input and generated responses.

[0607] "Analysis methods" refer to means of analyzing recorded data to understand customer needs and trends.

[0608] A "scanning method" refers to a means by which a user can obtain product information by scanning a barcode or QR code within a store.

[0609] A "speech recognition method" is a means for converting and analyzing content entered by a user via voice into text.

[0610] This invention is a system for users to obtain product information in a physical store and receive real-time responses to their questions. The following describes a specific configuration for realizing this system.

[0611] First, the user scans the product barcode using a smartphone app. The product barcode data is captured by the scanning device and transmitted to an analysis device, including a generative artificial intelligence system. Similarly, when voice input is used, the voice data is converted into text data via a voice recognition device and passed to the analysis device.

[0612] In the analysis method, a generative artificial intelligence analyzes the input data and uses a natural language processing engine to understand the user's question. For example, if a user asks "What is the price of this product?" by voice, a speech recognition device converts it to text, and that text is passed to the generative AI model.

[0613] Next, the generation mechanism operates and generates an appropriate response based on the analysis results. Response generation involves integration with inventory management systems and product databases. For example, product information is retrieved, and a response such as "The price of this product is 1980 yen" is generated.

[0614] The generated response is sent to the user's smartphone via the LINE API using the designated transmission method. The user can then view this response in real time using the smartphone app.

[0615] Furthermore, user input and generated responses are recorded by storage devices and stored on a data platform. The stored data can be analyzed by analytical devices to extract customer needs and trends. For example, if many users inquired about "new shoes," it could be analyzed that there is a high demand for that product.

[0616] A concrete example of a prompt would be a user asking, "How many days can I use this product?" In response to this input, the generative AI model analyzes the input and generates a response such as, "The usage period is approximately 30 days."

[0617] In this way, users can easily obtain product information and receive real-time answers to their questions, thereby improving customer satisfaction. Furthermore, by utilizing analytical tools, stores can accurately grasp customer demand and trends, enabling them to respond quickly.

[0618] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0619] Step 1:

[0620] The user uses their smartphone to scan product barcodes in the store. The scanned barcode data is captured by the scanning device and sent to a database within the smartphone app. The input is the identification data of the product barcode, and this data is ready to be sent to the server as output.

[0621] Step 2:

[0622] When a user performs voice input, they input their question by voice using their smartphone's microphone. This voice data passes through a speech recognition system, is converted into text data, and sent to an analysis system. The input is voice data, and the converted text data is passed to the generating AI model as output.

[0623] Step 3:

[0624] The server receives messages from users via the LINE API. These messages are received as text data and passed to a generative artificial intelligence (AI) system for analysis. The input is a text message, and the output is data ready for analysis, which is then passed to the generative AI model.

[0625] Step 4:

[0626] In the analysis method, a generative artificial intelligence analyzes the input data (text data or product barcode data). Using a natural language processing engine, it understands the user's question and extracts information to create an appropriate response. The input is either text data or product barcode data, and the analysis results are passed to the generation method as output.

[0627] Step 5:

[0628] The generation mechanism generates an appropriate response based on the analysis results. For example, it might refer to an inventory management system or product database to generate a response such as, "The price of this product is 1980 yen." The input is the analysis results, and the generated response is passed to the transmission mechanism as the output.

[0629] Step 6:

[0630] The transmission method uses the LINE API to send the generated response to the user's smartphone. The user can view this response in real time on the smartphone app. The input is the generated response, and the output is the response message that is displayed on the user's device.

[0631] Step 7:

[0632] The storage mechanism records and stores user input (text messages or voice input) and the generated responses. This data is stored on a data platform for later analysis. Inputs are user inputs and generated responses, and the recorded data is stored as output.

[0633] Step 8:

[0634] The analysis method involves analyzing stored data to understand customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that demand for that product is increasing. The input is stored data, and the output is the analysis results.

[0635] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0636] This invention relates to a system that analyzes user input and generates appropriate responses. In particular, by incorporating an emotion engine, this system can recognize the user's emotions and adjust its responses accordingly. The aim of this system is to provide a more sophisticated conversational experience and improve customer satisfaction.

[0637] System Configuration

[0638] A means (terminal) for receiving user input.

[0639] Users send questions and inquiries to the store's official account using messaging apps such as LINE. For example, a user might send a message saying, "Please tell me the availability of the product."

[0640] Analysis means (server)

[0641] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI) that acts as an analysis tool. This AI uses a natural language processing engine to analyze the content of the user's message. For example, it analyzes the input message, "Please tell me the stock status of the product."

[0642] Emotional Engine

[0643] Based on the message content analyzed by the generative artificial intelligence, the emotion engine recognizes the user's emotions (e.g., joy, anger, sadness). The emotion engine also adjusts the response according to the user's emotions. For example, if the user is expressing dissatisfaction, the response will include a polite apology.

[0644] Generation means (server)

[0645] Based on the analysis results and the emotion engine's findings, the generation mechanism generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "We currently have very little of that item in stock. Please hurry." If the emotion engine determines that the user is dissatisfied, it might generate a response such as, "We are sorry, but we have very little of this item in stock. We apologize for the inconvenience."

[0646] Transmission method (server and terminal)

[0647] The generated response is sent to the user using the LINE API. The server sends the generated response to the user's device, and the user receives the response in their messaging app. For example, the user's smartphone might display a message saying, "We currently have very little of this product in stock. Please hurry."

[0648] Storage method (server)

[0649] The server records user input and generated responses in a database, which serves as a storage mechanism. This ensures that all conversation data is stored in the database. The stored data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results from the sentiment engine.

[0650] Analysis method (server)

[0651] The stored data is analyzed using analytical tools. The server utilizes the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that the product is popular. It is also possible to analyze trends in customer satisfaction using data from the emotion engine.

[0652] Specific example

[0653] 1. The user asks a question.

[0654] User: "When will the new shoes be in stock?"

[0655] Device (user's smartphone): Send a message to the store's official LINE account.

[0656] 2. Receive and analyze user messages.

[0657] Server: Receives messages via LINE API.

[0658] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0659] 3. Emotion recognition by an emotion engine

[0660] Emotion Engine: Recognizes emotions from user messages and determines if the user is excited.

[0661] 4. Generate and send an appropriate response.

[0662] Server: Based on the analysis results and emotion recognition results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[0663] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0664] 5. Storage and analysis of conversation data

[0665] Server: Stores user questions, generated responses, and sentiment recognition results.

[0666] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing or to understand customers' emotional responses.

[0667] This system allows for more personalized responses to user inquiries, adjusting them based on their emotions. Furthermore, analysis of conversational data can reveal customer needs and emotional tendencies, which can then be incorporated into business strategies.

[0668] The following describes the processing flow.

[0669] Step 1:

[0670] Users send messages to the store's official account via the LINE app. These messages may contain questions or inquiries.

[0671] Step 2:

[0672] The device (the user's smartphone) sends this message to the LINE server. The LINE server receives the message and forwards it to the store's official account server.

[0673] Step 3:

[0674] The server receives messages from users using the LINE API. It then prepares the received messages for transmission to a generative artificial intelligence system, which is used for analysis.

[0675] Step 4:

[0676] The server sends the user's message to the generative artificial intelligence API. The generative artificial intelligence uses a natural language processing engine to analyze the message content. For example, it might analyze the message, "Please tell me the product's stock status."

[0677] Step 5:

[0678] The generative artificial intelligence outputs the analysis results and passes them to the emotion engine. The emotion engine recognizes the user's emotions based on the message content. For example, it might determine from the message that the user is feeling dissatisfied.

[0679] Step 6:

[0680] Based on the results of generative artificial intelligence and an emotion engine, the server generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "We currently have very little of that item in stock. Please hurry." If it determines that the user is expressing dissatisfaction, it will generate a response such as, "We are sorry, but we have very little of this item in stock. We apologize for the inconvenience."

[0681] Step 7:

[0682] The server sends the generated response to the user's device using the LINE API. The device (the user's smartphone) receives the message sent from the server and displays a response to the user such as, "We currently have very little of that item in stock. Please hurry," or "We are sorry, but we have very little of that item in stock. We apologize for the inconvenience."

[0683] Step 8:

[0684] The server records user input and generated responses in a database, which serves as a storage mechanism. The recorded data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results from the sentiment engine.

[0685] Step 9:

[0686] The server analyzes stored data using analytical tools. For example, it uses stored conversation data to extract customer needs and trends. It also uses emotion engine data to analyze trends in customer satisfaction.

[0687] Step 10:

[0688] The server provides feedback based on the extracted information to improve business strategies. This enables the development of appropriate inventory management and marketing strategies.

[0689] (Example 2)

[0690] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0691] Traditional user-system interactions suffered from a problem where responses were uniform because they did not take user emotions into account. As a result, users could not receive responses that suited their feelings, leading to decreased satisfaction. Furthermore, it was difficult to gain deep insights through the analysis of dialogue data, making it challenging to grasp customer needs and trends.

[0692] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0693] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing user input, recognition means including an emotion engine for recognizing user emotions based on the message content analyzed by the generative artificial intelligence, generation means for generating a response based on the analysis results and the results of the recognition means, transmission means for sending the generated response to the user, storage means for recording user input and generated responses, and analysis means for analyzing the recorded data. This makes it possible to generate personalized responses that correspond to the user's emotions, thereby improving user satisfaction. Furthermore, analysis based on the stored dialogue data allows for efficient understanding of customer needs and trends, contributing to business strategy.

[0694] "Means of receiving user input" refers to an interface for users to send messages or inquiries to a system, and typically refers to messaging applications or web forms.

[0695] "Analysis means" refers to a device or system that uses generative artificial intelligence to analyze the content of messages sent by users and performs processing to understand their intent and content.

[0696] "Generative artificial intelligence" refers to artificial intelligence that has the ability to analyze user input using a natural language processing engine, and to understand and interpret its content.

[0697] A "natural language processing engine" is a software system that analyzes text data as human language to understand its context and intent.

[0698] An "emotion engine" is a computer program or device that recognizes a user's emotions based on the analyzed message content and adjusts its response accordingly.

[0699] "Recognition means" refers to a function that includes an emotion engine and executes a process of recognizing the user's emotions based on information obtained from analysis means.

[0700] "Generation means" refers to a device or program that generates an appropriate response for the user based on the analysis results and emotion recognition results.

[0701] "Transmission means" refers to an interface for sending responses generated by the generation means to the user, and usually refers to the API of a messaging application.

[0702] "Storage means" refers to a database or storage device for recording user input and generated responses and saving them in a format that can be referenced later.

[0703] "Analysis means" refers to the process and apparatus for analyzing data stored in storage means to extract customer needs and trends.

[0704] A "data platform" is a foundational system or software for managing stored data and retrieving and analyzing it as needed.

[0705] This invention relates to a system that analyzes user input and generates appropriate responses. In particular, by incorporating an emotion engine, this system can recognize the user's emotions and adjust its responses accordingly. The aim of this system is to provide a more sophisticated conversational experience and improve customer satisfaction.

[0706] System Configuration

[0707] Means for receiving user input

[0708] Users send messages and inquiries to the store's official account using messaging applications (e.g., LINE). For this purpose, the user's device (smartphone, tablet, etc.) is used.

[0709] Analysis means

[0710] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI), which uses a natural language processing engine (e.g., Google Cloud Natural Language API) to analyze the message content.

[0711] Recognition means including an emotion engine

[0712] Based on the analyzed message content, an emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotions. For example, if a user sends the message "When will the new shoes be in stock?", it might be determined that the user is excited.

[0713] generation means

[0714] Based on the analysis results and emotion recognition results, the server generates an appropriate response. Generative artificial intelligence is used in this process; for example, it might refer to an inventory management database to generate a response such as, "New shoes are scheduled to arrive next Monday. Please look forward to them!"

[0715] Transmission method

[0716] The generated response is sent to the user's device using the LINE API. The server sends the generated response to the user's smartphone, and the user receives the response in their messaging app.

[0717] Preservation means

[0718] The server stores user input and generated responses in a database. This stored data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results.

[0719] analytical means

[0720] The stored data is analyzed using analytical tools. The server utilizes the stored conversation data to extract customer needs and trends. For example, if there are many inquiries about a particular product, it indicates that the product is popular. It is also possible to analyze trends in customer satisfaction using data from the emotion engine.

[0721] Specific example

[0722] 1. The user asks a question.

[0723] User: "When will the new shoes be in stock?"

[0724] Device (user's smartphone): Send a message to the store's official LINE account.

[0725] 2. Receive and analyze user messages.

[0726] Server: Receives messages via LINE API.

[0727] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0728] 3. Emotion recognition by an emotion engine

[0729] Emotion Engine: Recognizes "excitement" from the user's message.

[0730] 4. Generate and send an appropriate response.

[0731] Server: Based on the analysis results and emotion recognition results, it generates the response, "The new shoes will be in stock next Monday. Stay tuned!"

[0732] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0733] 5. Storage and analysis of conversation data

[0734] Server: Stores user questions, generated responses, and sentiment recognition results in a database.

[0735] Server: Based on stored data, it analyzes and understands trends in interest in specific products and customers' emotional responses.

[0736] Example of a prompt

[0737] "Generate a response for when a user inquires about the new shoes. The user is excited."

[0738] "Generate a response for when a user is dissatisfied with the stock situation."

[0739] This system allows for more personalized responses to user inquiries, as the response is adjusted according to the user's emotions. Furthermore, analysis of stored conversation data allows for the understanding of customer needs and emotional tendencies, which can then be reflected in business strategies.

[0740] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0741] Step 1:

[0742] The user sends a message using the LINE messaging app.

[0743] In terms of the specific action, the user uses the LINE app on their smartphone to type and send the text message "Please tell me the product's stock status" to the store's official account. At this point, the input is the text message the user entered into the LINE app. The device then sends this text message.

[0744] Step 2:

[0745] The server uses the LINE API to receive user messages.

[0746] In terms of specific operation, the server receives message data sent by the user via the LINE API. This message data includes the sender's user ID and the message body. The input is the text message sent by the user, and the output is the message data stored on the server.

[0747] Step 3:

[0748] The received message is analyzed by a generative artificial intelligence.

[0749] The server passes the received message data to a generative artificial intelligence (e.g., Google Cloud Natural Language API). Specifically, the generative AI analyzes the text message and extracts the main themes and content. The input is the user's message data, and the output is the analyzed message content (e.g., "Inquiry about product availability").

[0750] Step 4:

[0751] The emotion engine recognizes the user's emotions based on the analysis results.

[0752] Based on the analysis results, the server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to determine the emotions contained in the message. Specifically, the emotion engine analyzes the emotional characteristics of the message text and recognizes emotions such as "dissatisfaction," "excitement," and "joy." The input is the analyzed message content, and the output is the recognized emotion data.

[0753] Step 5:

[0754] The server generates a response based on the analysis results and emotion recognition results.

[0755] Based on the analysis results and sentiment recognition results, the server generates an appropriate response. Specifically, the response generation algorithm refers to the inventory management database and creates an appropriate response statement. For example, if the stock is low, it will generate a response such as, "We currently have very little of that product left in stock. Please hurry." The input is the analysis results and sentiment recognition results, and the output is the generated response.

[0756] Step 6:

[0757] The server sends the generated response to the user.

[0758] The generated response is sent to the user via the LINE API. Specifically, the server uses the LINE API to send the generated text message to the user's smartphone. The input is the generated response data, and the output is the message displayed on the user's device.

[0759] Step 7:

[0760] The server saves messages and responses to the database.

[0761] In practice, the server records all user messages and generated responses in a database. This includes metadata such as message content, timestamp, user ID, and sentiment recognition results. The input is conversation data and metadata, and the output is the record stored in the database.

[0762] Step 8:

[0763] The system analyzes data stored on the server to understand trends and customer needs.

[0764] In practice, the server uses stored data for analysis to extract customer needs and trends. For example, if many users are inquiring about a particular product, it indicates that the product is popular. The input is stored conversation data, and the output is trend information and customer needs information as a result of the analysis.

[0765] (Application Example 2)

[0766] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0767] Conventional user input analysis systems have the problem of not being able to respond while considering the user's emotions, making it difficult to achieve sufficient customer satisfaction. Furthermore, even in customer service by store staff in physical stores, it is difficult for staff to quickly grasp the emotional state of customers and respond appropriately. Especially in busy stores, it is difficult for staff to provide individual attention to each customer, which may lead to a decrease in customer satisfaction. In addition, because it is not possible to efficiently utilize the history of conversations with customers and reflect it in future service interactions, the same questions are asked repeatedly.

[0768] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0769] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results and emotion recognition results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, analysis means for analyzing the recorded data, and store clerk support means including smart glasses that recognize customer emotions in real time and suggest appropriate responses. This enables responses that take user emotions into consideration, allowing store clerks to efficiently handle customers in physical stores and improve customer satisfaction.

[0770] "Means of receiving user input" refers to an interface for receiving information and inquiries from users.

[0771] "Analysis means" refers to methods, including generative artificial intelligence, for interpreting user input and understanding its content.

[0772] "Generation means" refers to means for creating an appropriate response to be provided to the user based on the analysis results and emotion recognition results.

[0773] "Transmission means" refers to a communication mechanism for sending the generated response to the user.

[0774] A "storage mechanism" is a function that saves user input and generated responses so that they can be referenced later.

[0775] "Analysis methods" refer to means of analyzing stored data to understand user behavior patterns and trends.

[0776] "Smart glasses" are devices worn by store employees that display information and provide real-time customer emotion recognition results and response suggestions.

[0777] "Staff support tools" are technologies that assist store staff in physical stores and facilitate smooth interactions with customers.

[0778] "Generative artificial intelligence" is an AI technology that analyzes user input data and generates responses based on that data.

[0779] "Emotion recognition results" refer to information used to analyze user input and identify their emotional state.

[0780] "Response suggestions" are pieces of information that suggest the most appropriate answer or action based on the user's emotional state and the content of their inquiry.

[0781] This invention provides a system that analyzes user input, recognizes emotions, and generates a corresponding response. In particular, this system can perform exceptionally well in customer service at physical stores. The embodiments are described below in detail.

[0782] System Configuration

[0783] The system includes the following main components:

[0784] A means (terminal) for receiving user input.

[0785] This is an interface that allows users to make inquiries via voice input or text messages. Smartphones and tablets are examples of this.

[0786] Analysis means (server)

[0787] The server uses a natural language processing engine, including generative artificial intelligence (e.g., spaCy), to analyze user input. This analysis identifies the content of the user's inquiry.

[0788] Emotional Engine

[0789] Based on the user input data analyzed by the analysis tool, the user's emotions are recognized using an emotion engine (e.g., IBM Watson Tone Analyzer). This identifies the emotions the user is experiencing (joy, anger, sadness, etc.).

[0790] Generation means (server)

[0791] This is a means for generating an appropriate response based on the results obtained by the analysis means and the emotion engine. The generative artificial intelligence model creates a response that matches the user's emotions.

[0792] Transmission method (server and terminal)

[0793] The generated response is provided to the user via a transmission method. For example, the response message is sent to the user's smartphone via the LINE API.

[0794] Storage method (server)

[0795] User input and generated responses are stored in a database, which serves as a storage mechanism. This allows for the management of past conversation data and its use for later analysis.

[0796] Analysis method (server)

[0797] The stored data is analyzed using analytical tools. This allows for an understanding of customer behavior patterns and emotional tendencies.

[0798] Shop assistant support device (smart glasses)

[0799] Smart glasses and other store clerk assistance devices are devices that display emotion recognition results and response suggestions in real time when store clerks interact with customers. This allows store clerks to quickly provide the most appropriate response.

[0800] Program Processing Description

[0801] Hardware: Smartphones, tablets, smart glasses, servers

[0802] Software: Speech recognition software (e.g., Google Cloud Speech-to-Text), natural language processing engine (e.g., spaCy), emotion recognition engine (e.g., IBM Watson Tone Analyzer)

[0803] Data processing and calculations:

[0804] The server converts the user's voice input into text (speech recognition).

[0805] Analyze the content of the text using a natural language processing engine.

[0806] Identifying customer emotions with an emotion recognition engine

[0807] Based on the analysis results and emotions, a generative artificial intelligence model generates a response.

[0808] The generated response is provided to the user via a transmission method.

[0809] Specific example

[0810] For example, if a user asks, "How big is this product?" and simultaneously expresses anxiety, the system re-analyzes the question and displays a response on the salesperson's smart glasses saying, "Don't worry. This product should be the perfect size for you."

[0811] Example of a prompt:

[0812] "Design a system that receives voice input from customers expressing questions or complaints about a product, and generates and displays appropriate responses in real time through smart glasses."

[0813] The system as a whole provides a sophisticated conversational experience that takes user emotions into account, improving customer satisfaction in physical stores. Furthermore, analyzing the stored conversation data can be used to improve future business strategies.

[0814] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0815] Step 1:

[0816] The user makes an inquiry.

[0817] Users can use their smartphones or tablets to make inquiries via voice input or text message. For example, they might ask, "How big is this product?"

[0818] Input: User voice or text inquiry

[0819] Output: User voice data or text data

[0820] Step 2:

[0821] Voice input to text conversion

[0822] The device uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's voice input into text data.

[0823] Input: User's voice data

[0824] Output: Text data

[0825] Step 3:

[0826] Analyze user input

[0827] The server uses a natural language processing engine, including generative artificial intelligence (e.g., spaCy), to analyze the user's text input. This allows the content of the inquiry to be identified.

[0828] Input: User's text data

[0829] Output: Analysis results (inquiry details)

[0830] Step 4:

[0831] Recognizing user emotions

[0832] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotions based on the analysis results. For example, it might identify that the user is feeling anxious.

[0833] Input: Analysis results

[0834] Output: Emotion recognition result (user's emotional state)

[0835] Step 5:

[0836] Generate an appropriate response

[0837] Based on the analysis results and emotion recognition results, the server uses a generative artificial intelligence model to generate an appropriate response. For example, it might create a response such as, "Please rest assured. We believe this product is the perfect size for you."

[0838] Input: Analysis results, emotion recognition results

[0839] Output: Generated response

[0840] Step 6:

[0841] Send a response

[0842] The server sends the generated response to the user's terminal via a transmission method. For example, the response message is sent to the user's smartphone via the LINE API.

[0843] Input: Generated response

[0844] Output: Response message displayed on the user's terminal

[0845] Step 7:

[0846] Saving conversation data

[0847] The server saves user input and generated responses to a database. This records the conversation data for later reference.

[0848] Input: User input data, generated response

[0849] Output: Saved conversation data

[0850] Step 8:

[0851] Analysis of conversation data

[0852] The server uses stored conversation data to analyze customer behavior patterns and emotional tendencies. This analysis helps improve future customer interactions and business strategies.

[0853] Input: Saved conversation data

[0854] Output: Analysis results (customer behavior patterns, emotional tendencies)

[0855] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0856] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0857] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0858] [Third Embodiment]

[0859] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0860] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0861] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0862] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0863] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0864] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0865] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0866] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0867] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0868] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0869] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0870] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0871] This invention is a system designed to solve problems related to labor shortages and the collection of authentic customer feedback. This system receives user input, analyzes the input using generative artificial intelligence, generates an appropriate response and sends it to the user, and then analyzes the data to understand customer needs.

[0872] System Configuration

[0873] A means (terminal) for receiving user input.

[0874] Users send questions and inquiries to the store's official account using messaging apps such as LINE. For example, a user might send a message saying, "Please tell me the availability of the product."

[0875] Analysis means (server)

[0876] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI) that acts as an analysis tool. This AI uses a natural language processing engine to analyze the content of the user's message. For example, it analyzes the input message, "Please tell me the stock status of the product."

[0877] Generation means (server)

[0878] Based on the analysis performed by the analysis means, the generation means generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0879] Transmission method (server and terminal)

[0880] The generated response is sent to the user using the LINE API. The server sends the generated response to the user's device, and the user receives the response in their messaging app. For example, the user's smartphone might display a message saying, "We currently have very little of this product in stock. Please hurry."

[0881] Storage method (server)

[0882] The server records user input and generated responses. This ensures that all conversation data is stored in a database. The stored data includes, for example, a conversation timestamp and user ID.

[0883] Analysis method (server)

[0884] The stored data is analyzed by analytical tools. These tools utilize the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that the product is popular.

[0885] Specific example

[0886] 1. The user asks a question.

[0887] User: "When will the new shoes be in stock?"

[0888] Device (user's smartphone): Send a message to the store's official LINE account.

[0889] 2. Receive and analyze user messages.

[0890] Server: Receives messages via LINE API.

[0891] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0892] 3. Generate and send an appropriate response.

[0893] Server: Based on the analysis results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[0894] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0895] 4. Storage and analysis of conversation data

[0896] Server: Stores the user's questions and generated responses.

[0897] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing.

[0898] This system allows stores to effectively respond to user inquiries, analyze that data to understand customer needs, and thereby improve the operational efficiency of their stores.

[0899] The following describes the processing flow.

[0900] Step 1:

[0901] Users send messages to the store's official account via the LINE app. These messages may contain questions or inquiries.

[0902] Step 2:

[0903] The device (the user's smartphone) sends this message to the LINE server. The LINE server receives the message and forwards it to the store's official account server.

[0904] Step 3:

[0905] The server receives messages from users using the LINE API. It then prepares the received messages for transmission to a generative artificial intelligence system, which is used for analysis.

[0906] Step 4:

[0907] The server sends the user's message to the generative artificial intelligence API. The generative artificial intelligence uses a natural language processing engine to analyze the message content. For example, it might analyze the message, "Please tell me the product's stock status."

[0908] Step 5:

[0909] Generative artificial intelligence generates appropriate responses based on analysis results. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0910] Step 6:

[0911] The generative artificial intelligence generates a response, which is then sent back to the server's API. The server receives this response and prepares it for transmission to the user.

[0912] Step 7:

[0913] The server uses the LINE API to send a response generated by generative artificial intelligence to the user's device. The device (the user's smartphone) receives the message sent from the server and displays the message to the user saying, "There are only a few of that product left in stock. Please hurry."

[0914] Step 8:

[0915] The server records user input and generated responses in a database, which serves as a storage mechanism. The recorded data also includes metadata such as conversation timestamps and user IDs.

[0916] Step 9:

[0917] Data recorded by storage devices is analyzed by analysis devices. The server analyzes the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it indicates that the product is popular.

[0918] Step 10:

[0919] The server provides feedback based on the extracted information to improve business strategies. This enables the development of appropriate inventory management and marketing strategies.

[0920] (Example 1)

[0921] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0922] Traditional customer service systems face challenges such as a shortage of personnel and difficulty in collecting authentic customer feedback. In particular, responding quickly to a large volume of inquiries requires a large workforce, posing problems in terms of cost and efficiency. Furthermore, there is a lack of effective means to analyze collected customer feedback and understand customer needs. As a result, companies often miss opportunities to improve customer satisfaction.

[0923] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0924] In this invention, the server includes means for receiving user input, means for receiving the user input using communication technology, analysis means including a generative artificial intelligence that analyzes the received input using a natural language processing engine, generation means including an artificial intelligence model that generates a response based on the analysis results, transmission means for sending the generated response to the user using communication technology, data management means for recording the user input and the generated response, and data analysis means for analyzing the recorded data. This makes it possible to efficiently respond to customer inquiries and understand customer needs by analyzing the data.

[0925] "User input" refers to information, questions, or requests that a user submits through the system.

[0926] "Communication technology" refers to the technologies and protocols used to send and receive data between digital devices.

[0927] "Means of receiving" refers to a digital system that has the function of capturing input data sent from a user and transferring it to other components within the system for processing.

[0928] A "natural language processing engine" is a general term for the technologies and algorithms that enable computers to understand and analyze human language.

[0929] "Generative artificial intelligence" refers to artificial intelligence systems that can automatically generate text and data similar to those generated by humans.

[0930] "Analysis means" refers to a combination of software and hardware used to analyze received data and understand its contents.

[0931] "Analysis results" refers to the output of user input data analyzed by a natural language processing engine.

[0932] "Generation means" refers to the function of a system that generates appropriate responses or data based on analysis results.

[0933] An "artificial intelligence model" refers to a data processing model that has been trained to perform a specific task using machine learning or deep learning.

[0934] "Transmission means" refers to the function of a communication system that delivers the generated response to the user.

[0935] "Data management means" refers to the system's function for recording and storing user input and generated responses.

[0936] "Data analysis methods" refer to techniques for analyzing recorded data to extract important trends and information.

[0937] A "data platform" refers to the infrastructure and associated software systems used for storing, managing, and processing data.

[0938] This invention is a system designed to address challenges such as labor shortages and the collection of authentic customer feedback. It accepts user input, analyzes the input using generative artificial intelligence, generates an appropriate response and sends it to the user, and then analyzes the data to understand customer needs. An embodiment of this system will be described in detail below.

[0939] System Overview

[0940] This system includes the following components:

[0941] A means (terminal) for receiving user input.

[0942] A means (server) for receiving input data using communication technology.

[0943] Analysis means (server) including a generative artificial intelligence that analyzes input using a natural language processing engine.

[0944] A generation means (server) including an artificial intelligence model that generates a response based on the analysis results.

[0945] Means (server and terminal) for sending the generated response to the user.

[0946] A data management means (server) for recording user input and generated responses.

[0947] Data analysis means (server) for analyzing recorded data

[0948] Detailed explanation

[0949] User input reception

[0950] Users can use smartphones or other devices to send questions and inquiries to the store's official account via messaging apps. For example, they might send a message asking, "Please tell me the availability of this product."

[0951] Message reception and analysis

[0952] The server receives messages from users using the LINE API and other communication technologies. The received messages are passed to a generative artificial intelligence (AI), which is used for analysis by a natural language processing engine. For example, it analyzes a message such as "Please tell me the stock status of the product" and extracts keywords and context.

[0953] Response generation

[0954] Based on the analysis performed by the analysis tool, the artificial intelligence model, which acts as the generation tool, generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[0955] Sending a response

[0956] The generated response is then sent back to the user's device using the LINE API or other communication technologies. The server sends the generated response to the user's device, and the user receives and confirms the message in their messaging app. For example, a message such as "We currently have very little of this product in stock. Please hurry." might appear on the smartphone.

[0957] Data storage and analysis

[0958] The server records user input and generated responses. This ensures all conversation data is stored on the data platform. The stored data can then be analyzed by data analytics tools to extract customer needs and trends. For example, it might detect that many users are inquiring about a particular product, revealing its popularity.

[0959] Explanation of specific examples

[0960] The user asks a question.

[0961] User: "When will the new shoes be in stock?"

[0962] Device (user's smartphone): Send a message to the store's official account.

[0963] Receive and analyze user messages.

[0964] Server: Receives messages via LINE API.

[0965] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[0966] Generate and send an appropriate response.

[0967] Server: Based on the analysis results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[0968] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[0969] Storage and analysis of conversation data

[0970] Server: Stores the user's questions and generated responses.

[0971] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing.

[0972] Examples of prompts for generative AI models

[0973] When a user sends the message "When will the new shoes be in stock?", the following prompt is entered into the generative artificial intelligence:

[0974] A user sent the message, "When will the new shoes be in stock?" Please refer to the inventory database and generate a response.

[0975] This system allows stores to efficiently respond to customer inquiries, analyze the data to understand customer needs, and significantly improve the operational efficiency of their stores.

[0976] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0977] Step 1:

[0978] User input acceptance

[0979] User: Uses a smartphone or computer to send questions and inquiries to the store's official account via a messaging app. The user enters their specific question and presses the send button. Example: "Please tell me the product's stock status."

[0980] Input: Text data sent by the user via a messaging app.

[0981] Output: The message sent by the user arrives at the application server.

[0982] Step 2:

[0983] Message received

[0984] Server: Uses the LINE API to receive messages sent by users in real time. Upon receiving a message, it retrieves relevant data such as the message content, user ID, and timestamp.

[0985] Input: Metadata such as message text from the user, user ID, and timestamp.

[0986] Output: Structured data for passing received message data to the analysis tool.

[0987] Step 3:

[0988] Message parsing

[0989] Server: Passes received message data to a generative artificial intelligence system, which then analyzes it using a natural language processing engine. Specifically, it tokenizes the text and extracts context and keywords.

[0990] Input: Structured message data (user message, user ID, timestamp).

[0991] Output: Data including keywords and intent as analysis results.

[0992] Step 4:

[0993] Response generation

[0994] Server: Based on the analysis results, the generative artificial intelligence generates an appropriate response. Specifically, it refers to the inventory management database and retrieves information corresponding to the user's question. For example, in response to "Please tell me the inventory status of the product," it generates a sentence such as "Currently, there is only a small amount of stock left of that product."

[0995] Input: Analysis results and corresponding data (e.g., inventory status).

[0996] Output: The generated response message.

[0997] Step 5:

[0998] Sending a response

[0999] Server: The generated response is sent to the user's device using the LINE API. Before sending, a process is performed to verify the format and security of the response content.

[1000] Device: The user's smartphone receives the message via a messaging app and notifies the user.

[1001] Input: The generated response message.

[1002] Output: The response message displayed on the user's terminal.

[1003] Step 6:

[1004] Data storage

[1005] Server: Records user input and generated responses using data management means. For example, it stores them in a database along with the conversation timestamp and user ID.

[1006] Input: User input data and generated response data.

[1007] Output: Conversation history stored in the database.

[1008] Step 7:

[1009] Data analysis

[1010] Server: Analyzes stored conversation data using analysis tools. For example, machine learning algorithms are used to extract interest levels and trends for specific products.

[1011] Input: Conversation history stored in the database.

[1012] Output: Reports and data that clearly indicate customer needs and market trends.

[1013] The above steps result in a system that efficiently responds to user inquiries and allows for the understanding of customer needs based on that data.

[1014] (Application Example 1)

[1015] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1016] Traditional customer support systems faced challenges such as staff shortages and difficulty in gathering authentic customer feedback. Furthermore, limited access to product information within stores made prompt and appropriate responses difficult. This hindered improvements in customer satisfaction and accurate understanding of customer needs.

[1017] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1018] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, analysis means for analyzing the recorded data, scanning means for the user to identify products in the store, and speech recognition means for receiving and analyzing the user's voice input. This enables customers to easily obtain product information in the store using their smartphones and to receive quick and appropriate responses to their voice and text questions. Furthermore, by analyzing all inquiry data, it is possible to accurately grasp customer needs and trends and improve services.

[1019] "Means for receiving user input" refers to an interface that allows users to input text or voice from a device.

[1020] "Generative artificial intelligence" refers to artificial intelligence that analyzes input text or audio and generates appropriate responses based on that content.

[1021] "Analysis means" refers to a method for analyzing data input by a user using generative artificial intelligence.

[1022] "Generation means" refers to means for generating an appropriate response to the user based on the results analyzed by the analysis means.

[1023] "Transmission means" refers to the means for sending the generated response to the user.

[1024] "Storage means" refers to means for recording and saving user input and generated responses.

[1025] "Analysis methods" refer to means of analyzing recorded data to understand customer needs and trends.

[1026] A "scanning method" refers to a means by which a user can obtain product information by scanning a barcode or QR code within a store.

[1027] A "speech recognition method" is a means for converting and analyzing content entered by a user via voice into text.

[1028] This invention is a system for users to obtain product information in a physical store and receive real-time responses to their questions. The following describes a specific configuration for realizing this system.

[1029] First, the user scans the product barcode using a smartphone app. The product barcode data is captured by the scanning device and transmitted to an analysis device, including a generative artificial intelligence system. Similarly, when voice input is used, the voice data is converted into text data via a voice recognition device and passed to the analysis device.

[1030] In the analysis method, a generative artificial intelligence analyzes the input data and uses a natural language processing engine to understand the user's question. For example, if a user asks "What is the price of this product?" by voice, a speech recognition device converts it to text, and that text is passed to the generative AI model.

[1031] Next, the generation mechanism operates and generates an appropriate response based on the analysis results. Response generation involves integration with inventory management systems and product databases. For example, product information is retrieved, and a response such as "The price of this product is 1980 yen" is generated.

[1032] The generated response is sent to the user's smartphone via the LINE API using the designated transmission method. The user can then view this response in real time using the smartphone app.

[1033] Furthermore, user input and generated responses are recorded by storage devices and stored on a data platform. The stored data can be analyzed by analytical devices to extract customer needs and trends. For example, if many users inquired about "new shoes," it could be analyzed that there is a high demand for that product.

[1034] A concrete example of a prompt would be a user asking, "How many days can I use this product?" In response to this input, the generative AI model analyzes the input and generates a response such as, "The usage period is approximately 30 days."

[1035] In this way, users can easily obtain product information and receive real-time answers to their questions, thereby improving customer satisfaction. Furthermore, by utilizing analytical tools, stores can accurately grasp customer demand and trends, enabling them to respond quickly.

[1036] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1037] Step 1:

[1038] The user uses their smartphone to scan product barcodes in the store. The scanned barcode data is captured by the scanning device and sent to a database within the smartphone app. The input is the identification data of the product barcode, and this data is ready to be sent to the server as output.

[1039] Step 2:

[1040] When a user performs voice input, they input their question by voice using their smartphone's microphone. This voice data passes through a speech recognition system, is converted into text data, and sent to an analysis system. The input is voice data, and the converted text data is passed to the generating AI model as output.

[1041] Step 3:

[1042] The server receives messages from users via the LINE API. These messages are received as text data and passed to a generative artificial intelligence (AI) system for analysis. The input is a text message, and the output is data ready for analysis, which is then passed to the generative AI model.

[1043] Step 4:

[1044] In the analysis method, a generative artificial intelligence analyzes the input data (text data or product barcode data). Using a natural language processing engine, it understands the user's question and extracts information to create an appropriate response. The input is either text data or product barcode data, and the analysis results are passed to the generation method as output.

[1045] Step 5:

[1046] The generation mechanism generates an appropriate response based on the analysis results. For example, it might refer to an inventory management system or product database to generate a response such as, "The price of this product is 1980 yen." The input is the analysis results, and the generated response is passed to the transmission mechanism as the output.

[1047] Step 6:

[1048] The transmission method uses the LINE API to send the generated response to the user's smartphone. The user can view this response in real time on the smartphone app. The input is the generated response, and the output is the response message that is displayed on the user's device.

[1049] Step 7:

[1050] The storage mechanism records and stores user input (text messages or voice input) and the generated responses. This data is stored on a data platform for later analysis. Inputs are user inputs and generated responses, and the recorded data is stored as output.

[1051] Step 8:

[1052] The analysis method involves analyzing stored data to understand customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that demand for that product is increasing. The input is stored data, and the output is the analysis results.

[1053] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1054] This invention relates to a system that analyzes user input and generates appropriate responses. In particular, by incorporating an emotion engine, this system can recognize the user's emotions and adjust its responses accordingly. The aim of this system is to provide a more sophisticated conversational experience and improve customer satisfaction.

[1055] System Configuration

[1056] A means (terminal) for receiving user input.

[1057] Users send questions and inquiries to the store's official account using messaging apps such as LINE. For example, a user might send a message saying, "Please tell me the availability of the product."

[1058] Analysis means (server)

[1059] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI) that acts as an analysis tool. This AI uses a natural language processing engine to analyze the content of the user's message. For example, it analyzes the input message, "Please tell me the stock status of the product."

[1060] Emotional Engine

[1061] Based on the message content analyzed by the generative artificial intelligence, the emotion engine recognizes the user's emotions (e.g., joy, anger, sadness). The emotion engine also adjusts the response according to the user's emotions. For example, if the user is expressing dissatisfaction, the response will include a polite apology.

[1062] Generation means (server)

[1063] Based on the analysis results and the emotion engine's findings, the generation mechanism generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "We currently have very little of that item in stock. Please hurry." If the emotion engine determines that the user is dissatisfied, it might generate a response such as, "We are sorry, but we have very little of this item in stock. We apologize for the inconvenience."

[1064] Transmission method (server and terminal)

[1065] The generated response is sent to the user using the LINE API. The server sends the generated response to the user's device, and the user receives the response in their messaging app. For example, the user's smartphone might display a message saying, "We currently have very little of this product in stock. Please hurry."

[1066] Storage method (server)

[1067] The server records user input and generated responses in a database, which serves as a storage mechanism. This ensures that all conversation data is stored in the database. The stored data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results from the sentiment engine.

[1068] Analysis method (server)

[1069] The stored data is analyzed using analytical tools. The server utilizes the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that the product is popular. It is also possible to analyze trends in customer satisfaction using data from the emotion engine.

[1070] Specific example

[1071] 1. The user asks a question.

[1072] User: "When will the new shoes be in stock?"

[1073] Device (user's smartphone): Send a message to the store's official LINE account.

[1074] 2. Receive and analyze user messages.

[1075] Server: Receives messages via LINE API.

[1076] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[1077] 3. Emotion recognition by an emotion engine

[1078] Emotion Engine: Recognizes emotions from user messages and determines if the user is excited.

[1079] 4. Generate and send an appropriate response.

[1080] Server: Based on the analysis results and emotion recognition results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[1081] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[1082] 5. Storage and analysis of conversation data

[1083] Server: Stores user questions, generated responses, and sentiment recognition results.

[1084] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing or to understand customers' emotional responses.

[1085] This system allows for more personalized responses to user inquiries, adjusting them based on their emotions. Furthermore, analysis of conversational data can reveal customer needs and emotional tendencies, which can then be incorporated into business strategies.

[1086] The following describes the processing flow.

[1087] Step 1:

[1088] Users send messages to the store's official account via the LINE app. These messages may contain questions or inquiries.

[1089] Step 2:

[1090] The device (the user's smartphone) sends this message to the LINE server. The LINE server receives the message and forwards it to the store's official account server.

[1091] Step 3:

[1092] The server receives messages from users using the LINE API. It then prepares the received messages for transmission to a generative artificial intelligence system, which is used for analysis.

[1093] Step 4:

[1094] The server sends the user's message to the generative artificial intelligence API. The generative artificial intelligence uses a natural language processing engine to analyze the message content. For example, it might analyze the message, "Please tell me the product's stock status."

[1095] Step 5:

[1096] The generative artificial intelligence outputs the analysis results and passes them to the emotion engine. The emotion engine recognizes the user's emotions based on the message content. For example, it might determine from the message that the user is feeling dissatisfied.

[1097] Step 6:

[1098] Based on the results of generative artificial intelligence and an emotion engine, the server generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "We currently have very little of that item in stock. Please hurry." If it determines that the user is expressing dissatisfaction, it will generate a response such as, "We are sorry, but we have very little of this item in stock. We apologize for the inconvenience."

[1099] Step 7:

[1100] The server sends the generated response to the user's device using the LINE API. The device (the user's smartphone) receives the message sent from the server and displays a response to the user such as, "We currently have very little of that item in stock. Please hurry," or "We are sorry, but we have very little of that item in stock. We apologize for the inconvenience."

[1101] Step 8:

[1102] The server records user input and generated responses in a database, which serves as a storage mechanism. The recorded data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results from the sentiment engine.

[1103] Step 9:

[1104] The server analyzes stored data using analytical tools. For example, it uses stored conversation data to extract customer needs and trends. It also uses emotion engine data to analyze trends in customer satisfaction.

[1105] Step 10:

[1106] The server provides feedback based on the extracted information to improve business strategies. This enables the development of appropriate inventory management and marketing strategies.

[1107] (Example 2)

[1108] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1109] Traditional user-system interactions suffered from a problem where responses were uniform because they did not take user emotions into account. As a result, users could not receive responses that suited their feelings, leading to decreased satisfaction. Furthermore, it was difficult to gain deep insights through the analysis of dialogue data, making it challenging to grasp customer needs and trends.

[1110] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1111] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing user input, recognition means including an emotion engine for recognizing user emotions based on the message content analyzed by the generative artificial intelligence, generation means for generating a response based on the analysis results and the results of the recognition means, transmission means for sending the generated response to the user, storage means for recording user input and generated responses, and analysis means for analyzing the recorded data. This makes it possible to generate personalized responses that correspond to the user's emotions, thereby improving user satisfaction. Furthermore, analysis based on the stored dialogue data allows for efficient understanding of customer needs and trends, contributing to business strategy.

[1112] "Means of receiving user input" refers to an interface for users to send messages or inquiries to a system, and typically refers to messaging applications or web forms.

[1113] "Analysis means" refers to a device or system that uses generative artificial intelligence to analyze the content of messages sent by users and performs processing to understand their intent and content.

[1114] "Generative artificial intelligence" refers to artificial intelligence that has the ability to analyze user input using a natural language processing engine, and to understand and interpret its content.

[1115] A "natural language processing engine" is a software system that analyzes text data as human language to understand its context and intent.

[1116] An "emotion engine" is a computer program or device that recognizes a user's emotions based on the analyzed message content and adjusts its response accordingly.

[1117] "Recognition means" refers to a function that includes an emotion engine and executes a process of recognizing the user's emotions based on information obtained from analysis means.

[1118] "Generation means" refers to a device or program that generates an appropriate response for the user based on the analysis results and emotion recognition results.

[1119] "Transmission means" refers to an interface for sending responses generated by the generation means to the user, and usually refers to the API of a messaging application.

[1120] "Storage means" refers to a database or storage device for recording user input and generated responses and saving them in a format that can be referenced later.

[1121] "Analysis means" refers to the process and apparatus for analyzing data stored in storage means to extract customer needs and trends.

[1122] A "data platform" is a foundational system or software for managing stored data and retrieving and analyzing it as needed.

[1123] This invention relates to a system that analyzes user input and generates appropriate responses. In particular, by incorporating an emotion engine, this system can recognize the user's emotions and adjust its responses accordingly. The aim of this system is to provide a more sophisticated conversational experience and improve customer satisfaction.

[1124] System Configuration

[1125] Means for receiving user input

[1126] Users send messages and inquiries to the store's official account using messaging applications (e.g., LINE). For this purpose, the user's device (smartphone, tablet, etc.) is used.

[1127] Analysis means

[1128] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI), which uses a natural language processing engine (e.g., Google Cloud Natural Language API) to analyze the message content.

[1129] Recognition means including an emotion engine

[1130] Based on the analyzed message content, an emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotions. For example, if a user sends the message "When will the new shoes be in stock?", it might be determined that the user is excited.

[1131] generation means

[1132] Based on the analysis results and emotion recognition results, the server generates an appropriate response. Generative artificial intelligence is used in this process; for example, it might refer to an inventory management database to generate a response such as, "New shoes are scheduled to arrive next Monday. Please look forward to them!"

[1133] Transmission method

[1134] The generated response is sent to the user's device using the LINE API. The server sends the generated response to the user's smartphone, and the user receives the response in their messaging app.

[1135] Preservation means

[1136] The server stores user input and generated responses in a database. This stored data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results.

[1137] analytical means

[1138] The stored data is analyzed using analytical tools. The server utilizes the stored conversation data to extract customer needs and trends. For example, if there are many inquiries about a particular product, it indicates that the product is popular. It is also possible to analyze trends in customer satisfaction using data from the emotion engine.

[1139] Specific example

[1140] 1. The user asks a question.

[1141] User: "When will the new shoes be in stock?"

[1142] Device (user's smartphone): Send a message to the store's official LINE account.

[1143] 2. Receive and analyze user messages.

[1144] Server: Receives messages via LINE API.

[1145] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[1146] 3. Emotion recognition by an emotion engine

[1147] Emotion Engine: Recognizes "excitement" from the user's message.

[1148] 4. Generate and send an appropriate response.

[1149] Server: Based on the analysis results and emotion recognition results, it generates the response, "The new shoes will be in stock next Monday. Stay tuned!"

[1150] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[1151] 5. Storage and analysis of conversation data

[1152] Server: Stores user questions, generated responses, and sentiment recognition results in a database.

[1153] Server: Based on stored data, it analyzes and understands trends in interest in specific products and customers' emotional responses.

[1154] Example of a prompt

[1155] "Generate a response for when a user inquires about the new shoes. The user is excited."

[1156] "Generate a response for when a user is dissatisfied with the stock situation."

[1157] This system allows for more personalized responses to user inquiries, as the response is adjusted according to the user's emotions. Furthermore, analysis of stored conversation data allows for the understanding of customer needs and emotional tendencies, which can then be reflected in business strategies.

[1158] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1159] Step 1:

[1160] The user sends a message using the LINE messaging app.

[1161] In terms of the specific action, the user uses the LINE app on their smartphone to type and send the text message "Please tell me the product's stock status" to the store's official account. At this point, the input is the text message the user entered into the LINE app. The device then sends this text message.

[1162] Step 2:

[1163] The server uses the LINE API to receive user messages.

[1164] In terms of specific operation, the server receives message data sent by the user via the LINE API. This message data includes the sender's user ID and the message body. The input is the text message sent by the user, and the output is the message data stored on the server.

[1165] Step 3:

[1166] The received message is analyzed by a generative artificial intelligence.

[1167] The server passes the received message data to a generative artificial intelligence (e.g., Google Cloud Natural Language API). Specifically, the generative AI analyzes the text message and extracts the main themes and content. The input is the user's message data, and the output is the analyzed message content (e.g., "Inquiry about product availability").

[1168] Step 4:

[1169] The emotion engine recognizes the user's emotions based on the analysis results.

[1170] Based on the analysis results, the server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to determine the emotions contained in the message. Specifically, the emotion engine analyzes the emotional characteristics of the message text and recognizes emotions such as "dissatisfaction," "excitement," and "joy." The input is the analyzed message content, and the output is the recognized emotion data.

[1171] Step 5:

[1172] The server generates a response based on the analysis results and emotion recognition results.

[1173] Based on the analysis results and sentiment recognition results, the server generates an appropriate response. Specifically, the response generation algorithm refers to the inventory management database and creates an appropriate response statement. For example, if the stock is low, it will generate a response such as, "We currently have very little of that product left in stock. Please hurry." The input is the analysis results and sentiment recognition results, and the output is the generated response.

[1174] Step 6:

[1175] The server sends the generated response to the user.

[1176] The generated response is sent to the user via the LINE API. Specifically, the server uses the LINE API to send the generated text message to the user's smartphone. The input is the generated response data, and the output is the message displayed on the user's device.

[1177] Step 7:

[1178] The server saves messages and responses to the database.

[1179] In practice, the server records all user messages and generated responses in a database. This includes metadata such as message content, timestamp, user ID, and sentiment recognition results. The input is conversation data and metadata, and the output is the record stored in the database.

[1180] Step 8:

[1181] The system analyzes data stored on the server to understand trends and customer needs.

[1182] In practice, the server uses stored data for analysis to extract customer needs and trends. For example, if many users are inquiring about a particular product, it indicates that the product is popular. The input is stored conversation data, and the output is trend information and customer needs information as a result of the analysis.

[1183] (Application Example 2)

[1184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1185] Conventional user input analysis systems have the problem of not being able to respond while considering the user's emotions, making it difficult to achieve sufficient customer satisfaction. Furthermore, even in customer service by store staff in physical stores, it is difficult for staff to quickly grasp the emotional state of customers and respond appropriately. Especially in busy stores, it is difficult for staff to provide individual attention to each customer, which may lead to a decrease in customer satisfaction. In addition, because it is not possible to efficiently utilize the history of conversations with customers and reflect it in future service interactions, the same questions are asked repeatedly.

[1186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1187] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results and emotion recognition results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, analysis means for analyzing the recorded data, and store clerk support means including smart glasses that recognize customer emotions in real time and suggest appropriate responses. This enables responses that take user emotions into consideration, allowing store clerks to efficiently handle customers in physical stores and improve customer satisfaction.

[1188] "Means of receiving user input" refers to an interface for receiving information and inquiries from users.

[1189] "Analysis means" refers to methods, including generative artificial intelligence, for interpreting user input and understanding its content.

[1190] "Generation means" refers to means for creating an appropriate response to be provided to the user based on the analysis results and emotion recognition results.

[1191] "Transmission means" refers to a communication mechanism for sending the generated response to the user.

[1192] A "storage mechanism" is a function that saves user input and generated responses so that they can be referenced later.

[1193] "Analysis methods" refer to means of analyzing stored data to understand user behavior patterns and trends.

[1194] "Smart glasses" are devices worn by store employees that display information and provide real-time customer emotion recognition results and response suggestions.

[1195] "Staff support tools" are technologies that assist store staff in physical stores and facilitate smooth interactions with customers.

[1196] "Generative artificial intelligence" is an AI technology that analyzes user input data and generates responses based on that data.

[1197] "Emotion recognition results" refer to information used to analyze user input and identify their emotional state.

[1198] "Response suggestions" are pieces of information that suggest the most appropriate answer or action based on the user's emotional state and the content of their inquiry.

[1199] This invention provides a system that analyzes user input, recognizes emotions, and generates a corresponding response. In particular, this system can perform exceptionally well in customer service at physical stores. The embodiments are described below in detail.

[1200] System Configuration

[1201] The system includes the following main components:

[1202] A means (terminal) for receiving user input.

[1203] This is an interface that allows users to make inquiries via voice input or text messages. Smartphones and tablets are examples of this.

[1204] Analysis means (server)

[1205] The server uses a natural language processing engine, including generative artificial intelligence (e.g., spaCy), to analyze user input. This analysis identifies the content of the user's inquiry.

[1206] Emotional Engine

[1207] Based on the user input data analyzed by the analysis tool, the user's emotions are recognized using an emotion engine (e.g., IBM Watson Tone Analyzer). This identifies the emotions the user is experiencing (joy, anger, sadness, etc.).

[1208] Generation means (server)

[1209] This is a means for generating an appropriate response based on the results obtained by the analysis means and the emotion engine. The generative artificial intelligence model creates a response that matches the user's emotions.

[1210] Transmission method (server and terminal)

[1211] The generated response is provided to the user via a transmission method. For example, the response message is sent to the user's smartphone via the LINE API.

[1212] Storage method (server)

[1213] User input and generated responses are stored in a database, which serves as a storage mechanism. This allows for the management of past conversation data and its use for later analysis.

[1214] Analysis method (server)

[1215] The stored data is analyzed using analytical tools. This allows for an understanding of customer behavior patterns and emotional tendencies.

[1216] Shop assistant support device (smart glasses)

[1217] Smart glasses and other store clerk assistance devices are devices that display emotion recognition results and response suggestions in real time when store clerks interact with customers. This allows store clerks to quickly provide the most appropriate response.

[1218] Program Processing Description

[1219] Hardware: Smartphones, tablets, smart glasses, servers

[1220] Software: Speech recognition software (e.g., Google Cloud Speech-to-Text), natural language processing engine (e.g., spaCy), emotion recognition engine (e.g., IBM Watson Tone Analyzer)

[1221] Data processing and calculations:

[1222] The server converts the user's voice input into text (speech recognition).

[1223] Analyze the content of the text using a natural language processing engine.

[1224] Identifying customer emotions with an emotion recognition engine

[1225] Based on the analysis results and emotions, a generative artificial intelligence model generates a response.

[1226] The generated response is provided to the user via a transmission method.

[1227] Specific example

[1228] For example, if a user asks, "How big is this product?" and simultaneously expresses anxiety, the system re-analyzes the question and displays a response on the salesperson's smart glasses saying, "Don't worry. This product should be the perfect size for you."

[1229] Example of a prompt:

[1230] "Design a system that receives voice input from customers expressing questions or complaints about a product, and generates and displays appropriate responses in real time through smart glasses."

[1231] The system as a whole provides a sophisticated conversational experience that takes user emotions into account, improving customer satisfaction in physical stores. Furthermore, analyzing the stored conversation data can be used to improve future business strategies.

[1232] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1233] Step 1:

[1234] The user makes an inquiry.

[1235] Users can use their smartphones or tablets to make inquiries via voice input or text message. For example, they might ask, "How big is this product?"

[1236] Input: User voice or text inquiry

[1237] Output: User voice data or text data

[1238] Step 2:

[1239] Voice input to text conversion

[1240] The device uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's voice input into text data.

[1241] Input: User's voice data

[1242] Output: Text data

[1243] Step 3:

[1244] Analyze user input

[1245] The server uses a natural language processing engine, including generative artificial intelligence (e.g., spaCy), to analyze the user's text input. This allows the content of the inquiry to be identified.

[1246] Input: User's text data

[1247] Output: Analysis results (inquiry details)

[1248] Step 4:

[1249] Recognizing user emotions

[1250] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotions based on the analysis results. For example, it might identify that the user is feeling anxious.

[1251] Input: Analysis results

[1252] Output: Emotion recognition result (user's emotional state)

[1253] Step 5:

[1254] Generate an appropriate response

[1255] Based on the analysis results and emotion recognition results, the server uses a generative artificial intelligence model to generate an appropriate response. For example, it might create a response such as, "Please rest assured. We believe this product is the perfect size for you."

[1256] Input: Analysis results, emotion recognition results

[1257] Output: Generated response

[1258] Step 6:

[1259] Send a response

[1260] The server sends the generated response to the user's terminal via a transmission method. For example, the response message is sent to the user's smartphone via the LINE API.

[1261] Input: Generated response

[1262] Output: Response message displayed on the user's terminal

[1263] Step 7:

[1264] Saving conversation data

[1265] The server saves user input and generated responses to a database. This records the conversation data for later reference.

[1266] Input: User input data, generated response

[1267] Output: Saved conversation data

[1268] Step 8:

[1269] Analysis of conversation data

[1270] The server uses stored conversation data to analyze customer behavior patterns and emotional tendencies. This analysis helps improve future customer interactions and business strategies.

[1271] Input: Saved conversation data

[1272] Output: Analysis results (customer behavior patterns, emotional tendencies)

[1273] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1274] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1275] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1276] [Fourth Embodiment]

[1277] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1278] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1279] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1280] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1281] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1282] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1283] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1284] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1285] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1286] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1287] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1288] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1289] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1290] This invention is a system designed to solve problems related to labor shortages and the collection of authentic customer feedback. This system receives user input, analyzes the input using generative artificial intelligence, generates an appropriate response and sends it to the user, and then analyzes the data to understand customer needs.

[1291] System Configuration

[1292] A means (terminal) for receiving user input.

[1293] Users send questions and inquiries to the store's official account using messaging apps such as LINE. For example, a user might send a message saying, "Please tell me the availability of the product."

[1294] Analysis means (server)

[1295] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI) that acts as an analysis tool. This AI uses a natural language processing engine to analyze the content of the user's message. For example, it analyzes the input message, "Please tell me the stock status of the product."

[1296] Generation means (server)

[1297] Based on the analysis performed by the analysis means, the generation means generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[1298] Transmission method (server and terminal)

[1299] The generated response is sent to the user using the LINE API. The server sends the generated response to the user's device, and the user receives the response in their messaging app. For example, the user's smartphone might display a message saying, "We currently have very little of this product in stock. Please hurry."

[1300] Storage method (server)

[1301] The server records user input and generated responses. This ensures that all conversation data is stored in a database. The stored data includes, for example, a conversation timestamp and user ID.

[1302] Analysis method (server)

[1303] The stored data is analyzed by analytical tools. These tools utilize the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that the product is popular.

[1304] Specific example

[1305] 1. The user asks a question.

[1306] User: "When will the new shoes be in stock?"

[1307] Device (user's smartphone): Send a message to the store's official LINE account.

[1308] 2. Receive and analyze user messages.

[1309] Server: Receives messages via LINE API.

[1310] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[1311] 3. Generate and send an appropriate response.

[1312] Server: Based on the analysis results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[1313] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[1314] 4. Storage and analysis of conversation data

[1315] Server: Stores the user's questions and generated responses.

[1316] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing.

[1317] This system allows stores to effectively respond to user inquiries, analyze that data to understand customer needs, and thereby improve the operational efficiency of their stores.

[1318] The following describes the processing flow.

[1319] Step 1:

[1320] Users send messages to the store's official account via the LINE app. These messages may contain questions or inquiries.

[1321] Step 2:

[1322] The device (the user's smartphone) sends this message to the LINE server. The LINE server receives the message and forwards it to the store's official account server.

[1323] Step 3:

[1324] The server receives messages from users using the LINE API. It then prepares the received messages for transmission to a generative artificial intelligence system, which is used for analysis.

[1325] Step 4:

[1326] The server sends the user's message to the generative artificial intelligence API. The generative artificial intelligence uses a natural language processing engine to analyze the message content. For example, it might analyze the message, "Please tell me the product's stock status."

[1327] Step 5:

[1328] Generative artificial intelligence generates appropriate responses based on analysis results. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[1329] Step 6:

[1330] The generative artificial intelligence generates a response, which is then sent back to the server's API. The server receives this response and prepares it for transmission to the user.

[1331] Step 7:

[1332] The server uses the LINE API to send a response generated by generative artificial intelligence to the user's device. The device (the user's smartphone) receives the message sent from the server and displays the message to the user saying, "There are only a few of that product left in stock. Please hurry."

[1333] Step 8:

[1334] The server records user input and generated responses in a database, which serves as a storage mechanism. The recorded data also includes metadata such as conversation timestamps and user IDs.

[1335] Step 9:

[1336] Data recorded by storage devices is analyzed by analysis devices. The server analyzes the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it indicates that the product is popular.

[1337] Step 10:

[1338] The server provides feedback based on the extracted information to improve business strategies. This enables the development of appropriate inventory management and marketing strategies.

[1339] (Example 1)

[1340] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1341] Traditional customer service systems face challenges such as a shortage of personnel and difficulty in collecting authentic customer feedback. In particular, responding quickly to a large volume of inquiries requires a large workforce, posing problems in terms of cost and efficiency. Furthermore, there is a lack of effective means to analyze collected customer feedback and understand customer needs. As a result, companies often miss opportunities to improve customer satisfaction.

[1342] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1343] In this invention, the server includes means for receiving user input, means for receiving the user input using communication technology, analysis means including a generative artificial intelligence that analyzes the received input using a natural language processing engine, generation means including an artificial intelligence model that generates a response based on the analysis results, transmission means for sending the generated response to the user using communication technology, data management means for recording the user input and the generated response, and data analysis means for analyzing the recorded data. This makes it possible to efficiently respond to customer inquiries and understand customer needs by analyzing the data.

[1344] "User input" refers to information, questions, or requests that a user submits through the system.

[1345] "Communication technology" refers to the technologies and protocols used to send and receive data between digital devices.

[1346] "Means of receiving" refers to a digital system that has the function of capturing input data sent from a user and transferring it to other components within the system for processing.

[1347] A "natural language processing engine" is a general term for the technologies and algorithms that enable computers to understand and analyze human language.

[1348] "Generative artificial intelligence" refers to artificial intelligence systems that can automatically generate text and data similar to those generated by humans.

[1349] "Analysis means" refers to a combination of software and hardware used to analyze received data and understand its contents.

[1350] "Analysis results" refers to the output of user input data analyzed by a natural language processing engine.

[1351] "Generation means" refers to the function of a system that generates appropriate responses or data based on analysis results.

[1352] An "artificial intelligence model" refers to a data processing model that has been trained to perform a specific task using machine learning or deep learning.

[1353] "Transmission means" refers to the function of a communication system that delivers the generated response to the user.

[1354] "Data management means" refers to the system's function for recording and storing user input and generated responses.

[1355] "Data analysis methods" refer to techniques for analyzing recorded data to extract important trends and information.

[1356] A "data platform" refers to the infrastructure and associated software systems used for storing, managing, and processing data.

[1357] This invention is a system designed to address challenges such as labor shortages and the collection of authentic customer feedback. It accepts user input, analyzes the input using generative artificial intelligence, generates an appropriate response and sends it to the user, and then analyzes the data to understand customer needs. An embodiment of this system will be described in detail below.

[1358] System Overview

[1359] This system includes the following components:

[1360] A means (terminal) for receiving user input.

[1361] A means (server) for receiving input data using communication technology.

[1362] Analysis means (server) including a generative artificial intelligence that analyzes input using a natural language processing engine.

[1363] A generation means (server) including an artificial intelligence model that generates a response based on the analysis results.

[1364] Means (server and terminal) for sending the generated response to the user.

[1365] A data management means (server) for recording user input and generated responses.

[1366] Data analysis means (server) for analyzing recorded data

[1367] Detailed explanation

[1368] User input reception

[1369] Users can use smartphones or other devices to send questions and inquiries to the store's official account via messaging apps. For example, they might send a message asking, "Please tell me the availability of this product."

[1370] Message reception and analysis

[1371] The server receives messages from users using the LINE API and other communication technologies. The received messages are passed to a generative artificial intelligence (AI), which is used for analysis by a natural language processing engine. For example, it analyzes a message such as "Please tell me the stock status of the product" and extracts keywords and context.

[1372] Response generation

[1373] Based on the analysis performed by the analysis tool, the artificial intelligence model, which acts as the generation tool, generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "Currently, there is only a small amount of stock left of that product. Please hurry."

[1374] Sending a response

[1375] The generated response is then sent back to the user's device using the LINE API or other communication technologies. The server sends the generated response to the user's device, and the user receives and confirms the message in their messaging app. For example, a message such as "We currently have very little of this product in stock. Please hurry." might appear on the smartphone.

[1376] Data storage and analysis

[1377] The server records user input and generated responses. This ensures all conversation data is stored on the data platform. The stored data can then be analyzed by data analytics tools to extract customer needs and trends. For example, it might detect that many users are inquiring about a particular product, revealing its popularity.

[1378] Explanation of specific examples

[1379] The user asks a question.

[1380] User: "When will the new shoes be in stock?"

[1381] Device (user's smartphone): Send a message to the store's official account.

[1382] Receive and analyze user messages.

[1383] Server: Receives messages via LINE API.

[1384] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[1385] Generate and send an appropriate response.

[1386] Server: Based on the analysis results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[1387] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[1388] Storage and analysis of conversation data

[1389] Server: Stores the user's questions and generated responses.

[1390] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing.

[1391] Examples of prompts for generative AI models

[1392] When a user sends the message "When will the new shoes be in stock?", the following prompt is entered into the generative artificial intelligence:

[1393] A user sent the message, "When will the new shoes be in stock?" Please refer to the inventory database and generate a response.

[1394] This system allows stores to efficiently respond to customer inquiries, analyze the data to understand customer needs, and significantly improve the operational efficiency of their stores.

[1395] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1396] Step 1:

[1397] User input acceptance

[1398] User: Uses a smartphone or computer to send questions and inquiries to the store's official account via a messaging app. The user enters their specific question and presses the send button. Example: "Please tell me the product's stock status."

[1399] Input: Text data sent by the user via a messaging app.

[1400] Output: The message sent by the user arrives at the application server.

[1401] Step 2:

[1402] Message received

[1403] Server: Uses the LINE API to receive messages sent by users in real time. Upon receiving a message, it retrieves relevant data such as the message content, user ID, and timestamp.

[1404] Input: Metadata such as message text from the user, user ID, and timestamp.

[1405] Output: Structured data for passing received message data to the analysis tool.

[1406] Step 3:

[1407] Message parsing

[1408] Server: Passes received message data to a generative artificial intelligence system, which then analyzes it using a natural language processing engine. Specifically, it tokenizes the text and extracts context and keywords.

[1409] Input: Structured message data (user message, user ID, timestamp).

[1410] Output: Data including keywords and intent as analysis results.

[1411] Step 4:

[1412] Response generation

[1413] Server: Based on the analysis results, the generative artificial intelligence generates an appropriate response. Specifically, it refers to the inventory management database and retrieves information corresponding to the user's question. For example, in response to "Please tell me the inventory status of the product," it generates a sentence such as "Currently, there is only a small amount of stock left of that product."

[1414] Input: Analysis results and corresponding data (e.g., inventory status).

[1415] Output: The generated response message.

[1416] Step 5:

[1417] Sending a response

[1418] Server: The generated response is sent to the user's device using the LINE API. Before sending, a process is performed to verify the format and security of the response content.

[1419] Device: The user's smartphone receives the message via a messaging app and notifies the user.

[1420] Input: The generated response message.

[1421] Output: The response message displayed on the user's terminal.

[1422] Step 6:

[1423] Data storage

[1424] Server: Records user input and generated responses using data management means. For example, it stores them in a database along with the conversation timestamp and user ID.

[1425] Input: User input data and generated response data.

[1426] Output: Conversation history stored in the database.

[1427] Step 7:

[1428] Data analysis

[1429] Server: Analyzes stored conversation data using analysis tools. For example, machine learning algorithms are used to extract interest levels and trends for specific products.

[1430] Input: Conversation history stored in the database.

[1431] Output: Reports and data that clearly indicate customer needs and market trends.

[1432] The above steps result in a system that efficiently responds to user inquiries and allows for the understanding of customer needs based on that data.

[1433] (Application Example 1)

[1434] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1435] Traditional customer support systems faced challenges such as staff shortages and difficulty in gathering authentic customer feedback. Furthermore, limited access to product information within stores made prompt and appropriate responses difficult. This hindered improvements in customer satisfaction and accurate understanding of customer needs.

[1436] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1437] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, analysis means for analyzing the recorded data, scanning means for the user to identify products in the store, and speech recognition means for receiving and analyzing the user's voice input. This enables customers to easily obtain product information in the store using their smartphones and to receive quick and appropriate responses to their voice and text questions. Furthermore, by analyzing all inquiry data, it is possible to accurately grasp customer needs and trends and improve services.

[1438] "Means for receiving user input" refers to an interface that allows users to input text or voice from a device.

[1439] "Generative artificial intelligence" refers to artificial intelligence that analyzes input text or audio and generates appropriate responses based on that content.

[1440] "Analysis means" refers to a method for analyzing data input by a user using generative artificial intelligence.

[1441] "Generation means" refers to means for generating an appropriate response to the user based on the results analyzed by the analysis means.

[1442] "Transmission means" refers to the means for sending the generated response to the user.

[1443] "Storage means" refers to means for recording and saving user input and generated responses.

[1444] "Analysis methods" refer to means of analyzing recorded data to understand customer needs and trends.

[1445] A "scanning method" refers to a means by which a user can obtain product information by scanning a barcode or QR code within a store.

[1446] A "speech recognition method" is a means for converting and analyzing content entered by a user via voice into text.

[1447] This invention is a system for users to obtain product information in a physical store and receive real-time responses to their questions. The following describes a specific configuration for realizing this system.

[1448] First, the user scans the product barcode using a smartphone app. The product barcode data is captured by the scanning device and transmitted to an analysis device, including a generative artificial intelligence system. Similarly, when voice input is used, the voice data is converted into text data via a voice recognition device and passed to the analysis device.

[1449] In the analysis method, a generative artificial intelligence analyzes the input data and uses a natural language processing engine to understand the user's question. For example, if a user asks "What is the price of this product?" by voice, a speech recognition device converts it to text, and that text is passed to the generative AI model.

[1450] Next, the generation mechanism operates and generates an appropriate response based on the analysis results. Response generation involves integration with inventory management systems and product databases. For example, product information is retrieved, and a response such as "The price of this product is 1980 yen" is generated.

[1451] The generated response is sent to the user's smartphone via the LINE API using the designated transmission method. The user can then view this response in real time using the smartphone app.

[1452] Furthermore, user input and generated responses are recorded by storage devices and stored on a data platform. The stored data can be analyzed by analytical devices to extract customer needs and trends. For example, if many users inquired about "new shoes," it could be analyzed that there is a high demand for that product.

[1453] A concrete example of a prompt would be a user asking, "How many days can I use this product?" In response to this input, the generative AI model analyzes the input and generates a response such as, "The usage period is approximately 30 days."

[1454] In this way, users can easily obtain product information and receive real-time answers to their questions, thereby improving customer satisfaction. Furthermore, by utilizing analytical tools, stores can accurately grasp customer demand and trends, enabling them to respond quickly.

[1455] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1456] Step 1:

[1457] The user uses their smartphone to scan product barcodes in the store. The scanned barcode data is captured by the scanning device and sent to a database within the smartphone app. The input is the identification data of the product barcode, and this data is ready to be sent to the server as output.

[1458] Step 2:

[1459] When a user performs voice input, they input their question by voice using their smartphone's microphone. This voice data passes through a speech recognition system, is converted into text data, and sent to an analysis system. The input is voice data, and the converted text data is passed to the generating AI model as output.

[1460] Step 3:

[1461] The server receives messages from users via the LINE API. These messages are received as text data and passed to a generative artificial intelligence (AI) system for analysis. The input is a text message, and the output is data ready for analysis, which is then passed to the generative AI model.

[1462] Step 4:

[1463] In the analysis method, a generative artificial intelligence analyzes the input data (text data or product barcode data). Using a natural language processing engine, it understands the user's question and extracts information to create an appropriate response. The input is either text data or product barcode data, and the analysis results are passed to the generation method as output.

[1464] Step 5:

[1465] The generation mechanism generates an appropriate response based on the analysis results. For example, it might refer to an inventory management system or product database to generate a response such as, "The price of this product is 1980 yen." The input is the analysis results, and the generated response is passed to the transmission mechanism as the output.

[1466] Step 6:

[1467] The transmission method uses the LINE API to send the generated response to the user's smartphone. The user can view this response in real time on the smartphone app. The input is the generated response, and the output is the response message that is displayed on the user's device.

[1468] Step 7:

[1469] The storage mechanism records and stores user input (text messages or voice input) and the generated responses. This data is stored on a data platform for later analysis. Inputs are user inputs and generated responses, and the recorded data is stored as output.

[1470] Step 8:

[1471] The analysis method involves analyzing stored data to understand customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that demand for that product is increasing. The input is stored data, and the output is the analysis results.

[1472] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1473] This invention relates to a system that analyzes user input and generates appropriate responses. In particular, by incorporating an emotion engine, this system can recognize the user's emotions and adjust its responses accordingly. The aim of this system is to provide a more sophisticated conversational experience and improve customer satisfaction.

[1474] System Configuration

[1475] A means (terminal) for receiving user input.

[1476] Users send questions and inquiries to the store's official account using messaging apps such as LINE. For example, a user might send a message saying, "Please tell me the availability of the product."

[1477] Analysis means (server)

[1478] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI) that acts as an analysis tool. This AI uses a natural language processing engine to analyze the content of the user's message. For example, it analyzes the input message, "Please tell me the stock status of the product."

[1479] Emotional Engine

[1480] Based on the message content analyzed by the generative artificial intelligence, the emotion engine recognizes the user's emotions (e.g., joy, anger, sadness). The emotion engine also adjusts the response according to the user's emotions. For example, if the user is expressing dissatisfaction, the response will include a polite apology.

[1481] Generation means (server)

[1482] Based on the analysis results and the emotion engine's findings, the generation mechanism generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "We currently have very little of that item in stock. Please hurry." If the emotion engine determines that the user is dissatisfied, it might generate a response such as, "We are sorry, but we have very little of this item in stock. We apologize for the inconvenience."

[1483] Transmission method (server and terminal)

[1484] The generated response is sent to the user using the LINE API. The server sends the generated response to the user's device, and the user receives the response in their messaging app. For example, the user's smartphone might display a message saying, "We currently have very little of this product in stock. Please hurry."

[1485] Storage method (server)

[1486] The server records user input and generated responses in a database, which serves as a storage mechanism. This ensures that all conversation data is stored in the database. The stored data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results from the sentiment engine.

[1487] Analysis method (server)

[1488] The stored data is analyzed using analytical tools. The server utilizes the stored conversation data to extract customer needs and trends. For example, if many users are inquiring about a particular product, it can indicate that the product is popular. It is also possible to analyze trends in customer satisfaction using data from the emotion engine.

[1489] Specific example

[1490] 1. The user asks a question.

[1491] User: "When will the new shoes be in stock?"

[1492] Device (user's smartphone): Send a message to the store's official LINE account.

[1493] 2. Receive and analyze user messages.

[1494] Server: Receives messages via LINE API.

[1495] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[1496] 3. Emotion recognition by an emotion engine

[1497] Emotion Engine: Recognizes emotions from user messages and determines if the user is excited.

[1498] 4. Generate and send an appropriate response.

[1499] Server: Based on the analysis results and emotion recognition results, it generates the response, "The new shoes are scheduled to arrive next Monday."

[1500] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[1501] 5. Storage and analysis of conversation data

[1502] Server: Stores user questions, generated responses, and sentiment recognition results.

[1503] Server: Uses stored data to analyze and understand, for example, whether interest in a particular product is increasing or to understand customers' emotional responses.

[1504] This system allows for more personalized responses to user inquiries, adjusting them based on their emotions. Furthermore, analysis of conversational data can reveal customer needs and emotional tendencies, which can then be incorporated into business strategies.

[1505] The following describes the processing flow.

[1506] Step 1:

[1507] Users send messages to the store's official account via the LINE app. These messages may contain questions or inquiries.

[1508] Step 2:

[1509] The device (the user's smartphone) sends this message to the LINE server. The LINE server receives the message and forwards it to the store's official account server.

[1510] Step 3:

[1511] The server receives messages from users using the LINE API. It then prepares the received messages for transmission to a generative artificial intelligence system, which is used for analysis.

[1512] Step 4:

[1513] The server sends the user's message to the generative artificial intelligence API. The generative artificial intelligence uses a natural language processing engine to analyze the message content. For example, it might analyze the message, "Please tell me the product's stock status."

[1514] Step 5:

[1515] The generative artificial intelligence outputs the analysis results and passes them to the emotion engine. The emotion engine recognizes the user's emotions based on the message content. For example, it might determine from the message that the user is feeling dissatisfied.

[1516] Step 6:

[1517] Based on the results of generative artificial intelligence and an emotion engine, the server generates an appropriate response. For example, it might refer to an inventory management database and generate a response such as, "We currently have very little of that item in stock. Please hurry." If it determines that the user is expressing dissatisfaction, it will generate a response such as, "We are sorry, but we have very little of this item in stock. We apologize for the inconvenience."

[1518] Step 7:

[1519] The server sends the generated response to the user's device using the LINE API. The device (the user's smartphone) receives the message sent from the server and displays a response to the user such as, "We currently have very little of that item in stock. Please hurry," or "We are sorry, but we have very little of that item in stock. We apologize for the inconvenience."

[1520] Step 8:

[1521] The server records user input and generated responses in a database, which serves as a storage mechanism. The recorded data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results from the sentiment engine.

[1522] Step 9:

[1523] The server analyzes stored data using analytical tools. For example, it uses stored conversation data to extract customer needs and trends. It also uses emotion engine data to analyze trends in customer satisfaction.

[1524] Step 10:

[1525] The server provides feedback based on the extracted information to improve business strategies. This enables the development of appropriate inventory management and marketing strategies.

[1526] (Example 2)

[1527] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1528] Traditional user-system interactions suffered from a problem where responses were uniform because they did not take user emotions into account. As a result, users could not receive responses that suited their feelings, leading to decreased satisfaction. Furthermore, it was difficult to gain deep insights through the analysis of dialogue data, making it challenging to grasp customer needs and trends.

[1529] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1530] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing user input, recognition means including an emotion engine for recognizing user emotions based on the message content analyzed by the generative artificial intelligence, generation means for generating a response based on the analysis results and the results of the recognition means, transmission means for sending the generated response to the user, storage means for recording user input and generated responses, and analysis means for analyzing the recorded data. This makes it possible to generate personalized responses that correspond to the user's emotions, thereby improving user satisfaction. Furthermore, analysis based on the stored dialogue data allows for efficient understanding of customer needs and trends, contributing to business strategy.

[1531] "Means of receiving user input" refers to an interface for users to send messages or inquiries to a system, and typically refers to messaging applications or web forms.

[1532] "Analysis means" refers to a device or system that uses generative artificial intelligence to analyze the content of messages sent by users and performs processing to understand their intent and content.

[1533] "Generative artificial intelligence" refers to artificial intelligence that has the ability to analyze user input using a natural language processing engine, and to understand and interpret its content.

[1534] A "natural language processing engine" is a software system that analyzes text data as human language to understand its context and intent.

[1535] An "emotion engine" is a computer program or device that recognizes a user's emotions based on the analyzed message content and adjusts its response accordingly.

[1536] "Recognition means" refers to a function that includes an emotion engine and executes a process of recognizing the user's emotions based on information obtained from analysis means.

[1537] "Generation means" refers to a device or program that generates an appropriate response for the user based on the analysis results and emotion recognition results.

[1538] "Transmission means" refers to an interface for sending responses generated by the generation means to the user, and usually refers to the API of a messaging application.

[1539] "Storage means" refers to a database or storage device for recording user input and generated responses and saving them in a format that can be referenced later.

[1540] "Analysis means" refers to the process and apparatus for analyzing data stored in storage means to extract customer needs and trends.

[1541] A "data platform" is a foundational system or software for managing stored data and retrieving and analyzing it as needed.

[1542] This invention relates to a system that analyzes user input and generates appropriate responses. In particular, by incorporating an emotion engine, this system can recognize the user's emotions and adjust its responses accordingly. The aim of this system is to provide a more sophisticated conversational experience and improve customer satisfaction.

[1543] System Configuration

[1544] Means for receiving user input

[1545] Users send messages and inquiries to the store's official account using messaging applications (e.g., LINE). For this purpose, the user's device (smartphone, tablet, etc.) is used.

[1546] Analysis means

[1547] The server receives messages from users using the LINE API. The received messages are passed to a generative artificial intelligence (AI), which uses a natural language processing engine (e.g., Google Cloud Natural Language API) to analyze the message content.

[1548] Recognition means including an emotion engine

[1549] Based on the analyzed message content, an emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotions. For example, if a user sends the message "When will the new shoes be in stock?", it might be determined that the user is excited.

[1550] generation means

[1551] Based on the analysis results and emotion recognition results, the server generates an appropriate response. Generative artificial intelligence is used in this process; for example, it might refer to an inventory management database to generate a response such as, "New shoes are scheduled to arrive next Monday. Please look forward to them!"

[1552] Transmission method

[1553] The generated response is sent to the user's device using the LINE API. The server sends the generated response to the user's smartphone, and the user receives the response in their messaging app.

[1554] Preservation means

[1555] The server stores user input and generated responses in a database. This stored data includes metadata such as conversation timestamps, user IDs, and sentiment recognition results.

[1556] analytical means

[1557] The stored data is analyzed using analytical tools. The server utilizes the stored conversation data to extract customer needs and trends. For example, if there are many inquiries about a particular product, it indicates that the product is popular. It is also possible to analyze trends in customer satisfaction using data from the emotion engine.

[1558] Specific example

[1559] 1. The user asks a question.

[1560] User: "When will the new shoes be in stock?"

[1561] Device (user's smartphone): Send a message to the store's official LINE account.

[1562] 2. Receive and analyze user messages.

[1563] Server: Receives messages via LINE API.

[1564] Generative artificial intelligence: Analyzes the message content "When will the new shoes be in stock?".

[1565] 3. Emotion recognition by an emotion engine

[1566] Emotion Engine: Recognizes "excitement" from the user's message.

[1567] 4. Generate and send an appropriate response.

[1568] Server: Based on the analysis results and emotion recognition results, it generates the response, "The new shoes will be in stock next Monday. Stay tuned!"

[1569] Device (user's smartphone): Receives responses generated by the messaging app and displays them to the user.

[1570] 5. Storage and analysis of conversation data

[1571] Server: Stores user questions, generated responses, and sentiment recognition results in a database.

[1572] Server: Based on stored data, it analyzes and understands trends in interest in specific products and customers' emotional responses.

[1573] Example of a prompt

[1574] "Generate a response for when a user inquires about the new shoes. The user is excited."

[1575] "Generate a response for when a user is dissatisfied with the stock situation."

[1576] This system allows for more personalized responses to user inquiries, as the response is adjusted according to the user's emotions. Furthermore, analysis of stored conversation data allows for the understanding of customer needs and emotional tendencies, which can then be reflected in business strategies.

[1577] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1578] Step 1:

[1579] The user sends a message using the LINE messaging app.

[1580] In terms of the specific action, the user uses the LINE app on their smartphone to type and send the text message "Please tell me the product's stock status" to the store's official account. At this point, the input is the text message the user entered into the LINE app. The device then sends this text message.

[1581] Step 2:

[1582] The server uses the LINE API to receive user messages.

[1583] In terms of specific operation, the server receives message data sent by the user via the LINE API. This message data includes the sender's user ID and the message body. The input is the text message sent by the user, and the output is the message data stored on the server.

[1584] Step 3:

[1585] The received message is analyzed by a generative artificial intelligence.

[1586] The server passes the received message data to a generative artificial intelligence (e.g., Google Cloud Natural Language API). Specifically, the generative AI analyzes the text message and extracts the main themes and content. The input is the user's message data, and the output is the analyzed message content (e.g., "Inquiry about product availability").

[1587] Step 4:

[1588] The emotion engine recognizes the user's emotions based on the analysis results.

[1589] Based on the analysis results, the server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to determine the emotions contained in the message. Specifically, the emotion engine analyzes the emotional characteristics of the message text and recognizes emotions such as "dissatisfaction," "excitement," and "joy." The input is the analyzed message content, and the output is the recognized emotion data.

[1590] Step 5:

[1591] The server generates a response based on the analysis results and emotion recognition results.

[1592] Based on the analysis results and sentiment recognition results, the server generates an appropriate response. Specifically, the response generation algorithm refers to the inventory management database and creates an appropriate response statement. For example, if the stock is low, it will generate a response such as, "We currently have very little of that product left in stock. Please hurry." The input is the analysis results and sentiment recognition results, and the output is the generated response.

[1593] Step 6:

[1594] The server sends the generated response to the user.

[1595] The generated response is sent to the user via the LINE API. Specifically, the server uses the LINE API to send the generated text message to the user's smartphone. The input is the generated response data, and the output is the message displayed on the user's device.

[1596] Step 7:

[1597] The server saves messages and responses to the database.

[1598] In practice, the server records all user messages and generated responses in a database. This includes metadata such as message content, timestamp, user ID, and sentiment recognition results. The input is conversation data and metadata, and the output is the record stored in the database.

[1599] Step 8:

[1600] The system analyzes data stored on the server to understand trends and customer needs.

[1601] In practice, the server uses stored data for analysis to extract customer needs and trends. For example, if many users are inquiring about a particular product, it indicates that the product is popular. The input is stored conversation data, and the output is trend information and customer needs information as a result of the analysis.

[1602] (Application Example 2)

[1603] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1604] Conventional user input analysis systems have the problem of not being able to respond while considering the user's emotions, making it difficult to achieve sufficient customer satisfaction. Furthermore, even in customer service by store staff in physical stores, it is difficult for staff to quickly grasp the emotional state of customers and respond appropriately. Especially in busy stores, it is difficult for staff to provide individual attention to each customer, which may lead to a decrease in customer satisfaction. In addition, because it is not possible to efficiently utilize the history of conversations with customers and reflect it in future service interactions, the same questions are asked repeatedly.

[1605] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1606] In this invention, the server includes means for receiving user input, analysis means including generative artificial intelligence for analyzing the user input, generation means for generating a response based on the analysis results and emotion recognition results, transmission means for sending the generated response to the user, storage means for recording the user input and the generated response, analysis means for analyzing the recorded data, and store clerk support means including smart glasses that recognize customer emotions in real time and suggest appropriate responses. This enables responses that take user emotions into consideration, allowing store clerks to efficiently handle customers in physical stores and improve customer satisfaction.

[1607] "Means of receiving user input" refers to an interface for receiving information and inquiries from users.

[1608] "Analysis means" refers to methods, including generative artificial intelligence, for interpreting user input and understanding its content.

[1609] "Generation means" refers to means for creating an appropriate response to be provided to the user based on the analysis results and emotion recognition results.

[1610] "Transmission means" refers to a communication mechanism for sending the generated response to the user.

[1611] A "storage mechanism" is a function that saves user input and generated responses so that they can be referenced later.

[1612] "Analysis methods" refer to means of analyzing stored data to understand user behavior patterns and trends.

[1613] "Smart glasses" are devices worn by store employees that display information and provide real-time customer emotion recognition results and response suggestions.

[1614] "Staff support tools" are technologies that assist store staff in physical stores and facilitate smooth interactions with customers.

[1615] "Generative artificial intelligence" is an AI technology that analyzes user input data and generates responses based on that data.

[1616] "Emotion recognition results" refer to information used to analyze user input and identify their emotional state.

[1617] "Response suggestions" are pieces of information that suggest the most appropriate answer or action based on the user's emotional state and the content of their inquiry.

[1618] This invention provides a system that analyzes user input, recognizes emotions, and generates a corresponding response. In particular, this system can perform exceptionally well in customer service at physical stores. The embodiments are described below in detail.

[1619] System Configuration

[1620] The system includes the following main components:

[1621] A means (terminal) for receiving user input.

[1622] This is an interface that allows users to make inquiries via voice input or text messages. Smartphones and tablets are examples of this.

[1623] Analysis means (server)

[1624] The server uses a natural language processing engine, including generative artificial intelligence (e.g., spaCy), to analyze user input. This analysis identifies the content of the user's inquiry.

[1625] Emotional Engine

[1626] Based on the user input data analyzed by the analysis tool, the user's emotions are recognized using an emotion engine (e.g., IBM Watson Tone Analyzer). This identifies the emotions the user is experiencing (joy, anger, sadness, etc.).

[1627] Generation means (server)

[1628] This is a means for generating an appropriate response based on the results obtained by the analysis means and the emotion engine. The generative artificial intelligence model creates a response that matches the user's emotions.

[1629] Transmission method (server and terminal)

[1630] The generated response is provided to the user via a transmission method. For example, the response message is sent to the user's smartphone via the LINE API.

[1631] Storage method (server)

[1632] User input and generated responses are stored in a database, which serves as a storage mechanism. This allows for the management of past conversation data and its use for later analysis.

[1633] Analysis method (server)

[1634] The stored data is analyzed using analytical tools. This allows for an understanding of customer behavior patterns and emotional tendencies.

[1635] Shop assistant support device (smart glasses)

[1636] Smart glasses and other store clerk assistance devices are devices that display emotion recognition results and response suggestions in real time when store clerks interact with customers. This allows store clerks to quickly provide the most appropriate response.

[1637] Program Processing Description

[1638] Hardware: Smartphones, tablets, smart glasses, servers

[1639] Software: Speech recognition software (e.g., Google Cloud Speech-to-Text), natural language processing engine (e.g., spaCy), emotion recognition engine (e.g., IBM Watson Tone Analyzer)

[1640] Data processing and calculations:

[1641] The server converts the user's voice input into text (speech recognition).

[1642] Analyze the content of the text using a natural language processing engine.

[1643] Identifying customer emotions with an emotion recognition engine

[1644] Based on the analysis results and emotions, a generative artificial intelligence model generates a response.

[1645] The generated response is provided to the user via a transmission method.

[1646] Specific example

[1647] For example, if a user asks, "How big is this product?" and simultaneously expresses anxiety, the system re-analyzes the question and displays a response on the salesperson's smart glasses saying, "Don't worry. This product should be the perfect size for you."

[1648] Example of a prompt:

[1649] "Design a system that receives voice input from customers expressing questions or complaints about a product, and generates and displays appropriate responses in real time through smart glasses."

[1650] The system as a whole provides a sophisticated conversational experience that takes user emotions into account, improving customer satisfaction in physical stores. Furthermore, analyzing the stored conversation data can be used to improve future business strategies.

[1651] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1652] Step 1:

[1653] The user makes an inquiry.

[1654] Users can use their smartphones or tablets to make inquiries via voice input or text message. For example, they might ask, "How big is this product?"

[1655] Input: User voice or text inquiry

[1656] Output: User voice data or text data

[1657] Step 2:

[1658] Voice input to text conversion

[1659] The device uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's voice input into text data.

[1660] Input: User's voice data

[1661] Output: Text data

[1662] Step 3:

[1663] Analyze user input

[1664] The server uses a natural language processing engine, including generative artificial intelligence (e.g., spaCy), to analyze the user's text input. This allows the content of the inquiry to be identified.

[1665] Input: User's text data

[1666] Output: Analysis results (inquiry details)

[1667] Step 4:

[1668] Recognizing user emotions

[1669] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotions based on the analysis results. For example, it might identify that the user is feeling anxious.

[1670] Input: Analysis results

[1671] Output: Emotion recognition result (user's emotional state)

[1672] Step 5:

[1673] Generate an appropriate response

[1674] Based on the analysis results and emotion recognition results, the server uses a generative artificial intelligence model to generate an appropriate response. For example, it might create a response such as, "Please rest assured. We believe this product is the perfect size for you."

[1675] Input: Analysis results, emotion recognition results

[1676] Output: Generated response

[1677] Step 6:

[1678] Send a response

[1679] The server sends the generated response to the user's terminal via a transmission method. For example, the response message is sent to the user's smartphone via the LINE API.

[1680] Input: Generated response

[1681] Output: Response message displayed on the user's terminal

[1682] Step 7:

[1683] Saving conversation data

[1684] The server saves user input and generated responses to a database. This records the conversation data for later reference.

[1685] Input: User input data, generated response

[1686] Output: Saved conversation data

[1687] Step 8:

[1688] Analysis of conversation data

[1689] The server uses stored conversation data to analyze customer behavior patterns and emotional tendencies. This analysis helps improve future customer interactions and business strategies.

[1690] Input: Saved conversation data

[1691] Output: Analysis results (customer behavior patterns, emotional tendencies)

[1692] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1693] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1694] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1695] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1696] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1697] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1698] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1699] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1700] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1701] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1702] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1703] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1704] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1705] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1706] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1707] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1708] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1709] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1710] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1711] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1712] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1713] The following is further disclosed regarding the embodiments described above.

[1714] (Claim 1)

[1715] A means of receiving user input,

[1716] Analysis means including a generative artificial intelligence for analyzing the user's input,

[1717] A generation means for generating a response based on the analysis results,

[1718] A transmission means for sending the generated response to the user,

[1719] A storage means for recording the user's input and the generated response,

[1720] An analysis means for analyzing the recorded data,

[1721] A system that includes this.

[1722] (Claim 2)

[1723] The system according to claim 1, wherein the generative artificial intelligence includes a natural language processing engine.

[1724] (Claim 3)

[1725] The system according to claim 1, further comprising means for storing conversation data of the user input and the generated response in a data platform.

[1726] "Example 1"

[1727] (Claim 1)

[1728] A means of receiving user input,

[1729] A means for receiving the aforementioned user input using communication technology,

[1730] Analysis means including a generative artificial intelligence that analyzes the received input using a natural language processing engine,

[1731] A generation means including an artificial intelligence model that generates a response based on the analysis results,

[1732] A transmission means for transmitting the generated response to the user using communication technology,

[1733] A data management means for recording the user's input and the generated response,

[1734] A data analysis means for analyzing the recorded data,

[1735] A system that includes this.

[1736] (Claim 2)

[1737] The system according to claim 1, wherein the generative artificial intelligence includes a natural language processing engine.

[1738] (Claim 3)

[1739] The system according to claim 1, further comprising means for storing conversational data of the user input and the generated response in a data platform.

[1740] "Application Example 1"

[1741] (Claim 1)

[1742] A means of receiving user input,

[1743] Analysis means including a generative artificial intelligence for analyzing the user's input,

[1744] A generation means for generating a response based on the analysis results,

[1745] A transmission means for sending the generated response to the user,

[1746] A storage means for recording the user's input and the generated response,

[1747] An analysis means for analyzing the recorded data,

[1748] A scanning method for users to identify products within the store,

[1749] A speech recognition means that receives and analyzes the user's voice input,

[1750] A system that includes this.

[1751] (Claim 2)

[1752] The system according to claim 1, wherein the generative artificial intelligence includes a natural language processing engine.

[1753] (Claim 3)

[1754] The system according to claim 1, further comprising means for storing conversational data of the user input and the generated response in a data platform.

[1755] "Example 2 of combining an emotion engine"

[1756] (Claim 1)

[1757] A means of receiving user input,

[1758] Analysis means including a generative artificial intelligence for analyzing the user's input,

[1759] Based on the message content analyzed by the aforementioned generative artificial intelligence, a recognition means including an emotion engine that recognizes the user's emotions,

[1760] A generation means that generates a response based on the analysis results and the results of the recognition means,

[1761] A transmission means for sending the generated response to the user,

[1762] A storage means for recording the user's input and the generated response,

[1763] An analysis means for analyzing the recorded data,

[1764] A system that includes this.

[1765] (Claim 2)

[1766] The system according to claim 1, wherein the generative artificial intelligence includes a natural language processing engine.

[1767] (Claim 3)

[1768] The system according to claim 1, further comprising means for storing conversation data of the user input and the generated response in a data platform.

[1769] "Application example 2 when combining with an emotional engine"

[1770] (Claim 1)

[1771] A means of receiving user input,

[1772] Analysis means including a generative artificial intelligence for analyzing the user's input,

[1773] A generation means for generating a response based on the analysis results and emotion recognition results,

[1774] A transmission means for sending the generated response to the user,

[1775] A storage means for recording the user's input and the generated response,

[1776] An analysis means for analyzing the recorded data,

[1777] A store clerk support system including smart glasses that recognize customer emotions in real time and suggest appropriate responses,

[1778] A system that includes this.

[1779] (Claim 2)

[1780] The system according to claim 1, wherein the generative artificial intelligence includes a natural language processing engine.

[1781] (Claim 3)

[1782] The system according to claim 1, further comprising means for storing conversation data of the user input and the generated response in a data platform. [Explanation of Symbols]

[1783] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving user input, Analysis means including a generative artificial intelligence for analyzing the user's input, A generation means for generating a response based on the analysis results, A transmission means for sending the generated response to the user, A storage means for recording the user's input and the generated response, An analysis means for analyzing the recorded data, A system that includes this.

2. The system according to claim 1, wherein the generative artificial intelligence includes a natural language processing engine.

3. The system according to claim 1, further comprising means for storing conversation data of the user input and the generated response in a data platform.

Images