System

Abstract

A system is provided.SOLUTION: A system comprising: server means for generating an answer to a query using a generated AI model; terminal means for transmitting the query to the server means and receiving the answer; and means for updating the generated AI model based on feedback acquired from the terminal means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The purpose is to solve the following problems in customer support operations:

[0005] 1. Human error and inconsistent response quality due to the large volume of inquiries.

[0006] 2. Difficulty in responding quickly at night or on holidays.

[0007] 3. Reduce the human burden involved in responding to incidents and improve the quality of responses. [Means for solving the problem]

[0008] In order to solve the above problems, the present invention provides the following means.

[0009] The system includes a server means for generating answers to queries using a generative AI model, a terminal means for sending queries to the server means and receiving answers, and a means for updating the generative AI model based on feedback obtained from the terminal means.

[0010] Furthermore, by providing a means for analyzing the content of inquiries and classifying them into categories, it is possible to respond appropriately to each inquiry.

[0011] In addition, the server means is provided with means for referencing a database based on the contents of an inquiry and acquiring additional information, thereby making it possible to provide a more accurate and detailed answer.

[0012] A "generative AI model" is an algorithm that uses natural language processing technology to generate appropriate responses based on input information.

[0013] "Server means" refers to a device or system that has the function of generating a response based on the inquiry content using a generative AI model and transmitting it to the terminal means.

[0014] "Terminal means" refers to a device or application that allows a user to input inquiry details and receive a response from a server.

[0015] "Feedback" is information that allows users to input their evaluations and opinions on the answers provided, and is used to help improve the system.

[0016] A "database" is an information management system that efficiently stores related information such as inquiry details, user data, and reservation information, and can be referenced as needed.

[0017] An "inquiry" is information that a user inputs into the customer support system to request information or to solve a problem.

[0018] A "category" is a group of inquiries categorized to make it easier to process them, and examples include reservation confirmation, troubleshooting, and facility information. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] An embodiment of the present invention will be described in detail. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses.

[0041] server

[0042] 1. Loading and initializing the generative AI model

[0043] The server first loads the generative AI model on startup, which involves loading the model's weight files, configuration files, and associated resources into memory.

[0044] Next, it connects to the database and verifies that the necessary tables and indexes are set up correctly, including where to store user information, booking data, and feedback.

[0045] 2. Processing inquiries

[0046] The server receives the inquiry information sent from the terminal, including the category and detailed information of the inquiry.

[0047] First, the inquiry content is analyzed and classified into appropriate categories (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0048] The classified data is then passed to a generative AI model to generate an appropriate answer.

[0049] If necessary, retrieve additional information from the database, for example, look up data related to the reservation number to confirm the reservation information.

[0050] The generated answer is sent to the terminal and provided to the user.

[0051] Terminal

[0052] 1. Displaying the user interface

[0053] The terminal displays a customer support user interface to the user, which includes a contact form, a chat box, and a voice input option.

[0054] The terminal retrieves initial data from the server and displays it to the user. The initial data includes, for example, store opening hours and campaign information.

[0055] 2. Submitting and viewing inquiries

[0056] The inquiry entered by the user is sent to the server, at which point the device validates the input.

[0057] When a response is returned from the server, the terminal displays the response to the user.

[0058] Furthermore, a feedback input interface is displayed, and evaluations and opinions from users are sent to the server.

[0059] User

[0060] 1. Enter your inquiry

[0061] The user inputs a question or inquiry through the terminal interface. For example, they might input, "Please tell me the status of reservation number 12345."

[0062] If necessary, you can also input voice and upload images.

[0063] 2. Review answers and provide feedback

[0064] The user reviews the answers provided and evaluates whether the problem has been resolved.

[0065] Provide feedback as needed to help improve the system.

[0066] Specific examples

[0067] Let us take the example of a hotel reservation confirmation scenario.

[0068] 1. Reservation confirmation inquiries

[0069] User: Enter "Please tell me the reservation status for reservation number 12345" into the smartphone app.

[0070] Terminal: Send this input to the server.

[0071] 2. Server Processing

[0072] Server: Analyzes the received inquiry content and inputs it into the reservation confirmation category generation AI model.

[0073] Server: Search the database for information related to reservation number 12345.

[0074] Server: Generates a response saying, "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for your use." and sends it to the terminal.

[0075] 3. Providing answers

[0076] Terminal: Displays the answer from the server to the user.

[0077] User: Enters feedback such as "Thank you for your quick response" and sends it to the server.

[0078] As described above, the system of this invention uses a generative AI model to automate inquiry responses and provide high-quality customer support 24 hours a day. This improves user satisfaction and reduces the burden on personnel. Furthermore, the system constantly evolves based on feedback, improving its problem-solving capabilities.

[0079] The processing flow will be explained below.

[0080] Step 1:

[0081] Server: Responsible for loading and initializing the generative AI model. It loads the model weights, configuration files, and related resources from disk into memory. It also connects to the database and ensures that the necessary tables and indexes are set up correctly. This includes where to store user information, reservation data, and feedback.

[0082] Step 2:

[0083] Terminal: Displays the customer support user interface to the user. This includes an inquiry form, chat box, voice input options, etc. It also retrieves initial data (such as store opening hours and campaign information) from the server and displays it to the user.

[0084] Step 3:

[0085] User: Enters questions or inquiries through the device interface. For example, they can enter text such as "Please tell me the status of reservation number 12345." They can also input voice commands or upload images as needed.

[0086] Step 4:

[0087] Terminal: Validates the query entered by the user. Checks for errors or omissions in the input, and displays an error message to the user if there is a problem. If validation is successful, sends the query to the server.

[0088] Step 5:

[0089] Server: Receives inquiries sent from the terminal. The received data includes metadata such as the user ID, inquiry content, and timestamp.

[0090] Step 6:

[0091] Server: Analyzes the received inquiry and classifies it into an appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0092] Step 7:

[0093] Server: Passes the classified data to the generative AI model and generates the optimal answer to the inquiry. For example, the AI ​​model generates an answer such as "Reservation number 12345 is scheduled to check in on 2023-11-01."

[0094] Step 8:

[0095] Server: Retrieves additional information from the database if necessary. For example, to confirm the reservation, it searches and retrieves details related to reservation number 12345 from the database.

[0096] Step 9:

[0097] Server: Combines the generated answer with additional information to generate a final answer. For example, you can add a supplemental message such as "Thank you for using our service."

[0098] Step 10:

[0099] Server: Sends the final answer to the device.

[0100] Step 11:

[0101] Terminal: The answer obtained from the server is displayed to the user. To ensure the user can confirm, the screen displays "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for using our service."

[0102] Step 12:

[0103] User: Review the provided answer and provide feedback if necessary. For example, "Thank you for your quick response."

[0104] Step 13:

[0105] Terminal: Sends the feedback entered by the user to the server.

[0106] Step 14:

[0107] Server: Stores the received feedback in a database, which helps improve the generative AI model.

[0108] Step 15:

[0109] Server: Updates the generative AI model based on feedback to improve the accuracy of responses to future inquiries.

[0110] As described above, the system handles user inquiries through a series of processing steps and uses generative AI models to provide high-quality answers, achieving consistent responses and efficient processing, thereby improving the quality of customer support operations.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] Conventional customer support systems have difficulty responding to user inquiries effectively and quickly, often resulting in delayed responses, especially when complex inquiries or a large number of inquiries are received simultaneously. Furthermore, analyzing inquiries and generating responses requires a large amount of human resources, resulting in a high workload. Furthermore, there are insufficient means to effectively utilize feedback to improve the system. To solve these problems, a system that utilizes generative AI models to automate inquiry responses and provide efficient and prompt responses is needed.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes a means for reading a weight file and a setting file of the generative AI model, a means for connecting to a database, and a means for referencing a data storage device based on the query content and obtaining additional information. This allows the generative AI model to analyze the query content, classify it into categories, and quickly generate an appropriate answer, while also obtaining additional information from the database to provide a more detailed and accurate answer. Furthermore, the accuracy and effectiveness of the system can be improved by updating the generative AI model based on feedback obtained from users.

[0116] The "information processing device means" is a device that performs processing to generate an answer to an inquiry using a generative AI model.

[0117] The "display device means" is a device that sends inquiries from users and receives and displays responses from the generative AI model.

[0118] "Evaluation Information" means user-provided feedback or evaluations that are used to improve the performance of generative AI models.

[0119] A "weight file" is a data file that stores patterns and knowledge that a generative AI model has previously learned.

[0120] A "configuration file" is a file used to manage the behavior and parameters of a generative AI model.

[0121] A "data storage device" is a device for storing data necessary for system operation, such as user information, reservation data, and inquiry records.

[0122] "Inquiry content" refers to questions or requests that users input to the system.

[0123] A "generative AI model" is an algorithm that uses machine learning technology to automatically generate answers to user inquiries.

[0124] "Category" means a major group or area into which inquiries are classified.

[0125] "Additional information" is information obtained from a data repository to supplement the response to a query.

[0126] An embodiment of the present invention will be described in detail below. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses.

[0127] server

[0128] The server first loads the weight and configuration files of the generated AI model. This is typically done using a deep learning library such as TENSORFLOW (registered trademark) or PyTorch. For example, the model is loaded using torch.load('model.pth') or tf.keras.models.load_model('model.h5').

[0129] Next, the server connects to the database, which is most likely a relational database management system such as MySQL or PostgreSQL. This is done using the sqlalchemy library, for example with code like engine = create_engine('mysql: / / user:password@host / dbname').

[0130] The server receives the query information sent from the device. This is often done using an HTTP request. For example, the Flask framework is used to obtain data from request.json.

[0131] The server then analyzes the query using natural language processing techniques and categorizes it, using libraries such as nltk and spaCy to tokenize the text and predict the category using a classification model.

[0132] The parsed data is passed to a generative AI model to generate an appropriate answer. For example, a prompt is input to the generative AI model, such as gpt3_model.predict(prompt). If necessary, additional information is retrieved from the database. The SQL query uses syntax such as SELECT FROM reservations WHERE reservation_id = '12345'.

[0133] The generated answer is returned to the terminal as an HTTP response. jsonify(response) creates a JSON format answer and returns it as return response.

[0134] Terminal

[0135] The terminal displays a customer support user interface to the user, which includes a contact form, a chat box, and a voice input option. HTML and React are used as front-end technologies.

[0136] As initial data, display business hours and campaign information obtained from the server. Use an AJAX call to obtain and display the data. For example, this is often implemented as $.get(' / api / info', function(data) { $('info').html(data);});

[0137] When sending the query entered by the user to the server, the device performs validation to ensure that the input is correct. This is done with the following code: function sendQuery() { var query = $('query').val(); $.post(' / api / query', { query: query}, function(response) { displayResponse(response);});}

[0138] The response returned from the server is displayed to the user. The received data is embedded in HTML. Implement it as follows: function displayResponse(response) { $('response').html(response);}. In addition, a feedback input interface is displayed, and user ratings and opinions are sent to the server. The feedback form is: <textarea id="feedback">< / textarea> <button onclick="sendFeedback()"> Submit Feedback< / button> Arrange it as follows.

[0139] User

[0140] The user enters questions or inquiries through the terminal interface. For example, they can enter, "Please tell me the reservation status for reservation number 12345." The user types directly into the chat box or form using the keyboard.

[0141] It also allows voice input and image uploading as needed. The voice input function is implemented using the Web Speech API. <input type="file" id="imageUpload"> or <button onclick="startVoiceRecognition()"> Speak< / button> Set it as follows.

[0142] The user reviews the provided answer and evaluates whether the problem has been resolved, reads the answer to check accuracy and satisfaction, enters their thoughts and opinions in the feedback form, and presses the submit button.

[0143] Specific examples

[0144] Hotel booking confirmation scenario

[0145] 1. The user uses the smartphone app to enter "Please tell me the reservation status for reservation number 12345" into the terminal.

[0146] 2. The device sends this input to the server.

[0147] 3. The server analyzes the received inquiry and inputs it into the generative AI model as a "reservation confirmation" category.

[0148] 4. The server searches the database for information related to reservation number 12345, generates a response such as "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for your use," and sends it to the terminal.

[0149] 5. The terminal displays the response from the server to the user.

[0150] 6. The user enters feedback such as "The response was quick and helpful" and sends it to the server.

[0151] In this way, the system of this invention uses generative AI models to automate inquiry responses and provide high-quality customer support 24 hours a day. This improves user satisfaction and significantly reduces the human workload required for support. Furthermore, the system constantly evolves based on feedback, enabling it to provide optimal solutions.

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Step 1:

[0154] Loading and initializing the generative AI model

[0155] The server loads the weight and configuration files of the generative AI model from disk into memory. This process is performed using library functions from TensorFlow or PyTorch, for example. Specifically, commands such as torch.load('model.pth') or tf.keras.models.load_model('model.h5') are executed. The input is the weight and configuration files, and the output is the generative AI model deployed in memory.

[0156] Step 2:

[0157] Connecting to a Database

[0158] The server connects to a relational database such as MySQL or PostgreSQL. This connection is made using the sqlalchemy library. For example, run a command like engine = create_engine('mysql: / / user:password@host / dbname'). The input is the database connection information, and the output is the database instance to which the connection is established.

[0159] Step 3:

[0160] Receiving inquiry information

[0161] The server receives the query information sent from the terminal via an HTTP request. When using Flask as a framework, data is obtained from request.json, for example. The input is the query content sent as an HTTP request, and the output is the query information stored in variables on the server.

[0162] Step 4:

[0163] Analysis of inquiry content and categorization

[0164] The server analyzes the received query using natural language processing (NLP) techniques. This process involves tokenization using the nltk and spaCy libraries and applying classification models. The input is the query text data, and the output is classified category information.

[0165] Step 5:

[0166] Input to generative AI model and answer generation

[0167] The server inputs the parsed data as a prompt sentence into the generative AI model to generate an appropriate answer. For example, a command such as gpt3_model.predict(prompt) is executed. The input is the generated prompt sentence, and the output is the generated answer text.

[0168] Step 6:

[0169] Retrieving additional information from the database

[0170] If necessary, the server retrieves additional information from the database by executing an SQL query (e.g. SELECT FROM reservations WHERE reservation_id = '12345'). The input is the reservation number as an SQL query, and the output is the retrieved reservation record.

[0171] Step 7:

[0172] Submit your answer

[0173] The server sends the generated answer to the terminal as an HTTP response. Specifically, it creates a JSON format answer with jsonify(response) and returns it as return response. The input is the generated answer text, and the output is the HTTP response sent to the terminal.

[0174] Step 8:

[0175] User Interface Display

[0176] The terminal displays a customer support user interface to the user. For example, it generates an inquiry form or chat box using HTML and React. The input is an HTML template or stylesheet as initial data, and the output is the interface displayed in the user's browser.

[0177] Step 9:

[0178] Submit an inquiry

[0179] The user inputs and sends the inquiry through the terminal interface. For example, they might input "Please tell me the reservation status for reservation number 12345" and press the send button. The input is the text data entered by the user, and the output is an HTTP request sent to the server.

[0180] Step 10:

[0181] Check your answers

[0182] The user checks the answer sent from the server on the terminal interface. The terminal receives the response data from the server and displays it in the chat box. The input is the answer text from the server, and the output is the answer displayed on the user's screen.

[0183] (Application example 1)

[0184] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0185] There is a need for a system that can quickly and reliably respond to various inquiries and problems encountered by users of autonomous vehicles. There is also a lack of effective means to provide rapid response in emergencies and safety alert notifications. This is necessary to increase the safety and peace of mind of users.

[0186] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0187] In this invention, the server includes means for generating responses to inquiries using a generative AI model, terminal means for sending inquiries and receiving responses, means for updating the generative AI model based on feedback, means for responding to inquiries and problems encountered by users of autonomous vehicles, and means for generating emergency response measures and safety alert notifications, thereby enabling users of autonomous vehicles to receive support quickly and reliably and enabling effective emergency and safety responses.

[0188] A "generative AI model" is an artificial intelligence model that automatically generates appropriate answers based on given data and inquiry content.

[0189] "Server means" means a computing device for processing inquiries, generating and managing responses using a generative AI model.

[0190] "Terminal means" refers to a device through which a user inputs an inquiry and receives a response from the server means, and examples of such a device include a smartphone or tablet.

[0191] "Feedback" refers to the act and content of a user's evaluation or opinion regarding a system's response or answer.

[0192] An "autonomous vehicle" is a vehicle that operates autonomously with minimal driver intervention.

[0193] "Emergency response measures" refer to the course of action and procedures for responding quickly and appropriately in the event of an emergency such as an accident or breakdown.

[0194] "Safety alert notification" is a notification function that promptly notifies users of safety precautions and warnings.

[0195] "Inquiry analysis" is the process of understanding and classifying the inquiries provided by users and assigning them to appropriate categories.

[0196] "Category" refers to a group or classification item for systematically classifying inquiry content.

[0197] A "database" is a collection of information that systematically stores and manages various inquiries, their responses, user information, and so on.

[0198] "Acquisition of additional information" is the process of searching, extracting, and providing the necessary information from a database based on the content of the inquiry.

[0199] An embodiment of the present invention will now be described in detail. This system is a customer support and security support system that responds to inquiries and problems encountered by users of autonomous vehicles. The system is mainly composed of three components: a server, a terminal, and a user.

[0200] server

[0201] The server is the main device that generates answers to queries using the generative AI model. First, the server loads the generative AI model at startup and reads the configuration file, weight file, etc. into memory. Next, it connects to the database used by the system and checks whether the necessary tables and indexes, such as user information, reservation data, and feedback information, are set up correctly.

[0202] When the server receives an inquiry from a user, it analyzes the inquiry and classifies it into an appropriate category (e.g., reservation confirmation, emergency response measures, safety confirmation, etc.). It then uses a generative AI model to generate an optimal answer to the inquiry, retrieving additional information from a database if necessary. The generated answer is sent to the device and provided to the user.

[0203] Terminal

[0204] The terminal is a device such as a smartphone or tablet that allows users to enter inquiries and receive responses from the server. When a user enters and sends an inquiry into the terminal, the terminal sends the content to the server. The terminal receives and displays the response from the server.

[0205] Additionally, the device receives feedback from users and transmits it to the server, which uses it as data to improve the performance of the generative AI model.

[0206] User

[0207] When using an autonomous vehicle, a user inputs their inquiry using a terminal. For example, they can input, "Please tell me the reservation status for reservation number 67890." The user checks the answer provided by the server and provides feedback as needed.

[0208] Specific examples

[0209] For example, when confirming a reservation for an autonomous vehicle, a user opens a smartphone app and enters, "Please tell me the reservation status for reservation number 67890." The device sends this inquiry to a server, which processes the inquiry using a generative AI model in the reservation confirmation category. The server searches the database for information related to reservation number 67890, generates a response saying, "Reservation number 67890 is scheduled to check in on 2023-12-01," and sends it to the device. The device displays this response to the user, who then enters feedback such as, "The response was quick and helpful," and submits it.

[0210] In addition, if an emergency inquiry such as "My car won't start! What should I do?" is entered, the server will respond using a generative AI model in the emergency response category, generating a response such as "Emergency services have been notified. Please remain calm," and sending it to the device.

[0211] This system will enable users of autonomous vehicles to receive prompt and reliable support, and will also enable effective emergency and safety responses. An example of a prompt sentence is, "Please tell me the status of reservation number 12345."

[0212] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0213] Step 1:

[0214] The server launches the generative AI model and loads the model's weight and configuration files into memory. This process occurs during the system initialization phase and lays the groundwork for the generative AI model to function correctly. It uses the weight and configuration files as input and obtains the loaded model as output.

[0215] Step 2:

[0216] The server connects to the database and checks the consistency of the various tables and indexes that store user information, reservation data, feedback information, etc. This process maintains data consistency and ensures smooth query processing later. It uses database connection information as input and obtains the connection success or failure status as output.

[0217] Step 3:

[0218] The user uses a terminal to input a query. For example, "Please tell me the reservation status for reservation number 12345." This input data is sent to the server via the terminal. The user's query content is used as input, and the query data is sent to the server as output.

[0219] Step 4:

[0220] The server analyzes the received inquiry and classifies it into the appropriate category. This process is performed by a generative AI model, which categorizes the inquiry into categories such as "reservation confirmation," "emergency measures," and "security check." The server uses the inquiry as input and obtains the categorization result as output.

[0221] Step 5:

[0222] The server retrieves additional information from the database as needed based on the category. For example, in the case of a reservation confirmation, the server searches and retrieves the relevant reservation information from the database. It uses the category classification results and the inquiry content as input, and obtains the search results (additional information) as output.

[0223] Step 6:

[0224] The server uses a generative AI model to generate the optimal answer based on the acquired additional information and the query. This generation process results in the most appropriate and useful answer for the user. The additional information and the query are used as input, and the generated answer is obtained as output.

[0225] Step 7:

[0226] The terminal receives the answer sent from the server and displays it to the user, who then confirms this information. It uses the answer data from the server as input and gets the answer displayed to the user as output.

[0227] Step 8:

[0228] The user inputs feedback for the provided answer and sends it to the server via the terminal. For example, the user inputs feedback such as "The answer was quick and helpful." The user's feedback is used as input, and the feedback sent to the server is obtained as output.

[0229] Step 9:

[0230] The server stores the received feedback in a database and uses it to improve the performance of the generative AI model. This process allows the system to continuously improve. It uses user feedback as input and obtains an updated database as output.

[0231] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0232] An embodiment of the present invention will be described in detail. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses. Furthermore, it is equipped with an emotion engine that recognizes the user's emotions, enabling more appropriate and empathetic responses.

[0233] server

[0234] 1. Loading and initializing the generative AI model

[0235] When the server starts up, it loads the generative AI model, which includes loading the model's weight files, configuration files, and associated resources.

[0236] Connect to the database and check that the necessary tables and indexes are set up correctly. Check where user information, reservation data, and feedback are stored.

[0237] 2. Processing inquiries

[0238] The server receives the inquiry information sent from the terminal, including metadata such as the inquiry content, category, user ID, and timestamp.

[0239] Analyze the inquiry and categorize it into the appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0240] The classified data is passed to a generative AI model to generate the best answer, retrieving additional information from the database if necessary.

[0241] The generated answer is sent to the terminal and provided to the user.

[0242] 3. The Emotional Engine

[0243] The server uses an emotion engine to analyze the user's emotions, identifying the user's emotional state (e.g., joy, sadness, anger, tension, etc.) from voice and text.

[0244] The emotion engine's analysis results are reflected in the generative AI model, which then adjusts the tone and content of the response. For example, if the user is feeling angry, the response will be generated in a more polite and calm tone.

[0245] 4. Processing Feedback

[0246] The server receives feedback sent from the device and stores it in a database, which is used to update the generative AI model and improve the accuracy of the system.

[0247] Terminal

[0248] 1. Displaying the user interface

[0249] The device displays a customer support interface, including a contact form, chat box, and voice input options.

[0250] Retrieve and display initial data (e.g., store opening hours, campaign information, etc.) from the server.

[0251] 2. Submitting and viewing inquiries

[0252] Validate the query entered by the user and send it to the server. Only queries that pass validation are sent.

[0253] Receives the response from the server and displays it to the user, displaying all relevant information on the screen for the user to review.

[0254] A feedback input interface is provided to receive evaluations and opinions from users and transmit them to the server.

[0255] User

[0256] 1. Enter your inquiry

[0257] The user enters a question or inquiry through the terminal interface, for example, "Please tell me the status of reservation number 12345."

[0258] If necessary, you can also input voice and upload images.

[0259] 2. Review answers and provide feedback

[0260] The user reviews the answers provided and evaluates whether the problem has been resolved.

[0261] If necessary, enter your feedback to help improve the system. For example, enter "Thank you for your quick response."

[0262] Specific examples

[0263] Let us take the example of a hotel reservation confirmation scenario.

[0264] 1. Reservation confirmation inquiries

[0265] User: Enter "Please tell me the reservation status for reservation number 12345" into the smartphone app.

[0266] Terminal: Send this input to the server.

[0267] 2. Server Processing

[0268] Server: Analyzes the received inquiry content and inputs it into the reservation confirmation category generation AI model.

[0269] Server: Searches and retrieves information related to reservation number 12345 from the database.

[0270] Server: Analyzes the user's emotional state using an emotion engine, for example, sensing tension or anxiety from the text.

[0271] Server: Adjusts the answer based on the analysis results. For example, it generates an answer such as "Don't worry. Reservation number 12345 is scheduled to check in on 2023-11-01."

[0272] Server: Sends the generated answer to the device.

[0273] 3. Providing answers

[0274] Terminal: Displays the answer from the server to the user.

[0275] User: Enters feedback such as "Thank you for your quick response" and submits.

[0276] In this way, the system of the present invention is equipped with an emotion engine that recognizes user emotions and uses a generative AI model to automate inquiry responses, making it possible to provide more appropriate and empathetic customer support. This improves user satisfaction and reduces the burden on personnel. The system constantly evolves based on feedback, improving its problem-solving capabilities.

[0277] The processing flow will be explained below.

[0278] Step 1:

[0279] Server: Responsible for loading and initializing the generative AI model. This includes loading the model weights and configuration files and associated resources from disk into memory. It also connects to the database and checks tables and indexes.

[0280] Step 2:

[0281] Terminal: Displays the user interface to the user. This includes a contact form, chat box, and voice input options. It also retrieves and displays initial data from the server (e.g., store hours, campaign information, etc.).

[0282] Step 3:

[0283] User: Enters a question or inquiry through the device interface. For example, enters text such as "Please tell me the reservation status for reservation number 12345." If necessary, voice input and image upload are also performed.

[0284] Step 4:

[0285] Terminal: Validates the entered query. Performs error checking and displays an error message to the user if there is a problem. If validation is successful, sends the query to the server.

[0286] Step 5:

[0287] Server: Receives inquiries sent from the terminal. The received data includes metadata such as the user ID, inquiry content, and timestamp.

[0288] Step 6:

[0289] Server: Analyzes the received inquiry and classifies it into the appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0290] Step 7:

[0291] Server: Analyzes the user's emotions using an emotion engine. Identifies the emotional state (e.g., joy, sadness, anger, tension, etc.) from speech and text.

[0292] Step 8:

[0293] Server: The classified data and the results of the emotion engine are input into the generative AI model to generate the optimal answer, such as "Reservation number 12345 is scheduled to check in on 2023-11-01."

[0294] Step 9:

[0295] Server: If necessary, retrieve additional information from the database. For example, look up and retrieve details related to reservation number 12345 to confirm the reservation information.

[0296] Step 10:

[0297] Server: Combines the generated answer with additional information to generate the final answer. Adjusts the tone of the answer based on the results of the emotion engine. For example, it generates something like, "Don't worry, reservation number 12345 is scheduled to check in on 2023-11-01."

[0298] Step 11:

[0299] Server: Sends the final answer to the device.

[0300] Step 12:

[0301] Terminal: The answer obtained from the server is displayed to the user. To ensure the user can confirm, the screen displays "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for using our service."

[0302] Step 13:

[0303] User: Review the provided answer and provide feedback if necessary. For example, "Thank you for your quick response."

[0304] Step 14:

[0305] Terminal: Sends the feedback entered by the user to the server.

[0306] Step 15:

[0307] Server: Stores the received feedback in a database, which is used to improve the generative AI model.

[0308] Step 16:

[0309] Server: Updates the generative AI model and emotion engine based on feedback to improve response accuracy for future inquiries.

[0310] As described above, the system handles user inquiries through a series of processing steps and provides high-quality answers by utilizing generative AI models and an emotion engine. By enabling emotion-sensitive responses, user satisfaction can be further improved.

[0311] Example 2

[0312] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0313] Conventional customer support systems often lack consistency in their responses to inquiries. They also struggle to properly understand users' emotions and respond empathetically, potentially resulting in lower user satisfaction. In particular, conventional systems face challenges in providing adequate support in situations where rapid responses to changes in emotions are required.

[0314] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for generating an answer to a query using a generative AI model, terminal means for sending an inquiry to the server means and receiving an answer, means for updating the generative AI model based on feedback acquired from the terminal means, and means for analyzing the emotional state of the query using an emotion engine and adjusting the answer to be generated. This makes it possible to provide an appropriate and consistent answer according to the user's emotions and improve user satisfaction.

[0315] A "generative AI model" refers to an algorithm or data model that uses artificial intelligence technology to generate appropriate answers to natural language queries.

[0316] "Server means" refers to a computer system for receiving inquiry information, generating a response using a generative AI model, and performing emotion analysis using an emotion engine.

[0317] "Terminal means" refers to a device, such as a computer or smartphone, through which a user inputs inquiry details, transmits the information to a server, and receives a response from the server.

[0318] An "emotion engine" refers to software or algorithms that analyze a user's emotional state from text or voice and recognize specific emotions.

[0319] "Feedback" refers to users inputting their evaluations and opinions about the answers and services provided by the system, and this information is used to improve the system's performance.

[0320] "Database" refers to a set of data structures for storing information in response to queries and allowing server means to retrieve additional information as needed.

[0321] "Parsing" refers to the process of using natural language processing techniques to structure and categorize query content.

[0322] "Answer generation" refers to the process of using a generative AI model to generate an appropriate answer to a user's inquiry.

[0323] This invention is an advanced conversational robot system that uses generative AI models to automate customer support tasks and provide fast, consistent responses. It also features an emotion engine that recognizes the user's emotions and provides appropriate, empathetic responses. The following describes each component of the system and its operation in detail.

[0324] server

[0325] The server is the center of the system and plays the following roles:

[0326] 1. Load and initialize the generative AI model:

[0327] On startup, the server loads a generative AI model (e.g., a generic generative AI model), which includes loading the model's weight files, configuration files, and associated resources.

[0328] The server connects to a database system (e.g. a general purpose database) and checks whether the necessary tables and indexes are set up correctly, e.g., where to store user information, reservation data, feedback, etc.

[0329] 2. Handling inquiries:

[0330] The server receives the inquiry information sent from the terminal. This received data includes the inquiry content, category, user ID, timestamp, etc.

[0331] The server analyzes the query content using a natural language processing library (for example, a general natural language processing library) and classifies it into an appropriate category.

[0332] The classified data is passed to a generative AI model to generate the best answer, retrieving additional information from the database if necessary.

[0333] The generated answer is sent to the terminal and provided to the user.

[0334] 3. How the Emotional Engine Works:

[0335] The server analyzes the user's emotions using an emotion engine (e.g., a general emotion analysis tool) and identifies the user's emotional state from the voice and text.

[0336] The emotion engine's analysis results are reflected in the generative AI model, which then adjusts the tone and content of the response. For example, if the user is feeling angry, the response will be generated in a more polite and calm tone.

[0337] 4. Feedback Processing:

[0338] The server receives feedback sent from the device and stores it in a database, which is used to update the generative AI model and improve the accuracy of the system.

[0339] Terminal

[0340] The terminal provides an interface for the user to access customer support.

[0341] 1. Display the user interface:

[0342] The device displays a customer support interface, including a contact form, a chat box, and a voice input option.

[0343] As initial data, store opening hours and campaign information are obtained from the server and displayed.

[0344] 2. Submitting and viewing inquiries:

[0345] Validate the query entered by the user and send it to the server. Only queries that pass validation are sent.

[0346] Receives the response from the server and displays it to the user, displaying all relevant information on the screen for the user to review.

[0347] A feedback input interface is provided to receive evaluations and opinions from users and transmit them to the server.

[0348] User

[0349] Users submit queries through the system and provide feedback on the answers provided.

[0350] 1. Enter your inquiry:

[0351] The user enters a question or inquiry through the terminal interface, for example, "Please tell me the status of reservation number 12345."

[0352] If necessary, you can also input voice and upload images.

[0353] 2. Review answers and provide feedback:

[0354] The user reviews the answers provided and evaluates whether the problem has been resolved.

[0355] If necessary, enter your feedback to help improve the system. For example, enter "Thank you for your quick response."

[0356] Specific examples

[0357] Let us take the example of a hotel reservation confirmation scenario.

[0358] 1. Booking confirmation inquiries:

[0359] User: Enter "Please tell me the status of reservation number 12345" into the smartphone app.

[0360] Terminal: Send this input to the server.

[0361] 2. Server processing:

[0362] The server analyzes the received inquiry and inputs it into a generation AI model for reservation confirmation categories.

[0363] The server searches the database for information related to reservation number 12345 and retrieves it.

[0364] The server uses an emotion engine to analyze the user's emotional state, for example, sensing tension or anxiety from the text.

[0365] The server will then adjust the answer based on the analysis results, for example, generating a response such as "Don't worry, reservation number 12345 is scheduled to check in on 2023-11-01."

[0366] The server sends the generated response to the terminal.

[0367] 3. Providing answers:

[0368] The terminal displays the response from the server to the user.

[0369] User: Enters feedback such as "Thank you for your quick response" and submits.

[0370] The terminal sends the feedback to the server.

[0371] In this way, by using a generative AI model and an emotion engine, the system of the present invention can provide appropriate and consistent answers that correspond to the user's emotions, enabling a high level of automation in customer support operations. This improves user satisfaction and reduces the workload on personnel. Furthermore, the system can constantly evolve based on feedback, improving its problem-solving capabilities.

[0372] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0373] Step 1: Server - Initialize the system and load the AI ​​model

[0374] The server loads the generative AI model at startup, specifically by loading and initializing the generative AI model's weight and configuration files, as well as related resources.

[0375] Input: Server initialization command

[0376] Data processing: Loading and initializing the generative AI model and reading relevant files into memory.

[0377] Output: Initialized generative AI model

[0378] Action: The server logs that the generative AI model has finished loading.

[0379] Step 2: Server - Check database connection

[0380] The server connects to the database (e.g., a common database system) and verifies that the necessary tables and indexes are set up correctly.

[0381] Input: Database connection information

[0382] Data Processing: Checking the database connection and settings

[0383] Output: Successful connection and confirmation result

[0384] Action: The server verifies that the database connection is successful and logs that there are no errors.

[0385] Step 3: Terminal - Initial Display of User Interface

[0386] The device displays a customer support interface to the user, which includes a contact form, a chat box, and a voice input option.

[0387] Input: Initial Data Request

[0388] Data processing: Obtaining initial data from the server

[0389] Output: Initial data to be displayed (e.g. store hours, campaign information)

[0390] Operation: The terminal requests initial data from the server and displays the retrieved data in the interface.

[0391] Step 4: User - Enter your inquiry

[0392] The user inputs a question or inquiry through the terminal interface. For example, they might input, "Please tell me the status of reservation number 12345."

[0393] Input: User's inquiry

[0394] Data processing: Validation of input inquiry details

[0395] Output: Validated query content

[0396] How it works: The user enters a query, which is validated by the terminal.

[0397] Step 5: Device - Submit your inquiry

[0398] The terminal transmits the inquiry content that passes validation to the server.

[0399] Input: Validated inquiry content

[0400] Data processing: Sending inquiry details to the server

[0401] Output: The query sent to the server

[0402] Operation: The terminal sends the user's query to the server in the appropriate format.

[0403] Step 6: Server - Parse and categorize the query

[0404] The server analyzes the received inquiry and categorizes it into an appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0405] Input: Received inquiry content

[0406] Data processing: Analysis and categorization using natural language processing

[0407] Output: Inquiry content with identified category

[0408] How it works: The server analyzes the query and passes the classification results to the generative AI model.

[0409] Step 7: Server - Retrieving Information from the Database

[0410] The server retrieves any additional information needed from the database based on the query, for example, searching for and retrieving reservation information related to the reservation number.

[0411] Input: Category-specific inquiry content

[0412] Data processing: Information retrieval and retrieval from databases

[0413] Output: Relevant information obtained

[0414] What happens: The server performs a database query to retrieve the required information.

[0415] Step 8: Server - Answer Generation

[0416] The server uses a generative AI model to generate the optimal answer based on the additional information obtained.

[0417] Input: Acquired relevant information and a generative AI model

[0418] Data processing: Answer generation using generative AI models

[0419] Output: The generated answer

[0420] How it works: The server uses a generative AI model to generate answers to user queries.

[0421] Step 9: Server - Sentiment Analysis and Response Adjustment

[0422] The server uses an emotion engine to analyze the user's emotions and adjust the tone and content of the response accordingly.

[0423] Input: User's query and generated answer

[0424] Data processing: sentiment analysis and response adjustment

[0425] Output: Adjusted answer

[0426] How it works: The server adjusts the response based on the analysis results of the emotion engine.

[0427] Step 10: Server - Sending the response to the device

[0428] The server sends the generated response to the terminal.

[0429] Input: Adjusted Answer

[0430] Data processing: sending adjusted responses

[0431] Output: Answer sent to terminal

[0432] Operation: The server sends the adjusted response to the terminal and records the transmission log.

[0433] Step 11: Terminal - View answers and receive feedback

[0434] The terminal displays the response from the server to the user and provides a feedback input interface.

[0435] Input: Response from the server

[0436] Data processing: Displaying answers and accepting feedback

[0437] Output: User feedback

[0438] Action: The device displays the answer for the user to review and accepts feedback.

[0439] Step 12: User - Review answers and provide feedback

[0440] Users review the answers provided and provide feedback to help improve the system.

[0441] Input: Provided Answer

[0442] Data processing: Feedback input

[0443] Output:Completed feedback

[0444] Action: The user reviews the answer, enters feedback, and sends it to the device.

[0445] Step 13: Device - Send Feedback

[0446] The terminal validates the feedback entered by the user and sends it to the server.

[0447] Input: User-entered feedback

[0448] Data Processing: Feedback validation and submission

[0449] Output: Feedback sent to the server

[0450] Action: The device validates the feedback and sends it to the server.

[0451] Step 14: Server - Receiving and storing feedback

[0452] The server receives the feedback sent from the terminal and stores it in a database.

[0453] Input: Feedback sent from the device

[0454] Data processing: receiving feedback and storing it in a database

[0455] Output: Saved feedback

[0456] How it works: The server stores the received feedback in a database and uses it to update the generative AI model in the future.

[0457] This enables the entire system to function, providing quick and appropriate answers to user inquiries, improving user satisfaction and operational efficiency.

[0458] (Application example 2)

[0459] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0460] Inquiries and troubles at logistics centers require quick and appropriate responses, but because they rely on human labor, responses can be delayed and lack consistency. Furthermore, it can be difficult for staff to respond empathetically based on their emotions, which can reduce user satisfaction. The present invention aims to solve these problems and provide a system that automates efficient and empathetic responses.

[0461] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating a response to an inquiry using a generative AI model, emotion analysis means for analyzing the user's emotions and adjusting the content of the response based on the analysis results, and means for referencing a database and acquiring additional information based on the content of the inquiry. This enables a quick and consistent response, and can improve user satisfaction through empathetic responses.

[0462] A "generative AI model" is an artificial intelligence that uses machine learning algorithms to automatically generate appropriate answers to any inquiry.

[0463] "Server means" refers to a server system that provides the execution environment for the generative AI model and the computational resources for processing the inquiry content and generating and delivering the answer.

[0464] "Terminal means" refers to a device through which a user inputs an inquiry and receives and displays a response from the server means, and includes a smartphone, tablet, or computer.

[0465] "Means for updating generative AI models based on feedback" refers to the process of collecting user ratings and opinions and using them as training data for generative AI models to continuously improve their performance.

[0466] The "emotion analysis means" is a function for analyzing the emotions of a user from text or voice and adjusting the content of the response generated according to that emotional state.

[0467] "Categorization" is the process of analyzing the inquiry and classifying it into a specific category (e.g., inventory check, delivery status, problem report, etc.).

[0468] "Means for referencing a database and obtaining additional information" refers to the process of searching a database based on the inquiry content and obtaining the necessary additional information (e.g., stock status, delivery details, etc.).

[0469] This section describes in detail an embodiment of the present invention. This system is designed to automate inquiries and troubleshooting at logistics centers and provide efficient and empathetic responses. Specifically, this system uses a robot equipped with a generative AI model and emotion analysis means to handle inquiries.

[0470] Hardware:

[0471] Robot Platform:

[0472] A robot platform is a piece of hardware that physically moves around a logistics center and receives and responds to inquiries from staff. Representative examples include Pepper and NAO.

[0473] Camera and Microphone:

[0474] The robot is equipped with a camera and microphone, which are used to analyze the user's emotions from their facial expressions and voice.

[0475] display:

[0476] The robot is equipped with a display that is used to visually display the answers generated by the generative AI model.

[0477] software:

[0478] Generative AI models:

[0479] A generative AI model is software that generates appropriate answers to inquiries. A typical example is a machine learning algorithm such as GPT-4 (registered trademark).

[0480] Emotion analysis means:

[0481] The emotion analysis means is software that analyzes the user's emotions and adjusts the content of the response based on that state. Examples of such software include Emotion API and IBM Watson (registered trademark) Tone Analyzer.

[0482] Database:

[0483] A database is a data storage for storing additional information required for inquiries, such as inventory information at a logistics center, delivery status, etc. Typical examples include MySQL and PostgreSQL.

[0484] Robot control software:

[0485] Robot control software is a platform for controlling robots, receiving queries from users, and processing data. A typical example is ROS (Robot Operating System).

[0486] Data processing and calculation:

[0487] Server Action:

[0488] The server generates answers using a generative AI model, analyzes emotions, and performs database lookups. Details are explained below.

[0489] 1. Startup and initialization:

[0490] When the server starts up, it loads the generative AI model and sentiment analysis method, connects to the database, checks the model weight file, configuration file, necessary indexes and tables, and loads login information, reservation data, etc.

[0491] 2. Inquiry reception and analysis:

[0492] When a user (staff member or customer) sends an inquiry to the robot, the content is transferred to the server, which analyzes the content of the inquiry and classifies it into categories (e.g., inventory check, delivery status, trouble report).

[0493] 3. Emotion analysis:

[0494] The server uses emotion analysis means to analyze the user's emotions, for example, to identify the user's anxiety or anger from the text and voice data.

[0495] 4. Answer generation and additional information acquisition:

[0496] The server uses a generative AI model to generate an appropriate response based on the analyzed inquiry content and emotion data, and retrieves additional information from the database (e.g., stock availability, delivery details, etc.) as needed to reflect the response.

[0497] 5. Provide answers:

[0498] The generated answers are provided to the user via the robot, either in voice or text format, and are also displayed on the robot's display.

[0499] Examples:

[0500] 1. Inventory Check Scenario

[0501] Staff: "Please let me know the stock status of this item."

[0502] Robot: Converts speech to text and feeds it into a generative AI model of inventory check categories.

[0503] Server: Obtains inventory information from the database and uses emotion analysis to analyze staff emotions as "worried."

[0504] Server: Generates a response and provides it to the robot: "Don't worry, we have plenty of this item in stock."

[0505] 2. Trouble Reporting Scenario

[0506] Staff: "The delivery is delayed, what's going on?"

[0507] Robot: Converts speech to text and feeds it into a generative AI model of trouble report categories.

[0508] Server: Obtains delivery status from the database and analyzes staff emotion as "anger" using emotion analysis means.

[0509] Server: "We apologize for the delay. We are currently investigating the cause of the delivery delay and will let you know the results shortly." This is the generated response and provided by the robot.

[0510] Prompt Sentence Examples

[0511] User: "What is the stock status of this item?"

[0512] Sentiment Analysis: "Worried"

[0513] Generative AI model output: "Don't worry, we have plenty of this item in stock."

[0514] In this way, by using generative AI models and emotion analysis means, embodiments of the present invention can automate inquiry responses at logistics centers and provide fast and empathetic service.

[0515] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0516] Step 1:

[0517] When the server starts up, it loads the generative AI model and sentiment analysis means and connects to the database. The server reads the model weight file and configuration file and checks whether the necessary tables and indexes are set up correctly. The inputs include the model weight file, configuration file, and database connection information, and the output is that the system will start operating normally by initializing the generative AI model and sentiment analysis means based on these.

[0518] Step 2:

[0519] The user inputs a query into the terminal (robot). The input includes the query content in voice or text format. The robot converts the voice into text and sends the query content as text data to the server. The query content is transferred to the server as output.

[0520] Step 3:

[0521] The server analyzes the received inquiry and classifies it into categories. Specifically, the text data is input into a string analysis algorithm, which classifies it into categories such as "inventory check," "delivery status," and "trouble report." The input is the text of the inquiry, and the output is the data classified into categories. This classification data is used in the next step.

[0522] Step 4:

[0523] The server uses the emotion analysis means to analyze the user's emotions. The input includes the text data of the inquiry received earlier. The emotion analysis means identifies the emotional state (e.g., worry, anger, joy, etc.) from the text and outputs the result. The output is the emotional data obtained by the emotion analysis means.

[0524] Step 5:

[0525] The server generates an appropriate answer using a generative AI model based on the analyzed query content and emotional data. The input is classified category data and emotional data, which are fed into the generative AI model to generate an answer in natural language format. The output is the generated answer.

[0526] Step 6:

[0527] If necessary, the server retrieves additional information related to the query from the database. The input is the query and category data, and based on this, it executes a database query to retrieve the target data (e.g., inventory information, delivery status, etc.). The output is the retrieved additional information.

[0528] Step 7:

[0529] The server integrates additional information into the generated answer and sends the final answer to the robot terminal. The input is the integrated answer data, and the output is the final answer sent to the robot terminal.

[0530] Step 8:

[0531] The robot terminal provides the answer received from the server to the user. Specifically, it displays or plays back the answer in text or audio format. The input is the answer data received from the server, and the output is the answer provided to the user.

[0532] Step 9:

[0533] The user checks the provided answers and enters feedback if necessary. The feedback is sent back to the server and used as update data for the generative AI model. The input is the feedback data from the user, and the output is the updated generative AI model.

[0534] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0535] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0536] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0537] [Second embodiment]

[0538] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0539] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0540] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0541] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0542] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0543] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0544] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0545] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0546] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0547] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0548] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0549] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0550] An embodiment of the present invention will be described in detail. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses.

[0551] server

[0552] 1. Loading and initializing the generative AI model

[0553] The server first loads the generative AI model on startup, which involves loading the model's weight files, configuration files, and associated resources into memory.

[0554] Next, it connects to the database and verifies that the necessary tables and indexes are set up correctly, including where to store user information, booking data, and feedback.

[0555] 2. Processing inquiries

[0556] The server receives the inquiry information sent from the terminal, including the category and detailed information of the inquiry.

[0557] First, the inquiry content is analyzed and classified into appropriate categories (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0558] The classified data is then passed to a generative AI model to generate an appropriate answer.

[0559] If necessary, retrieve additional information from the database, for example, look up data related to the reservation number to confirm the reservation information.

[0560] The generated answer is sent to the terminal and provided to the user.

[0561] Terminal

[0562] 1. Displaying the user interface

[0563] The terminal displays a customer support user interface to the user, which includes a contact form, a chat box, and a voice input option.

[0564] The terminal retrieves initial data from the server and displays it to the user. The initial data includes, for example, store opening hours and campaign information.

[0565] 2. Submitting and viewing inquiries

[0566] The inquiry entered by the user is sent to the server, at which point the device validates the input.

[0567] When a response is returned from the server, the terminal displays the response to the user.

[0568] Furthermore, a feedback input interface is displayed, and evaluations and opinions from users are sent to the server.

[0569] User

[0570] 1. Enter your inquiry

[0571] The user inputs a question or inquiry through the terminal interface. For example, they might input, "Please tell me the status of reservation number 12345."

[0572] If necessary, you can also input voice and upload images.

[0573] 2. Review answers and provide feedback

[0574] The user reviews the answers provided and evaluates whether the problem has been resolved.

[0575] Provide feedback as needed to help improve the system.

[0576] Specific examples

[0577] Let us take the example of a hotel reservation confirmation scenario.

[0578] 1. Reservation confirmation inquiries

[0579] User: Enter "Please tell me the reservation status for reservation number 12345" into the smartphone app.

[0580] Terminal: Send this input to the server.

[0581] 2. Server Processing

[0582] Server: Analyzes the received inquiry content and inputs it into the reservation confirmation category generation AI model.

[0583] Server: Search the database for information related to reservation number 12345.

[0584] Server: Generates a response saying, "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for your use." and sends it to the terminal.

[0585] 3. Providing answers

[0586] Terminal: Displays the answer from the server to the user.

[0587] User: Enters feedback such as "Thank you for your quick response" and sends it to the server.

[0588] As described above, the system of this invention uses a generative AI model to automate inquiry responses and provide high-quality customer support 24 hours a day. This improves user satisfaction and reduces the burden on personnel. Furthermore, the system constantly evolves based on feedback, improving its problem-solving capabilities.

[0589] The processing flow will be explained below.

[0590] Step 1:

[0591] Server: Responsible for loading and initializing the generative AI model. It loads the model weights, configuration files, and related resources from disk into memory. It also connects to the database and ensures that the necessary tables and indexes are set up correctly. This includes where to store user information, reservation data, and feedback.

[0592] Step 2:

[0593] Terminal: Displays the customer support user interface to the user. This includes an inquiry form, chat box, voice input options, etc. It also retrieves initial data (such as store opening hours and campaign information) from the server and displays it to the user.

[0594] Step 3:

[0595] User: Enters questions or inquiries through the device interface. For example, they can enter text such as "Please tell me the status of reservation number 12345." They can also input voice commands or upload images as needed.

[0596] Step 4:

[0597] Terminal: Validates the query entered by the user. Checks for errors or omissions in the input, and displays an error message to the user if there is a problem. If validation is successful, sends the query to the server.

[0598] Step 5:

[0599] Server: Receives inquiries sent from the terminal. The received data includes metadata such as the user ID, inquiry content, and timestamp.

[0600] Step 6:

[0601] Server: Analyzes the received inquiry and classifies it into an appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0602] Step 7:

[0603] Server: Passes the classified data to the generative AI model and generates the optimal answer to the inquiry. For example, the AI ​​model generates an answer such as "Reservation number 12345 is scheduled to check in on 2023-11-01."

[0604] Step 8:

[0605] Server: Retrieves additional information from the database if necessary. For example, to confirm the reservation, it searches and retrieves details related to reservation number 12345 from the database.

[0606] Step 9:

[0607] Server: Combines the generated answer with additional information to generate a final answer. For example, you can add a supplemental message such as "Thank you for using our service."

[0608] Step 10:

[0609] Server: Sends the final answer to the device.

[0610] Step 11:

[0611] Terminal: The answer obtained from the server is displayed to the user. To ensure the user can confirm, the screen displays "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for using our service."

[0612] Step 12:

[0613] User: Review the provided answer and provide feedback if necessary. For example, "Thank you for your quick response."

[0614] Step 13:

[0615] Terminal: Sends the feedback entered by the user to the server.

[0616] Step 14:

[0617] Server: Stores the received feedback in a database, which helps improve the generative AI model.

[0618] Step 15:

[0619] Server: Updates the generative AI model based on feedback to improve the accuracy of responses to future inquiries.

[0620] As described above, the system handles user inquiries through a series of processing steps and uses generative AI models to provide high-quality answers, achieving consistent responses and efficient processing, thereby improving the quality of customer support operations.

[0621] Example 1

[0622] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0623] Conventional customer support systems have difficulty responding to user inquiries effectively and quickly, often resulting in delayed responses, especially when complex inquiries or a large number of inquiries are received simultaneously. Furthermore, analyzing inquiries and generating responses requires a large amount of human resources, resulting in a high workload. Furthermore, there are insufficient means to effectively utilize feedback to improve the system. To solve these problems, a system that utilizes generative AI models to automate inquiry responses and provide efficient and prompt responses is needed.

[0624] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0625] In this invention, the server includes a means for reading a weight file and a setting file of the generative AI model, a means for connecting to a database, and a means for referencing a data storage device based on the query content and obtaining additional information. This allows the generative AI model to analyze the query content, classify it into categories, and quickly generate an appropriate answer, while also obtaining additional information from the database to provide a more detailed and accurate answer. Furthermore, the accuracy and effectiveness of the system can be improved by updating the generative AI model based on feedback obtained from users.

[0626] The "information processing device means" is a device that performs processing to generate an answer to an inquiry using a generative AI model.

[0627] The "display device means" is a device that sends inquiries from users and receives and displays responses from the generative AI model.

[0628] "Evaluation Information" means user-provided feedback or evaluations that are used to improve the performance of generative AI models.

[0629] A "weight file" is a data file that stores patterns and knowledge that a generative AI model has previously learned.

[0630] A "configuration file" is a file used to manage the behavior and parameters of a generative AI model.

[0631] A "data storage device" is a device for storing data necessary for system operation, such as user information, reservation data, and inquiry records.

[0632] "Inquiry content" refers to questions or requests that users input to the system.

[0633] A "generative AI model" is an algorithm that uses machine learning technology to automatically generate answers to user inquiries.

[0634] "Category" means a major group or area into which inquiries are classified.

[0635] "Additional information" is information obtained from a data repository to supplement the response to a query.

[0636] An embodiment of the present invention will be described in detail below. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses.

[0637] server

[0638] The server first loads the weight and configuration files of the generated AI model. This is typically done using a deep learning library such as TensorFlow or PyTorch. For example, the model is loaded using torch.load('model.pth') or tf.keras.models.load_model('model.h5').

[0639] Next, the server connects to a database, which is most likely a relational database management system like MySQL or PostgreSQL. This is done using the sqlalchemy library, for example with code like engine = create_engine('mysql: / / user:password@host / dbname').

[0640] The server receives the query information sent from the device. This is often done using an HTTP request. For example, the Flask framework is used to obtain data from request.json.

[0641] The server then analyzes the query using natural language processing techniques and categorizes it, using libraries such as nltk and spaCy to tokenize the text and predict the category using a classification model.

[0642] The parsed data is passed to a generative AI model to generate an appropriate answer. For example, a prompt is input to the generative AI model, such as gpt3_model.predict(prompt). If necessary, additional information is retrieved from the database. The SQL query uses syntax such as SELECT FROM reservations WHERE reservation_id = '12345'.

[0643] The generated answer is returned to the terminal as an HTTP response. jsonify(response) creates a JSON format answer and returns it as return response.

[0644] Terminal

[0645] The terminal displays a customer support user interface to the user, which includes a contact form, a chat box, and a voice input option. HTML and React are used as front-end technologies.

[0646] As initial data, display business hours and campaign information obtained from the server. Use an AJAX call to obtain and display the data. For example, this is often implemented as $.get(' / api / info', function(data) { $('info').html(data);});

[0647] When sending the query entered by the user to the server, the device performs validation to ensure that the input is correct. This is done with the following code: function sendQuery() { var query = $('query').val(); $.post(' / api / query', { query: query}, function(response) { displayResponse(response);});}

[0648] The response returned from the server is displayed to the user. The received data is embedded in HTML. Implement it as follows: function displayResponse(response) { $('response').html(response);}. In addition, a feedback input interface is displayed, and user ratings and opinions are sent to the server. The feedback form is: <textarea id="feedback">< / textarea> <button onclick="sendFeedback()"> Submit Feedback< / button> Arrange it as follows.

[0649] User

[0650] The user enters questions or inquiries through the terminal interface. For example, they can enter, "Please tell me the reservation status for reservation number 12345." The user types directly into the chat box or form using the keyboard.

[0651] It also allows voice input and image uploading as needed. The voice input function is implemented using the Web Speech API. <input type="file" id="imageUpload"> or <button onclick="startVoiceRecognition()"> Speak< / button> Set it as follows.

[0652] The user reviews the provided answer and evaluates whether the problem has been resolved, reads the answer to check accuracy and satisfaction, enters their thoughts and opinions in the feedback form, and presses the submit button.

[0653] Specific examples

[0654] Hotel booking confirmation scenario

[0655] 1. The user uses the smartphone app to enter "Please tell me the reservation status for reservation number 12345" into the terminal.

[0656] 2. The device sends this input to the server.

[0657] 3. The server analyzes the received inquiry and inputs it into the generative AI model as a "reservation confirmation" category.

[0658] 4. The server searches the database for information related to reservation number 12345, generates a response such as "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for your use," and sends it to the terminal.

[0659] 5. The terminal displays the response from the server to the user.

[0660] 6. The user enters feedback such as "The response was quick and helpful" and sends it to the server.

[0661] In this way, the system of this invention uses generative AI models to automate inquiry responses and provide high-quality customer support 24 hours a day. This improves user satisfaction and significantly reduces the human workload required for support. Furthermore, the system constantly evolves based on feedback, enabling it to provide optimal solutions.

[0662] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0663] Step 1:

[0664] Loading and initializing the generative AI model

[0665] The server loads the weight and configuration files of the generative AI model from disk into memory. This process is performed using library functions from TensorFlow or PyTorch, for example. Specifically, commands such as torch.load('model.pth') or tf.keras.models.load_model('model.h5') are executed. The input is the weight and configuration files, and the output is the generative AI model deployed in memory.

[0666] Step 2:

[0667] Connecting to a Database

[0668] The server connects to a relational database such as MySQL or PostgreSQL. This connection is made using the sqlalchemy library. For example, run a command like engine = create_engine('mysql: / / user:password@host / dbname'). The input is the database connection information, and the output is the database instance to which the connection is established.

[0669] Step 3:

[0670] Receiving inquiry information

[0671] The server receives the query information sent from the terminal via an HTTP request. When using Flask as a framework, data is obtained from request.json, for example. The input is the query content sent as an HTTP request, and the output is the query information stored in variables on the server.

[0672] Step 4:

[0673] Analysis of inquiry content and categorization

[0674] The server analyzes the received query using natural language processing (NLP) techniques. This process involves tokenization using the nltk and spaCy libraries and applying classification models. The input is the query text data, and the output is classified category information.

[0675] Step 5:

[0676] Input to generative AI model and answer generation

[0677] The server inputs the parsed data as a prompt sentence into the generative AI model to generate an appropriate answer. For example, a command such as gpt3_model.predict(prompt) is executed. The input is the generated prompt sentence, and the output is the generated answer text.

[0678] Step 6:

[0679] Retrieving additional information from the database

[0680] If necessary, the server retrieves additional information from the database by executing an SQL query (e.g. SELECT FROM reservations WHERE reservation_id = '12345'). The input is the reservation number as an SQL query, and the output is the retrieved reservation record.

[0681] Step 7:

[0682] Submit your answer

[0683] The server sends the generated answer to the terminal as an HTTP response. Specifically, it creates a JSON format answer with jsonify(response) and returns it as return response. The input is the generated answer text, and the output is the HTTP response sent to the terminal.

[0684] Step 8:

[0685] User Interface Display

[0686] The terminal displays a customer support user interface to the user. For example, it generates an inquiry form or chat box using HTML and React. The input is an HTML template or stylesheet as initial data, and the output is the interface displayed in the user's browser.

[0687] Step 9:

[0688] Submit an inquiry

[0689] The user inputs and sends the inquiry through the terminal interface. For example, they might input "Please tell me the reservation status for reservation number 12345" and press the send button. The input is the text data entered by the user, and the output is an HTTP request sent to the server.

[0690] Step 10:

[0691] Check your answers

[0692] The user checks the answer sent from the server on the terminal interface. The terminal receives the response data from the server and displays it in the chat box. The input is the answer text from the server, and the output is the answer displayed on the user's screen.

[0693] (Application example 1)

[0694] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0695] There is a need for a system that can quickly and reliably respond to various inquiries and problems encountered by users of autonomous vehicles. There is also a lack of effective means to provide rapid response in emergencies and safety alert notifications. This is necessary to increase the safety and peace of mind of users.

[0696] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0697] In this invention, the server includes means for generating responses to inquiries using a generative AI model, terminal means for sending inquiries and receiving responses, means for updating the generative AI model based on feedback, means for responding to inquiries and problems encountered by users of autonomous vehicles, and means for generating emergency response measures and safety alert notifications, thereby enabling users of autonomous vehicles to receive support quickly and reliably and enabling effective emergency and safety responses.

[0698] A "generative AI model" is an artificial intelligence model that automatically generates appropriate answers based on given data and inquiry content.

[0699] "Server means" means a computing device for processing inquiries, generating and managing responses using a generative AI model.

[0700] "Terminal means" refers to a device through which a user inputs an inquiry and receives a response from the server means, and examples of such a device include a smartphone or tablet.

[0701] "Feedback" refers to the act and content of a user's evaluation or opinion regarding a system's response or answer.

[0702] An "autonomous vehicle" is a vehicle that operates autonomously with minimal driver intervention.

[0703] "Emergency response measures" refer to the course of action and procedures for responding quickly and appropriately in the event of an emergency such as an accident or breakdown.

[0704] "Safety alert notification" is a notification function that promptly notifies users of safety precautions and warnings.

[0705] "Inquiry analysis" is the process of understanding and classifying the inquiries provided by users and assigning them to appropriate categories.

[0706] "Category" refers to a group or classification item for systematically classifying inquiry content.

[0707] A "database" is a collection of information that systematically stores and manages various inquiries, their responses, user information, and so on.

[0708] "Acquisition of additional information" is the process of searching, extracting, and providing the necessary information from a database based on the content of the inquiry.

[0709] An embodiment of the present invention will now be described in detail. This system is a customer support and security support system that responds to inquiries and problems encountered by users of autonomous vehicles. The system is mainly composed of three components: a server, a terminal, and a user.

[0710] server

[0711] The server is the main device that generates answers to queries using the generative AI model. First, the server loads the generative AI model at startup and reads the configuration file, weight file, etc. into memory. Next, it connects to the database used by the system and checks whether the necessary tables and indexes, such as user information, reservation data, and feedback information, are set up correctly.

[0712] When the server receives an inquiry from a user, it analyzes the inquiry and classifies it into an appropriate category (e.g., reservation confirmation, emergency response measures, safety confirmation, etc.). It then uses a generative AI model to generate an optimal answer to the inquiry, retrieving additional information from a database if necessary. The generated answer is sent to the device and provided to the user.

[0713] Terminal

[0714] The terminal is a device such as a smartphone or tablet that allows users to enter inquiries and receive responses from the server. When a user enters and sends an inquiry into the terminal, the terminal sends the content to the server. The terminal receives and displays the response from the server.

[0715] Additionally, the device receives feedback from users and transmits it to the server, which uses it as data to improve the performance of the generative AI model.

[0716] User

[0717] When using an autonomous vehicle, a user inputs their inquiry using a terminal. For example, they can input, "Please tell me the reservation status for reservation number 67890." The user checks the answer provided by the server and provides feedback as needed.

[0718] Specific examples

[0719] For example, when confirming a reservation for an autonomous vehicle, a user opens a smartphone app and enters, "Please tell me the reservation status for reservation number 67890." The device sends this inquiry to a server, which processes the inquiry using a generative AI model in the reservation confirmation category. The server searches the database for information related to reservation number 67890, generates a response saying, "Reservation number 67890 is scheduled to check in on 2023-12-01," and sends it to the device. The device displays this response to the user, who then enters feedback such as, "The response was quick and helpful," and submits it.

[0720] In addition, if an emergency inquiry such as "My car won't start! What should I do?" is entered, the server will respond using a generative AI model in the emergency response category, generating a response such as "Emergency services have been notified. Please remain calm," and sending it to the device.

[0721] This system will enable users of autonomous vehicles to receive prompt and reliable support, and will also enable effective emergency and safety responses. An example of a prompt sentence is, "Please tell me the status of reservation number 12345."

[0722] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0723] Step 1:

[0724] The server launches the generative AI model and loads the model's weight and configuration files into memory. This process occurs during the system initialization phase and lays the groundwork for the generative AI model to function correctly. It uses the weight and configuration files as input and obtains the loaded model as output.

[0725] Step 2:

[0726] The server connects to the database and checks the consistency of the various tables and indexes that store user information, reservation data, feedback information, etc. This process maintains data consistency and ensures smooth query processing later. It uses database connection information as input and obtains the connection success or failure status as output.

[0727] Step 3:

[0728] The user uses a terminal to input a query. For example, "Please tell me the reservation status for reservation number 12345." This input data is sent to the server via the terminal. The user's query content is used as input, and the query data is sent to the server as output.

[0729] Step 4:

[0730] The server analyzes the received inquiry and classifies it into the appropriate category. This process is performed by a generative AI model, which categorizes the inquiry into categories such as "reservation confirmation," "emergency measures," and "security check." The server uses the inquiry as input and obtains the categorization result as output.

[0731] Step 5:

[0732] The server retrieves additional information from the database as needed based on the category. For example, in the case of a reservation confirmation, the server searches and retrieves the relevant reservation information from the database. It uses the category classification results and the inquiry content as input, and obtains the search results (additional information) as output.

[0733] Step 6:

[0734] The server uses a generative AI model to generate the optimal answer based on the acquired additional information and the query. This generation process results in the most appropriate and useful answer for the user. The additional information and the query are used as input, and the generated answer is obtained as output.

[0735] Step 7:

[0736] The terminal receives the answer sent from the server and displays it to the user, who then confirms this information. It uses the answer data from the server as input and gets the answer displayed to the user as output.

[0737] Step 8:

[0738] The user inputs feedback for the provided answer and sends it to the server via the terminal. For example, the user inputs feedback such as "The answer was quick and helpful." The user's feedback is used as input, and the feedback sent to the server is obtained as output.

[0739] Step 9:

[0740] The server stores the received feedback in a database and uses it to improve the performance of the generative AI model. This process allows the system to continuously improve. It uses user feedback as input and obtains an updated database as output.

[0741] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0742] An embodiment of the present invention will be described in detail. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses. Furthermore, it is equipped with an emotion engine that recognizes the user's emotions, enabling more appropriate and empathetic responses.

[0743] server

[0744] 1. Loading and initializing the generative AI model

[0745] When the server starts up, it loads the generative AI model, which includes loading the model's weight files, configuration files, and associated resources.

[0746] Connect to the database and check that the necessary tables and indexes are set up correctly. Check where user information, reservation data, and feedback are stored.

[0747] 2. Processing inquiries

[0748] The server receives the inquiry information sent from the terminal, including metadata such as the inquiry content, category, user ID, and timestamp.

[0749] Analyze the inquiry and categorize it into the appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0750] The classified data is passed to a generative AI model to generate the best answer, retrieving additional information from the database if necessary.

[0751] The generated answer is sent to the terminal and provided to the user.

[0752] 3. The Emotional Engine

[0753] The server uses an emotion engine to analyze the user's emotions, identifying the user's emotional state (e.g., joy, sadness, anger, tension, etc.) from voice and text.

[0754] The emotion engine's analysis results are reflected in the generative AI model, which then adjusts the tone and content of the response. For example, if the user is feeling angry, the response will be generated in a more polite and calm tone.

[0755] 4. Processing Feedback

[0756] The server receives feedback sent from the device and stores it in a database, which is used to update the generative AI model and improve the accuracy of the system.

[0757] Terminal

[0758] 1. Displaying the user interface

[0759] The device displays a customer support interface, including a contact form, chat box, and voice input options.

[0760] Retrieve and display initial data (e.g., store opening hours, campaign information, etc.) from the server.

[0761] 2. Submitting and viewing inquiries

[0762] Validate the query entered by the user and send it to the server. Only queries that pass validation are sent.

[0763] Receives the response from the server and displays it to the user, displaying all relevant information on the screen for the user to review.

[0764] A feedback input interface is provided to receive evaluations and opinions from users and transmit them to the server.

[0765] User

[0766] 1. Enter your inquiry

[0767] The user enters a question or inquiry through the terminal interface, for example, "Please tell me the status of reservation number 12345."

[0768] If necessary, you can also input voice and upload images.

[0769] 2. Review answers and provide feedback

[0770] The user reviews the answers provided and evaluates whether the problem has been resolved.

[0771] If necessary, enter your feedback to help improve the system. For example, enter "Thank you for your quick response."

[0772] Specific examples

[0773] Let us take the example of a hotel reservation confirmation scenario.

[0774] 1. Reservation confirmation inquiries

[0775] User: Enter "Please tell me the reservation status for reservation number 12345" into the smartphone app.

[0776] Terminal: Send this input to the server.

[0777] 2. Server Processing

[0778] Server: Analyzes the received inquiry content and inputs it into the reservation confirmation category generation AI model.

[0779] Server: Searches and retrieves information related to reservation number 12345 from the database.

[0780] Server: Analyzes the user's emotional state using an emotion engine, for example, sensing tension or anxiety from the text.

[0781] Server: Adjusts the answer based on the analysis results. For example, it generates an answer such as "Don't worry. Reservation number 12345 is scheduled to check in on 2023-11-01."

[0782] Server: Sends the generated answer to the device.

[0783] 3. Providing answers

[0784] Terminal: Displays the answer from the server to the user.

[0785] User: Enters feedback such as "Thank you for your quick response" and submits.

[0786] In this way, the system of the present invention is equipped with an emotion engine that recognizes user emotions and uses a generative AI model to automate inquiry responses, making it possible to provide more appropriate and empathetic customer support. This improves user satisfaction and reduces the burden on personnel. The system constantly evolves based on feedback, improving its problem-solving capabilities.

[0787] The processing flow will be explained below.

[0788] Step 1:

[0789] Server: Responsible for loading and initializing the generative AI model. This includes loading the model weights and configuration files and associated resources from disk into memory. It also connects to the database and checks tables and indexes.

[0790] Step 2:

[0791] Terminal: Displays the user interface to the user. This includes a contact form, chat box, and voice input options. It also retrieves and displays initial data from the server (e.g., store hours, campaign information, etc.).

[0792] Step 3:

[0793] User: Enters a question or inquiry through the device interface. For example, enters text such as "Please tell me the reservation status for reservation number 12345." If necessary, voice input and image upload are also performed.

[0794] Step 4:

[0795] Terminal: Validates the entered query. Performs error checking and displays an error message to the user if there is a problem. If validation is successful, sends the query to the server.

[0796] Step 5:

[0797] Server: Receives inquiries sent from the terminal. The received data includes metadata such as the user ID, inquiry content, and timestamp.

[0798] Step 6:

[0799] Server: Analyzes the received inquiry and classifies it into the appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0800] Step 7:

[0801] Server: Analyzes the user's emotions using an emotion engine. Identifies the emotional state (e.g., joy, sadness, anger, tension, etc.) from speech and text.

[0802] Step 8:

[0803] Server: The classified data and the results of the emotion engine are input into the generative AI model to generate the optimal answer, such as "Reservation number 12345 is scheduled to check in on 2023-11-01."

[0804] Step 9:

[0805] Server: If necessary, retrieve additional information from the database. For example, look up and retrieve details related to reservation number 12345 to confirm the reservation information.

[0806] Step 10:

[0807] Server: Combines the generated answer with additional information to generate the final answer. Adjusts the tone of the answer based on the results of the emotion engine. For example, it generates something like, "Don't worry, reservation number 12345 is scheduled to check in on 2023-11-01."

[0808] Step 11:

[0809] Server: Sends the final answer to the device.

[0810] Step 12:

[0811] Terminal: The answer obtained from the server is displayed to the user. To ensure the user can confirm, the screen displays "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for using our service."

[0812] Step 13:

[0813] User: Review the provided answer and provide feedback if necessary. For example, "Thank you for your quick response."

[0814] Step 14:

[0815] Terminal: Sends the feedback entered by the user to the server.

[0816] Step 15:

[0817] Server: Stores the received feedback in a database, which is used to improve the generative AI model.

[0818] Step 16:

[0819] Server: Updates the generative AI model and emotion engine based on feedback to improve response accuracy for future inquiries.

[0820] As described above, the system handles user inquiries through a series of processing steps and provides high-quality answers by utilizing generative AI models and an emotion engine. By enabling emotion-sensitive responses, user satisfaction can be further improved.

[0821] Example 2

[0822] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0823] Conventional customer support systems often lack consistency in their responses to inquiries. They also struggle to properly understand users' emotions and respond empathetically, potentially resulting in lower user satisfaction. In particular, conventional systems face challenges in providing adequate support in situations where rapid responses to changes in emotions are required.

[0824] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for generating an answer to a query using a generative AI model, terminal means for sending an inquiry to the server means and receiving an answer, means for updating the generative AI model based on feedback acquired from the terminal means, and means for analyzing the emotional state of the query using an emotion engine and adjusting the answer to be generated. This makes it possible to provide an appropriate and consistent answer according to the user's emotions and improve user satisfaction.

[0825] A "generative AI model" refers to an algorithm or data model that uses artificial intelligence technology to generate appropriate answers to natural language queries.

[0826] "Server means" refers to a computer system for receiving inquiry information, generating a response using a generative AI model, and performing emotion analysis using an emotion engine.

[0827] "Terminal means" refers to a device, such as a computer or smartphone, through which a user inputs inquiry details, transmits the information to a server, and receives a response from the server.

[0828] An "emotion engine" refers to software or algorithms that analyze a user's emotional state from text or voice and recognize specific emotions.

[0829] "Feedback" refers to users inputting their evaluations and opinions about the answers and services provided by the system, and this information is used to improve the system's performance.

[0830] "Database" refers to a set of data structures for storing information in response to queries and allowing server means to retrieve additional information as needed.

[0831] "Parsing" refers to the process of using natural language processing techniques to structure and categorize query content.

[0832] "Answer generation" refers to the process of using a generative AI model to generate an appropriate answer to a user's inquiry.

[0833] This invention is an advanced conversational robot system that uses generative AI models to automate customer support tasks and provide fast, consistent responses. It also features an emotion engine that recognizes the user's emotions and provides appropriate, empathetic responses. The following describes each component of the system and its operation in detail.

[0834] server

[0835] The server is the center of the system and plays the following roles:

[0836] 1. Load and initialize the generative AI model:

[0837] On startup, the server loads a generative AI model (e.g., a generic generative AI model), which includes loading the model's weight files, configuration files, and associated resources.

[0838] The server connects to a database system (e.g. a general purpose database) and checks whether the necessary tables and indexes are set up correctly, e.g., where to store user information, reservation data, feedback, etc.

[0839] 2. Handling inquiries:

[0840] The server receives the inquiry information sent from the terminal. This received data includes the inquiry content, category, user ID, timestamp, etc.

[0841] The server analyzes the query content using a natural language processing library (for example, a general natural language processing library) and classifies it into an appropriate category.

[0842] The classified data is passed to a generative AI model to generate the best answer, retrieving additional information from the database if necessary.

[0843] The generated answer is sent to the terminal and provided to the user.

[0844] 3. How the Emotional Engine Works:

[0845] The server analyzes the user's emotions using an emotion engine (e.g., a general emotion analysis tool) and identifies the user's emotional state from the voice and text.

[0846] The emotion engine's analysis results are reflected in the generative AI model, which then adjusts the tone and content of the response. For example, if the user is feeling angry, the response will be generated in a more polite and calm tone.

[0847] 4. Feedback Processing:

[0848] The server receives feedback sent from the device and stores it in a database, which is used to update the generative AI model and improve the accuracy of the system.

[0849] Terminal

[0850] The terminal provides an interface for the user to access customer support.

[0851] 1. Display the user interface:

[0852] The device displays a customer support interface, including a contact form, a chat box, and a voice input option.

[0853] As initial data, store opening hours and campaign information are obtained from the server and displayed.

[0854] 2. Submitting and viewing inquiries:

[0855] Validate the query entered by the user and send it to the server. Only queries that pass validation are sent.

[0856] Receives the response from the server and displays it to the user, displaying all relevant information on the screen for the user to review.

[0857] A feedback input interface is provided to receive evaluations and opinions from users and transmit them to the server.

[0858] User

[0859] Users submit queries through the system and provide feedback on the answers provided.

[0860] 1. Enter your inquiry:

[0861] The user enters a question or inquiry through the terminal interface, for example, "Please tell me the status of reservation number 12345."

[0862] If necessary, you can also input voice and upload images.

[0863] 2. Review answers and provide feedback:

[0864] The user reviews the answers provided and evaluates whether the problem has been resolved.

[0865] If necessary, enter your feedback to help improve the system. For example, enter "Thank you for your quick response."

[0866] Specific examples

[0867] Let us take the example of a hotel reservation confirmation scenario.

[0868] 1. Booking confirmation inquiries:

[0869] User: Enter "Please tell me the status of reservation number 12345" into the smartphone app.

[0870] Terminal: Send this input to the server.

[0871] 2. Server processing:

[0872] The server analyzes the received inquiry and inputs it into a generation AI model for reservation confirmation categories.

[0873] The server searches the database for information related to reservation number 12345 and retrieves it.

[0874] The server uses an emotion engine to analyze the user's emotional state, for example, sensing tension or anxiety from the text.

[0875] The server will then adjust the answer based on the analysis results, for example, generating a response such as "Don't worry, reservation number 12345 is scheduled to check in on 2023-11-01."

[0876] The server sends the generated response to the terminal.

[0877] 3. Providing answers:

[0878] The terminal displays the response from the server to the user.

[0879] User: Enters feedback such as "Thank you for your quick response" and submits.

[0880] The terminal sends the feedback to the server.

[0881] In this way, by using a generative AI model and an emotion engine, the system of the present invention can provide appropriate and consistent answers that correspond to the user's emotions, enabling a high level of automation in customer support operations. This improves user satisfaction and reduces the workload on personnel. Furthermore, the system can constantly evolve based on feedback, improving its problem-solving capabilities.

[0882] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0883] Step 1: Server - Initialize the system and load the AI ​​model

[0884] The server loads the generative AI model at startup, specifically by loading and initializing the generative AI model's weight and configuration files, as well as related resources.

[0885] Input: Server initialization command

[0886] Data processing: Loading and initializing the generative AI model and reading relevant files into memory.

[0887] Output: Initialized generative AI model

[0888] Action: The server logs that the generative AI model has finished loading.

[0889] Step 2: Server - Check database connection

[0890] The server connects to the database (e.g., a common database system) and verifies that the necessary tables and indexes are set up correctly.

[0891] Input: Database connection information

[0892] Data Processing: Checking the database connection and settings

[0893] Output: Successful connection and confirmation result

[0894] Action: The server verifies that the database connection is successful and logs that there are no errors.

[0895] Step 3: Terminal - Initial Display of User Interface

[0896] The device displays a customer support interface to the user, which includes a contact form, a chat box, and a voice input option.

[0897] Input: Initial Data Request

[0898] Data processing: Obtaining initial data from the server

[0899] Output: Initial data to be displayed (e.g. store hours, campaign information)

[0900] Operation: The terminal requests initial data from the server and displays the retrieved data in the interface.

[0901] Step 4: User - Enter your inquiry

[0902] The user inputs a question or inquiry through the terminal interface. For example, they might input, "Please tell me the status of reservation number 12345."

[0903] Input: User's inquiry

[0904] Data processing: Validation of input inquiry details

[0905] Output: Validated query content

[0906] How it works: The user enters a query, which is validated by the terminal.

[0907] Step 5: Device - Submit your inquiry

[0908] The terminal transmits the inquiry content that passes validation to the server.

[0909] Input: Validated inquiry content

[0910] Data processing: Sending inquiry details to the server

[0911] Output: The query sent to the server

[0912] Operation: The terminal sends the user's query to the server in the appropriate format.

[0913] Step 6: Server - Parse and categorize the query

[0914] The server analyzes the received inquiry and categorizes it into an appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[0915] Input: Received inquiry content

[0916] Data processing: Analysis and categorization using natural language processing

[0917] Output: Inquiry content with identified category

[0918] How it works: The server analyzes the query and passes the classification results to the generative AI model.

[0919] Step 7: Server - Retrieving Information from the Database

[0920] The server retrieves any additional information needed from the database based on the query, for example, searching for and retrieving reservation information related to the reservation number.

[0921] Input: Category-specific inquiry content

[0922] Data processing: Information retrieval and retrieval from databases

[0923] Output: Relevant information obtained

[0924] What happens: The server performs a database query to retrieve the required information.

[0925] Step 8: Server - Answer Generation

[0926] The server uses a generative AI model to generate the optimal answer based on the additional information obtained.

[0927] Input: Acquired relevant information and a generative AI model

[0928] Data processing: Answer generation using generative AI models

[0929] Output: The generated answer

[0930] How it works: The server uses a generative AI model to generate answers to user queries.

[0931] Step 9: Server - Sentiment Analysis and Response Adjustment

[0932] The server uses an emotion engine to analyze the user's emotions and adjust the tone and content of the response accordingly.

[0933] Input: User's query and generated answer

[0934] Data processing: sentiment analysis and response adjustment

[0935] Output: Adjusted answer

[0936] How it works: The server adjusts the response based on the analysis results of the emotion engine.

[0937] Step 10: Server - Sending the response to the device

[0938] The server sends the generated response to the terminal.

[0939] Input: Adjusted Answer

[0940] Data processing: sending adjusted responses

[0941] Output: Answer sent to terminal

[0942] Operation: The server sends the adjusted response to the terminal and records the transmission log.

[0943] Step 11: Terminal - View answers and receive feedback

[0944] The terminal displays the response from the server to the user and provides a feedback input interface.

[0945] Input: Response from the server

[0946] Data processing: Displaying answers and accepting feedback

[0947] Output: User feedback

[0948] Action: The device displays the answer for the user to review and accepts feedback.

[0949] Step 12: User - Review answers and provide feedback

[0950] Users review the answers provided and provide feedback to help improve the system.

[0951] Input: Provided Answer

[0952] Data processing: Feedback input

[0953] Output:Completed feedback

[0954] Action: The user reviews the answer, enters feedback, and sends it to the device.

[0955] Step 13: Device - Send Feedback

[0956] The terminal validates the feedback entered by the user and sends it to the server.

[0957] Input: User-entered feedback

[0958] Data Processing: Feedback validation and submission

[0959] Output: Feedback sent to the server

[0960] Action: The device validates the feedback and sends it to the server.

[0961] Step 14: Server - Receiving and storing feedback

[0962] The server receives the feedback sent from the terminal and stores it in a database.

[0963] Input: Feedback sent from the device

[0964] Data processing: receiving feedback and storing it in a database

[0965] Output: Saved feedback

[0966] How it works: The server stores the received feedback in a database and uses it to update the generative AI model in the future.

[0967] This enables the entire system to function, providing quick and appropriate answers to user inquiries, improving user satisfaction and operational efficiency.

[0968] (Application example 2)

[0969] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0970] Inquiries and troubles at logistics centers require quick and appropriate responses, but because they rely on human labor, responses can be delayed and lack consistency. Furthermore, it can be difficult for staff to respond empathetically based on their emotions, which can reduce user satisfaction. The present invention aims to solve these problems and provide a system that automates efficient and empathetic responses.

[0971] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating a response to an inquiry using a generative AI model, emotion analysis means for analyzing the user's emotions and adjusting the content of the response based on the analysis results, and means for referencing a database and acquiring additional information based on the content of the inquiry. This enables a quick and consistent response, and can improve user satisfaction through empathetic responses.

[0972] A "generative AI model" is an artificial intelligence that uses machine learning algorithms to automatically generate appropriate answers to any inquiry.

[0973] "Server means" refers to a server system that provides the execution environment for the generative AI model and the computational resources for processing the inquiry content and generating and delivering the answer.

[0974] "Terminal means" refers to a device through which a user inputs an inquiry and receives and displays a response from the server means, and includes a smartphone, tablet, or computer.

[0975] "Means for updating generative AI models based on feedback" refers to the process of collecting user ratings and opinions and using them as training data for generative AI models to continuously improve their performance.

[0976] The "emotion analysis means" is a function for analyzing the emotions of a user from text or voice and adjusting the content of the response generated according to that emotional state.

[0977] "Categorization" is the process of analyzing the inquiry and classifying it into a specific category (e.g., inventory check, delivery status, problem report, etc.).

[0978] "Means for referencing a database and obtaining additional information" refers to the process of searching a database based on the inquiry content and obtaining the necessary additional information (e.g., stock status, delivery details, etc.).

[0979] This section describes in detail an embodiment of the present invention. This system is designed to automate inquiries and troubleshooting at logistics centers and provide efficient and empathetic responses. Specifically, this system uses a robot equipped with a generative AI model and emotion analysis means to handle inquiries.

[0980] Hardware:

[0981] Robot Platform:

[0982] A robot platform is a piece of hardware that physically moves around a logistics center and receives and responds to inquiries from staff. Representative examples include Pepper and NAO.

[0983] Camera and Microphone:

[0984] The robot is equipped with a camera and microphone, which are used to analyze the user's emotions from their facial expressions and voice.

[0985] display:

[0986] The robot is equipped with a display that is used to visually display the answers generated by the generative AI model.

[0987] software:

[0988] Generative AI models:

[0989] A generative AI model is software that generates appropriate answers to queries. A typical example is a machine learning algorithm such as GPT-4.

[0990] Emotion analysis means:

[0991] The emotion analysis tool is software that analyzes the user's emotions and adjusts the response content based on that state. Tools such as Emotion API and IBM Watson Tone Analyzer are used.

[0992] Database:

[0993] A database is a data storage for storing additional information required for inquiries, such as inventory information at a logistics center, delivery status, etc. Typical examples include MySQL and PostgreSQL.

[0994] Robot control software:

[0995] Robot control software is a platform for controlling robots, receiving queries from users, and processing data. A typical example is ROS (Robot Operating System).

[0996] Data processing and calculation:

[0997] Server Action:

[0998] The server generates answers using a generative AI model, analyzes emotions, and performs database lookups. Details are explained below.

[0999] 1. Startup and initialization:

[1000] When the server starts up, it loads the generative AI model and sentiment analysis method, connects to the database, checks the model weight file, configuration file, necessary indexes and tables, and loads login information, reservation data, etc.

[1001] 2. Inquiry reception and analysis:

[1002] When a user (staff member or customer) sends an inquiry to the robot, the content is transferred to the server, which analyzes the content of the inquiry and classifies it into categories (e.g., inventory check, delivery status, trouble report).

[1003] 3. Emotion analysis:

[1004] The server uses emotion analysis means to analyze the user's emotions, for example, to identify the user's anxiety or anger from the text and voice data.

[1005] 4. Answer generation and additional information acquisition:

[1006] The server uses a generative AI model to generate an appropriate response based on the analyzed inquiry content and emotion data, and retrieves additional information from the database (e.g., stock availability, delivery details, etc.) as needed to reflect the response.

[1007] 5. Provide answers:

[1008] The generated answers are provided to the user via the robot, either in voice or text format, and are also displayed on the robot's display.

[1009] Examples:

[1010] 1. Inventory Check Scenario

[1011] Staff: "Please let me know the stock status of this item."

[1012] Robot: Converts speech to text and feeds it into a generative AI model of inventory check categories.

[1013] Server: Obtains inventory information from the database and uses emotion analysis to analyze staff emotions as "worried."

[1014] Server: Generates a response and provides it to the robot: "Don't worry, we have plenty of this item in stock."

[1015] 2. Trouble Reporting Scenario

[1016] Staff: "The delivery is delayed, what's going on?"

[1017] Robot: Converts speech to text and feeds it into a generative AI model of trouble report categories.

[1018] Server: Obtains delivery status from the database and analyzes staff emotion as "anger" using emotion analysis means.

[1019] Server: "We apologize for the delay. We are currently investigating the cause of the delivery delay and will let you know the results shortly." This is the generated response and provided by the robot.

[1020] Prompt Sentence Examples

[1021] User: "What is the stock status of this item?"

[1022] Sentiment Analysis: "Worried"

[1023] Generative AI model output: "Don't worry, we have plenty of this item in stock."

[1024] In this way, by using generative AI models and emotion analysis means, embodiments of the present invention can automate inquiry responses at logistics centers and provide fast and empathetic service.

[1025] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1026] Step 1:

[1027] When the server starts up, it loads the generative AI model and sentiment analysis means and connects to the database. The server reads the model weight file and configuration file and checks whether the necessary tables and indexes are set up correctly. The inputs include the model weight file, configuration file, and database connection information, and the output is that the system will start operating normally by initializing the generative AI model and sentiment analysis means based on these.

[1028] Step 2:

[1029] The user inputs a query into the terminal (robot). The input includes the query content in voice or text format. The robot converts the voice into text and sends the query content as text data to the server. The query content is transferred to the server as output.

[1030] Step 3:

[1031] The server analyzes the received inquiry and classifies it into categories. Specifically, the text data is input into a string analysis algorithm, which classifies it into categories such as "inventory check," "delivery status," and "trouble report." The input is the text of the inquiry, and the output is the data classified into categories. This classification data is used in the next step.

[1032] Step 4:

[1033] The server uses the emotion analysis means to analyze the user's emotions. The input includes the text data of the inquiry received earlier. The emotion analysis means identifies the emotional state (e.g., worry, anger, joy, etc.) from the text and outputs the result. The output is the emotional data obtained by the emotion analysis means.

[1034] Step 5:

[1035] The server generates an appropriate answer using a generative AI model based on the analyzed query content and emotional data. The input is classified category data and emotional data, which are fed into the generative AI model to generate an answer in natural language format. The output is the generated answer.

[1036] Step 6:

[1037] If necessary, the server retrieves additional information related to the query from the database. The input is the query and category data, and based on this, it executes a database query to retrieve the target data (e.g., inventory information, delivery status, etc.). The output is the retrieved additional information.

[1038] Step 7:

[1039] The server integrates additional information into the generated answer and sends the final answer to the robot terminal. The input is the integrated answer data, and the output is the final answer sent to the robot terminal.

[1040] Step 8:

[1041] The robot terminal provides the answer received from the server to the user. Specifically, it displays or plays back the answer in text or audio format. The input is the answer data received from the server, and the output is the answer provided to the user.

[1042] Step 9:

[1043] The user checks the provided answers and enters feedback if necessary. The feedback is sent back to the server and used as update data for the generative AI model. The input is the feedback data from the user, and the output is the updated generative AI model.

[1044] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1045] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1046] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1047] [Third embodiment]

[1048] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1049] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1050] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1051] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1052] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1053] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1054] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1055] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1056] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1057] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1058] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1059] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1060] An embodiment of the present invention will be described in detail. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses.

[1061] server

[1062] 1. Loading and initializing the generative AI model

[1063] The server first loads the generative AI model on startup, which involves loading the model's weight files, configuration files, and associated resources into memory.

[1064] Next, it connects to the database and verifies that the necessary tables and indexes are set up correctly, including where to store user information, booking data, and feedback.

[1065] 2. Processing inquiries

[1066] The server receives the inquiry information sent from the terminal, including the category and detailed information of the inquiry.

[1067] First, the inquiry content is analyzed and classified into appropriate categories (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1068] The classified data is then passed to a generative AI model to generate an appropriate answer.

[1069] If necessary, retrieve additional information from the database, for example, look up data related to the reservation number to confirm the reservation information.

[1070] The generated answer is sent to the terminal and provided to the user.

[1071] Terminal

[1072] 1. Displaying the user interface

[1073] The terminal displays a customer support user interface to the user, which includes a contact form, a chat box, and a voice input option.

[1074] The terminal retrieves initial data from the server and displays it to the user. The initial data includes, for example, store opening hours and campaign information.

[1075] 2. Submitting and viewing inquiries

[1076] The inquiry entered by the user is sent to the server, at which point the device validates the input.

[1077] When a response is returned from the server, the terminal displays the response to the user.

[1078] Furthermore, a feedback input interface is displayed, and evaluations and opinions from users are sent to the server.

[1079] User

[1080] 1. Enter your inquiry

[1081] The user inputs a question or inquiry through the terminal interface. For example, they might input, "Please tell me the status of reservation number 12345."

[1082] If necessary, you can also input voice and upload images.

[1083] 2. Review answers and provide feedback

[1084] The user reviews the answers provided and evaluates whether the problem has been resolved.

[1085] Provide feedback as needed to help improve the system.

[1086] Specific examples

[1087] Let us take the example of a hotel reservation confirmation scenario.

[1088] 1. Reservation confirmation inquiries

[1089] User: Enter "Please tell me the reservation status for reservation number 12345" into the smartphone app.

[1090] Terminal: Send this input to the server.

[1091] 2. Server Processing

[1092] Server: Analyzes the received inquiry content and inputs it into the reservation confirmation category generation AI model.

[1093] Server: Search the database for information related to reservation number 12345.

[1094] Server: Generates a response saying, "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for your use." and sends it to the terminal.

[1095] 3. Providing answers

[1096] Terminal: Displays the answer from the server to the user.

[1097] User: Enters feedback such as "Thank you for your quick response" and sends it to the server.

[1098] As described above, the system of this invention uses a generative AI model to automate inquiry responses and provide high-quality customer support 24 hours a day. This improves user satisfaction and reduces the burden on personnel. Furthermore, the system constantly evolves based on feedback, improving its problem-solving capabilities.

[1099] The processing flow will be explained below.

[1100] Step 1:

[1101] Server: Responsible for loading and initializing the generative AI model. It loads the model weights, configuration files, and related resources from disk into memory. It also connects to the database and ensures that the necessary tables and indexes are set up correctly. This includes where to store user information, reservation data, and feedback.

[1102] Step 2:

[1103] Terminal: Displays the customer support user interface to the user. This includes an inquiry form, chat box, voice input options, etc. It also retrieves initial data (such as store opening hours and campaign information) from the server and displays it to the user.

[1104] Step 3:

[1105] User: Enters questions or inquiries through the device interface. For example, they can enter text such as "Please tell me the status of reservation number 12345." They can also input voice commands or upload images as needed.

[1106] Step 4:

[1107] Terminal: Validates the query entered by the user. Checks for errors or omissions in the input, and displays an error message to the user if there is a problem. If validation is successful, sends the query to the server.

[1108] Step 5:

[1109] Server: Receives inquiries sent from the terminal. The received data includes metadata such as the user ID, inquiry content, and timestamp.

[1110] Step 6:

[1111] Server: Analyzes the received inquiry and classifies it into an appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1112] Step 7:

[1113] Server: Passes the classified data to the generative AI model and generates the optimal answer to the inquiry. For example, the AI ​​model generates an answer such as "Reservation number 12345 is scheduled to check in on 2023-11-01."

[1114] Step 8:

[1115] Server: Retrieves additional information from the database if necessary. For example, to confirm the reservation, it searches and retrieves details related to reservation number 12345 from the database.

[1116] Step 9:

[1117] Server: Combines the generated answer with additional information to generate a final answer. For example, you can add a supplemental message such as "Thank you for using our service."

[1118] Step 10:

[1119] Server: Sends the final answer to the device.

[1120] Step 11:

[1121] Terminal: The answer obtained from the server is displayed to the user. To ensure the user can confirm, the screen displays "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for using our service."

[1122] Step 12:

[1123] User: Review the provided answer and provide feedback if necessary. For example, "Thank you for your quick response."

[1124] Step 13:

[1125] Terminal: Sends the feedback entered by the user to the server.

[1126] Step 14:

[1127] Server: Stores the received feedback in a database, which helps improve the generative AI model.

[1128] Step 15:

[1129] Server: Updates the generative AI model based on feedback to improve the accuracy of responses to future inquiries.

[1130] As described above, the system handles user inquiries through a series of processing steps and uses generative AI models to provide high-quality answers, achieving consistent responses and efficient processing, thereby improving the quality of customer support operations.

[1131] Example 1

[1132] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1133] Conventional customer support systems have difficulty responding to user inquiries effectively and quickly, often resulting in delayed responses, especially when complex inquiries or a large number of inquiries are received simultaneously. Furthermore, analyzing inquiries and generating responses requires a large amount of human resources, resulting in a high workload. Furthermore, there are insufficient means to effectively utilize feedback to improve the system. To solve these problems, a system that utilizes generative AI models to automate inquiry responses and provide efficient and prompt responses is needed.

[1134] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1135] In this invention, the server includes a means for reading a weight file and a setting file of the generative AI model, a means for connecting to a database, and a means for referencing a data storage device based on the query content and obtaining additional information. This allows the generative AI model to analyze the query content, classify it into categories, and quickly generate an appropriate answer, while also obtaining additional information from the database to provide a more detailed and accurate answer. Furthermore, the accuracy and effectiveness of the system can be improved by updating the generative AI model based on feedback obtained from users.

[1136] The "information processing device means" is a device that performs processing to generate an answer to an inquiry using a generative AI model.

[1137] The "display device means" is a device that sends inquiries from users and receives and displays responses from the generative AI model.

[1138] "Evaluation Information" means user-provided feedback or evaluations that are used to improve the performance of generative AI models.

[1139] A "weight file" is a data file that stores patterns and knowledge that a generative AI model has previously learned.

[1140] A "configuration file" is a file used to manage the behavior and parameters of a generative AI model.

[1141] A "data storage device" is a device for storing data necessary for system operation, such as user information, reservation data, and inquiry records.

[1142] "Inquiry content" refers to questions or requests that users input to the system.

[1143] A "generative AI model" is an algorithm that uses machine learning technology to automatically generate answers to user inquiries.

[1144] "Category" means a major group or area into which inquiries are classified.

[1145] "Additional information" is information obtained from a data repository to supplement the response to a query.

[1146] An embodiment of the present invention will be described in detail below. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses.

[1147] server

[1148] The server first loads the weight and configuration files of the generated AI model. This is typically done using a deep learning library such as TensorFlow or PyTorch. For example, the model is loaded using torch.load('model.pth') or tf.keras.models.load_model('model.h5').

[1149] Next, the server connects to a database, which is most likely a relational database management system like MySQL or PostgreSQL. This is done using the sqlalchemy library, for example with code like engine = create_engine('mysql: / / user:password@host / dbname').

[1150] The server receives the query information sent from the device. This is often done using an HTTP request. For example, the Flask framework is used to obtain data from request.json.

[1151] The server then analyzes the query using natural language processing techniques and categorizes it, using libraries such as nltk and spaCy to tokenize the text and predict the category using a classification model.

[1152] The parsed data is passed to a generative AI model to generate an appropriate answer. For example, a prompt is input to the generative AI model, such as gpt3_model.predict(prompt). If necessary, additional information is retrieved from the database. The SQL query uses syntax such as SELECT FROM reservations WHERE reservation_id = '12345'.

[1153] The generated answer is returned to the terminal as an HTTP response. jsonify(response) creates a JSON format answer and returns it as return response.

[1154] Terminal

[1155] The terminal displays a customer support user interface to the user, which includes a contact form, a chat box, and a voice input option. HTML and React are used as front-end technologies.

[1156] As initial data, display business hours and campaign information obtained from the server. Use an AJAX call to obtain and display the data. For example, this is often implemented as $.get(' / api / info', function(data) { $('info').html(data);});

[1157] When sending the query entered by the user to the server, the device performs validation to ensure that the input is correct. This is done with the following code: function sendQuery() { var query = $('query').val(); $.post(' / api / query', { query: query}, function(response) { displayResponse(response);});}

[1158] The response returned from the server is displayed to the user. The received data is embedded in HTML. Implement it as follows: function displayResponse(response) { $('response').html(response);}. In addition, a feedback input interface is displayed, and user ratings and opinions are sent to the server. The feedback form is: <textarea id="feedback">< / textarea> <button onclick="sendFeedback()"> Submit Feedback< / button> Arrange it as follows.

[1159] User

[1160] The user enters questions or inquiries through the terminal interface. For example, they can enter, "Please tell me the reservation status for reservation number 12345." The user types directly into the chat box or form using the keyboard.

[1161] It also allows voice input and image uploading as needed. The voice input function is implemented using the Web Speech API. <input type="file" id="imageUpload"> or <button onclick="startVoiceRecognition()"> Speak< / button> Set it as follows.

[1162] The user reviews the provided answer and evaluates whether the problem has been resolved, reads the answer to check accuracy and satisfaction, enters their thoughts and opinions in the feedback form, and presses the submit button.

[1163] Specific examples

[1164] Hotel booking confirmation scenario

[1165] 1. The user uses the smartphone app to enter "Please tell me the reservation status for reservation number 12345" into the terminal.

[1166] 2. The device sends this input to the server.

[1167] 3. The server analyzes the received inquiry and inputs it into the generative AI model as a "reservation confirmation" category.

[1168] 4. The server searches the database for information related to reservation number 12345, generates a response such as "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for your use," and sends it to the terminal.

[1169] 5. The terminal displays the response from the server to the user.

[1170] 6. The user enters feedback such as "The response was quick and helpful" and sends it to the server.

[1171] In this way, the system of this invention uses generative AI models to automate inquiry responses and provide high-quality customer support 24 hours a day. This improves user satisfaction and significantly reduces the human workload required for support. Furthermore, the system constantly evolves based on feedback, enabling it to provide optimal solutions.

[1172] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1173] Step 1:

[1174] Loading and initializing the generative AI model

[1175] The server loads the weight and configuration files of the generative AI model from disk into memory. This process is performed using library functions from TensorFlow or PyTorch, for example. Specifically, commands such as torch.load('model.pth') or tf.keras.models.load_model('model.h5') are executed. The input is the weight and configuration files, and the output is the generative AI model deployed in memory.

[1176] Step 2:

[1177] Connecting to a Database

[1178] The server connects to a relational database such as MySQL or PostgreSQL. This connection is made using the sqlalchemy library. For example, run a command like engine = create_engine('mysql: / / user:password@host / dbname'). The input is the database connection information, and the output is the database instance to which the connection is established.

[1179] Step 3:

[1180] Receiving inquiry information

[1181] The server receives the query information sent from the terminal via an HTTP request. When using Flask as a framework, data is obtained from request.json, for example. The input is the query content sent as an HTTP request, and the output is the query information stored in variables on the server.

[1182] Step 4:

[1183] Analysis of inquiry content and categorization

[1184] The server analyzes the received query using natural language processing (NLP) techniques. This process involves tokenization using the nltk and spaCy libraries and applying classification models. The input is the query text data, and the output is classified category information.

[1185] Step 5:

[1186] Input to generative AI model and answer generation

[1187] The server inputs the parsed data as a prompt sentence into the generative AI model to generate an appropriate answer. For example, a command such as gpt3_model.predict(prompt) is executed. The input is the generated prompt sentence, and the output is the generated answer text.

[1188] Step 6:

[1189] Retrieving additional information from the database

[1190] If necessary, the server retrieves additional information from the database by executing an SQL query (e.g. SELECT FROM reservations WHERE reservation_id = '12345'). The input is the reservation number as an SQL query, and the output is the retrieved reservation record.

[1191] Step 7:

[1192] Submit your answer

[1193] The server sends the generated answer to the terminal as an HTTP response. Specifically, it creates a JSON format answer with jsonify(response) and returns it as return response. The input is the generated answer text, and the output is the HTTP response sent to the terminal.

[1194] Step 8:

[1195] User Interface Display

[1196] The terminal displays a customer support user interface to the user. For example, it generates an inquiry form or chat box using HTML and React. The input is an HTML template or stylesheet as initial data, and the output is the interface displayed in the user's browser.

[1197] Step 9:

[1198] Submit an inquiry

[1199] The user inputs and sends the inquiry through the terminal interface. For example, they might input "Please tell me the reservation status for reservation number 12345" and press the send button. The input is the text data entered by the user, and the output is an HTTP request sent to the server.

[1200] Step 10:

[1201] Check your answers

[1202] The user checks the answer sent from the server on the terminal interface. The terminal receives the response data from the server and displays it in the chat box. The input is the answer text from the server, and the output is the answer displayed on the user's screen.

[1203] (Application example 1)

[1204] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1205] There is a need for a system that can quickly and reliably respond to various inquiries and problems encountered by users of autonomous vehicles. There is also a lack of effective means to provide rapid response in emergencies and safety alert notifications. This is necessary to increase the safety and peace of mind of users.

[1206] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1207] In this invention, the server includes means for generating responses to inquiries using a generative AI model, terminal means for sending inquiries and receiving responses, means for updating the generative AI model based on feedback, means for responding to inquiries and problems encountered by users of autonomous vehicles, and means for generating emergency response measures and safety alert notifications, thereby enabling users of autonomous vehicles to receive support quickly and reliably and enabling effective emergency and safety responses.

[1208] A "generative AI model" is an artificial intelligence model that automatically generates appropriate answers based on given data and inquiry content.

[1209] "Server means" means a computing device for processing inquiries, generating and managing responses using a generative AI model.

[1210] "Terminal means" refers to a device through which a user inputs an inquiry and receives a response from the server means, and examples of such a device include a smartphone or tablet.

[1211] "Feedback" refers to the act and content of a user's evaluation or opinion regarding a system's response or answer.

[1212] An "autonomous vehicle" is a vehicle that operates autonomously with minimal driver intervention.

[1213] "Emergency response measures" refer to the course of action and procedures for responding quickly and appropriately in the event of an emergency such as an accident or breakdown.

[1214] "Safety alert notification" is a notification function that promptly notifies users of safety precautions and warnings.

[1215] "Inquiry analysis" is the process of understanding and classifying the inquiries provided by users and assigning them to appropriate categories.

[1216] "Category" refers to a group or classification item for systematically classifying inquiry content.

[1217] A "database" is a collection of information that systematically stores and manages various inquiries, their responses, user information, and so on.

[1218] "Acquisition of additional information" is the process of searching, extracting, and providing the necessary information from a database based on the content of the inquiry.

[1219] An embodiment of the present invention will now be described in detail. This system is a customer support and security support system that responds to inquiries and problems encountered by users of autonomous vehicles. The system is mainly composed of three components: a server, a terminal, and a user.

[1220] server

[1221] The server is the main device that generates answers to queries using the generative AI model. First, the server loads the generative AI model at startup and reads the configuration file, weight file, etc. into memory. Next, it connects to the database used by the system and checks whether the necessary tables and indexes, such as user information, reservation data, and feedback information, are set up correctly.

[1222] When the server receives an inquiry from a user, it analyzes the inquiry and classifies it into an appropriate category (e.g., reservation confirmation, emergency response measures, safety confirmation, etc.). It then uses a generative AI model to generate an optimal answer to the inquiry, retrieving additional information from a database if necessary. The generated answer is sent to the device and provided to the user.

[1223] Terminal

[1224] The terminal is a device such as a smartphone or tablet that allows users to enter inquiries and receive responses from the server. When a user enters and sends an inquiry into the terminal, the terminal sends the content to the server. The terminal receives and displays the response from the server.

[1225] Additionally, the device receives feedback from users and transmits it to the server, which uses it as data to improve the performance of the generative AI model.

[1226] User

[1227] When using an autonomous vehicle, a user inputs their inquiry using a terminal. For example, they can input, "Please tell me the reservation status for reservation number 67890." The user checks the answer provided by the server and provides feedback as needed.

[1228] Specific examples

[1229] For example, when confirming a reservation for an autonomous vehicle, a user opens a smartphone app and enters, "Please tell me the reservation status for reservation number 67890." The device sends this inquiry to a server, which processes the inquiry using a generative AI model in the reservation confirmation category. The server searches the database for information related to reservation number 67890, generates a response saying, "Reservation number 67890 is scheduled to check in on 2023-12-01," and sends it to the device. The device displays this response to the user, who then enters feedback such as, "The response was quick and helpful," and submits it.

[1230] In addition, if an emergency inquiry such as "My car won't start! What should I do?" is entered, the server will respond using a generative AI model in the emergency response category, generating a response such as "Emergency services have been notified. Please remain calm," and sending it to the device.

[1231] This system will enable users of autonomous vehicles to receive prompt and reliable support, and will also enable effective emergency and safety responses. An example of a prompt sentence is, "Please tell me the status of reservation number 12345."

[1232] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1233] Step 1:

[1234] The server launches the generative AI model and loads the model's weight and configuration files into memory. This process occurs during the system initialization phase and lays the groundwork for the generative AI model to function correctly. It uses the weight and configuration files as input and obtains the loaded model as output.

[1235] Step 2:

[1236] The server connects to the database and checks the consistency of the various tables and indexes that store user information, reservation data, feedback information, etc. This process maintains data consistency and ensures smooth query processing later. It uses database connection information as input and obtains the connection success or failure status as output.

[1237] Step 3:

[1238] The user uses a terminal to input a query. For example, "Please tell me the reservation status for reservation number 12345." This input data is sent to the server via the terminal. The user's query content is used as input, and the query data is sent to the server as output.

[1239] Step 4:

[1240] The server analyzes the received inquiry and classifies it into the appropriate category. This process is performed by a generative AI model, which categorizes the inquiry into categories such as "reservation confirmation," "emergency measures," and "security check." The server uses the inquiry as input and obtains the categorization result as output.

[1241] Step 5:

[1242] The server retrieves additional information from the database as needed based on the category. For example, in the case of a reservation confirmation, the server searches and retrieves the relevant reservation information from the database. It uses the category classification results and the inquiry content as input, and obtains the search results (additional information) as output.

[1243] Step 6:

[1244] The server uses a generative AI model to generate the optimal answer based on the acquired additional information and the query. This generation process results in the most appropriate and useful answer for the user. The additional information and the query are used as input, and the generated answer is obtained as output.

[1245] Step 7:

[1246] The terminal receives the answer sent from the server and displays it to the user, who then confirms this information. It uses the answer data from the server as input and gets the answer displayed to the user as output.

[1247] Step 8:

[1248] The user inputs feedback for the provided answer and sends it to the server via the terminal. For example, the user inputs feedback such as "The answer was quick and helpful." The user's feedback is used as input, and the feedback sent to the server is obtained as output.

[1249] Step 9:

[1250] The server stores the received feedback in a database and uses it to improve the performance of the generative AI model. This process allows the system to continuously improve. It uses user feedback as input and obtains an updated database as output.

[1251] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1252] An embodiment of the present invention will be described in detail. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses. Furthermore, it is equipped with an emotion engine that recognizes the user's emotions, enabling more appropriate and empathetic responses.

[1253] server

[1254] 1. Loading and initializing the generative AI model

[1255] When the server starts up, it loads the generative AI model, which includes loading the model's weight files, configuration files, and associated resources.

[1256] Connect to the database and check that the necessary tables and indexes are set up correctly. Check where user information, reservation data, and feedback are stored.

[1257] 2. Processing inquiries

[1258] The server receives the inquiry information sent from the terminal, including metadata such as the inquiry content, category, user ID, and timestamp.

[1259] Analyze the inquiry and categorize it into the appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1260] The classified data is passed to a generative AI model to generate the best answer, retrieving additional information from the database if necessary.

[1261] The generated answer is sent to the terminal and provided to the user.

[1262] 3. The Emotional Engine

[1263] The server uses an emotion engine to analyze the user's emotions, identifying the user's emotional state (e.g., joy, sadness, anger, tension, etc.) from voice and text.

[1264] The emotion engine's analysis results are reflected in the generative AI model, which then adjusts the tone and content of the response. For example, if the user is feeling angry, the response will be generated in a more polite and calm tone.

[1265] 4. Processing Feedback

[1266] The server receives feedback sent from the device and stores it in a database, which is used to update the generative AI model and improve the accuracy of the system.

[1267] Terminal

[1268] 1. Displaying the user interface

[1269] The device displays a customer support interface, including a contact form, chat box, and voice input options.

[1270] Retrieve and display initial data (e.g., store opening hours, campaign information, etc.) from the server.

[1271] 2. Submitting and viewing inquiries

[1272] Validate the query entered by the user and send it to the server. Only queries that pass validation are sent.

[1273] Receives the response from the server and displays it to the user, displaying all relevant information on the screen for the user to review.

[1274] A feedback input interface is provided to receive evaluations and opinions from users and transmit them to the server.

[1275] User

[1276] 1. Enter your inquiry

[1277] The user enters a question or inquiry through the terminal interface, for example, "Please tell me the status of reservation number 12345."

[1278] If necessary, you can also input voice and upload images.

[1279] 2. Review answers and provide feedback

[1280] The user reviews the answers provided and evaluates whether the problem has been resolved.

[1281] If necessary, enter your feedback to help improve the system. For example, enter "Thank you for your quick response."

[1282] Specific examples

[1283] Let us take the example of a hotel reservation confirmation scenario.

[1284] 1. Reservation confirmation inquiries

[1285] User: Enter "Please tell me the reservation status for reservation number 12345" into the smartphone app.

[1286] Terminal: Send this input to the server.

[1287] 2. Server Processing

[1288] Server: Analyzes the received inquiry content and inputs it into the reservation confirmation category generation AI model.

[1289] Server: Searches and retrieves information related to reservation number 12345 from the database.

[1290] Server: Analyzes the user's emotional state using an emotion engine, for example, sensing tension or anxiety from the text.

[1291] Server: Adjusts the answer based on the analysis results. For example, it generates an answer such as "Don't worry. Reservation number 12345 is scheduled to check in on 2023-11-01."

[1292] Server: Sends the generated answer to the device.

[1293] 3. Providing answers

[1294] Terminal: Displays the answer from the server to the user.

[1295] User: Enters feedback such as "Thank you for your quick response" and submits.

[1296] In this way, the system of the present invention is equipped with an emotion engine that recognizes user emotions and uses a generative AI model to automate inquiry responses, making it possible to provide more appropriate and empathetic customer support. This improves user satisfaction and reduces the burden on personnel. The system constantly evolves based on feedback, improving its problem-solving capabilities.

[1297] The processing flow will be explained below.

[1298] Step 1:

[1299] Server: Responsible for loading and initializing the generative AI model. This includes loading the model weights and configuration files and associated resources from disk into memory. It also connects to the database and checks tables and indexes.

[1300] Step 2:

[1301] Terminal: Displays the user interface to the user. This includes a contact form, chat box, and voice input options. It also retrieves and displays initial data from the server (e.g., store hours, campaign information, etc.).

[1302] Step 3:

[1303] User: Enters a question or inquiry through the device interface. For example, enters text such as "Please tell me the reservation status for reservation number 12345." If necessary, voice input and image upload are also performed.

[1304] Step 4:

[1305] Terminal: Validates the entered query. Performs error checking and displays an error message to the user if there is a problem. If validation is successful, sends the query to the server.

[1306] Step 5:

[1307] Server: Receives inquiries sent from the terminal. The received data includes metadata such as the user ID, inquiry content, and timestamp.

[1308] Step 6:

[1309] Server: Analyzes the received inquiry and classifies it into the appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1310] Step 7:

[1311] Server: Analyzes the user's emotions using an emotion engine. Identifies the emotional state (e.g., joy, sadness, anger, tension, etc.) from speech and text.

[1312] Step 8:

[1313] Server: The classified data and the results of the emotion engine are input into the generative AI model to generate the optimal answer, such as "Reservation number 12345 is scheduled to check in on 2023-11-01."

[1314] Step 9:

[1315] Server: If necessary, retrieve additional information from the database. For example, look up and retrieve details related to reservation number 12345 to confirm the reservation information.

[1316] Step 10:

[1317] Server: Combines the generated answer with additional information to generate the final answer. Adjusts the tone of the answer based on the results of the emotion engine. For example, it generates something like, "Don't worry, reservation number 12345 is scheduled to check in on 2023-11-01."

[1318] Step 11:

[1319] Server: Sends the final answer to the device.

[1320] Step 12:

[1321] Terminal: The answer obtained from the server is displayed to the user. To ensure the user can confirm, the screen displays "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for using our service."

[1322] Step 13:

[1323] User: Review the provided answer and provide feedback if necessary. For example, "Thank you for your quick response."

[1324] Step 14:

[1325] Terminal: Sends the feedback entered by the user to the server.

[1326] Step 15:

[1327] Server: Stores the received feedback in a database, which is used to improve the generative AI model.

[1328] Step 16:

[1329] Server: Updates the generative AI model and emotion engine based on feedback to improve response accuracy for future inquiries.

[1330] As described above, the system handles user inquiries through a series of processing steps and provides high-quality answers by utilizing generative AI models and an emotion engine. By enabling emotion-sensitive responses, user satisfaction can be further improved.

[1331] Example 2

[1332] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1333] Conventional customer support systems often lack consistency in their responses to inquiries. They also struggle to properly understand users' emotions and respond empathetically, potentially resulting in lower user satisfaction. In particular, conventional systems face challenges in providing adequate support in situations where rapid responses to changes in emotions are required.

[1334] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for generating an answer to a query using a generative AI model, terminal means for sending an inquiry to the server means and receiving an answer, means for updating the generative AI model based on feedback acquired from the terminal means, and means for analyzing the emotional state of the query using an emotion engine and adjusting the answer to be generated. This makes it possible to provide an appropriate and consistent answer according to the user's emotions and improve user satisfaction.

[1335] A "generative AI model" refers to an algorithm or data model that uses artificial intelligence technology to generate appropriate answers to natural language queries.

[1336] "Server means" refers to a computer system for receiving inquiry information, generating a response using a generative AI model, and performing emotion analysis using an emotion engine.

[1337] "Terminal means" refers to a device, such as a computer or smartphone, through which a user inputs inquiry details, transmits the information to a server, and receives a response from the server.

[1338] An "emotion engine" refers to software or algorithms that analyze a user's emotional state from text or voice and recognize specific emotions.

[1339] "Feedback" refers to users inputting their evaluations and opinions about the answers and services provided by the system, and this information is used to improve the system's performance.

[1340] "Database" refers to a set of data structures for storing information in response to queries and allowing server means to retrieve additional information as needed.

[1341] "Parsing" refers to the process of using natural language processing techniques to structure and categorize query content.

[1342] "Answer generation" refers to the process of using a generative AI model to generate an appropriate answer to a user's inquiry.

[1343] This invention is an advanced conversational robot system that uses generative AI models to automate customer support tasks and provide fast, consistent responses. It also features an emotion engine that recognizes the user's emotions and provides appropriate, empathetic responses. The following describes each component of the system and its operation in detail.

[1344] server

[1345] The server is the center of the system and plays the following roles:

[1346] 1. Load and initialize the generative AI model:

[1347] On startup, the server loads a generative AI model (e.g., a generic generative AI model), which includes loading the model's weight files, configuration files, and associated resources.

[1348] The server connects to a database system (e.g. a general purpose database) and checks whether the necessary tables and indexes are set up correctly, e.g., where to store user information, reservation data, feedback, etc.

[1349] 2. Handling inquiries:

[1350] The server receives the inquiry information sent from the terminal. This received data includes the inquiry content, category, user ID, timestamp, etc.

[1351] The server analyzes the query content using a natural language processing library (for example, a general natural language processing library) and classifies it into an appropriate category.

[1352] The classified data is passed to a generative AI model to generate the best answer, retrieving additional information from the database if necessary.

[1353] The generated answer is sent to the terminal and provided to the user.

[1354] 3. How the Emotional Engine Works:

[1355] The server analyzes the user's emotions using an emotion engine (e.g., a general emotion analysis tool) and identifies the user's emotional state from the voice and text.

[1356] The emotion engine's analysis results are reflected in the generative AI model, which then adjusts the tone and content of the response. For example, if the user is feeling angry, the response will be generated in a more polite and calm tone.

[1357] 4. Feedback Processing:

[1358] The server receives feedback sent from the device and stores it in a database, which is used to update the generative AI model and improve the accuracy of the system.

[1359] Terminal

[1360] The terminal provides an interface for the user to access customer support.

[1361] 1. Display the user interface:

[1362] The device displays a customer support interface, including a contact form, a chat box, and a voice input option.

[1363] As initial data, store opening hours and campaign information are obtained from the server and displayed.

[1364] 2. Submitting and viewing inquiries:

[1365] Validate the query entered by the user and send it to the server. Only queries that pass validation are sent.

[1366] Receives the response from the server and displays it to the user, displaying all relevant information on the screen for the user to review.

[1367] A feedback input interface is provided to receive evaluations and opinions from users and transmit them to the server.

[1368] User

[1369] Users submit queries through the system and provide feedback on the answers provided.

[1370] 1. Enter your inquiry:

[1371] The user enters a question or inquiry through the terminal interface, for example, "Please tell me the status of reservation number 12345."

[1372] If necessary, you can also input voice and upload images.

[1373] 2. Review answers and provide feedback:

[1374] The user reviews the answers provided and evaluates whether the problem has been resolved.

[1375] If necessary, enter your feedback to help improve the system. For example, enter "Thank you for your quick response."

[1376] Specific examples

[1377] Let us take the example of a hotel reservation confirmation scenario.

[1378] 1. Booking confirmation inquiries:

[1379] User: Enter "Please tell me the status of reservation number 12345" into the smartphone app.

[1380] Terminal: Send this input to the server.

[1381] 2. Server processing:

[1382] The server analyzes the received inquiry and inputs it into a generation AI model for reservation confirmation categories.

[1383] The server searches the database for information related to reservation number 12345 and retrieves it.

[1384] The server uses an emotion engine to analyze the user's emotional state, for example, sensing tension or anxiety from the text.

[1385] The server will then adjust the answer based on the analysis results, for example, generating a response such as "Don't worry, reservation number 12345 is scheduled to check in on 2023-11-01."

[1386] The server sends the generated response to the terminal.

[1387] 3. Providing answers:

[1388] The terminal displays the response from the server to the user.

[1389] User: Enters feedback such as "Thank you for your quick response" and submits.

[1390] The terminal sends the feedback to the server.

[1391] In this way, by using a generative AI model and an emotion engine, the system of the present invention can provide appropriate and consistent answers that correspond to the user's emotions, enabling a high level of automation in customer support operations. This improves user satisfaction and reduces the workload on personnel. Furthermore, the system can constantly evolve based on feedback, improving its problem-solving capabilities.

[1392] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1393] Step 1: Server - Initialize the system and load the AI ​​model

[1394] The server loads the generative AI model at startup, specifically by loading and initializing the generative AI model's weight and configuration files, as well as related resources.

[1395] Input: Server initialization command

[1396] Data processing: Loading and initializing the generative AI model and reading relevant files into memory.

[1397] Output: Initialized generative AI model

[1398] Action: The server logs that the generative AI model has finished loading.

[1399] Step 2: Server - Check database connection

[1400] The server connects to the database (e.g., a common database system) and verifies that the necessary tables and indexes are set up correctly.

[1401] Input: Database connection information

[1402] Data Processing: Checking the database connection and settings

[1403] Output: Successful connection and confirmation result

[1404] Action: The server verifies that the database connection is successful and logs that there are no errors.

[1405] Step 3: Terminal - Initial Display of User Interface

[1406] The device displays a customer support interface to the user, which includes a contact form, a chat box, and a voice input option.

[1407] Input: Initial Data Request

[1408] Data processing: Obtaining initial data from the server

[1409] Output: Initial data to be displayed (e.g. store hours, campaign information)

[1410] Operation: The terminal requests initial data from the server and displays the retrieved data in the interface.

[1411] Step 4: User - Enter your inquiry

[1412] The user inputs a question or inquiry through the terminal interface. For example, they might input, "Please tell me the status of reservation number 12345."

[1413] Input: User's inquiry

[1414] Data processing: Validation of input inquiry details

[1415] Output: Validated query content

[1416] How it works: The user enters a query, which is validated by the terminal.

[1417] Step 5: Device - Submit your inquiry

[1418] The terminal transmits the inquiry content that passes validation to the server.

[1419] Input: Validated inquiry content

[1420] Data processing: Sending inquiry details to the server

[1421] Output: The query sent to the server

[1422] Operation: The terminal sends the user's query to the server in the appropriate format.

[1423] Step 6: Server - Parse and categorize the query

[1424] The server analyzes the received inquiry and categorizes it into an appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1425] Input: Received inquiry content

[1426] Data processing: Analysis and categorization using natural language processing

[1427] Output: Inquiry content with identified category

[1428] How it works: The server analyzes the query and passes the classification results to the generative AI model.

[1429] Step 7: Server - Retrieving Information from the Database

[1430] The server retrieves any additional information needed from the database based on the query, for example, searching for and retrieving reservation information related to the reservation number.

[1431] Input: Category-specific inquiry content

[1432] Data processing: Information retrieval and retrieval from databases

[1433] Output: Relevant information obtained

[1434] What happens: The server performs a database query to retrieve the required information.

[1435] Step 8: Server - Answer Generation

[1436] The server uses a generative AI model to generate the optimal answer based on the additional information obtained.

[1437] Input: Acquired relevant information and a generative AI model

[1438] Data processing: Answer generation using generative AI models

[1439] Output: The generated answer

[1440] How it works: The server uses a generative AI model to generate answers to user queries.

[1441] Step 9: Server - Sentiment Analysis and Response Adjustment

[1442] The server uses an emotion engine to analyze the user's emotions and adjust the tone and content of the response accordingly.

[1443] Input: User's query and generated answer

[1444] Data processing: sentiment analysis and response adjustment

[1445] Output: Adjusted answer

[1446] How it works: The server adjusts the response based on the analysis results of the emotion engine.

[1447] Step 10: Server - Sending the response to the device

[1448] The server sends the generated response to the terminal.

[1449] Input: Adjusted Answer

[1450] Data processing: sending adjusted responses

[1451] Output: Answer sent to terminal

[1452] Operation: The server sends the adjusted response to the terminal and records the transmission log.

[1453] Step 11: Terminal - View answers and receive feedback

[1454] The terminal displays the response from the server to the user and provides a feedback input interface.

[1455] Input: Response from the server

[1456] Data processing: Displaying answers and accepting feedback

[1457] Output: User feedback

[1458] Action: The device displays the answer for the user to review and accepts feedback.

[1459] Step 12: User - Review answers and provide feedback

[1460] Users review the answers provided and provide feedback to help improve the system.

[1461] Input: Provided Answer

[1462] Data processing: Feedback input

[1463] Output:Completed feedback

[1464] Action: The user reviews the answer, enters feedback, and sends it to the device.

[1465] Step 13: Device - Send Feedback

[1466] The terminal validates the feedback entered by the user and sends it to the server.

[1467] Input: User-entered feedback

[1468] Data Processing: Feedback validation and submission

[1469] Output: Feedback sent to the server

[1470] Action: The device validates the feedback and sends it to the server.

[1471] Step 14: Server - Receiving and storing feedback

[1472] The server receives the feedback sent from the terminal and stores it in a database.

[1473] Input: Feedback sent from the device

[1474] Data processing: receiving feedback and storing it in a database

[1475] Output: Saved feedback

[1476] How it works: The server stores the received feedback in a database and uses it to update the generative AI model in the future.

[1477] This enables the entire system to function, providing quick and appropriate answers to user inquiries, improving user satisfaction and operational efficiency.

[1478] (Application example 2)

[1479] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1480] Inquiries and troubles at logistics centers require quick and appropriate responses, but because they rely on human labor, responses can be delayed and lack consistency. Furthermore, it can be difficult for staff to respond empathetically based on their emotions, which can reduce user satisfaction. The present invention aims to solve these problems and provide a system that automates efficient and empathetic responses.

[1481] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating a response to an inquiry using a generative AI model, emotion analysis means for analyzing the user's emotions and adjusting the content of the response based on the analysis results, and means for referencing a database and acquiring additional information based on the content of the inquiry. This enables a quick and consistent response, and can improve user satisfaction through empathetic responses.

[1482] A "generative AI model" is an artificial intelligence that uses machine learning algorithms to automatically generate appropriate answers to any inquiry.

[1483] "Server means" refers to a server system that provides the execution environment for the generative AI model and the computational resources for processing the inquiry content and generating and delivering the answer.

[1484] "Terminal means" refers to a device through which a user inputs an inquiry and receives and displays a response from the server means, and includes a smartphone, tablet, or computer.

[1485] "Means for updating generative AI models based on feedback" refers to the process of collecting user ratings and opinions and using them as training data for generative AI models to continuously improve their performance.

[1486] The "emotion analysis means" is a function for analyzing the emotions of a user from text or voice and adjusting the content of the response generated according to that emotional state.

[1487] "Categorization" is the process of analyzing the inquiry and classifying it into a specific category (e.g., inventory check, delivery status, problem report, etc.).

[1488] "Means for referencing a database and obtaining additional information" refers to the process of searching a database based on the inquiry content and obtaining the necessary additional information (e.g., stock status, delivery details, etc.).

[1489] This section describes in detail an embodiment of the present invention. This system is designed to automate inquiries and troubleshooting at logistics centers and provide efficient and empathetic responses. Specifically, this system uses a robot equipped with a generative AI model and emotion analysis means to handle inquiries.

[1490] Hardware:

[1491] Robot Platform:

[1492] A robot platform is a piece of hardware that physically moves around a logistics center and receives and responds to inquiries from staff. Representative examples include Pepper and NAO.

[1493] Camera and Microphone:

[1494] The robot is equipped with a camera and microphone, which are used to analyze the user's emotions from their facial expressions and voice.

[1495] display:

[1496] The robot is equipped with a display that is used to visually display the answers generated by the generative AI model.

[1497] software:

[1498] Generative AI models:

[1499] A generative AI model is software that generates appropriate answers to queries. A typical example is a machine learning algorithm such as GPT-4.

[1500] Emotion analysis means:

[1501] The emotion analysis tool is software that analyzes the user's emotions and adjusts the response content based on that state. Tools such as Emotion API and IBM Watson Tone Analyzer are used.

[1502] Database:

[1503] A database is a data storage for storing additional information required for inquiries, such as inventory information at a logistics center, delivery status, etc. Typical examples include MySQL and PostgreSQL.

[1504] Robot control software:

[1505] Robot control software is a platform for controlling robots, receiving queries from users, and processing data. A typical example is ROS (Robot Operating System).

[1506] Data processing and calculation:

[1507] Server Action:

[1508] The server generates answers using a generative AI model, analyzes emotions, and performs database lookups. Details are explained below.

[1509] 1. Startup and initialization:

[1510] When the server starts up, it loads the generative AI model and sentiment analysis method, connects to the database, checks the model weight file, configuration file, necessary indexes and tables, and loads login information, reservation data, etc.

[1511] 2. Inquiry reception and analysis:

[1512] When a user (staff member or customer) sends an inquiry to the robot, the content is transferred to the server, which analyzes the content of the inquiry and classifies it into categories (e.g., inventory check, delivery status, trouble report).

[1513] 3. Emotion analysis:

[1514] The server uses emotion analysis means to analyze the user's emotions, for example, to identify the user's anxiety or anger from the text and voice data.

[1515] 4. Answer generation and additional information acquisition:

[1516] The server uses a generative AI model to generate an appropriate response based on the analyzed inquiry content and emotion data, and retrieves additional information from the database (e.g., stock availability, delivery details, etc.) as needed to reflect the response.

[1517] 5. Provide answers:

[1518] The generated answers are provided to the user via the robot, either in voice or text format, and are also displayed on the robot's display.

[1519] Examples:

[1520] 1. Inventory Check Scenario

[1521] Staff: "Please let me know the stock status of this item."

[1522] Robot: Converts speech to text and feeds it into a generative AI model of inventory check categories.

[1523] Server: Obtains inventory information from the database and uses emotion analysis to analyze staff emotions as "worried."

[1524] Server: Generates a response and provides it to the robot: "Don't worry, we have plenty of this item in stock."

[1525] 2. Trouble Reporting Scenario

[1526] Staff: "The delivery is delayed, what's going on?"

[1527] Robot: Converts speech to text and feeds it into a generative AI model of trouble report categories.

[1528] Server: Obtains delivery status from the database and analyzes staff emotion as "anger" using emotion analysis means.

[1529] Server: "We apologize for the delay. We are currently investigating the cause of the delivery delay and will let you know the results shortly." This is the generated response and provided by the robot.

[1530] Prompt Sentence Examples

[1531] User: "What is the stock status of this item?"

[1532] Sentiment Analysis: "Worried"

[1533] Generative AI model output: "Don't worry, we have plenty of this item in stock."

[1534] In this way, by using generative AI models and emotion analysis means, embodiments of the present invention can automate inquiry responses at logistics centers and provide fast and empathetic service.

[1535] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1536] Step 1:

[1537] When the server starts up, it loads the generative AI model and sentiment analysis means and connects to the database. The server reads the model weight file and configuration file and checks whether the necessary tables and indexes are set up correctly. The inputs include the model weight file, configuration file, and database connection information, and the output is that the system will start operating normally by initializing the generative AI model and sentiment analysis means based on these.

[1538] Step 2:

[1539] The user inputs a query into the terminal (robot). The input includes the query content in voice or text format. The robot converts the voice into text and sends the query content as text data to the server. The query content is transferred to the server as output.

[1540] Step 3:

[1541] The server analyzes the received inquiry and classifies it into categories. Specifically, the text data is input into a string analysis algorithm, which classifies it into categories such as "inventory check," "delivery status," and "trouble report." The input is the text of the inquiry, and the output is the data classified into categories. This classification data is used in the next step.

[1542] Step 4:

[1543] The server uses the emotion analysis means to analyze the user's emotions. The input includes the text data of the inquiry received earlier. The emotion analysis means identifies the emotional state (e.g., worry, anger, joy, etc.) from the text and outputs the result. The output is the emotional data obtained by the emotion analysis means.

[1544] Step 5:

[1545] The server generates an appropriate answer using a generative AI model based on the analyzed query content and emotional data. The input is classified category data and emotional data, which are fed into the generative AI model to generate an answer in natural language format. The output is the generated answer.

[1546] Step 6:

[1547] If necessary, the server retrieves additional information related to the query from the database. The input is the query and category data, and based on this, it executes a database query to retrieve the target data (e.g., inventory information, delivery status, etc.). The output is the retrieved additional information.

[1548] Step 7:

[1549] The server integrates additional information into the generated answer and sends the final answer to the robot terminal. The input is the integrated answer data, and the output is the final answer sent to the robot terminal.

[1550] Step 8:

[1551] The robot terminal provides the answer received from the server to the user. Specifically, it displays or plays back the answer in text or audio format. The input is the answer data received from the server, and the output is the answer provided to the user.

[1552] Step 9:

[1553] The user checks the provided answers and enters feedback if necessary. The feedback is sent back to the server and used as update data for the generative AI model. The input is the feedback data from the user, and the output is the updated generative AI model.

[1554] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1555] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1556] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1557] [Fourth embodiment]

[1558] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1559] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1560] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1561] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1562] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1563] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1564] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1565] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1566] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1567] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1568] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1569] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1570] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1571] An embodiment of the present invention will be described in detail. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses.

[1572] server

[1573] 1. Loading and initializing the generative AI model

[1574] The server first loads the generative AI model on startup, which involves loading the model's weight files, configuration files, and associated resources into memory.

[1575] Next, it connects to the database and verifies that the necessary tables and indexes are set up correctly, including where to store user information, booking data, and feedback.

[1576] 2. Processing inquiries

[1577] The server receives the inquiry information sent from the terminal, including the category and detailed information of the inquiry.

[1578] First, the inquiry content is analyzed and classified into appropriate categories (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1579] The classified data is then passed to a generative AI model to generate an appropriate answer.

[1580] If necessary, retrieve additional information from the database, for example, look up data related to the reservation number to confirm the reservation information.

[1581] The generated answer is sent to the terminal and provided to the user.

[1582] Terminal

[1583] 1. Displaying the user interface

[1584] The terminal displays a customer support user interface to the user, which includes a contact form, a chat box, and a voice input option.

[1585] The terminal retrieves initial data from the server and displays it to the user. The initial data includes, for example, store opening hours and campaign information.

[1586] 2. Submitting and viewing inquiries

[1587] The inquiry entered by the user is sent to the server, at which point the device validates the input.

[1588] When a response is returned from the server, the terminal displays the response to the user.

[1589] Furthermore, a feedback input interface is displayed, and evaluations and opinions from users are sent to the server.

[1590] User

[1591] 1. Enter your inquiry

[1592] The user inputs a question or inquiry through the terminal interface. For example, they might input, "Please tell me the status of reservation number 12345."

[1593] If necessary, you can also input voice and upload images.

[1594] 2. Review answers and provide feedback

[1595] The user reviews the answers provided and evaluates whether the problem has been resolved.

[1596] Provide feedback as needed to help improve the system.

[1597] Specific examples

[1598] Let us take the example of a hotel reservation confirmation scenario.

[1599] 1. Reservation confirmation inquiries

[1600] User: Enter "Please tell me the reservation status for reservation number 12345" into the smartphone app.

[1601] Terminal: Send this input to the server.

[1602] 2. Server Processing

[1603] Server: Analyzes the received inquiry content and inputs it into the reservation confirmation category generation AI model.

[1604] Server: Search the database for information related to reservation number 12345.

[1605] Server: Generates a response saying, "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for your use." and sends it to the terminal.

[1606] 3. Providing answers

[1607] Terminal: Displays the answer from the server to the user.

[1608] User: Enters feedback such as "Thank you for your quick response" and sends it to the server.

[1609] As described above, the system of this invention uses a generative AI model to automate inquiry responses and provide high-quality customer support 24 hours a day. This improves user satisfaction and reduces the burden on personnel. Furthermore, the system constantly evolves based on feedback, improving its problem-solving capabilities.

[1610] The processing flow will be explained below.

[1611] Step 1:

[1612] Server: Responsible for loading and initializing the generative AI model. It loads the model weights, configuration files, and related resources from disk into memory. It also connects to the database and ensures that the necessary tables and indexes are set up correctly. This includes where to store user information, reservation data, and feedback.

[1613] Step 2:

[1614] Terminal: Displays the customer support user interface to the user. This includes an inquiry form, chat box, voice input options, etc. It also retrieves initial data (such as store opening hours and campaign information) from the server and displays it to the user.

[1615] Step 3:

[1616] User: Enters questions or inquiries through the device interface. For example, they can enter text such as "Please tell me the status of reservation number 12345." They can also input voice commands or upload images as needed.

[1617] Step 4:

[1618] Terminal: Validates the query entered by the user. Checks for errors or omissions in the input, and displays an error message to the user if there is a problem. If validation is successful, sends the query to the server.

[1619] Step 5:

[1620] Server: Receives inquiries sent from the terminal. The received data includes metadata such as the user ID, inquiry content, and timestamp.

[1621] Step 6:

[1622] Server: Analyzes the received inquiry and classifies it into an appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1623] Step 7:

[1624] Server: Passes the classified data to the generative AI model and generates the optimal answer to the inquiry. For example, the AI ​​model generates an answer such as "Reservation number 12345 is scheduled to check in on 2023-11-01."

[1625] Step 8:

[1626] Server: Retrieves additional information from the database if necessary. For example, to confirm the reservation, it searches and retrieves details related to reservation number 12345 from the database.

[1627] Step 9:

[1628] Server: Combines the generated answer with additional information to generate a final answer. For example, you can add a supplemental message such as "Thank you for using our service."

[1629] Step 10:

[1630] Server: Sends the final answer to the device.

[1631] Step 11:

[1632] Terminal: The answer obtained from the server is displayed to the user. To ensure the user can confirm, the screen displays "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for using our service."

[1633] Step 12:

[1634] User: Review the provided answer and provide feedback if necessary. For example, "Thank you for your quick response."

[1635] Step 13:

[1636] Terminal: Sends the feedback entered by the user to the server.

[1637] Step 14:

[1638] Server: Stores the received feedback in a database, which helps improve the generative AI model.

[1639] Step 15:

[1640] Server: Updates the generative AI model based on feedback to improve the accuracy of responses to future inquiries.

[1641] As described above, the system handles user inquiries through a series of processing steps and uses generative AI models to provide high-quality answers, achieving consistent responses and efficient processing, thereby improving the quality of customer support operations.

[1642] Example 1

[1643] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1644] Conventional customer support systems have difficulty responding to user inquiries effectively and quickly, often resulting in delayed responses, especially when complex inquiries or a large number of inquiries are received simultaneously. Furthermore, analyzing inquiries and generating responses requires a large amount of human resources, resulting in a high workload. Furthermore, there are insufficient means to effectively utilize feedback to improve the system. To solve these problems, a system that utilizes generative AI models to automate inquiry responses and provide efficient and prompt responses is needed.

[1645] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1646] In this invention, the server includes a means for reading a weight file and a setting file of the generative AI model, a means for connecting to a database, and a means for referencing a data storage device based on the query content and obtaining additional information. This allows the generative AI model to analyze the query content, classify it into categories, and quickly generate an appropriate answer, while also obtaining additional information from the database to provide a more detailed and accurate answer. Furthermore, the accuracy and effectiveness of the system can be improved by updating the generative AI model based on feedback obtained from users.

[1647] The "information processing device means" is a device that performs processing to generate an answer to an inquiry using a generative AI model.

[1648] The "display device means" is a device that sends inquiries from users and receives and displays responses from the generative AI model.

[1649] "Evaluation Information" means user-provided feedback or evaluations that are used to improve the performance of generative AI models.

[1650] A "weight file" is a data file that stores patterns and knowledge that a generative AI model has previously learned.

[1651] A "configuration file" is a file used to manage the behavior and parameters of a generative AI model.

[1652] A "data storage device" is a device for storing data necessary for system operation, such as user information, reservation data, and inquiry records.

[1653] "Inquiry content" refers to questions or requests that users input to the system.

[1654] A "generative AI model" is an algorithm that uses machine learning technology to automatically generate answers to user inquiries.

[1655] "Category" means a major group or area into which inquiries are classified.

[1656] "Additional information" is information obtained from a data repository to supplement the response to a query.

[1657] An embodiment of the present invention will be described in detail below. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses.

[1658] server

[1659] The server first loads the weight and configuration files of the generated AI model. This is typically done using a deep learning library such as TensorFlow or PyTorch. For example, the model is loaded using torch.load('model.pth') or tf.keras.models.load_model('model.h5').

[1660] Next, the server connects to a database, which is most likely a relational database management system like MySQL or PostgreSQL. This is done using the sqlalchemy library, for example with code like engine = create_engine('mysql: / / user:password@host / dbname').

[1661] The server receives the query information sent from the device. This is often done using an HTTP request. For example, the Flask framework is used to obtain data from request.json.

[1662] The server then analyzes the query using natural language processing techniques and categorizes it, using libraries such as nltk and spaCy to tokenize the text and predict the category using a classification model.

[1663] The parsed data is passed to a generative AI model to generate an appropriate answer. For example, a prompt is input to the generative AI model, such as gpt3_model.predict(prompt). If necessary, additional information is retrieved from the database. The SQL query uses syntax such as SELECT FROM reservations WHERE reservation_id = '12345'.

[1664] The generated answer is returned to the terminal as an HTTP response. jsonify(response) creates a JSON format answer and returns it as return response.

[1665] Terminal

[1666] The terminal displays a customer support user interface to the user, which includes a contact form, a chat box, and a voice input option. HTML and React are used as front-end technologies.

[1667] As initial data, display business hours and campaign information obtained from the server. Use an AJAX call to obtain and display the data. For example, this is often implemented as $.get(' / api / info', function(data) { $('info').html(data);});

[1668] When sending the query entered by the user to the server, the device performs validation to ensure that the input is correct. This is done with the following code: function sendQuery() { var query = $('query').val(); $.post(' / api / query', { query: query}, function(response) { displayResponse(response);});}

[1669] The response returned from the server is displayed to the user. The received data is embedded in HTML. Implement it as follows: function displayResponse(response) { $('response').html(response);}. In addition, a feedback input interface is displayed, and user ratings and opinions are sent to the server. The feedback form is: <textarea id="feedback">< / textarea> <button onclick="sendFeedback()"> Submit Feedback< / button> Arrange it as follows.

[1670] User

[1671] The user enters questions or inquiries through the terminal interface. For example, they can enter, "Please tell me the reservation status for reservation number 12345." The user types directly into the chat box or form using the keyboard.

[1672] It also allows voice input and image uploading as needed. The voice input function is implemented using the Web Speech API. <input type="file" id="imageUpload"> or <button onclick="startVoiceRecognition()"> Speak< / button> Set it as follows.

[1673] The user reviews the provided answer and evaluates whether the problem has been resolved, reads the answer to check accuracy and satisfaction, enters their thoughts and opinions in the feedback form, and presses the submit button.

[1674] Specific examples

[1675] Hotel booking confirmation scenario

[1676] 1. The user uses the smartphone app to enter "Please tell me the reservation status for reservation number 12345" into the terminal.

[1677] 2. The device sends this input to the server.

[1678] 3. The server analyzes the received inquiry and inputs it into the generative AI model as a "reservation confirmation" category.

[1679] 4. The server searches the database for information related to reservation number 12345, generates a response such as "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for your use," and sends it to the terminal.

[1680] 5. The terminal displays the response from the server to the user.

[1681] 6. The user enters feedback such as "The response was quick and helpful" and sends it to the server.

[1682] In this way, the system of this invention uses generative AI models to automate inquiry responses and provide high-quality customer support 24 hours a day. This improves user satisfaction and significantly reduces the human workload required for support. Furthermore, the system constantly evolves based on feedback, enabling it to provide optimal solutions.

[1683] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1684] Step 1:

[1685] Loading and initializing the generative AI model

[1686] The server loads the weight and configuration files of the generative AI model from disk into memory. This process is performed using library functions from TensorFlow or PyTorch, for example. Specifically, commands such as torch.load('model.pth') or tf.keras.models.load_model('model.h5') are executed. The input is the weight and configuration files, and the output is the generative AI model deployed in memory.

[1687] Step 2:

[1688] Connecting to a Database

[1689] The server connects to a relational database such as MySQL or PostgreSQL. This connection is made using the sqlalchemy library. For example, run a command like engine = create_engine('mysql: / / user:password@host / dbname'). The input is the database connection information, and the output is the database instance to which the connection is established.

[1690] Step 3:

[1691] Receiving inquiry information

[1692] The server receives the query information sent from the terminal via an HTTP request. When using Flask as a framework, data is obtained from request.json, for example. The input is the query content sent as an HTTP request, and the output is the query information stored in variables on the server.

[1693] Step 4:

[1694] Analysis of inquiry content and categorization

[1695] The server analyzes the received query using natural language processing (NLP) techniques. This process involves tokenization using the nltk and spaCy libraries and applying classification models. The input is the query text data, and the output is classified category information.

[1696] Step 5:

[1697] Input to generative AI model and answer generation

[1698] The server inputs the parsed data as a prompt sentence into the generative AI model to generate an appropriate answer. For example, a command such as gpt3_model.predict(prompt) is executed. The input is the generated prompt sentence, and the output is the generated answer text.

[1699] Step 6:

[1700] Retrieving additional information from the database

[1701] If necessary, the server retrieves additional information from the database by executing an SQL query (e.g. SELECT FROM reservations WHERE reservation_id = '12345'). The input is the reservation number as an SQL query, and the output is the retrieved reservation record.

[1702] Step 7:

[1703] Submit your answer

[1704] The server sends the generated answer to the terminal as an HTTP response. Specifically, it creates a JSON format answer with jsonify(response) and returns it as return response. The input is the generated answer text, and the output is the HTTP response sent to the terminal.

[1705] Step 8:

[1706] User Interface Display

[1707] The terminal displays a customer support user interface to the user. For example, it generates an inquiry form or chat box using HTML and React. The input is an HTML template or stylesheet as initial data, and the output is the interface displayed in the user's browser.

[1708] Step 9:

[1709] Submit an inquiry

[1710] The user inputs and sends the inquiry through the terminal interface. For example, they might input "Please tell me the reservation status for reservation number 12345" and press the send button. The input is the text data entered by the user, and the output is an HTTP request sent to the server.

[1711] Step 10:

[1712] Check your answers

[1713] The user checks the answer sent from the server on the terminal interface. The terminal receives the response data from the server and displays it in the chat box. The input is the answer text from the server, and the output is the answer displayed on the user's screen.

[1714] (Application example 1)

[1715] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1716] There is a need for a system that can quickly and reliably respond to various inquiries and problems encountered by users of autonomous vehicles. There is also a lack of effective means to provide rapid response in emergencies and safety alert notifications. This is necessary to increase the safety and peace of mind of users.

[1717] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1718] In this invention, the server includes means for generating responses to inquiries using a generative AI model, terminal means for sending inquiries and receiving responses, means for updating the generative AI model based on feedback, means for responding to inquiries and problems encountered by users of autonomous vehicles, and means for generating emergency response measures and safety alert notifications, thereby enabling users of autonomous vehicles to receive support quickly and reliably and enabling effective emergency and safety responses.

[1719] A "generative AI model" is an artificial intelligence model that automatically generates appropriate answers based on given data and inquiry content.

[1720] "Server means" means a computing device for processing inquiries, generating and managing responses using a generative AI model.

[1721] "Terminal means" refers to a device through which a user inputs an inquiry and receives a response from the server means, and examples of such a device include a smartphone or tablet.

[1722] "Feedback" refers to the act and content of a user's evaluation or opinion regarding a system's response or answer.

[1723] An "autonomous vehicle" is a vehicle that operates autonomously with minimal driver intervention.

[1724] "Emergency response measures" refer to the course of action and procedures for responding quickly and appropriately in the event of an emergency such as an accident or breakdown.

[1725] "Safety alert notification" is a notification function that promptly notifies users of safety precautions and warnings.

[1726] "Inquiry analysis" is the process of understanding and classifying the inquiries provided by users and assigning them to appropriate categories.

[1727] "Category" refers to a group or classification item for systematically classifying inquiry content.

[1728] A "database" is a collection of information that systematically stores and manages various inquiries, their responses, user information, and so on.

[1729] "Acquisition of additional information" is the process of searching, extracting, and providing the necessary information from a database based on the content of the inquiry.

[1730] An embodiment of the present invention will now be described in detail. This system is a customer support and security support system that responds to inquiries and problems encountered by users of autonomous vehicles. The system is mainly composed of three components: a server, a terminal, and a user.

[1731] server

[1732] The server is the main device that generates answers to queries using the generative AI model. First, the server loads the generative AI model at startup and reads the configuration file, weight file, etc. into memory. Next, it connects to the database used by the system and checks whether the necessary tables and indexes, such as user information, reservation data, and feedback information, are set up correctly.

[1733] When the server receives an inquiry from a user, it analyzes the inquiry and classifies it into an appropriate category (e.g., reservation confirmation, emergency response measures, safety confirmation, etc.). It then uses a generative AI model to generate an optimal answer to the inquiry, retrieving additional information from a database if necessary. The generated answer is sent to the device and provided to the user.

[1734] Terminal

[1735] The terminal is a device such as a smartphone or tablet that allows users to enter inquiries and receive responses from the server. When a user enters and sends an inquiry into the terminal, the terminal sends the content to the server. The terminal receives and displays the response from the server.

[1736] Additionally, the device receives feedback from users and transmits it to the server, which uses it as data to improve the performance of the generative AI model.

[1737] User

[1738] When using an autonomous vehicle, a user inputs their inquiry using a terminal. For example, they can input, "Please tell me the reservation status for reservation number 67890." The user checks the answer provided by the server and provides feedback as needed.

[1739] Specific examples

[1740] For example, when confirming a reservation for an autonomous vehicle, a user opens a smartphone app and enters, "Please tell me the reservation status for reservation number 67890." The device sends this inquiry to a server, which processes the inquiry using a generative AI model in the reservation confirmation category. The server searches the database for information related to reservation number 67890, generates a response saying, "Reservation number 67890 is scheduled to check in on 2023-12-01," and sends it to the device. The device displays this response to the user, who then enters feedback such as, "The response was quick and helpful," and submits it.

[1741] In addition, if an emergency inquiry such as "My car won't start! What should I do?" is entered, the server will respond using a generative AI model in the emergency response category, generating a response such as "Emergency services have been notified. Please remain calm," and sending it to the device.

[1742] This system will enable users of autonomous vehicles to receive prompt and reliable support, and will also enable effective emergency and safety responses. An example of a prompt sentence is, "Please tell me the status of reservation number 12345."

[1743] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1744] Step 1:

[1745] The server launches the generative AI model and loads the model's weight and configuration files into memory. This process occurs during the system initialization phase and lays the groundwork for the generative AI model to function correctly. It uses the weight and configuration files as input and obtains the loaded model as output.

[1746] Step 2:

[1747] The server connects to the database and checks the consistency of the various tables and indexes that store user information, reservation data, feedback information, etc. This process maintains data consistency and ensures smooth query processing later. It uses database connection information as input and obtains the connection success or failure status as output.

[1748] Step 3:

[1749] The user uses a terminal to input a query. For example, "Please tell me the reservation status for reservation number 12345." This input data is sent to the server via the terminal. The user's query content is used as input, and the query data is sent to the server as output.

[1750] Step 4:

[1751] The server analyzes the received inquiry and classifies it into the appropriate category. This process is performed by a generative AI model, which categorizes the inquiry into categories such as "reservation confirmation," "emergency measures," and "security check." The server uses the inquiry as input and obtains the categorization result as output.

[1752] Step 5:

[1753] The server retrieves additional information from the database as needed based on the category. For example, in the case of a reservation confirmation, the server searches and retrieves the relevant reservation information from the database. It uses the category classification results and the inquiry content as input, and obtains the search results (additional information) as output.

[1754] Step 6:

[1755] The server uses a generative AI model to generate the optimal answer based on the acquired additional information and the query. This generation process results in the most appropriate and useful answer for the user. The additional information and the query are used as input, and the generated answer is obtained as output.

[1756] Step 7:

[1757] The terminal receives the answer sent from the server and displays it to the user, who then confirms this information. It uses the answer data from the server as input and gets the answer displayed to the user as output.

[1758] Step 8:

[1759] The user inputs feedback for the provided answer and sends it to the server via the terminal. For example, the user inputs feedback such as "The answer was quick and helpful." The user's feedback is used as input, and the feedback sent to the server is obtained as output.

[1760] Step 9:

[1761] The server stores the received feedback in a database and uses it to improve the performance of the generative AI model. This process allows the system to continuously improve. It uses user feedback as input and obtains an updated database as output.

[1762] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1763] An embodiment of the present invention will be described in detail. This system is a conversational robot system that uses generative AI models to highly automate customer support tasks and provide fast and consistent responses. Furthermore, it is equipped with an emotion engine that recognizes the user's emotions, enabling more appropriate and empathetic responses.

[1764] server

[1765] 1. Loading and initializing the generative AI model

[1766] When the server starts up, it loads the generative AI model, which includes loading the model's weight files, configuration files, and associated resources.

[1767] Connect to the database and check that the necessary tables and indexes are set up correctly. Check where user information, reservation data, and feedback are stored.

[1768] 2. Processing inquiries

[1769] The server receives the inquiry information sent from the terminal, including metadata such as the inquiry content, category, user ID, and timestamp.

[1770] Analyze the inquiry and categorize it into the appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1771] The classified data is passed to a generative AI model to generate the best answer, retrieving additional information from the database if necessary.

[1772] The generated answer is sent to the terminal and provided to the user.

[1773] 3. The Emotional Engine

[1774] The server uses an emotion engine to analyze the user's emotions, identifying the user's emotional state (e.g., joy, sadness, anger, tension, etc.) from voice and text.

[1775] The emotion engine's analysis results are reflected in the generative AI model, which then adjusts the tone and content of the response. For example, if the user is feeling angry, the response will be generated in a more polite and calm tone.

[1776] 4. Processing Feedback

[1777] The server receives feedback sent from the device and stores it in a database, which is used to update the generative AI model and improve the accuracy of the system.

[1778] Terminal

[1779] 1. Displaying the user interface

[1780] The device displays a customer support interface, including a contact form, chat box, and voice input options.

[1781] Retrieve and display initial data (e.g., store opening hours, campaign information, etc.) from the server.

[1782] 2. Submitting and viewing inquiries

[1783] Validate the query entered by the user and send it to the server. Only queries that pass validation are sent.

[1784] Receives the response from the server and displays it to the user, displaying all relevant information on the screen for the user to review.

[1785] A feedback input interface is provided to receive evaluations and opinions from users and transmit them to the server.

[1786] User

[1787] 1. Enter your inquiry

[1788] The user enters a question or inquiry through the terminal interface, for example, "Please tell me the status of reservation number 12345."

[1789] If necessary, you can also input voice and upload images.

[1790] 2. Review answers and provide feedback

[1791] The user reviews the answers provided and evaluates whether the problem has been resolved.

[1792] If necessary, enter your feedback to help improve the system. For example, enter "Thank you for your quick response."

[1793] Specific examples

[1794] Let us take the example of a hotel reservation confirmation scenario.

[1795] 1. Reservation confirmation inquiries

[1796] User: Enter "Please tell me the reservation status for reservation number 12345" into the smartphone app.

[1797] Terminal: Send this input to the server.

[1798] 2. Server Processing

[1799] Server: Analyzes the received inquiry content and inputs it into the reservation confirmation category generation AI model.

[1800] Server: Searches and retrieves information related to reservation number 12345 from the database.

[1801] Server: Analyzes the user's emotional state using an emotion engine, for example, sensing tension or anxiety from the text.

[1802] Server: Adjusts the answer based on the analysis results. For example, it generates an answer such as "Don't worry. Reservation number 12345 is scheduled to check in on 2023-11-01."

[1803] Server: Sends the generated answer to the device.

[1804] 3. Providing answers

[1805] Terminal: Displays the answer from the server to the user.

[1806] User: Enters feedback such as "Thank you for your quick response" and submits.

[1807] In this way, the system of the present invention is equipped with an emotion engine that recognizes user emotions and uses a generative AI model to automate inquiry responses, making it possible to provide more appropriate and empathetic customer support. This improves user satisfaction and reduces the burden on personnel. The system constantly evolves based on feedback, improving its problem-solving capabilities.

[1808] The processing flow will be explained below.

[1809] Step 1:

[1810] Server: Responsible for loading and initializing the generative AI model. This includes loading the model weights and configuration files and associated resources from disk into memory. It also connects to the database and checks tables and indexes.

[1811] Step 2:

[1812] Terminal: Displays the user interface to the user. This includes a contact form, chat box, and voice input options. It also retrieves and displays initial data from the server (e.g., store hours, campaign information, etc.).

[1813] Step 3:

[1814] User: Enters a question or inquiry through the device interface. For example, enters text such as "Please tell me the reservation status for reservation number 12345." If necessary, voice input and image upload are also performed.

[1815] Step 4:

[1816] Terminal: Validates the entered query. Performs error checking and displays an error message to the user if there is a problem. If validation is successful, sends the query to the server.

[1817] Step 5:

[1818] Server: Receives inquiries sent from the terminal. The received data includes metadata such as the user ID, inquiry content, and timestamp.

[1819] Step 6:

[1820] Server: Analyzes the received inquiry and classifies it into the appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1821] Step 7:

[1822] Server: Analyzes the user's emotions using an emotion engine. Identifies the emotional state (e.g., joy, sadness, anger, tension, etc.) from speech and text.

[1823] Step 8:

[1824] Server: The classified data and the results of the emotion engine are input into the generative AI model to generate the optimal answer, such as "Reservation number 12345 is scheduled to check in on 2023-11-01."

[1825] Step 9:

[1826] Server: If necessary, retrieve additional information from the database. For example, look up and retrieve details related to reservation number 12345 to confirm the reservation information.

[1827] Step 10:

[1828] Server: Combines the generated answer with additional information to generate the final answer. Adjusts the tone of the answer based on the results of the emotion engine. For example, it generates something like, "Don't worry, reservation number 12345 is scheduled to check in on 2023-11-01."

[1829] Step 11:

[1830] Server: Sends the final answer to the device.

[1831] Step 12:

[1832] Terminal: The answer obtained from the server is displayed to the user. To ensure the user can confirm, the screen displays "Reservation number 12345 is scheduled to check in on 2023-11-01. Thank you for using our service."

[1833] Step 13:

[1834] User: Review the provided answer and provide feedback if necessary. For example, "Thank you for your quick response."

[1835] Step 14:

[1836] Terminal: Sends the feedback entered by the user to the server.

[1837] Step 15:

[1838] Server: Stores the received feedback in a database, which is used to improve the generative AI model.

[1839] Step 16:

[1840] Server: Updates the generative AI model and emotion engine based on feedback to improve response accuracy for future inquiries.

[1841] As described above, the system handles user inquiries through a series of processing steps and provides high-quality answers by utilizing generative AI models and an emotion engine. By enabling emotion-sensitive responses, user satisfaction can be further improved.

[1842] Example 2

[1843] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1844] Conventional customer support systems often lack consistency in their responses to inquiries. They also struggle to properly understand users' emotions and respond empathetically, potentially resulting in lower user satisfaction. In particular, conventional systems face challenges in providing adequate support in situations where rapid responses to changes in emotions are required.

[1845] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for generating an answer to a query using a generative AI model, terminal means for sending an inquiry to the server means and receiving an answer, means for updating the generative AI model based on feedback acquired from the terminal means, and means for analyzing the emotional state of the query using an emotion engine and adjusting the answer to be generated. This makes it possible to provide an appropriate and consistent answer according to the user's emotions and improve user satisfaction.

[1846] A "generative AI model" refers to an algorithm or data model that uses artificial intelligence technology to generate appropriate answers to natural language queries.

[1847] "Server means" refers to a computer system for receiving inquiry information, generating a response using a generative AI model, and performing emotion analysis using an emotion engine.

[1848] "Terminal means" refers to a device, such as a computer or smartphone, through which a user inputs inquiry details, transmits the information to a server, and receives a response from the server.

[1849] An "emotion engine" refers to software or algorithms that analyze a user's emotional state from text or voice and recognize specific emotions.

[1850] "Feedback" refers to users inputting their evaluations and opinions about the answers and services provided by the system, and this information is used to improve the system's performance.

[1851] "Database" refers to a set of data structures for storing information in response to queries and allowing server means to retrieve additional information as needed.

[1852] "Parsing" refers to the process of using natural language processing techniques to structure and categorize query content.

[1853] "Answer generation" refers to the process of using a generative AI model to generate an appropriate answer to a user's inquiry.

[1854] This invention is an advanced conversational robot system that uses generative AI models to automate customer support tasks and provide fast, consistent responses. It also features an emotion engine that recognizes the user's emotions and provides appropriate, empathetic responses. The following describes each component of the system and its operation in detail.

[1855] server

[1856] The server is the center of the system and plays the following roles:

[1857] 1. Load and initialize the generative AI model:

[1858] On startup, the server loads a generative AI model (e.g., a generic generative AI model), which includes loading the model's weight files, configuration files, and associated resources.

[1859] The server connects to a database system (e.g. a general purpose database) and checks whether the necessary tables and indexes are set up correctly, e.g., where to store user information, reservation data, feedback, etc.

[1860] 2. Handling inquiries:

[1861] The server receives the inquiry information sent from the terminal. This received data includes the inquiry content, category, user ID, timestamp, etc.

[1862] The server analyzes the query content using a natural language processing library (for example, a general natural language processing library) and classifies it into an appropriate category.

[1863] The classified data is passed to a generative AI model to generate the best answer, retrieving additional information from the database if necessary.

[1864] The generated answer is sent to the terminal and provided to the user.

[1865] 3. How the Emotional Engine Works:

[1866] The server analyzes the user's emotions using an emotion engine (e.g., a general emotion analysis tool) and identifies the user's emotional state from the voice and text.

[1867] The emotion engine's analysis results are reflected in the generative AI model, which then adjusts the tone and content of the response. For example, if the user is feeling angry, the response will be generated in a more polite and calm tone.

[1868] 4. Feedback Processing:

[1869] The server receives feedback sent from the device and stores it in a database, which is used to update the generative AI model and improve the accuracy of the system.

[1870] Terminal

[1871] The terminal provides an interface for the user to access customer support.

[1872] 1. Display the user interface:

[1873] The device displays a customer support interface, including a contact form, a chat box, and a voice input option.

[1874] As initial data, store opening hours and campaign information are obtained from the server and displayed.

[1875] 2. Submitting and viewing inquiries:

[1876] Validate the query entered by the user and send it to the server. Only queries that pass validation are sent.

[1877] Receives the response from the server and displays it to the user, displaying all relevant information on the screen for the user to review.

[1878] A feedback input interface is provided to receive evaluations and opinions from users and transmit them to the server.

[1879] User

[1880] Users submit queries through the system and provide feedback on the answers provided.

[1881] 1. Enter your inquiry:

[1882] The user enters a question or inquiry through the terminal interface, for example, "Please tell me the status of reservation number 12345."

[1883] If necessary, you can also input voice and upload images.

[1884] 2. Review answers and provide feedback:

[1885] The user reviews the answers provided and evaluates whether the problem has been resolved.

[1886] If necessary, enter your feedback to help improve the system. For example, enter "Thank you for your quick response."

[1887] Specific examples

[1888] Let us take the example of a hotel reservation confirmation scenario.

[1889] 1. Booking confirmation inquiries:

[1890] User: Enter "Please tell me the status of reservation number 12345" into the smartphone app.

[1891] Terminal: Send this input to the server.

[1892] 2. Server processing:

[1893] The server analyzes the received inquiry and inputs it into a generation AI model for reservation confirmation categories.

[1894] The server searches the database for information related to reservation number 12345 and retrieves it.

[1895] The server uses an emotion engine to analyze the user's emotional state, for example, sensing tension or anxiety from the text.

[1896] The server will then adjust the answer based on the analysis results, for example, generating a response such as "Don't worry, reservation number 12345 is scheduled to check in on 2023-11-01."

[1897] The server sends the generated response to the terminal.

[1898] 3. Providing answers:

[1899] The terminal displays the response from the server to the user.

[1900] User: Enters feedback such as "Thank you for your quick response" and submits.

[1901] The terminal sends the feedback to the server.

[1902] In this way, by using a generative AI model and an emotion engine, the system of the present invention can provide appropriate and consistent answers that correspond to the user's emotions, enabling a high level of automation in customer support operations. This improves user satisfaction and reduces the workload on personnel. Furthermore, the system can constantly evolve based on feedback, improving its problem-solving capabilities.

[1903] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1904] Step 1: Server - Initialize the system and load the AI ​​model

[1905] The server loads the generative AI model at startup, specifically by loading and initializing the generative AI model's weight and configuration files, as well as related resources.

[1906] Input: Server initialization command

[1907] Data processing: Loading and initializing the generative AI model and reading relevant files into memory.

[1908] Output: Initialized generative AI model

[1909] Action: The server logs that the generative AI model has finished loading.

[1910] Step 2: Server - Check database connection

[1911] The server connects to the database (e.g., a common database system) and verifies that the necessary tables and indexes are set up correctly.

[1912] Input: Database connection information

[1913] Data Processing: Checking the database connection and settings

[1914] Output: Successful connection and confirmation result

[1915] Action: The server verifies that the database connection is successful and logs that there are no errors.

[1916] Step 3: Terminal - Initial Display of User Interface

[1917] The device displays a customer support interface to the user, which includes a contact form, a chat box, and a voice input option.

[1918] Input: Initial Data Request

[1919] Data processing: Obtaining initial data from the server

[1920] Output: Initial data to be displayed (e.g. store hours, campaign information)

[1921] Operation: The terminal requests initial data from the server and displays the retrieved data in the interface.

[1922] Step 4: User - Enter your inquiry

[1923] The user inputs a question or inquiry through the terminal interface. For example, they might input, "Please tell me the status of reservation number 12345."

[1924] Input: User's inquiry

[1925] Data processing: Validation of input inquiry details

[1926] Output: Validated query content

[1927] How it works: The user enters a query, which is validated by the terminal.

[1928] Step 5: Device - Submit your inquiry

[1929] The terminal transmits the inquiry content that passes validation to the server.

[1930] Input: Validated inquiry content

[1931] Data processing: Sending inquiry details to the server

[1932] Output: The query sent to the server

[1933] Operation: The terminal sends the user's query to the server in the appropriate format.

[1934] Step 6: Server - Parse and categorize the query

[1935] The server analyzes the received inquiry and categorizes it into an appropriate category (e.g., reservation confirmation, troubleshooting, facility information, etc.).

[1936] Input: Received inquiry content

[1937] Data processing: Analysis and categorization using natural language processing

[1938] Output: Inquiry content with identified category

[1939] How it works: The server analyzes the query and passes the classification results to the generative AI model.

[1940] Step 7: Server - Retrieving Information from the Database

[1941] The server retrieves any additional information needed from the database based on the query, for example, searching for and retrieving reservation information related to the reservation number.

[1942] Input: Category-specific inquiry content

[1943] Data processing: Information retrieval and retrieval from databases

[1944] Output: Relevant information obtained

[1945] What happens: The server performs a database query to retrieve the required information.

[1946] Step 8: Server - Answer Generation

[1947] The server uses a generative AI model to generate the optimal answer based on the additional information obtained.

[1948] Input: Acquired relevant information and a generative AI model

[1949] Data processing: Answer generation using generative AI models

[1950] Output: The generated answer

[1951] How it works: The server uses a generative AI model to generate answers to user queries.

[1952] Step 9: Server - Sentiment Analysis and Response Adjustment

[1953] The server uses an emotion engine to analyze the user's emotions and adjust the tone and content of the response accordingly.

[1954] Input: User's query and generated answer

[1955] Data processing: sentiment analysis and response adjustment

[1956] Output: Adjusted answer

[1957] How it works: The server adjusts the response based on the analysis results of the emotion engine.

[1958] Step 10: Server - Sending the response to the device

[1959] The server sends the generated response to the terminal.

[1960] Input: Adjusted Answer

[1961] Data processing: sending adjusted responses

[1962] Output: Answer sent to terminal

[1963] Operation: The server sends the adjusted response to the terminal and records the transmission log.

[1964] Step 11: Terminal - View answers and receive feedback

[1965] The terminal displays the response from the server to the user and provides a feedback input interface.

[1966] Input: Response from the server

[1967] Data processing: Displaying answers and accepting feedback

[1968] Output: User feedback

[1969] Action: The device displays the answer for the user to review and accepts feedback.

[1970] Step 12: User - Review answers and provide feedback

[1971] Users review the answers provided and provide feedback to help improve the system.

[1972] Input: Provided Answer

[1973] Data processing: Feedback input

[1974] Output:Completed feedback

[1975] Action: The user reviews the answer, enters feedback, and sends it to the device.

[1976] Step 13: Device - Send Feedback

[1977] The terminal validates the feedback entered by the user and sends it to the server.

[1978] Input: User-entered feedback

[1979] Data Processing: Feedback validation and submission

[1980] Output: Feedback sent to the server

[1981] Action: The device validates the feedback and sends it to the server.

[1982] Step 14: Server - Receiving and storing feedback

[1983] The server receives the feedback sent from the terminal and stores it in a database.

[1984] Input: Feedback sent from the device

[1985] Data processing: receiving feedback and storing it in a database

[1986] Output: Saved feedback

[1987] How it works: The server stores the received feedback in a database and uses it to update the generative AI model in the future.

[1988] This enables the entire system to function, providing quick and appropriate answers to user inquiries, improving user satisfaction and operational efficiency.

[1989] (Application example 2)

[1990] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1991] Inquiries and troubles at logistics centers require quick and appropriate responses, but because they rely on human labor, responses can be delayed and lack consistency. Furthermore, it can be difficult for staff to respond empathetically based on their emotions, which can reduce user satisfaction. The present invention aims to solve these problems and provide a system that automates efficient and empathetic responses.

[1992] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating a response to an inquiry using a generative AI model, emotion analysis means for analyzing the user's emotions and adjusting the content of the response based on the analysis results, and means for referencing a database and acquiring additional information based on the content of the inquiry. This enables a quick and consistent response, and can improve user satisfaction through empathetic responses.

[1993] A "generative AI model" is an artificial intelligence that uses machine learning algorithms to automatically generate appropriate answers to any inquiry.

[1994] "Server means" refers to a server system that provides the execution environment for the generative AI model and the computational resources for processing the inquiry content and generating and delivering the answer.

[1995] "Terminal means" refers to a device through which a user inputs an inquiry and receives and displays a response from the server means, and includes a smartphone, tablet, or computer.

[1996] "Means for updating generative AI models based on feedback" refers to the process of collecting user ratings and opinions and using them as training data for generative AI models to continuously improve their performance.

[1997] The "emotion analysis means" is a function for analyzing the emotions of a user from text or voice and adjusting the content of the response generated according to that emotional state.

[1998] "Categorization" is the process of analyzing the inquiry and classifying it into a specific category (e.g., inventory check, delivery status, problem report, etc.).

[1999] "Means for referencing a database and obtaining additional information" refers to the process of searching a database based on the inquiry content and obtaining the necessary additional information (e.g., stock status, delivery details, etc.).

[2000] This section describes in detail an embodiment of the present invention. This system is designed to automate inquiries and troubleshooting at logistics centers and provide efficient and empathetic responses. Specifically, this system uses a robot equipped with a generative AI model and emotion analysis means to handle inquiries.

[2001] Hardware:

[2002] Robot Platform:

[2003] A robot platform is a piece of hardware that physically moves around a logistics center and receives and responds to inquiries from staff. Representative examples include Pepper and NAO.

[2004] Camera and Microphone:

[2005] The robot is equipped with a camera and microphone, which are used to analyze the user's emotions from their facial expressions and voice.

[2006] display:

[2007] The robot is equipped with a display that is used to visually display the answers generated by the generative AI model.

[2008] software:

[2009] Generative AI models:

[2010] A generative AI model is software that generates appropriate answers to queries. A typical example is a machine learning algorithm such as GPT-4.

[2011] Emotion analysis means:

[2012] The emotion analysis tool is software that analyzes the user's emotions and adjusts the response content based on that state. Tools such as Emotion API and IBM Watson Tone Analyzer are used.

[2013] Database:

[2014] A database is a data storage for storing additional information required for inquiries, such as inventory information at a logistics center, delivery status, etc. Typical examples include MySQL and PostgreSQL.

[2015] Robot control software:

[2016] Robot control software is a platform for controlling robots, receiving queries from users, and processing data. A typical example is ROS (Robot Operating System).

[2017] Data processing and calculation:

[2018] Server Action:

[2019] The server generates answers using a generative AI model, analyzes emotions, and performs database lookups. Details are explained below.

[2020] 1. Startup and initialization:

[2021] When the server starts up, it loads the generative AI model and sentiment analysis method, connects to the database, checks the model weight file, configuration file, necessary indexes and tables, and loads login information, reservation data, etc.

[2022] 2. Inquiry reception and analysis:

[2023] When a user (staff member or customer) sends an inquiry to the robot, the content is transferred to the server, which analyzes the content of the inquiry and classifies it into categories (e.g., inventory check, delivery status, trouble report).

[2024] 3. Emotion analysis:

[2025] The server uses emotion analysis means to analyze the user's emotions, for example, to identify the user's anxiety or anger from the text and voice data.

[2026] 4. Answer generation and additional information acquisition:

[2027] The server uses a generative AI model to generate an appropriate response based on the analyzed inquiry content and emotion data, and retrieves additional information from the database (e.g., stock availability, delivery details, etc.) as needed to reflect the response.

[2028] 5. Provide answers:

[2029] The generated answers are provided to the user via the robot, either in voice or text format, and are also displayed on the robot's display.

[2030] Examples:

[2031] 1. Inventory Check Scenario

[2032] Staff: "Please let me know the stock status of this item."

[2033] Robot: Converts speech to text and feeds it into a generative AI model of inventory check categories.

[2034] Server: Obtains inventory information from the database and uses emotion analysis to analyze staff emotions as "worried."

[2035] Server: Generates a response and provides it to the robot: "Don't worry, we have plenty of this item in stock."

[2036] 2. Trouble Reporting Scenario

[2037] Staff: "The delivery is delayed, what's going on?"

[2038] Robot: Converts speech to text and feeds it into a generative AI model of trouble report categories.

[2039] Server: Obtains delivery status from the database and analyzes staff emotion as "anger" using emotion analysis means.

[2040] Server: "We apologize for the delay. We are currently investigating the cause of the delivery delay and will let you know the results shortly." This is the generated response and provided by the robot.

[2041] Prompt Sentence Examples

[2042] User: "What is the stock status of this item?"

[2043] Sentiment Analysis: "Worried"

[2044] Generative AI model output: "Don't worry, we have plenty of this item in stock."

[2045] In this way, by using generative AI models and emotion analysis means, embodiments of the present invention can automate inquiry responses at logistics centers and provide fast and empathetic service.

[2046] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2047] Step 1:

[2048] When the server starts up, it loads the generative AI model and sentiment analysis means and connects to the database. The server reads the model weight file and configuration file and checks whether the necessary tables and indexes are set up correctly. The inputs include the model weight file, configuration file, and database connection information, and the output is that the system will start operating normally by initializing the generative AI model and sentiment analysis means based on these.

[2049] Step 2:

[2050] The user inputs a query into the terminal (robot). The input includes the query content in voice or text format. The robot converts the voice into text and sends the query content as text data to the server. The query content is transferred to the server as output.

[2051] Step 3:

[2052] The server analyzes the received inquiry and classifies it into categories. Specifically, the text data is input into a string analysis algorithm, which classifies it into categories such as "inventory check," "delivery status," and "trouble report." The input is the text of the inquiry, and the output is the data classified into categories. This classification data is used in the next step.

[2053] Step 4:

[2054] The server uses the emotion analysis means to analyze the user's emotions. The input includes the text data of the inquiry received earlier. The emotion analysis means identifies the emotional state (e.g., worry, anger, joy, etc.) from the text and outputs the result. The output is the emotional data obtained by the emotion analysis means.

[2055] Step 5:

[2056] The server generates an appropriate answer using a generative AI model based on the analyzed query content and emotional data. The input is classified category data and emotional data, which are fed into the generative AI model to generate an answer in natural language format. The output is the generated answer.

[2057] Step 6:

[2058] If necessary, the server retrieves additional information related to the query from the database. The input is the query and category data, and based on this, it executes a database query to retrieve the target data (e.g., inventory information, delivery status, etc.). The output is the retrieved additional information.

[2059] Step 7:

[2060] The server integrates additional information into the generated answer and sends the final answer to the robot terminal. The input is the integrated answer data, and the output is the final answer sent to the robot terminal.

[2061] Step 8:

[2062] The robot terminal provides the answer received from the server to the user. Specifically, it displays or plays back the answer in text or audio format. The input is the answer data received from the server, and the output is the answer provided to the user.

[2063] Step 9:

[2064] The user checks the provided answers and enters feedback if necessary. The feedback is sent back to the server and used as update data for the generative AI model. The input is the feedback data from the user, and the output is the updated generative AI model.

[2065] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2066] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2067] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2068] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2069] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2070] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2071] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2072] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2073] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2074] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2075] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2076] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2077] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2078] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2079] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in ...

Claims

1. A server means for generating an answer to a query using a generative AI model; terminal means for sending inquiries to the server means and receiving replies; means for updating a generative AI model based on feedback obtained from the terminal means; A system including:

2. 2. The system according to claim 1, further comprising means for analyzing and categorizing the content of the inquiry.

3. 2. The system according to claim 1, wherein said server means further comprises means for referencing a database based on the contents of the inquiry and acquiring additional information.

Images