System

Abstract

A system is provided.SOLUTION: A system comprising: means for allowing a virtual assistant to automatically appear on a talk screen when a user inputs a specific keyword on a message app; means for allowing the virtual assistant to arrange a schedule of a participant; means for allowing the virtual assistant to hear preferences of the participant and search for a store via the Internet; and means for allowing the virtual assistant to suggest a candidate store and automatically make a reservation based on a selection made by the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, users want to easily schedule social gatherings such as drinking parties, but often spend time and effort on tedious organizing tasks such as coordinating dates, selecting restaurants, and making reservations. Even if people say "let's go out for drinks" as a social courtesy, the drinking party often doesn't actually happen. To solve this situation, a system that allows users to easily organize drinking parties without the hassle is needed. [Means for solving the problem]

[0005] The present invention provides a system in which, when a user inputs a specific keyword into a messaging app, a virtual assistant automatically appears and handles all of the arrangements for a drinking party. Specifically, the system includes a means for the user to input a specific keyword, such as "Let's go drinking!", and a means for the virtual assistant to appear on the chat screen. The virtual assistant has a means for arranging the schedule of participants, listening to the participants' preferences, searching for restaurants via the Internet, proposing the most suitable restaurant, and automatically making a reservation based on the user's selection. This provides a system that allows a user to organize a drinking party without going through any complicated procedures, simply by inputting "Let's go drinking!"

[0006] A "messaging app" is a software platform that allows users to send and receive messages, including text, images, and audio.

[0007] A "specific keyword" is a specific character string that is preset and entered by the user to activate the virtual assistant.

[0008] A "virtual assistant" is artificial intelligence software that automatically handles specific tasks through interaction with the user.

[0009] "Scheduling" is the process of checking the schedules of multiple users and determining the best date and time.

[0010] "Searching for stores via the Internet" refers to the process of gathering store information that meets specific criteria from online databases and review sites.

[0011] "Means for virtual assistants to appear on the chat screen" refers to a mechanism that automatically displays a virtual assistant on the messaging app screen when a user enters a specific keyword.

[0012] "Means for suggesting stores" is the process by which the virtual assistant selects several candidates from the store information collected and presents them to the user.

[0013] The "means of making a reservation" is a mechanism by which the virtual assistant automatically confirms a reservation at the selected store based on information such as the date, time, and number of people. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention provides a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app, and automatically handles all steps from scheduling appointments to selecting a restaurant and making a reservation. This system operates as follows.

[0036] Explanation of program processing

[0037] User-entered keywords

[0038] A user types a specific keyword, such as "Let's go for a drink!", through a messaging app.

[0039] The server receives and analyzes this keyword through the messaging app's API.

[0040] The rise of virtual assistants

[0041] When the server recognizes a specific keyword, it triggers the appearance of a virtual assistant.

[0042] The server launches a virtual assistant on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules of the participants."

[0043] Schedule adjustment

[0044] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th."

[0045] The server receives this schedule information and stores it in a database.

[0046] The server sends similar messages to other participants to gather schedule information, for example, "Please tell me your free dates."

[0047] When the user (other participant) responds, the server also receives that information and stores it in the database.

[0048] The server processes the schedule information of all participants and determines the best time and date.

[0049] The server notifies the terminal of the optimal schedule.

[0050] Shop selection

[0051] The AI ​​will display on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0052] The user types, "Izakaya is good."

[0053] The server receives the information and searches for izakayas in the specified area via the Internet.

[0054] The server uses the review site's API to select multiple candidates and send that information to the device.

[0055] The AI ​​will display on the device, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0056] Choosing a restaurant and making a reservation

[0057] The user selects "2. Shop B is better."

[0058] The server receives this selection information and stores it in a database.

[0059] The server makes a reservation at store B using the reservation API.

[0060] The server sends a reservation completion message to the terminal and displays detailed information.

[0061] Specific examples

[0062] For example, if User A types "Let's go for drinks!" into a messaging app, the server responds immediately and a virtual assistant appears. The virtual assistant then displays, "Hello, let's go for drinks! This is AI. Let's first arrange the dates for the participants." User A enters a date, such as "May 10th after 7 PM," and the optimal date is determined. The virtual assistant then asks, "What are your preferences, such as izakaya, bar, or yakiniku?" and the user answers, "izakaya." The virtual assistant then searches for possible izakayas via the internet and automatically makes a reservation at the restaurant selected by the user.

[0063] In this way, users can set up a drinking party without going through any complicated procedures, simply by typing "Let's go drinking!"

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] A user types "Let's go for a drink!" into a messaging app. This message is sent to the server via the messaging app's API.

[0067] Step 2:

[0068] The server analyzes the received message and recognizes the specific keyword "Let's go for a drink!"

[0069] Step 3:

[0070] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0071] Step 4:

[0072] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0073] Step 5:

[0074] The user inputs the desired date and time into the virtual assistant. For example, they might input "I'm available after 7 PM on May 10th."

[0075] Step 6:

[0076] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[0077] Step 7:

[0078] The server receives the responses of other participants and also stores their itinerary information in the itinerary database. Once all participants' itineraries are collected, the server analyzes the data to determine the optimal itinerary.

[0079] Step 8:

[0080] The server determines the optimal schedule and sends that information to the terminal, which then notifies the terminal of the optimal schedule.

[0081] Step 9:

[0082] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0083] Step 10:

[0084] The user types, "Izakaya is good."

[0085] Step 11:

[0086] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[0087] Step 12:

[0088] The server selects some of the candidates it has obtained and sends that information to the terminal.

[0089] Step 13:

[0090] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0091] Step 14:

[0092] The user selects "2. Shop B is better."

[0093] Step 15:

[0094] The server receives this selection information and stores it in a database.

[0095] Step 16:

[0096] The server uses the reservation API to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[0097] Step 17:

[0098] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[0099] Step 18:

[0100] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[0101] Step 19:

[0102] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[0103] This allows users to easily set up drinking parties.

[0104] Example 1

[0105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] The conventional process of scheduling and reserving a restaurant using a messaging app requires users to manually perform multiple steps, which is time-consuming and inefficient.There is a need for a system that can greatly improve user convenience by automating the entire process, from scheduling to selecting a restaurant and making a reservation, simply by entering specific keywords.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0108] In this invention, the server includes a means for a virtual assistant to automatically appear on the chat screen when a user inputs a specific keyword in a messaging app, a means for the virtual assistant to arrange schedules for participants, a means for the virtual assistant to hear the preferences of participants, search for stores via the Internet, and present multiple candidates, a means for the virtual assistant to automatically reserve a store based on the user's selection, and a means for the virtual assistant to send a reservation completion notification to the user's terminal. This automates a series of procedures from scheduling to reserving a store simply by inputting a specific keyword, making it possible to significantly reduce the user's effort and time.

[0109] A "user" is an entity that uses a messaging app to input specific keywords and make various reservations and adjustments through a virtual assistant.

[0110] A "messaging app" is a software application that enables users to send and receive text messages.

[0111] A "specific keyword" is a trigger word that a user can enter in the messaging app to activate the virtual assistant.

[0112] A "virtual assistant" is an artificial intelligence system that automates tasks such as scheduling, selecting stores, and making reservations based on user input.

[0113] The "talk screen" is the display area where messages are sent and received in a messaging app.

[0114] "Schedule adjustment" is the process of collecting the desired dates of multiple participants and determining the optimal date and time.

[0115] A "participant" is another individual who participates in a particular event (such as a drinking party) with the user.

[0116] The "Internet" is an information and communications network that connects computers and networks around the world.

[0117] "Store" refers to a commercial facility, restaurant, etc. that a user selects to visit.

[0118] "Candidates" are multiple store options that the virtual assistant suggests to the user.

[0119] "User selection" refers to the act of the user selecting a store from the candidates presented by the virtual assistant.

[0120] "Reservation" refers to securing a seat or service in advance at a store selected by the user.

[0121] "Notification" is the act of a virtual assistant sending information to a user's terminal.

[0122] A "terminal" is a device (e.g., a smartphone, tablet, or PC) on which a user uses a messaging app.

[0123] This invention is a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app, and automatically handles all steps from scheduling appointments to selecting a restaurant and making a reservation. This system is realized by a server, a user terminal, a messaging app, and virtual assistant software.

[0124] Explanation of program processing

[0125] User-entered keywords

[0126] A user enters a specific keyword, such as "Let's go for a drink!", through a messaging app (e.g., a general chat app). This message is sent to the server through the messaging app's API. The server receives and analyzes the keyword.

[0127] The rise of virtual assistants

[0128] When the server recognizes a specific keyword, it issues an instruction to trigger the appearance of the virtual assistant. The server requests the device to start the virtual assistant. Using virtual assistant software (e.g., Dialogflow), a message is displayed on the user's device saying, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants."

[0129] Schedule adjustment

[0130] The user inputs the desired date and time into the virtual assistant. For example, "I'm free after 7 PM on May 10th." The server receives this schedule information and stores it in a database (e.g., MySQL, PostgreSQL). The server then sends a similar message to other participants to collect their schedule information. For example, it sends a message saying, "Please tell me your free dates." When other participants respond, the server also receives that information and stores it in the database. The server aggregates the schedule information of all participants and determines the optimal date and time.

[0131] Shop selection

[0132] The server, through a virtual assistant, displays on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc." The user enters, "Izakaya is good." The server receives this information and searches for izakayas in the specified area using the Google Maps API, Yelp API, etc. The server uses review sites and store information APIs (such as Gurunavi and Hot Pepper) to generate a list of multiple candidates with high satisfaction rates, and sends this information to the user's device. The device then displays, "How about the following izakayas? 1. Store A 2. Store B 3. Store C."

[0133] Choosing a restaurant and making a reservation

[0134] The user selects "2. Restaurant B is good." The server saves this selection information in the database. The server uses the reservation API to reserve Restaurant B. The server sends a reservation completion message to the terminal and displays detailed information. This message includes the details, "Reservation for Restaurant B has been completed. The date and time is May 10th at 7 PM."

[0135] Specific examples

[0136] For example, if User A types "Let's go for a drink!" into a messaging app, the server receives and analyzes the keyword, and the virtual assistant launches. The process then automatically adjusts the date, selects a restaurant, and makes a reservation. This allows User A to get everything ready with just a few inputs.

[0137] Prompt Sentence Examples

[0138] You can simulate the functionality of a virtual assistant by inputting the following prompt sentences into a generative AI model (e.g., ChatGPT):

[0139] The user types "Let's go out for drinks!" into a messaging app. As a virtual assistant, you should display a message saying "Hello, this is Let's go out for drinks! AI. Let's first arrange the dates for the participants." After that, collect the participants' preferred dates and times, decide on the optimal date, and then proceed with selecting a restaurant and making a reservation.

[0140] This prompt can then be followed by a response or next step from the virtual assistant.

[0141] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0142] Step 1:

[0143] A user uses a messaging app to input a specific keyword, such as "Let's go for a drink!" The input keyword is sent to the server through the messaging app's API. The server receives the keyword and uses a text analysis algorithm to analyze it. As an output, the analysis results generate a flag that triggers the launch of the virtual assistant.

[0144] Step 2:

[0145] The server issues an instruction to start the virtual assistant. It sends message data to the device to display, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants." The device displays this message to the user. It uses virtual assistant software (e.g., Dialogflow) to output the message in text format.

[0146] Step 3:

[0147] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th." This date and time information is sent to the server and saved in a database. The server receives this as input and generates a message template to send similar messages to other participants. As output, it sends a message waiting for input from other participants.

[0148] Step 4:

[0149] The server receives the schedule information of other participants and stores it in a database. It aggregates the schedule information of all participants and runs an algorithm to calculate the optimal date and time. This algorithm takes participant response data as input and performs duplication and optimization. The determined optimal date and time is generated as output.

[0150] Step 5:

[0151] The server notifies the device of the optimal date and displays a message to the user through a virtual assistant saying, "Please tell us your preferences. For example, izakaya, bar, yakiniku, etc." The device displays this message to the user, who then inputs their preferences. The server receives this preference information as input.

[0152] Step 6:

[0153] The server searches for izakayas in a specified area via the Internet using the Google Maps API, Yelp API, etc. The user's preferences and local information are used as input. The server collects highly rated restaurant candidates through review sites and restaurant information APIs, generates a list of candidates, and sends it to the device. The output is a list containing multiple restaurant candidates.

[0154] Step 7:

[0155] The terminal displays "Would you like to try one of the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C," and the user selects "2. Restaurant B." The selection information is sent to the server and saved in the database. The server receives this selection information as input and begins the process of making a reservation for Restaurant B using the reservation API.

[0156] Step 8:

[0157] The server confirms the reservation for Store B through the reservation API, generates a reservation completion message, and sends it to the device. Detailed information such as "Reservation for Store B has been completed. The date and time is May 10th at 7 PM" is displayed on the device. This allows the user to confirm that the reservation has been confirmed.

[0158] Step 9:

[0159] The server sequentially verifies and updates the status of schedule adjustment, store selection, and reservation, and sends notifications to the user's device so that the user can check the progress.

[0160] (Application example 1)

[0161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] Conventional food delivery services require users to go through several cumbersome procedures when using them. Specifically, users must launch the delivery service app, search for the food and restaurant that suits their preferences, and then order and pay separately. Furthermore, users are sometimes not notified of the order status or estimated delivery time, which can be inconvenient for them. Given this current situation, there is a demand for a system that allows users to use delivery services simply.

[0163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0164] In this invention, the server includes: means for a virtual assistant to automatically appear on the chat screen when a user inputs a specific keyword in a messaging app; means for the virtual assistant to arrange schedules for participants; means for the virtual assistant to listen to participants' preferences and search for delivery locations via the Internet; means for the virtual assistant to suggest candidate delivery locations and automatically place orders and reservations based on the user's selection; means for the virtual assistant to prioritize highly rated delivery locations when searching for delivery locations in a specific area using an API of a review site; means for the virtual assistant to use an electronic payment service to process payment for the order; and means for the virtual assistant to notify the user of the completion of the order and the estimated delivery time. This enables users to easily use food delivery services from a messaging app, avoiding cumbersome procedures and keeping track of the situation in real time.

[0165] A "messaging app" is software that users use to send and receive text messages and other information.

[0166] "Specific keywords" are specific words or phrases that users type into messaging apps to indicate specific intent or requests.

[0167] A "virtual assistant" is a software agent that automatically responds to user input and performs specific tasks.

[0168] "Participant scheduling" is the process of checking the available times and days of multiple participants to determine the best date and time.

[0169] "Hearing participant preferences" is the process of asking and collecting the preferences and wishes of users and participants.

[0170] "Searching via the Internet" refers to the act of searching for information or stores through online networks.

[0171] "Search for locations" means searching for service locations such as restaurants and delivery services based on specified conditions.

[0172] "Proposing destination candidates" is a process of presenting multiple options to the user from the search results.

[0173] "Placing an order or reservation based on the user's selection" means automatically placing an order or reservation with the provider selected by the user.

[0174] "Using the API of a rating site" means using the application programming interface provided by other websites that provide ratings and reviews.

[0175] "Preferentially selecting providers with high ratings" means giving priority to selecting service providers with high user ratings and reviews.

[0176] "Using electronic payment services to process payments" means using digital payment methods such as credit cards or electronic money to make online payments.

[0177] "Notifying the user of order completion and estimated delivery time one by one" is a process in which the user is notified of information about the estimated delivery time one by one from the time the order is confirmed.

[0178] This invention is a system in which a virtual assistant automatically appears on the chat screen when a user inputs a specific keyword into a messaging app, and automatically handles everything from scheduling to selecting a delivery destination, placing an order, making a payment, and notifying the user of the estimated delivery time. The following describes in detail an embodiment of the invention.

[0179] Generating a Program

[0180] The system for realizing the present invention consists of a server, a terminal, and a user. The server receives specific keywords from the user and launches the virtual assistant. Specifically, it provides an API endpoint using a lightweight web framework called Flask. It also uses a library called Requests to call external APIs (for example, the API of a rating site or the API of an electronic payment service).

[0181] Natural language explanation of the process

[0182] The server consists of a means to:

[0183] 1. Message reception and analysis: When a user enters a specific keyword (e.g., "Delivery Master, I'm hungry") into the messaging app, the server receives and analyzes the message via API. Depending on the results of this analysis, it launches the virtual assistant.

[0184] 2. Launching a virtual assistant: When the server recognizes a specific keyword, a virtual assistant will automatically appear on the device and interact with the user. For example, if a user types "I want to eat pizza," the assistant will respond, "Hello, this is Delivery Master. Please tell us what you want to eat."

[0185] 3. Search and selection of restaurants: The server obtains the user's location information and searches for restaurants (for example, nearby pizza restaurants). At this time, it uses the API of rating sites such as Tabelog to prioritize restaurants with high ratings.

[0186] 4. Ordering and reservation procedures: The virtual assistant automatically places an order and makes a reservation for the delivery location selected by the user. For example, if the user selects "Pizza Hut Margherita," the server processes the order details and places the order with the delivery location.

[0187] 5. Electronic Payment: The server uses an electronic payment service (e.g., PayPal) to process payment for the order. The server processes the payment securely based on the user's credit card information.

[0188] 6. Status Notification: Once the order is completed, the virtual assistant will notify the user of important information such as the estimated delivery time. For example, "Your order is complete. The estimated delivery time is 7:00 PM."

[0189] Adding specific examples

[0190] For example, if a user types "Delivery Master, I'm hungry" into a messaging app, the following sequence of events will be processed by the server:

[0191] The server recognizes the user's input and the virtual assistant displays, "Hello, this is Delivery Master. Please tell us what you would like to eat."

[0192] If the user answers "pizza," the server searches for the best pizza place via the Internet based on the user's location information.

[0193] It then suggests multiple options to the user (e.g., "Pizza Hut A, Domino's Pizza B, Salvatore C").

[0194] When a user selects "Pizza Hut A, Margherita," the server places the order and processes the payment using PayPal.

[0195] Finally, the virtual assistant announces, "Your order is complete. Estimated delivery time is 7:00 PM."

[0196] Example prompt sentence:

[0197] "Delivery Master, I'm hungry."

[0198] This allows users to easily use food delivery services and eliminates complicated procedures.

[0199] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0200] Step 1:

[0201] The server receives a message in which the user enters a specific keyword, "Delivery Master, I'm hungry," into a messaging app. The server retrieves this message via an API and analyzes the keyword. The input data is the user's message, and the output is the recognition of the specific keyword. Based on this recognition, the launch of the virtual assistant is triggered.

[0202] Step 2:

[0203] The terminal receives instructions from the server, launches the virtual assistant, and displays a message to the user saying, "Hello, this is Delivery Master. Please tell us what you would like to eat." The input here is the instruction from the server, and the output is the message displayed on the terminal.

[0204] Step 3:

[0205] The user responds to the virtual assistant's message with "pizza." This user input is sent to the server via the terminal. The input data is the user's response message and is sent to the server.

[0206] Step 4:

[0207] The server obtains the user's location information based on the dish information received from the user. Using this location information, it searches for restaurants via the Internet. The server calls a rating site API to search for highly rated pizza restaurants in the specified area. The input data is the user's location information and preferred dishes, and the output data is a list of candidate restaurants.

[0208] Step 5:

[0209] The server proposes a list of providers obtained from the results of the rating site API to the user through a virtual assistant. For example, it displays a message such as, "Which would you like: Pizza Hut A, Domino's Pizza B, or Salvatore C?" The input here is a list of candidate providers, and the output is a proposal message to the user.

[0210] Step 6:

[0211] The user selects "Pizza Hut A, Margherita" from the presented list of destinations and inputs it into the virtual assistant. This information is sent to the server via the terminal. The input data is the user's selection information and is sent to the server.

[0212] Step 7:

[0213] The server obtains the user's selection information and places an order with the specified delivery destination. Using the order API, it executes an order for "Pizza Hut A, Margherita" with the delivery destination. The input data is the product information selected by the user, and the output data is the order completion status.

[0214] Step 8:

[0215] After the order is completed, the server processes the payment using an electronic payment service such as PayPal. When the payment is successfully completed, the result is obtained. The input data is the payment information, and the output data is the payment completion status.

[0216] Step 9:

[0217] After the server confirms that the order and payment procedures have been completed, it sends a message to the user through the virtual assistant saying, "The order has been completed. The estimated delivery time is 7:00 PM." The input here is the order and payment completion status, and the output is a notification message to the user.

[0218] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0219] This invention combines a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app and automatically performs a series of processes such as scheduling, selecting a restaurant, and making a reservation, with an emotion engine. The emotion engine has the function of recognizing the user's emotional state and providing appropriate dialogue based on that. This system operates as follows.

[0220] Explanation of program processing

[0221] User-entered keywords

[0222] A user types a specific keyword, such as "Let's go for a drink!", through a messaging app.

[0223] The server receives and analyzes this keyword through the messaging app's API.

[0224] The rise of virtual assistants

[0225] When the server recognizes a specific keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0226] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0227] Emotion Engine Operation

[0228] The server receives the user's message and analyzes the user's emotional state using an emotion engine.

[0229] The emotion engine analyzes the user's emotional state and provides the results to the virtual assistant, for example, identifying whether the user is happy or stressed.

[0230] The virtual assistant on the device will adjust the tone and content of messages to match the user's emotions. For example, if the user is feeling stressed, it will prioritize suggestions for places to relax.

[0231] Schedule adjustment

[0232] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th."

[0233] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[0234] Shop selection

[0235] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0236] The user types, "Izakaya is good."

[0237] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[0238] The server selects some of the candidates it has obtained and sends that information to the terminal.

[0239] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0240] Choosing a restaurant and making a reservation

[0241] The user selects "2. Shop B is better."

[0242] The server receives this selection information and stores it in a database.

[0243] The server uses the reservation API to make a reservation at Restaurant B. The server sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[0244] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[0245] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[0246] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[0247] Specific examples

[0248] For example, if User A types "Let's go out for drinks!" into a messaging app, the server responds immediately and a virtual assistant appears. The virtual assistant then displays, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants." User A enters a date, such as "May 10th after 7 PM," and the optimal date is determined. The virtual assistant then asks, "What are your preferences, such as izakaya, bar, or yakiniku?" and the user answers, "izakaya." The virtual assistant then searches for potential izakayas via the internet and automatically makes a reservation at the restaurant selected by the user. The emotion engine analyzes the user's emotions and makes optimal suggestions, providing a more personalized service.

[0249] In this way, users can set up a drinking party without going through any complicated procedures, simply by typing "Let's go drinking!". In addition, the system makes suggestions and responds based on the user's emotions, resulting in a service that provides a higher level of satisfaction.

[0250] The processing flow will be explained below.

[0251] Step 1:

[0252] A user types "Let's go for a drink!" into a messaging app. This message is sent to the server via the messaging app's API.

[0253] Step 2:

[0254] The server analyzes the received message and recognizes the specific keyword "Let's go for a drink!"

[0255] Step 3:

[0256] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0257] Step 4:

[0258] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0259] Step 5:

[0260] The user inputs the desired date and time into the virtual assistant. For example, they might input "I'm available after 7 PM on May 10th."

[0261] Step 6:

[0262] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[0263] Step 7:

[0264] The server receives the responses of other participants and also stores their itinerary information in the itinerary database. Once all participants' itineraries are collected, the server analyzes the data to determine the optimal itinerary.

[0265] Step 8:

[0266] The server determines the optimal schedule and sends that information to the terminal, which then notifies the terminal of the optimal schedule.

[0267] Step 9:

[0268] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0269] Step 10:

[0270] The user types, "Izakaya is good."

[0271] Step 11:

[0272] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[0273] Step 12:

[0274] The server selects some of the candidates it has obtained and sends that information to the terminal.

[0275] Step 13:

[0276] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0277] Step 14:

[0278] The user selects "2. Shop B is better."

[0279] Step 15:

[0280] The server receives this selection information and stores it in a database.

[0281] Step 16:

[0282] The server uses the reservation API to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[0283] Step 17:

[0284] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[0285] Step 18:

[0286] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[0287] Step 19:

[0288] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[0289] Step 20:

[0290] The server activates an emotion engine to identify emotions from the user's messages and inputs.

[0291] Step 21:

[0292] The server uses an emotion engine to analyze the user's message and detect their emotional state (e.g., joy, stress, frustration, etc.).

[0293] Step 22:

[0294] The virtual assistant on the device will respond appropriately based on the results of emotion analysis. For example, it will suggest a store where a user can relax if they are feeling stressed.

[0295] Step 23:

[0296] The server tracks the user's emotional state and adjusts the tone and content of the message in real time.

[0297] Step 24:

[0298] The user receives suggestions and responses that correspond to their emotions, and the drinking party planning proceeds in an emotionally satisfied state.

[0299] Example 2

[0300] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0301] Currently, when using a messaging app to schedule a drinking party, the tasks of scheduling, selecting a restaurant, and making a reservation are extremely cumbersome and time-consuming. Furthermore, the system does not respond to the user's emotional state, which can lead to a decrease in participant satisfaction. Conventional systems do not offer a method for efficiently resolving these issues. Therefore, the present invention aims to solve these problems.

[0302] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0303] In this invention, the server includes: a means for automatically having a virtual assistant appear on the chat screen when a user inputs a specific keyword into a messaging app; a means for the virtual assistant to arrange schedules for participants; a means for the virtual assistant to listen to participants' preferences and search for restaurants via the Internet; a means for the virtual assistant to suggest candidate restaurants and automatically make reservations based on the user's selection; and a means for analyzing the user's emotional state using emotion analysis technology installed in the system and providing appropriate dialogue based on the analysis results. This allows users to easily and efficiently set up drinking parties using a messaging app and enables responses according to the user's emotional state.

[0304] "User" refers to an individual or group that uses this system.

[0305] "Messaging app" refers to a software application for sending and receiving text and multimedia messages over the Internet.

[0306] A "virtual assistant" refers to a computer program that automatically performs tasks through user interaction.

[0307] A "server" refers to a computer system that provides services to clients over a network.

[0308] "Emotion analysis technology" refers to technology for analyzing a user's emotional state from text or voice data.

[0309] "Schedule adjustment" refers to the process of checking the free times of multiple users and determining the optimal date and time.

[0310] "Store" refers to a place that a user visits, and in this case, it mainly refers to a restaurant.

[0311] "Hearing" refers to the act of collecting information from users.

[0312] "Internet" refers to the global system of computer networks and infrastructure that enables information sharing and communication.

[0313] "Reservation" refers to the act of a user making a reservation in advance to receive a specific service at a specific date and time.

[0314] "Candidates" refers to a list of options that a user can choose from.

[0315] "Analysis" refers to the process of examining data in detail to reveal its meaning and structure.

[0316] "Dialogue" refers to the exchange of information between a user and a virtual assistant.

[0317] This invention is a system in which, when a user inputs a specific keyword into a messaging app, a virtual assistant automatically appears and performs a series of steps such as scheduling, selecting a restaurant, making a reservation, etc. Furthermore, by incorporating emotion analysis technology, this system provides optimal dialogue according to the user's emotional state.

[0318] The system is implemented using the following hardware and software:

[0319] Server: A computer system that provides services to clients over a network, especially cloud services such as Amazon Web Services (AWS).

[0320] Device: Any device used by a user, including internet-enabled devices such as smartphones and tablets.

[0321] Messaging app: A software application for sending and receiving text and multimedia messages, such as LINE or WhatsApp.

[0322] Emotion analysis technology: Technology for analyzing the user's emotional state. For example, IBM Watson Tone Analyzer is used.

[0323] Schedule Database: A database system for storing schedule information. It uses the Google Calendar API.

[0324] Review site API: An API for obtaining store information. For example, the Yelp API is used.

[0325] Reservation API: An API for making restaurant reservations. For example, the OpenTable API is used.

[0326] As a specific example of operation, consider the following scenario.

[0327] User-entered keywords

[0328] When a user types a specific keyword, such as "Let's go for a drink!", through a messaging app, the device sends the input to a server, which receives the keyword through the messaging app's API and analyzes it.

[0329] The rise of virtual assistants

[0330] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device. The virtual assistant then appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0331] How emotion analysis technology works

[0332] The server receives the user's message and uses emotion analysis technology to analyze the user's emotional state. For example, if a user sends a message saying, "I've been busy and stressed lately," the server sends the message to emotion analysis technology and identifies the user as feeling stressed. The virtual assistant on the device adjusts the tone and content of the message to match the user's emotions, displaying, "I'll find you a great izakaya to relieve your stress!"

[0333] Schedule adjustment

[0334] When a user inputs a desired date and time into the virtual assistant, for example, "I'm free after 7 PM on May 10th," the server saves the date and time information in a schedule database and, if necessary, sends similar messages to other participants to collect their schedule information.

[0335] Store selection

[0336] The virtual assistant displays on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc." The user enters, "Izakaya is good." The server receives this information, searches for izakaya in the specified area via the Internet, and retrieves multiple candidates using the review site's API. The server selects several from the retrieved candidates and sends this information to the device. The message displayed is, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0337] Choosing a store and making a reservation

[0338] The user selects "2. Restaurant B is good." The server receives this selection and saves it in a database. The server uses the reservation API to send the necessary reservation information (date and time, number of people, names, etc.). If the reservation is successful, the server receives the reservation confirmation information and also performs error handling. The reservation confirmation information is saved in a database and sent to the terminal. The virtual assistant displays a message on the terminal saying, "Reservation for Restaurant B has been completed. Details are below."

[0339] Example prompts for generative AI models

[0340] "Please tell me the step-by-step process for hosting a drinking party at an izakaya after 7 PM on May 10th. Please include everything from coordinating the schedules of the participants to selecting the restaurant and making the reservation. Please also take into consideration the feelings of the participants."

[0341] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0342] Step 1: User enters specific keywords

[0343] A user types a specific keyword into a messaging app, such as "Let's go for drinks!"

[0344] The terminal sends the input to the application server.

[0345] Input: The keyword "Let's go for a drink!"

[0346] Output: The entered keyword is sent to the server

[0347] Specific actions: User A opens LINE on their smartphone and sends a message to a friend saying, "Let's go for drinks!"

[0348] Step 2: The server receives and parses the keyword

[0349] The server receives the keywords through the messaging app's API and analyzes the content. The analysis process is implemented in Python and JavaScript.

[0350] Input: Message data containing keywords

[0351] Output: Data that determines actions based on keywords

[0352] What it does: The server uses Amazon Web Services (AWS) Lambda to analyze the received message and detect the keyword "Let's go for a drink!"

[0353] Step 3: The virtual assistant arrives

[0354] When the server recognizes a specific keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0355] A virtual assistant appears on the device and displays an initial message.

[0356] Input: Keyword analysis results

[0357] Output: Virtual assistant trigger data, initial message

[0358] Specific operation: The server generates data and displays the AI ​​assistant on User A's smartphone. The message displayed is, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants."

[0359] Step 4: Analyze user emotions using emotion analysis technology

[0360] The server receives the user's additional message and analyzes the user's emotional state using emotion analysis technology, such as IBM Watson Tone Analyzer.

[0361] Input: Message from the user (e.g., "I've been busy and stressed lately.")

[0362] Output: Sentiment analysis result (e.g. "I feel stressed")

[0363] What it does: The server sends an additional message to the emotion analysis technology, identifying it as stressed.

[0364] Step 5: Adjust the dialogue based on the sentiment results

[0365] The device receives the emotion analysis results, and the virtual assistant responds appropriately, adjusting the tone and message content according to the emotion.

[0366] Input: Sentiment analysis results

[0367] Output: Adjusted message content

[0368] Specific operation: The virtual assistant displays a message on the device saying, "I'll find you a great izakaya to relieve your stress!"

[0369] Step 6: Propose a schedule

[0370] The user inputs the desired date and time into the virtual assistant. For example, "I'm available after 7 PM on May 10th."

[0371] The server receives the schedule information and stores it in a schedule database (e.g., Google Calendar API). It also sends similar messages to other participants to collect their schedule information.

[0372] Input: Date information

[0373] Output: Date information stored in the database, messages sent to other participants

[0374] Specific behavior: User A enters "I'm free after 7pm on May 10th," and the server saves this to the Google Calendar API.

[0375] Step 7: Store Selection

[0376] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0377] The user types, "Izakaya is good."

[0378] The server receives the information and searches for izakayas in the specified area via the Internet, using the API of a review site (e.g., Yelp API) to obtain multiple candidates.

[0379] Input: Type of store (e.g. "Izakaya")

[0380] Output: Multiple store candidates

[0381] Specific operation: The user enters "izakaya" (Japanese pub) on their device, and the information is sent to the server. The server uses the Yelp API to search for izakayas, selects "Store A," "Store B," or "Store C," and sends the information back to the user's device.

[0382] Step 8: Virtual assistant suggests stores

[0383] The virtual assistant will display on the device, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0384] Input: Multiple store candidates

[0385] Output: List of store candidates

[0386] Specific operation: A list of "1. Store A," "2. Store B," and "3. Store C" is displayed on the user's device.

[0387] Step 9: User selects store

[0388] The user selects "2. Shop B is better."

[0389] The server receives this selection information and stores it in a database.

[0390] Input: Selected store information

[0391] Output: Selections stored in a database

[0392] Specific operation: The user enters "2. Shop B is good," and the server saves the data in the database.

[0393] Step 10: Server reserves the store

[0394] The server uses a reservation API (e.g., OpenTable API) to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[0395] If the reservation is successful, the server receives the reservation confirmation information, performs error handling, saves the reservation confirmation information in the database, and sends the information to the terminal.

[0396] Input: Selected store information, reservation information

[0397] Output: Booking confirmation information, error message (if necessary)

[0398] Specific operation: The server makes a reservation for Restaurant B using the OpenTable API, and the reservation is confirmed.

[0399] Step 11: Notify user of reservation confirmation

[0400] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[0401] Input: Confirmed reservation information

[0402] Output: Display a confirmation message

[0403] Specific operation: Detailed information such as "Reservation at Restaurant B completed. Date, time, location, reservation name" will be displayed on the user's device.

[0404] (Application example 2)

[0405] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0406] In modern society, where people are busy every day, coordinating group schedules and making restaurant reservations can be cumbersome and time-consuming. To solve these problems, there is a need for a system that allows users to easily coordinate schedules and make restaurant reservations. Furthermore, there is a need to provide more personalized and comfortable services by responding to each user's emotional state.

[0407] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for automatically causing a virtual agent to appear on the screen when a user inputs a specific keyword into a communication application; means for the virtual agent to arrange schedules for participants; means for the virtual agent to collect participants' preferences and search for stores via a digital network; means for the virtual agent to suggest candidate stores and automatically make reservations based on the user's selection; and means for the virtual agent to recognize the user's emotional state using an emotion analysis engine and provide appropriate dialogue. This allows users to easily arrange schedules and make store reservations without hassle, and to receive personalized services tailored to their emotions, simply by inputting a specific keyword.

[0408] A "communications application" is software that users use to exchange messages, such as text, voice, or images.

[0409] A "virtual agent" is a software agent that interacts with a user and automatically performs various tasks.

[0410] An "emotion analysis engine" is a system for analyzing and identifying a user's emotional state from text or voice.

[0411] A "digital network" refers to a digital communications network such as the Internet, which is a medium for transmitting and receiving data.

[0412] "User's Emotional State" means the momentary or ongoing psychological or emotional state of a User.

[0413] "User preferences" refers to the tastes and preferences that a user exhibits for a particular category or condition.

[0414] "Store candidates" is a list of multiple stores that may be suitable for the user's request.

[0415] "Reservation" refers to the advance procedure to reserve a specific service or location for a specified date and time.

[0416] The system of this invention allows a user to input a specific keyword into a communication application, and a virtual agent appears and automatically executes a series of subsequent tasks. It also uses an emotion analysis engine that recognizes the user's emotional state and adjusts the content of the dialogue accordingly.

[0417] System configuration

[0418] The system consists of three main components: the server, the terminal, and the user. The following is a detailed description of each component.

[0419] User

[0420] A user inputs a specific keyword through a communication application. For example, it is an everyday keyword such as "Let's go for drinks!" or "Want to go to dinner?". The user uses a device such as a smartphone or tablet.

[0421] Terminal

[0422] The terminal functions as an interface with the user, providing a screen for the virtual agent to interact with the user and collecting input from the user. Based on the user's input, the virtual agent then converses with the user to arrange a schedule or select a store.

[0423] server

[0424] The server performs the main processing of the virtual agent and the emotion analysis engine. It receives input from the user and analyzes their emotional state using the emotion analysis engine. This includes the following main processes:

[0425] 1. Keyword analysis: The server detects specific keywords in the user's input and triggers the virtual agent.

[0426] 2. Sentiment Analysis: Identify the user's emotional state using an emotion analysis engine (e.g., TextBlob). Adjust the virtual agent's dialogue accordingly.

[0427] 3. Schedule adjustment: Collects the user's schedule information and automatically selects the optimal schedule.

[0428] 4. Store Selection and Reservation: The virtual agent collects the user's preferences, searches for stores via the Internet, suggests suitable store candidates to the user, and makes a store reservation based on the user's selection.

[0429] Specific examples

[0430] For example, if User A types "Let's go for drinks!" into a communication application, the server recognizes the specific keyword and a virtual agent appears. The virtual agent displays the message, "Hello, let's go for drinks! This is AI. Let's first arrange the schedule for the participants." If User A types "May 10th after 7 PM," the server collects schedule information and determines the optimal date. The virtual agent then asks, "What are your preferences? For example, izakaya, bar, yakiniku, etc." The user responds, "Izakaya." The server searches for potential izakayas via the Internet, and the virtual agent suggests, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C." If the user selects "2. Restaurant B is good," the server makes the reservation, and once the reservation is complete, the virtual agent notifies the user, "Your reservation for Restaurant B has been completed. Details are below."

[0431] Example prompts for generative AI models

[0432] Below are some examples of prompts for generative AI models:

[0433] Analyze the following user messages and generate appropriate dialogue messages depending on the user's emotional state:

[0434] User message: "Tired after work, let's go for a drink!"

[0435] User data: {"date": "After 7 PM on May 10th", "preference": "Izakaya"}

[0436] Example of a generated conversation message:

[0437] """

[0438] Hello, let's go drinking! This is AI. Thank you for your hard work. I found a place where you can relax. How about going to an izakaya after 7pm on May 10th?

[0439] 1. Izakaya A (Rating: 4.3)

[0440] 2. Izakaya B (Rating: 4.5)

[0441] Which restaurant would you like to book?

[0442] """

[0443] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0444] Step 1:

[0445] Input: A user types a specific keyword (e.g., "Let's go for a drink!") into a communication application.

[0446] Operation: The terminal sends the user's input message to the server.

[0447] Output: Messages containing the keyword are sent to the server.

[0448] Step 2:

[0449] Input: The server receives the user's input message.

[0450] How it works: The server analyzes specific keywords (e.g., "Let's go for a drink!") and triggers the virtual agent.

[0451] Output: A virtual agent appears on the terminal and says, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0452] Step 3:

[0453] Input: The user who receives the message from the virtual agent inputs the desired date (e.g., "After 7 PM on May 10th").

[0454] Operation: The terminal sends the user's input schedule to the server.

[0455] Output: The schedule information is sent to the server.

[0456] Step 4:

[0457] Input: The server receives the user's schedule information.

[0458] How it works: The server parses the schedule information and stores it in its internal schedule database. It also sends similar messages to other participants as needed to collect schedule information.

[0459] Output: The optimal date is determined, and the virtual agent displays a message on the terminal saying, "Please tell us your preferences. For example, izakaya, bar, yakiniku, etc."

[0460] Step 5:

[0461] Input: The user inputs the type of store they want (e.g., "izakaya").

[0462] Operation: The terminal sends the user's input information to the server.

[0463] Output: Store preference information is sent to the server.

[0464] Step 6:

[0465] Input: The server receives the user's store preference information.

[0466] How it works: The server searches for establishments (e.g., "izakaya") in a specified area via the Internet, retrieves multiple candidates using the review site's API, analyzes the user's emotional state using a sentiment analysis engine, and selects the best candidate.

[0467] Output: Multiple store candidates (e.g., "Store A," "Store B," and "Store C") are obtained, and the virtual agent displays a message on the terminal saying, "Would you like to try one of the following izakayas? 1. Izakaya A 2. Izakaya B 3. Izakaya C."

[0468] Step 7:

[0469] Input: The user enters the number of the desired store (e.g., "2. Izakaya B").

[0470] Operation: The terminal sends the user's selection information to the server.

[0471] Output: The selection is sent to the server.

[0472] Step 8:

[0473] Input: The server receives the user's selection.

[0474] Operation: The server uses the reservation API to make a reservation at the specified store. It sends the necessary reservation information (e.g., date and time, number of people, names, etc.) to the reservation API. If the reservation is successful, it receives the reservation confirmation information and also performs error handling.

[0475] Output: The reservation confirmation information is saved on the server and sent to the device.

[0476] Step 9:

[0477] Input: The terminal that receives the reservation confirmation information displays a message to the user through the virtual agent.

[0478] Action: The virtual agent displays the message "Your reservation at Store B has been completed. Details below."

[0479] Output: The user is notified that the reservation has been completed.

[0480] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0481] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0482] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0483] [Second embodiment]

[0484] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0485] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0486] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0487] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0488] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0489] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0490] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0491] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0492] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0493] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0494] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0495] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0496] The present invention provides a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app, and automatically handles all steps from scheduling appointments to selecting a restaurant and making a reservation. This system operates as follows.

[0497] Explanation of program processing

[0498] User-entered keywords

[0499] A user types a specific keyword, such as "Let's go for a drink!", through a messaging app.

[0500] The server receives and analyzes this keyword through the messaging app's API.

[0501] The rise of virtual assistants

[0502] When the server recognizes a specific keyword, it triggers the appearance of a virtual assistant.

[0503] The server launches a virtual assistant on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules of the participants."

[0504] Schedule adjustment

[0505] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th."

[0506] The server receives this schedule information and stores it in a database.

[0507] The server sends similar messages to other participants to gather schedule information, for example, "Please tell me your free dates."

[0508] When the user (other participant) responds, the server also receives that information and stores it in the database.

[0509] The server processes the schedule information of all participants and determines the best time and date.

[0510] The server notifies the terminal of the optimal schedule.

[0511] Shop selection

[0512] The AI ​​will display on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0513] The user types, "Izakaya is good."

[0514] The server receives the information and searches for izakayas in the specified area via the Internet.

[0515] The server uses the review site's API to select multiple candidates and send that information to the device.

[0516] The AI ​​will display on the device, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0517] Choosing a restaurant and making a reservation

[0518] The user selects "2. Shop B is better."

[0519] The server receives this selection information and stores it in a database.

[0520] The server makes a reservation at store B using the reservation API.

[0521] The server sends a reservation completion message to the terminal and displays detailed information.

[0522] Specific examples

[0523] For example, if User A types "Let's go for drinks!" into a messaging app, the server responds immediately and a virtual assistant appears. The virtual assistant then displays, "Hello, let's go for drinks! This is AI. Let's first arrange the dates for the participants." User A enters a date, such as "May 10th after 7 PM," and the optimal date is determined. The virtual assistant then asks, "What are your preferences, such as izakaya, bar, or yakiniku?" and the user answers, "izakaya." The virtual assistant then searches for possible izakayas via the internet and automatically makes a reservation at the restaurant selected by the user.

[0524] In this way, users can set up a drinking party without going through any complicated procedures, simply by typing "Let's go drinking!"

[0525] The processing flow will be explained below.

[0526] Step 1:

[0527] A user types "Let's go for a drink!" into a messaging app. This message is sent to the server via the messaging app's API.

[0528] Step 2:

[0529] The server analyzes the received message and recognizes the specific keyword "Let's go for a drink!"

[0530] Step 3:

[0531] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0532] Step 4:

[0533] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0534] Step 5:

[0535] The user inputs the desired date and time into the virtual assistant. For example, they might input "I'm available after 7 PM on May 10th."

[0536] Step 6:

[0537] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[0538] Step 7:

[0539] The server receives the responses of other participants and also stores their itinerary information in the itinerary database. Once all participants' itineraries are collected, the server analyzes the data to determine the optimal itinerary.

[0540] Step 8:

[0541] The server determines the optimal schedule and sends that information to the terminal, which then notifies the terminal of the optimal schedule.

[0542] Step 9:

[0543] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0544] Step 10:

[0545] The user types, "Izakaya is good."

[0546] Step 11:

[0547] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[0548] Step 12:

[0549] The server selects some of the candidates it has obtained and sends that information to the terminal.

[0550] Step 13:

[0551] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0552] Step 14:

[0553] The user selects "2. Shop B is better."

[0554] Step 15:

[0555] The server receives this selection information and stores it in a database.

[0556] Step 16:

[0557] The server uses the reservation API to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[0558] Step 17:

[0559] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[0560] Step 18:

[0561] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[0562] Step 19:

[0563] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[0564] This allows users to easily set up drinking parties.

[0565] Example 1

[0566] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0567] The conventional process of scheduling and reserving a restaurant using a messaging app requires users to manually perform multiple steps, which is time-consuming and inefficient.There is a need for a system that can greatly improve user convenience by automating the entire process, from scheduling to selecting a restaurant and making a reservation, simply by entering specific keywords.

[0568] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0569] In this invention, the server includes a means for a virtual assistant to automatically appear on the chat screen when a user inputs a specific keyword in a messaging app, a means for the virtual assistant to arrange schedules for participants, a means for the virtual assistant to hear the preferences of participants, search for stores via the Internet, and present multiple candidates, a means for the virtual assistant to automatically reserve a store based on the user's selection, and a means for the virtual assistant to send a reservation completion notification to the user's terminal. This automates a series of procedures from scheduling to reserving a store simply by inputting a specific keyword, making it possible to significantly reduce the user's effort and time.

[0570] A "user" is an entity that uses a messaging app to input specific keywords and make various reservations and adjustments through a virtual assistant.

[0571] A "messaging app" is a software application that enables users to send and receive text messages.

[0572] A "specific keyword" is a trigger word that a user can enter in the messaging app to activate the virtual assistant.

[0573] A "virtual assistant" is an artificial intelligence system that automates tasks such as scheduling, selecting stores, and making reservations based on user input.

[0574] The "talk screen" is the display area where messages are sent and received in a messaging app.

[0575] "Schedule adjustment" is the process of collecting the desired dates of multiple participants and determining the optimal date and time.

[0576] A "participant" is another individual who participates in a particular event (such as a drinking party) with the user.

[0577] The "Internet" is an information and communications network that connects computers and networks around the world.

[0578] "Store" refers to a commercial facility, restaurant, etc. that a user selects to visit.

[0579] "Candidates" are multiple store options that the virtual assistant suggests to the user.

[0580] "User selection" refers to the act of the user selecting a store from the candidates presented by the virtual assistant.

[0581] "Reservation" refers to securing a seat or service in advance at a store selected by the user.

[0582] "Notification" is the act of a virtual assistant sending information to a user's terminal.

[0583] A "terminal" is a device (e.g., a smartphone, tablet, or PC) on which a user uses a messaging app.

[0584] This invention is a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app, and automatically handles all steps from scheduling appointments to selecting a restaurant and making a reservation. This system is realized by a server, a user terminal, a messaging app, and virtual assistant software.

[0585] Explanation of program processing

[0586] User-entered keywords

[0587] A user enters a specific keyword, such as "Let's go for a drink!", through a messaging app (e.g., a general chat app). This message is sent to the server through the messaging app's API. The server receives and analyzes the keyword.

[0588] The rise of virtual assistants

[0589] When the server recognizes a specific keyword, it issues an instruction to trigger the appearance of the virtual assistant. The server requests the device to start the virtual assistant. Using virtual assistant software (e.g., Dialogflow), a message is displayed on the user's device saying, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants."

[0590] Schedule adjustment

[0591] The user inputs the desired date and time into the virtual assistant. For example, "I'm free after 7 PM on May 10th." The server receives this schedule information and stores it in a database (e.g., MySQL, PostgreSQL). The server then sends a similar message to other participants to collect their schedule information. For example, it sends a message saying, "Please tell me your free dates." When other participants respond, the server also receives that information and stores it in the database. The server aggregates the schedule information of all participants and determines the optimal date and time.

[0592] Shop selection

[0593] The server, through a virtual assistant, displays on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc." The user enters, "Izakaya is good." The server receives this information and searches for izakayas in the specified area using the Google Maps API, Yelp API, etc. The server uses review sites and store information APIs (such as Gurunavi and Hot Pepper) to generate a list of multiple candidates with high satisfaction rates, and sends this information to the user's device. The device then displays, "How about the following izakayas? 1. Store A 2. Store B 3. Store C."

[0594] Choosing a restaurant and making a reservation

[0595] The user selects "2. Restaurant B is good." The server saves this selection information in the database. The server uses the reservation API to reserve Restaurant B. The server sends a reservation completion message to the terminal and displays detailed information. This message includes the details, "Reservation for Restaurant B has been completed. The date and time is May 10th at 7 PM."

[0596] Specific examples

[0597] For example, if User A types "Let's go for a drink!" into a messaging app, the server receives and analyzes the keyword, and the virtual assistant launches. The process then automatically adjusts the date, selects a restaurant, and makes a reservation. This allows User A to get everything ready with just a few inputs.

[0598] Prompt Sentence Examples

[0599] You can simulate the functionality of a virtual assistant by inputting the following prompt sentences into a generative AI model (e.g., ChatGPT):

[0600] The user types "Let's go out for drinks!" into a messaging app. As a virtual assistant, you should display a message saying "Hello, this is Let's go out for drinks! AI. Let's first arrange the dates for the participants." After that, collect the participants' preferred dates and times, decide on the optimal date, and then proceed with selecting a restaurant and making a reservation.

[0601] This prompt can then be followed by a response or next step from the virtual assistant.

[0602] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0603] Step 1:

[0604] A user uses a messaging app to input a specific keyword, such as "Let's go for a drink!" The input keyword is sent to the server through the messaging app's API. The server receives the keyword and uses a text analysis algorithm to analyze it. As an output, the analysis results generate a flag that triggers the launch of the virtual assistant.

[0605] Step 2:

[0606] The server issues an instruction to start the virtual assistant. It sends message data to the device to display, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants." The device displays this message to the user. It uses virtual assistant software (e.g., Dialogflow) to output the message in text format.

[0607] Step 3:

[0608] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th." This date and time information is sent to the server and saved in a database. The server receives this as input and generates a message template to send similar messages to other participants. As output, it sends a message waiting for input from other participants.

[0609] Step 4:

[0610] The server receives the schedule information of other participants and stores it in a database. It aggregates the schedule information of all participants and runs an algorithm to calculate the optimal date and time. This algorithm takes participant response data as input and performs duplication and optimization. The determined optimal date and time is generated as output.

[0611] Step 5:

[0612] The server notifies the device of the optimal date and displays a message to the user through a virtual assistant saying, "Please tell us your preferences. For example, izakaya, bar, yakiniku, etc." The device displays this message to the user, who then inputs their preferences. The server receives this preference information as input.

[0613] Step 6:

[0614] The server searches for izakayas in a specified area via the Internet using the Google Maps API, Yelp API, etc. The user's preferences and local information are used as input. The server collects highly rated restaurant candidates through review sites and restaurant information APIs, generates a list of candidates, and sends it to the device. The output is a list containing multiple restaurant candidates.

[0615] Step 7:

[0616] The terminal displays "Would you like to try one of the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C," and the user selects "2. Restaurant B." The selection information is sent to the server and saved in the database. The server receives this selection information as input and begins the process of making a reservation for Restaurant B using the reservation API.

[0617] Step 8:

[0618] The server confirms the reservation for Store B through the reservation API, generates a reservation completion message, and sends it to the device. Detailed information such as "Reservation for Store B has been completed. The date and time is May 10th at 7 PM" is displayed on the device. This allows the user to confirm that the reservation has been confirmed.

[0619] Step 9:

[0620] The server sequentially verifies and updates the status of schedule adjustment, store selection, and reservation, and sends notifications to the user's device so that the user can check the progress.

[0621] (Application example 1)

[0622] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0623] Conventional food delivery services require users to go through several cumbersome procedures when using them. Specifically, users must launch the delivery service app, search for the food and restaurant that suits their preferences, and then order and pay separately. Furthermore, users are sometimes not notified of the order status or estimated delivery time, which can be inconvenient for them. Given this current situation, there is a demand for a system that allows users to use delivery services simply.

[0624] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0625] In this invention, the server includes: means for a virtual assistant to automatically appear on the chat screen when a user inputs a specific keyword in a messaging app; means for the virtual assistant to arrange schedules for participants; means for the virtual assistant to listen to participants' preferences and search for delivery locations via the Internet; means for the virtual assistant to suggest candidate delivery locations and automatically place orders and reservations based on the user's selection; means for the virtual assistant to prioritize highly rated delivery locations when searching for delivery locations in a specific area using an API of a review site; means for the virtual assistant to use an electronic payment service to process payment for the order; and means for the virtual assistant to notify the user of the completion of the order and the estimated delivery time. This enables users to easily use food delivery services from a messaging app, avoiding cumbersome procedures and keeping track of the situation in real time.

[0626] A "messaging app" is software that users use to send and receive text messages and other information.

[0627] "Specific keywords" are specific words or phrases that users type into messaging apps to indicate specific intent or requests.

[0628] A "virtual assistant" is a software agent that automatically responds to user input and performs specific tasks.

[0629] "Participant scheduling" is the process of checking the available times and days of multiple participants to determine the best date and time.

[0630] "Hearing participant preferences" is the process of asking and collecting the preferences and wishes of users and participants.

[0631] "Searching via the Internet" refers to the act of searching for information or stores through online networks.

[0632] "Search for locations" means searching for service locations such as restaurants and delivery services based on specified conditions.

[0633] "Proposing destination candidates" is a process of presenting multiple options to the user from the search results.

[0634] "Placing an order or reservation based on the user's selection" means automatically placing an order or reservation with the provider selected by the user.

[0635] "Using the API of a rating site" means using the application programming interface provided by other websites that provide ratings and reviews.

[0636] "Preferentially selecting providers with high ratings" means giving priority to selecting service providers with high user ratings and reviews.

[0637] "Using electronic payment services to process payments" means using digital payment methods such as credit cards or electronic money to make online payments.

[0638] "Notifying the user of order completion and estimated delivery time one by one" is a process in which the user is notified of information about the estimated delivery time one by one from the time the order is confirmed.

[0639] This invention is a system in which a virtual assistant automatically appears on the chat screen when a user inputs a specific keyword into a messaging app, and automatically handles everything from scheduling to selecting a delivery destination, placing an order, making a payment, and notifying the user of the estimated delivery time. The following describes in detail an embodiment of the invention.

[0640] Generating a Program

[0641] The system for realizing the present invention consists of a server, a terminal, and a user. The server receives specific keywords from the user and launches the virtual assistant. Specifically, it provides an API endpoint using a lightweight web framework called Flask. It also uses a library called Requests to call external APIs (for example, the API of a rating site or the API of an electronic payment service).

[0642] Natural language explanation of the process

[0643] The server consists of a means to:

[0644] 1. Message reception and analysis: When a user enters a specific keyword (e.g., "Delivery Master, I'm hungry") into the messaging app, the server receives and analyzes the message via API. Depending on the results of this analysis, it launches the virtual assistant.

[0645] 2. Launching a virtual assistant: When the server recognizes a specific keyword, a virtual assistant will automatically appear on the device and interact with the user. For example, if a user types "I want to eat pizza," the assistant will respond, "Hello, this is Delivery Master. Please tell us what you want to eat."

[0646] 3. Search and selection of restaurants: The server obtains the user's location information and searches for restaurants (for example, nearby pizza restaurants). At this time, it uses the API of rating sites such as Tabelog to prioritize restaurants with high ratings.

[0647] 4. Ordering and reservation procedures: The virtual assistant automatically places an order and makes a reservation for the delivery location selected by the user. For example, if the user selects "Pizza Hut Margherita," the server processes the order details and places the order with the delivery location.

[0648] 5. Electronic Payment: The server uses an electronic payment service (e.g., PayPal) to process payment for the order. The server processes the payment securely based on the user's credit card information.

[0649] 6. Status Notification: Once the order is completed, the virtual assistant will notify the user of important information such as the estimated delivery time. For example, "Your order is complete. The estimated delivery time is 7:00 PM."

[0650] Adding specific examples

[0651] For example, if a user types "Delivery Master, I'm hungry" into a messaging app, the following sequence of events will be processed by the server:

[0652] The server recognizes the user's input and the virtual assistant displays, "Hello, this is Delivery Master. Please tell us what you would like to eat."

[0653] If the user answers "pizza," the server searches for the best pizza place via the Internet based on the user's location information.

[0654] It then suggests multiple options to the user (e.g., "Pizza Hut A, Domino's Pizza B, Salvatore C").

[0655] When a user selects "Pizza Hut A, Margherita," the server places the order and processes the payment using PayPal.

[0656] Finally, the virtual assistant announces, "Your order is complete. Estimated delivery time is 7:00 PM."

[0657] Example prompt sentence:

[0658] "Delivery Master, I'm hungry."

[0659] This allows users to easily use food delivery services and eliminates complicated procedures.

[0660] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0661] Step 1:

[0662] The server receives a message in which the user enters a specific keyword, "Delivery Master, I'm hungry," into a messaging app. The server retrieves this message via an API and analyzes the keyword. The input data is the user's message, and the output is the recognition of the specific keyword. Based on this recognition, the launch of the virtual assistant is triggered.

[0663] Step 2:

[0664] The terminal receives instructions from the server, launches the virtual assistant, and displays a message to the user saying, "Hello, this is Delivery Master. Please tell us what you would like to eat." The input here is the instruction from the server, and the output is the message displayed on the terminal.

[0665] Step 3:

[0666] The user responds to the virtual assistant's message with "pizza." This user input is sent to the server via the terminal. The input data is the user's response message and is sent to the server.

[0667] Step 4:

[0668] The server obtains the user's location information based on the dish information received from the user. Using this location information, it searches for restaurants via the Internet. The server calls a rating site API to search for highly rated pizza restaurants in the specified area. The input data is the user's location information and preferred dishes, and the output data is a list of candidate restaurants.

[0669] Step 5:

[0670] The server proposes a list of providers obtained from the results of the rating site API to the user through a virtual assistant. For example, it displays a message such as, "Which would you like: Pizza Hut A, Domino's Pizza B, or Salvatore C?" The input here is a list of candidate providers, and the output is a proposal message to the user.

[0671] Step 6:

[0672] The user selects "Pizza Hut A, Margherita" from the presented list of destinations and inputs it into the virtual assistant. This information is sent to the server via the terminal. The input data is the user's selection information and is sent to the server.

[0673] Step 7:

[0674] The server obtains the user's selection information and places an order with the specified delivery destination. Using the order API, it executes an order for "Pizza Hut A, Margherita" with the delivery destination. The input data is the product information selected by the user, and the output data is the order completion status.

[0675] Step 8:

[0676] After the order is completed, the server processes the payment using an electronic payment service such as PayPal. When the payment is successfully completed, the result is obtained. The input data is the payment information, and the output data is the payment completion status.

[0677] Step 9:

[0678] After the server confirms that the order and payment procedures have been completed, it sends a message to the user through the virtual assistant saying, "The order has been completed. The estimated delivery time is 7:00 PM." The input here is the order and payment completion status, and the output is a notification message to the user.

[0679] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0680] This invention combines a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app and automatically performs a series of processes such as scheduling, selecting a restaurant, and making a reservation, with an emotion engine. The emotion engine has the function of recognizing the user's emotional state and providing appropriate dialogue based on that. This system operates as follows.

[0681] Explanation of program processing

[0682] User-entered keywords

[0683] A user types a specific keyword, such as "Let's go for a drink!", through a messaging app.

[0684] The server receives and analyzes this keyword through the messaging app's API.

[0685] The rise of virtual assistants

[0686] When the server recognizes a specific keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0687] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0688] Emotion Engine Operation

[0689] The server receives the user's message and analyzes the user's emotional state using an emotion engine.

[0690] The emotion engine analyzes the user's emotional state and provides the results to the virtual assistant, for example, identifying whether the user is happy or stressed.

[0691] The virtual assistant on the device will adjust the tone and content of messages to match the user's emotions. For example, if the user is feeling stressed, it will prioritize suggestions for places to relax.

[0692] Schedule adjustment

[0693] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th."

[0694] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[0695] Shop selection

[0696] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0697] The user types, "Izakaya is good."

[0698] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[0699] The server selects some of the candidates it has obtained and sends that information to the terminal.

[0700] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0701] Choosing a restaurant and making a reservation

[0702] The user selects "2. Shop B is better."

[0703] The server receives this selection information and stores it in a database.

[0704] The server uses the reservation API to make a reservation at Restaurant B. The server sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[0705] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[0706] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[0707] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[0708] Specific examples

[0709] For example, if User A types "Let's go out for drinks!" into a messaging app, the server responds immediately and a virtual assistant appears. The virtual assistant then displays, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants." User A enters a date, such as "May 10th after 7 PM," and the optimal date is determined. The virtual assistant then asks, "What are your preferences, such as izakaya, bar, or yakiniku?" and the user answers, "izakaya." The virtual assistant then searches for potential izakayas via the internet and automatically makes a reservation at the restaurant selected by the user. The emotion engine analyzes the user's emotions and makes optimal suggestions, providing a more personalized service.

[0710] In this way, users can set up a drinking party without going through any complicated procedures, simply by typing "Let's go drinking!". In addition, the system makes suggestions and responds based on the user's emotions, resulting in a service that provides a higher level of satisfaction.

[0711] The processing flow will be explained below.

[0712] Step 1:

[0713] A user types "Let's go for a drink!" into a messaging app. This message is sent to the server via the messaging app's API.

[0714] Step 2:

[0715] The server analyzes the received message and recognizes the specific keyword "Let's go for a drink!"

[0716] Step 3:

[0717] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0718] Step 4:

[0719] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0720] Step 5:

[0721] The user inputs the desired date and time into the virtual assistant. For example, they might input "I'm available after 7 PM on May 10th."

[0722] Step 6:

[0723] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[0724] Step 7:

[0725] The server receives the responses of other participants and also stores their itinerary information in the itinerary database. Once all participants' itineraries are collected, the server analyzes the data to determine the optimal itinerary.

[0726] Step 8:

[0727] The server determines the optimal schedule and sends that information to the terminal, which then notifies the terminal of the optimal schedule.

[0728] Step 9:

[0729] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0730] Step 10:

[0731] The user types, "Izakaya is good."

[0732] Step 11:

[0733] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[0734] Step 12:

[0735] The server selects some of the candidates it has obtained and sends that information to the terminal.

[0736] Step 13:

[0737] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0738] Step 14:

[0739] The user selects "2. Shop B is better."

[0740] Step 15:

[0741] The server receives this selection information and stores it in a database.

[0742] Step 16:

[0743] The server uses the reservation API to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[0744] Step 17:

[0745] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[0746] Step 18:

[0747] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[0748] Step 19:

[0749] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[0750] Step 20:

[0751] The server activates an emotion engine to identify emotions from the user's messages and inputs.

[0752] Step 21:

[0753] The server uses an emotion engine to analyze the user's message and detect their emotional state (e.g., joy, stress, frustration, etc.).

[0754] Step 22:

[0755] The virtual assistant on the device will respond appropriately based on the results of emotion analysis. For example, it will suggest a store where a user can relax if they are feeling stressed.

[0756] Step 23:

[0757] The server tracks the user's emotional state and adjusts the tone and content of the message in real time.

[0758] Step 24:

[0759] The user receives suggestions and responses that correspond to their emotions, and the drinking party planning proceeds in an emotionally satisfied state.

[0760] Example 2

[0761] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0762] Currently, when using a messaging app to schedule a drinking party, the tasks of scheduling, selecting a restaurant, and making a reservation are extremely cumbersome and time-consuming. Furthermore, the system does not respond to the user's emotional state, which can lead to a decrease in participant satisfaction. Conventional systems do not offer a method for efficiently resolving these issues. Therefore, the present invention aims to solve these problems.

[0763] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0764] In this invention, the server includes: a means for automatically having a virtual assistant appear on the chat screen when a user inputs a specific keyword into a messaging app; a means for the virtual assistant to arrange schedules for participants; a means for the virtual assistant to listen to participants' preferences and search for restaurants via the Internet; a means for the virtual assistant to suggest candidate restaurants and automatically make reservations based on the user's selection; and a means for analyzing the user's emotional state using emotion analysis technology installed in the system and providing appropriate dialogue based on the analysis results. This allows users to easily and efficiently set up drinking parties using a messaging app and enables responses according to the user's emotional state.

[0765] "User" refers to an individual or group that uses this system.

[0766] "Messaging app" refers to a software application for sending and receiving text and multimedia messages over the Internet.

[0767] A "virtual assistant" refers to a computer program that automatically performs tasks through user interaction.

[0768] A "server" refers to a computer system that provides services to clients over a network.

[0769] "Emotion analysis technology" refers to technology for analyzing a user's emotional state from text or voice data.

[0770] "Schedule adjustment" refers to the process of checking the free times of multiple users and determining the optimal date and time.

[0771] "Store" refers to a place that a user visits, and in this case, it mainly refers to a restaurant.

[0772] "Hearing" refers to the act of collecting information from users.

[0773] "Internet" refers to the global system of computer networks and infrastructure that enables information sharing and communication.

[0774] "Reservation" refers to the act of a user making a reservation in advance to receive a specific service at a specific date and time.

[0775] "Candidates" refers to a list of options that a user can choose from.

[0776] "Analysis" refers to the process of examining data in detail to reveal its meaning and structure.

[0777] "Dialogue" refers to the exchange of information between a user and a virtual assistant.

[0778] This invention is a system in which, when a user inputs a specific keyword into a messaging app, a virtual assistant automatically appears and performs a series of steps such as scheduling, selecting a restaurant, making a reservation, etc. Furthermore, by incorporating emotion analysis technology, this system provides optimal dialogue according to the user's emotional state.

[0779] The system is implemented using the following hardware and software:

[0780] Server: A computer system that provides services to clients over a network, especially cloud services such as Amazon Web Services (AWS).

[0781] Device: Any device used by a user, including internet-enabled devices such as smartphones and tablets.

[0782] Messaging app: A software application for sending and receiving text and multimedia messages, such as LINE or WhatsApp.

[0783] Emotion analysis technology: Technology for analyzing the user's emotional state. For example, IBM Watson Tone Analyzer is used.

[0784] Schedule Database: A database system for storing schedule information. It uses the Google Calendar API.

[0785] Review site API: An API for obtaining store information. For example, the Yelp API is used.

[0786] Reservation API: An API for making restaurant reservations. For example, the OpenTable API is used.

[0787] As a specific example of operation, consider the following scenario.

[0788] User-entered keywords

[0789] When a user types a specific keyword, such as "Let's go for a drink!", through a messaging app, the device sends the input to a server, which receives the keyword through the messaging app's API and analyzes it.

[0790] The rise of virtual assistants

[0791] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device. The virtual assistant then appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0792] How emotion analysis technology works

[0793] The server receives the user's message and uses emotion analysis technology to analyze the user's emotional state. For example, if a user sends a message saying, "I've been busy and stressed lately," the server sends the message to emotion analysis technology and identifies the user as feeling stressed. The virtual assistant on the device adjusts the tone and content of the message to match the user's emotions, displaying, "I'll find you a great izakaya to relieve your stress!"

[0794] Schedule adjustment

[0795] When a user inputs a desired date and time into the virtual assistant, for example, "I'm free after 7 PM on May 10th," the server saves the date and time information in a schedule database and, if necessary, sends similar messages to other participants to collect their schedule information.

[0796] Store selection

[0797] The virtual assistant displays on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc." The user enters, "Izakaya is good." The server receives this information, searches for izakaya in the specified area via the Internet, and retrieves multiple candidates using the review site's API. The server selects several from the retrieved candidates and sends this information to the device. The message displayed is, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0798] Choosing a store and making a reservation

[0799] The user selects "2. Restaurant B is good." The server receives this selection and saves it in a database. The server uses the reservation API to send the necessary reservation information (date and time, number of people, names, etc.). If the reservation is successful, the server receives the reservation confirmation information and also performs error handling. The reservation confirmation information is saved in a database and sent to the terminal. The virtual assistant displays a message on the terminal saying, "Reservation for Restaurant B has been completed. Details are below."

[0800] Example prompts for generative AI models

[0801] "Please tell me the step-by-step process for hosting a drinking party at an izakaya after 7 PM on May 10th. Please include everything from coordinating the schedules of the participants to selecting the restaurant and making the reservation. Please also take into consideration the feelings of the participants."

[0802] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0803] Step 1: User enters specific keywords

[0804] A user types a specific keyword into a messaging app, such as "Let's go for drinks!"

[0805] The terminal sends the input to the application server.

[0806] Input: The keyword "Let's go for a drink!"

[0807] Output: The entered keyword is sent to the server

[0808] Specific actions: User A opens LINE on their smartphone and sends a message to a friend saying, "Let's go for drinks!"

[0809] Step 2: The server receives and parses the keyword

[0810] The server receives the keywords through the messaging app's API and analyzes the content. The analysis process is implemented in Python and JavaScript.

[0811] Input: Message data containing keywords

[0812] Output: Data that determines actions based on keywords

[0813] What it does: The server uses Amazon Web Services (AWS) Lambda to analyze the received message and detect the keyword "Let's go for a drink!"

[0814] Step 3: The virtual assistant arrives

[0815] When the server recognizes a specific keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0816] A virtual assistant appears on the device and displays an initial message.

[0817] Input: Keyword analysis results

[0818] Output: Virtual assistant trigger data, initial message

[0819] Specific operation: The server generates data and displays the AI ​​assistant on User A's smartphone. The message displayed is, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants."

[0820] Step 4: Analyze user emotions using emotion analysis technology

[0821] The server receives the user's additional message and analyzes the user's emotional state using emotion analysis technology, such as IBM Watson Tone Analyzer.

[0822] Input: Message from the user (e.g., "I've been busy and stressed lately.")

[0823] Output: Sentiment analysis result (e.g. "I feel stressed")

[0824] What it does: The server sends an additional message to the emotion analysis technology, identifying it as stressed.

[0825] Step 5: Adjust the dialogue based on the sentiment results

[0826] The device receives the emotion analysis results, and the virtual assistant responds appropriately, adjusting the tone and message content according to the emotion.

[0827] Input: Sentiment analysis results

[0828] Output: Adjusted message content

[0829] Specific operation: The virtual assistant displays a message on the device saying, "I'll find you a great izakaya to relieve your stress!"

[0830] Step 6: Propose a schedule

[0831] The user inputs the desired date and time into the virtual assistant. For example, "I'm available after 7 PM on May 10th."

[0832] The server receives the schedule information and stores it in a schedule database (e.g., Google Calendar API). It also sends similar messages to other participants to collect their schedule information.

[0833] Input: Date information

[0834] Output: Date information stored in the database, messages sent to other participants

[0835] Specific behavior: User A enters "I'm free after 7pm on May 10th," and the server saves this to the Google Calendar API.

[0836] Step 7: Store Selection

[0837] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0838] The user types, "Izakaya is good."

[0839] The server receives the information and searches for izakayas in the specified area via the Internet, using the API of a review site (e.g., Yelp API) to obtain multiple candidates.

[0840] Input: Type of store (e.g. "Izakaya")

[0841] Output: Multiple store candidates

[0842] Specific operation: The user enters "izakaya" (Japanese pub) on their device, and the information is sent to the server. The server uses the Yelp API to search for izakayas, selects "Store A," "Store B," or "Store C," and sends the information back to the user's device.

[0843] Step 8: Virtual assistant suggests stores

[0844] The virtual assistant will display on the device, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0845] Input: Multiple store candidates

[0846] Output: List of store candidates

[0847] Specific operation: A list of "1. Store A," "2. Store B," and "3. Store C" is displayed on the user's device.

[0848] Step 9: User selects store

[0849] The user selects "2. Shop B is better."

[0850] The server receives this selection information and stores it in a database.

[0851] Input: Selected store information

[0852] Output: Selections stored in a database

[0853] Specific operation: The user enters "2. Shop B is good," and the server saves the data in the database.

[0854] Step 10: Server reserves the store

[0855] The server uses a reservation API (e.g., OpenTable API) to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[0856] If the reservation is successful, the server receives the reservation confirmation information, performs error handling, saves the reservation confirmation information in the database, and sends the information to the terminal.

[0857] Input: Selected store information, reservation information

[0858] Output: Booking confirmation information, error message (if necessary)

[0859] Specific operation: The server makes a reservation for Restaurant B using the OpenTable API, and the reservation is confirmed.

[0860] Step 11: Notify user of reservation confirmation

[0861] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[0862] Input: Confirmed reservation information

[0863] Output: Display a confirmation message

[0864] Specific operation: Detailed information such as "Reservation at Restaurant B completed. Date, time, location, reservation name" will be displayed on the user's device.

[0865] (Application example 2)

[0866] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0867] In modern society, where people are busy every day, coordinating group schedules and making restaurant reservations can be cumbersome and time-consuming. To solve these problems, there is a need for a system that allows users to easily coordinate schedules and make restaurant reservations. Furthermore, there is a need to provide more personalized and comfortable services by responding to each user's emotional state.

[0868] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for automatically causing a virtual agent to appear on the screen when a user inputs a specific keyword into a communication application; means for the virtual agent to arrange schedules for participants; means for the virtual agent to collect participants' preferences and search for stores via a digital network; means for the virtual agent to suggest candidate stores and automatically make reservations based on the user's selection; and means for the virtual agent to recognize the user's emotional state using an emotion analysis engine and provide appropriate dialogue. This allows users to easily arrange schedules and make store reservations without hassle, and to receive personalized services tailored to their emotions, simply by inputting a specific keyword.

[0869] A "communications application" is software that users use to exchange messages, such as text, voice, or images.

[0870] A "virtual agent" is a software agent that interacts with a user and automatically performs various tasks.

[0871] An "emotion analysis engine" is a system for analyzing and identifying a user's emotional state from text or voice.

[0872] A "digital network" refers to a digital communications network such as the Internet, which is a medium for transmitting and receiving data.

[0873] "User's Emotional State" means the momentary or ongoing psychological or emotional state of a User.

[0874] "User preferences" refers to the tastes and preferences that a user exhibits for a particular category or condition.

[0875] "Store candidates" is a list of multiple stores that may be suitable for the user's request.

[0876] "Reservation" refers to the advance procedure to reserve a specific service or location for a specified date and time.

[0877] The system of this invention allows a user to input a specific keyword into a communication application, and a virtual agent appears and automatically executes a series of subsequent tasks. It also uses an emotion analysis engine that recognizes the user's emotional state and adjusts the content of the dialogue accordingly.

[0878] System configuration

[0879] The system consists of three main components: the server, the terminal, and the user. The following is a detailed description of each component.

[0880] User

[0881] A user inputs a specific keyword through a communication application. For example, it is an everyday keyword such as "Let's go for drinks!" or "Want to go to dinner?". The user uses a device such as a smartphone or tablet.

[0882] Terminal

[0883] The terminal functions as an interface with the user, providing a screen for the virtual agent to interact with the user and collecting input from the user. Based on the user's input, the virtual agent then converses with the user to arrange a schedule or select a store.

[0884] server

[0885] The server performs the main processing of the virtual agent and the emotion analysis engine. It receives input from the user and analyzes their emotional state using the emotion analysis engine. This includes the following main processes:

[0886] 1. Keyword analysis: The server detects specific keywords in the user's input and triggers the virtual agent.

[0887] 2. Sentiment Analysis: Identify the user's emotional state using an emotion analysis engine (e.g., TextBlob). Adjust the virtual agent's dialogue accordingly.

[0888] 3. Schedule adjustment: Collects the user's schedule information and automatically selects the optimal schedule.

[0889] 4. Store Selection and Reservation: The virtual agent collects the user's preferences, searches for stores via the Internet, suggests suitable store candidates to the user, and makes a store reservation based on the user's selection.

[0890] Specific examples

[0891] For example, if User A types "Let's go for drinks!" into a communication application, the server recognizes the specific keyword and a virtual agent appears. The virtual agent displays the message, "Hello, let's go for drinks! This is AI. Let's first arrange the schedule for the participants." If User A types "May 10th after 7 PM," the server collects schedule information and determines the optimal date. The virtual agent then asks, "What are your preferences? For example, izakaya, bar, yakiniku, etc." The user responds, "Izakaya." The server searches for potential izakayas via the Internet, and the virtual agent suggests, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C." If the user selects "2. Restaurant B is good," the server makes the reservation, and once the reservation is complete, the virtual agent notifies the user, "Your reservation for Restaurant B has been completed. Details are below."

[0892] Example prompts for generative AI models

[0893] Below are some examples of prompts for generative AI models:

[0894] Analyze the following user messages and generate appropriate dialogue messages depending on the user's emotional state:

[0895] User message: "Tired after work, let's go for a drink!"

[0896] User data: {"date": "After 7 PM on May 10th", "preference": "Izakaya"}

[0897] Example of a generated conversation message:

[0898] """

[0899] Hello, let's go drinking! This is AI. Thank you for your hard work. I found a place where you can relax. How about going to an izakaya after 7pm on May 10th?

[0900] 1. Izakaya A (Rating: 4.3)

[0901] 2. Izakaya B (Rating: 4.5)

[0902] Which restaurant would you like to book?

[0903] """

[0904] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0905] Step 1:

[0906] Input: A user types a specific keyword (e.g., "Let's go for a drink!") into a communication application.

[0907] Operation: The terminal sends the user's input message to the server.

[0908] Output: Messages containing the keyword are sent to the server.

[0909] Step 2:

[0910] Input: The server receives the user's input message.

[0911] How it works: The server analyzes specific keywords (e.g., "Let's go for a drink!") and triggers the virtual agent.

[0912] Output: A virtual agent appears on the terminal and says, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0913] Step 3:

[0914] Input: The user who receives the message from the virtual agent inputs the desired date (e.g., "After 7 PM on May 10th").

[0915] Operation: The terminal sends the user's input schedule to the server.

[0916] Output: The schedule information is sent to the server.

[0917] Step 4:

[0918] Input: The server receives the user's schedule information.

[0919] How it works: The server parses the schedule information and stores it in its internal schedule database. It also sends similar messages to other participants as needed to collect schedule information.

[0920] Output: The optimal date is determined, and the virtual agent displays a message on the terminal saying, "Please tell us your preferences. For example, izakaya, bar, yakiniku, etc."

[0921] Step 5:

[0922] Input: The user inputs the type of store they want (e.g., "izakaya").

[0923] Operation: The terminal sends the user's input information to the server.

[0924] Output: Store preference information is sent to the server.

[0925] Step 6:

[0926] Input: The server receives the user's store preference information.

[0927] How it works: The server searches for establishments (e.g., "izakaya") in a specified area via the Internet, retrieves multiple candidates using the review site's API, analyzes the user's emotional state using a sentiment analysis engine, and selects the best candidate.

[0928] Output: Multiple store candidates (e.g., "Store A," "Store B," and "Store C") are obtained, and the virtual agent displays a message on the terminal saying, "Would you like to try one of the following izakayas? 1. Izakaya A 2. Izakaya B 3. Izakaya C."

[0929] Step 7:

[0930] Input: The user enters the number of the desired store (e.g., "2. Izakaya B").

[0931] Operation: The terminal sends the user's selection information to the server.

[0932] Output: The selection is sent to the server.

[0933] Step 8:

[0934] Input: The server receives the user's selection.

[0935] Operation: The server uses the reservation API to make a reservation at the specified store. It sends the necessary reservation information (e.g., date and time, number of people, names, etc.) to the reservation API. If the reservation is successful, it receives the reservation confirmation information and also performs error handling.

[0936] Output: The reservation confirmation information is saved on the server and sent to the device.

[0937] Step 9:

[0938] Input: The terminal that receives the reservation confirmation information displays a message to the user through the virtual agent.

[0939] Action: The virtual agent displays the message "Your reservation at Store B has been completed. Details below."

[0940] Output: The user is notified that the reservation has been completed.

[0941] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0942] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0943] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0944] [Third embodiment]

[0945] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0946] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0947] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0948] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0949] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0950] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0951] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0952] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0953] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0954] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0955] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0956] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0957] The present invention provides a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app, and automatically handles all steps from scheduling appointments to selecting a restaurant and making a reservation. This system operates as follows.

[0958] Explanation of program processing

[0959] User-entered keywords

[0960] A user types a specific keyword, such as "Let's go for a drink!", through a messaging app.

[0961] The server receives and analyzes this keyword through the messaging app's API.

[0962] The rise of virtual assistants

[0963] When the server recognizes a specific keyword, it triggers the appearance of a virtual assistant.

[0964] The server launches a virtual assistant on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules of the participants."

[0965] Schedule adjustment

[0966] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th."

[0967] The server receives this schedule information and stores it in a database.

[0968] The server sends similar messages to other participants to gather schedule information, for example, "Please tell me your free dates."

[0969] When the user (other participant) responds, the server also receives that information and stores it in the database.

[0970] The server processes the schedule information of all participants and determines the best time and date.

[0971] The server notifies the terminal of the optimal schedule.

[0972] Shop selection

[0973] The AI ​​will display on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[0974] The user types, "Izakaya is good."

[0975] The server receives the information and searches for izakayas in the specified area via the Internet.

[0976] The server uses the review site's API to select multiple candidates and send that information to the device.

[0977] The AI ​​will display on the device, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[0978] Choosing a restaurant and making a reservation

[0979] The user selects "2. Shop B is better."

[0980] The server receives this selection information and stores it in a database.

[0981] The server makes a reservation at store B using the reservation API.

[0982] The server sends a reservation completion message to the terminal and displays detailed information.

[0983] Specific examples

[0984] For example, if User A types "Let's go for drinks!" into a messaging app, the server responds immediately and a virtual assistant appears. The virtual assistant then displays, "Hello, let's go for drinks! This is AI. Let's first arrange the dates for the participants." User A enters a date, such as "May 10th after 7 PM," and the optimal date is determined. The virtual assistant then asks, "What are your preferences, such as izakaya, bar, or yakiniku?" and the user answers, "izakaya." The virtual assistant then searches for possible izakayas via the internet and automatically makes a reservation at the restaurant selected by the user.

[0985] In this way, users can set up a drinking party without going through any complicated procedures, simply by typing "Let's go drinking!"

[0986] The processing flow will be explained below.

[0987] Step 1:

[0988] A user types "Let's go for a drink!" into a messaging app. This message is sent to the server via the messaging app's API.

[0989] Step 2:

[0990] The server analyzes the received message and recognizes the specific keyword "Let's go for a drink!"

[0991] Step 3:

[0992] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device.

[0993] Step 4:

[0994] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[0995] Step 5:

[0996] The user inputs the desired date and time into the virtual assistant. For example, they might input "I'm available after 7 PM on May 10th."

[0997] Step 6:

[0998] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[0999] Step 7:

[1000] The server receives the responses of other participants and also stores their itinerary information in the itinerary database. Once all participants' itineraries are collected, the server analyzes the data to determine the optimal itinerary.

[1001] Step 8:

[1002] The server determines the optimal schedule and sends that information to the terminal, which then notifies the terminal of the optimal schedule.

[1003] Step 9:

[1004] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1005] Step 10:

[1006] The user types, "Izakaya is good."

[1007] Step 11:

[1008] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[1009] Step 12:

[1010] The server selects some of the candidates it has obtained and sends that information to the terminal.

[1011] Step 13:

[1012] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1013] Step 14:

[1014] The user selects "2. Shop B is better."

[1015] Step 15:

[1016] The server receives this selection information and stores it in a database.

[1017] Step 16:

[1018] The server uses the reservation API to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[1019] Step 17:

[1020] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[1021] Step 18:

[1022] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[1023] Step 19:

[1024] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[1025] This allows users to easily set up drinking parties.

[1026] Example 1

[1027] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1028] The conventional process of scheduling and reserving a restaurant using a messaging app requires users to manually perform multiple steps, which is time-consuming and inefficient.There is a need for a system that can greatly improve user convenience by automating the entire process, from scheduling to selecting a restaurant and making a reservation, simply by entering specific keywords.

[1029] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1030] In this invention, the server includes a means for a virtual assistant to automatically appear on the chat screen when a user inputs a specific keyword in a messaging app, a means for the virtual assistant to arrange schedules for participants, a means for the virtual assistant to hear the preferences of participants, search for stores via the Internet, and present multiple candidates, a means for the virtual assistant to automatically reserve a store based on the user's selection, and a means for the virtual assistant to send a reservation completion notification to the user's terminal. This automates a series of procedures from scheduling to reserving a store simply by inputting a specific keyword, making it possible to significantly reduce the user's effort and time.

[1031] A "user" is an entity that uses a messaging app to input specific keywords and make various reservations and adjustments through a virtual assistant.

[1032] A "messaging app" is a software application that enables users to send and receive text messages.

[1033] A "specific keyword" is a trigger word that a user can enter in the messaging app to activate the virtual assistant.

[1034] A "virtual assistant" is an artificial intelligence system that automates tasks such as scheduling, selecting stores, and making reservations based on user input.

[1035] The "talk screen" is the display area where messages are sent and received in a messaging app.

[1036] "Schedule adjustment" is the process of collecting the desired dates of multiple participants and determining the optimal date and time.

[1037] A "participant" is another individual who participates in a particular event (such as a drinking party) with the user.

[1038] The "Internet" is an information and communications network that connects computers and networks around the world.

[1039] "Store" refers to a commercial facility, restaurant, etc. that a user selects to visit.

[1040] "Candidates" are multiple store options that the virtual assistant suggests to the user.

[1041] "User selection" refers to the act of the user selecting a store from the candidates presented by the virtual assistant.

[1042] "Reservation" refers to securing a seat or service in advance at a store selected by the user.

[1043] "Notification" is the act of a virtual assistant sending information to a user's terminal.

[1044] A "terminal" is a device (e.g., a smartphone, tablet, or PC) on which a user uses a messaging app.

[1045] This invention is a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app, and automatically handles all steps from scheduling appointments to selecting a restaurant and making a reservation. This system is realized by a server, a user terminal, a messaging app, and virtual assistant software.

[1046] Explanation of program processing

[1047] User-entered keywords

[1048] A user enters a specific keyword, such as "Let's go for a drink!", through a messaging app (e.g., a general chat app). This message is sent to the server through the messaging app's API. The server receives and analyzes the keyword.

[1049] The rise of virtual assistants

[1050] When the server recognizes a specific keyword, it issues an instruction to trigger the appearance of the virtual assistant. The server requests the device to start the virtual assistant. Using virtual assistant software (e.g., Dialogflow), a message is displayed on the user's device saying, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants."

[1051] Schedule adjustment

[1052] The user inputs the desired date and time into the virtual assistant. For example, "I'm free after 7 PM on May 10th." The server receives this schedule information and stores it in a database (e.g., MySQL, PostgreSQL). The server then sends a similar message to other participants to collect their schedule information. For example, it sends a message saying, "Please tell me your free dates." When other participants respond, the server also receives that information and stores it in the database. The server aggregates the schedule information of all participants and determines the optimal date and time.

[1053] Shop selection

[1054] The server, through a virtual assistant, displays on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc." The user enters, "Izakaya is good." The server receives this information and searches for izakayas in the specified area using the Google Maps API, Yelp API, etc. The server uses review sites and store information APIs (such as Gurunavi and Hot Pepper) to generate a list of multiple candidates with high satisfaction rates, and sends this information to the user's device. The device then displays, "How about the following izakayas? 1. Store A 2. Store B 3. Store C."

[1055] Choosing a restaurant and making a reservation

[1056] The user selects "2. Restaurant B is good." The server saves this selection information in the database. The server uses the reservation API to reserve Restaurant B. The server sends a reservation completion message to the terminal and displays detailed information. This message includes the details, "Reservation for Restaurant B has been completed. The date and time is May 10th at 7 PM."

[1057] Specific examples

[1058] For example, if User A types "Let's go for a drink!" into a messaging app, the server receives and analyzes the keyword, and the virtual assistant launches. The process then automatically adjusts the date, selects a restaurant, and makes a reservation. This allows User A to get everything ready with just a few inputs.

[1059] Prompt Sentence Examples

[1060] You can simulate the functionality of a virtual assistant by inputting the following prompt sentences into a generative AI model (e.g., ChatGPT):

[1061] The user types "Let's go out for drinks!" into a messaging app. As a virtual assistant, you should display a message saying "Hello, this is Let's go out for drinks! AI. Let's first arrange the dates for the participants." After that, collect the participants' preferred dates and times, decide on the optimal date, and then proceed with selecting a restaurant and making a reservation.

[1062] This prompt can then be followed by a response or next step from the virtual assistant.

[1063] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1064] Step 1:

[1065] A user uses a messaging app to input a specific keyword, such as "Let's go for a drink!" The input keyword is sent to the server through the messaging app's API. The server receives the keyword and uses a text analysis algorithm to analyze it. As an output, the analysis results generate a flag that triggers the launch of the virtual assistant.

[1066] Step 2:

[1067] The server issues an instruction to start the virtual assistant. It sends message data to the device to display, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants." The device displays this message to the user. It uses virtual assistant software (e.g., Dialogflow) to output the message in text format.

[1068] Step 3:

[1069] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th." This date and time information is sent to the server and saved in a database. The server receives this as input and generates a message template to send similar messages to other participants. As output, it sends a message waiting for input from other participants.

[1070] Step 4:

[1071] The server receives the schedule information of other participants and stores it in a database. It aggregates the schedule information of all participants and runs an algorithm to calculate the optimal date and time. This algorithm takes participant response data as input and performs duplication and optimization. The determined optimal date and time is generated as output.

[1072] Step 5:

[1073] The server notifies the device of the optimal date and displays a message to the user through a virtual assistant saying, "Please tell us your preferences. For example, izakaya, bar, yakiniku, etc." The device displays this message to the user, who then inputs their preferences. The server receives this preference information as input.

[1074] Step 6:

[1075] The server searches for izakayas in a specified area via the Internet using the Google Maps API, Yelp API, etc. The user's preferences and local information are used as input. The server collects highly rated restaurant candidates through review sites and restaurant information APIs, generates a list of candidates, and sends it to the device. The output is a list containing multiple restaurant candidates.

[1076] Step 7:

[1077] The terminal displays "Would you like to try one of the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C," and the user selects "2. Restaurant B." The selection information is sent to the server and saved in the database. The server receives this selection information as input and begins the process of making a reservation for Restaurant B using the reservation API.

[1078] Step 8:

[1079] The server confirms the reservation for Store B through the reservation API, generates a reservation completion message, and sends it to the device. Detailed information such as "Reservation for Store B has been completed. The date and time is May 10th at 7 PM" is displayed on the device. This allows the user to confirm that the reservation has been confirmed.

[1080] Step 9:

[1081] The server sequentially verifies and updates the status of schedule adjustment, store selection, and reservation, and sends notifications to the user's device so that the user can check the progress.

[1082] (Application example 1)

[1083] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1084] Conventional food delivery services require users to go through several cumbersome procedures when using them. Specifically, users must launch the delivery service app, search for the food and restaurant that suits their preferences, and then order and pay separately. Furthermore, users are sometimes not notified of the order status or estimated delivery time, which can be inconvenient for them. Given this current situation, there is a demand for a system that allows users to use delivery services simply.

[1085] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1086] In this invention, the server includes: means for a virtual assistant to automatically appear on the chat screen when a user inputs a specific keyword in a messaging app; means for the virtual assistant to arrange schedules for participants; means for the virtual assistant to listen to participants' preferences and search for delivery locations via the Internet; means for the virtual assistant to suggest candidate delivery locations and automatically place orders and reservations based on the user's selection; means for the virtual assistant to prioritize highly rated delivery locations when searching for delivery locations in a specific area using an API of a review site; means for the virtual assistant to use an electronic payment service to process payment for the order; and means for the virtual assistant to notify the user of the completion of the order and the estimated delivery time. This enables users to easily use food delivery services from a messaging app, avoiding cumbersome procedures and keeping track of the situation in real time.

[1087] A "messaging app" is software that users use to send and receive text messages and other information.

[1088] "Specific keywords" are specific words or phrases that users type into messaging apps to indicate specific intent or requests.

[1089] A "virtual assistant" is a software agent that automatically responds to user input and performs specific tasks.

[1090] "Participant scheduling" is the process of checking the available times and days of multiple participants to determine the best date and time.

[1091] "Hearing participant preferences" is the process of asking and collecting the preferences and wishes of users and participants.

[1092] "Searching via the Internet" refers to the act of searching for information or stores through online networks.

[1093] "Search for locations" means searching for service locations such as restaurants and delivery services based on specified conditions.

[1094] "Proposing destination candidates" is a process of presenting multiple options to the user from the search results.

[1095] "Placing an order or reservation based on the user's selection" means automatically placing an order or reservation with the provider selected by the user.

[1096] "Using the API of a rating site" means using the application programming interface provided by other websites that provide ratings and reviews.

[1097] "Preferentially selecting providers with high ratings" means giving priority to selecting service providers with high user ratings and reviews.

[1098] "Using electronic payment services to process payments" means using digital payment methods such as credit cards or electronic money to make online payments.

[1099] "Notifying the user of order completion and estimated delivery time one by one" is a process in which the user is notified of information about the estimated delivery time one by one from the time the order is confirmed.

[1100] This invention is a system in which a virtual assistant automatically appears on the chat screen when a user inputs a specific keyword into a messaging app, and automatically handles everything from scheduling to selecting a delivery destination, placing an order, making a payment, and notifying the user of the estimated delivery time. The following describes in detail an embodiment of the invention.

[1101] Generating a Program

[1102] The system for realizing the present invention consists of a server, a terminal, and a user. The server receives specific keywords from the user and launches the virtual assistant. Specifically, it provides an API endpoint using a lightweight web framework called Flask. It also uses a library called Requests to call external APIs (for example, the API of a rating site or the API of an electronic payment service).

[1103] Natural language explanation of the process

[1104] The server consists of a means to:

[1105] 1. Message reception and analysis: When a user enters a specific keyword (e.g., "Delivery Master, I'm hungry") into the messaging app, the server receives and analyzes the message via API. Depending on the results of this analysis, it launches the virtual assistant.

[1106] 2. Launching a virtual assistant: When the server recognizes a specific keyword, a virtual assistant will automatically appear on the device and interact with the user. For example, if a user types "I want to eat pizza," the assistant will respond, "Hello, this is Delivery Master. Please tell us what you want to eat."

[1107] 3. Search and selection of restaurants: The server obtains the user's location information and searches for restaurants (for example, nearby pizza restaurants). At this time, it uses the API of rating sites such as Tabelog to prioritize restaurants with high ratings.

[1108] 4. Ordering and reservation procedures: The virtual assistant automatically places an order and makes a reservation for the delivery location selected by the user. For example, if the user selects "Pizza Hut Margherita," the server processes the order details and places the order with the delivery location.

[1109] 5. Electronic Payment: The server uses an electronic payment service (e.g., PayPal) to process payment for the order. The server processes the payment securely based on the user's credit card information.

[1110] 6. Status Notification: Once the order is completed, the virtual assistant will notify the user of important information such as the estimated delivery time. For example, "Your order is complete. The estimated delivery time is 7:00 PM."

[1111] Adding specific examples

[1112] For example, if a user types "Delivery Master, I'm hungry" into a messaging app, the following sequence of events will be processed by the server:

[1113] The server recognizes the user's input and the virtual assistant displays, "Hello, this is Delivery Master. Please tell us what you would like to eat."

[1114] If the user answers "pizza," the server searches for the best pizza place via the Internet based on the user's location information.

[1115] It then suggests multiple options to the user (e.g., "Pizza Hut A, Domino's Pizza B, Salvatore C").

[1116] When a user selects "Pizza Hut A, Margherita," the server places the order and processes the payment using PayPal.

[1117] Finally, the virtual assistant announces, "Your order is complete. Estimated delivery time is 7:00 PM."

[1118] Example prompt sentence:

[1119] "Delivery Master, I'm hungry."

[1120] This allows users to easily use food delivery services and eliminates complicated procedures.

[1121] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1122] Step 1:

[1123] The server receives a message in which the user enters a specific keyword, "Delivery Master, I'm hungry," into a messaging app. The server retrieves this message via an API and analyzes the keyword. The input data is the user's message, and the output is the recognition of the specific keyword. Based on this recognition, the launch of the virtual assistant is triggered.

[1124] Step 2:

[1125] The terminal receives instructions from the server, launches the virtual assistant, and displays a message to the user saying, "Hello, this is Delivery Master. Please tell us what you would like to eat." The input here is the instruction from the server, and the output is the message displayed on the terminal.

[1126] Step 3:

[1127] The user responds to the virtual assistant's message with "pizza." This user input is sent to the server via the terminal. The input data is the user's response message and is sent to the server.

[1128] Step 4:

[1129] The server obtains the user's location information based on the dish information received from the user. Using this location information, it searches for restaurants via the Internet. The server calls a rating site API to search for highly rated pizza restaurants in the specified area. The input data is the user's location information and preferred dishes, and the output data is a list of candidate restaurants.

[1130] Step 5:

[1131] The server proposes a list of providers obtained from the results of the rating site API to the user through a virtual assistant. For example, it displays a message such as, "Which would you like: Pizza Hut A, Domino's Pizza B, or Salvatore C?" The input here is a list of candidate providers, and the output is a proposal message to the user.

[1132] Step 6:

[1133] The user selects "Pizza Hut A, Margherita" from the presented list of destinations and inputs it into the virtual assistant. This information is sent to the server via the terminal. The input data is the user's selection information and is sent to the server.

[1134] Step 7:

[1135] The server obtains the user's selection information and places an order with the specified delivery destination. Using the order API, it executes an order for "Pizza Hut A, Margherita" with the delivery destination. The input data is the product information selected by the user, and the output data is the order completion status.

[1136] Step 8:

[1137] After the order is completed, the server processes the payment using an electronic payment service such as PayPal. When the payment is successfully completed, the result is obtained. The input data is the payment information, and the output data is the payment completion status.

[1138] Step 9:

[1139] After the server confirms that the order and payment procedures have been completed, it sends a message to the user through the virtual assistant saying, "The order has been completed. The estimated delivery time is 7:00 PM." The input here is the order and payment completion status, and the output is a notification message to the user.

[1140] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1141] This invention combines a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app and automatically performs a series of processes such as scheduling, selecting a restaurant, and making a reservation, with an emotion engine. The emotion engine has the function of recognizing the user's emotional state and providing appropriate dialogue based on that. This system operates as follows.

[1142] Explanation of program processing

[1143] User-entered keywords

[1144] A user types a specific keyword, such as "Let's go for a drink!", through a messaging app.

[1145] The server receives and analyzes this keyword through the messaging app's API.

[1146] The rise of virtual assistants

[1147] When the server recognizes a specific keyword, it generates data to trigger the virtual assistant and sends it to the device.

[1148] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1149] Emotion Engine Operation

[1150] The server receives the user's message and analyzes the user's emotional state using an emotion engine.

[1151] The emotion engine analyzes the user's emotional state and provides the results to the virtual assistant, for example, identifying whether the user is happy or stressed.

[1152] The virtual assistant on the device will adjust the tone and content of messages to match the user's emotions. For example, if the user is feeling stressed, it will prioritize suggestions for places to relax.

[1153] Schedule adjustment

[1154] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th."

[1155] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[1156] Shop selection

[1157] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1158] The user types, "Izakaya is good."

[1159] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[1160] The server selects some of the candidates it has obtained and sends that information to the terminal.

[1161] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1162] Choosing a restaurant and making a reservation

[1163] The user selects "2. Shop B is better."

[1164] The server receives this selection information and stores it in a database.

[1165] The server uses the reservation API to make a reservation at Restaurant B. The server sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[1166] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[1167] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[1168] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[1169] Specific examples

[1170] For example, if User A types "Let's go out for drinks!" into a messaging app, the server responds immediately and a virtual assistant appears. The virtual assistant then displays, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants." User A enters a date, such as "May 10th after 7 PM," and the optimal date is determined. The virtual assistant then asks, "What are your preferences, such as izakaya, bar, or yakiniku?" and the user answers, "izakaya." The virtual assistant then searches for potential izakayas via the internet and automatically makes a reservation at the restaurant selected by the user. The emotion engine analyzes the user's emotions and makes optimal suggestions, providing a more personalized service.

[1171] In this way, users can set up a drinking party without going through any complicated procedures, simply by typing "Let's go drinking!". In addition, the system makes suggestions and responds based on the user's emotions, resulting in a service that provides a higher level of satisfaction.

[1172] The processing flow will be explained below.

[1173] Step 1:

[1174] A user types "Let's go for a drink!" into a messaging app. This message is sent to the server via the messaging app's API.

[1175] Step 2:

[1176] The server analyzes the received message and recognizes the specific keyword "Let's go for a drink!"

[1177] Step 3:

[1178] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device.

[1179] Step 4:

[1180] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1181] Step 5:

[1182] The user inputs the desired date and time into the virtual assistant. For example, they might input "I'm available after 7 PM on May 10th."

[1183] Step 6:

[1184] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[1185] Step 7:

[1186] The server receives the responses of other participants and also stores their itinerary information in the itinerary database. Once all participants' itineraries are collected, the server analyzes the data to determine the optimal itinerary.

[1187] Step 8:

[1188] The server determines the optimal schedule and sends that information to the terminal, which then notifies the terminal of the optimal schedule.

[1189] Step 9:

[1190] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1191] Step 10:

[1192] The user types, "Izakaya is good."

[1193] Step 11:

[1194] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[1195] Step 12:

[1196] The server selects some of the candidates it has obtained and sends that information to the terminal.

[1197] Step 13:

[1198] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1199] Step 14:

[1200] The user selects "2. Shop B is better."

[1201] Step 15:

[1202] The server receives this selection information and stores it in a database.

[1203] Step 16:

[1204] The server uses the reservation API to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[1205] Step 17:

[1206] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[1207] Step 18:

[1208] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[1209] Step 19:

[1210] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[1211] Step 20:

[1212] The server activates an emotion engine to identify emotions from the user's messages and inputs.

[1213] Step 21:

[1214] The server uses an emotion engine to analyze the user's message and detect their emotional state (e.g., joy, stress, frustration, etc.).

[1215] Step 22:

[1216] The virtual assistant on the device will respond appropriately based on the results of emotion analysis. For example, it will suggest a store where a user can relax if they are feeling stressed.

[1217] Step 23:

[1218] The server tracks the user's emotional state and adjusts the tone and content of the message in real time.

[1219] Step 24:

[1220] The user receives suggestions and responses that correspond to their emotions, and the drinking party planning proceeds in an emotionally satisfied state.

[1221] Example 2

[1222] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1223] Currently, when using a messaging app to schedule a drinking party, the tasks of scheduling, selecting a restaurant, and making a reservation are extremely cumbersome and time-consuming. Furthermore, the system does not respond to the user's emotional state, which can lead to a decrease in participant satisfaction. Conventional systems do not offer a method for efficiently resolving these issues. Therefore, the present invention aims to solve these problems.

[1224] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1225] In this invention, the server includes: a means for automatically having a virtual assistant appear on the chat screen when a user inputs a specific keyword into a messaging app; a means for the virtual assistant to arrange schedules for participants; a means for the virtual assistant to listen to participants' preferences and search for restaurants via the Internet; a means for the virtual assistant to suggest candidate restaurants and automatically make reservations based on the user's selection; and a means for analyzing the user's emotional state using emotion analysis technology installed in the system and providing appropriate dialogue based on the analysis results. This allows users to easily and efficiently set up drinking parties using a messaging app and enables responses according to the user's emotional state.

[1226] "User" refers to an individual or group that uses this system.

[1227] "Messaging app" refers to a software application for sending and receiving text and multimedia messages over the Internet.

[1228] A "virtual assistant" refers to a computer program that automatically performs tasks through user interaction.

[1229] A "server" refers to a computer system that provides services to clients over a network.

[1230] "Emotion analysis technology" refers to technology for analyzing a user's emotional state from text or voice data.

[1231] "Schedule adjustment" refers to the process of checking the free times of multiple users and determining the optimal date and time.

[1232] "Store" refers to a place that a user visits, and in this case, it mainly refers to a restaurant.

[1233] "Hearing" refers to the act of collecting information from users.

[1234] "Internet" refers to the global system of computer networks and infrastructure that enables information sharing and communication.

[1235] "Reservation" refers to the act of a user making a reservation in advance to receive a specific service at a specific date and time.

[1236] "Candidates" refers to a list of options that a user can choose from.

[1237] "Analysis" refers to the process of examining data in detail to reveal its meaning and structure.

[1238] "Dialogue" refers to the exchange of information between a user and a virtual assistant.

[1239] This invention is a system in which, when a user inputs a specific keyword into a messaging app, a virtual assistant automatically appears and performs a series of steps such as scheduling, selecting a restaurant, making a reservation, etc. Furthermore, by incorporating emotion analysis technology, this system provides optimal dialogue according to the user's emotional state.

[1240] The system is implemented using the following hardware and software:

[1241] Server: A computer system that provides services to clients over a network, especially cloud services such as Amazon Web Services (AWS).

[1242] Device: Any device used by a user, including internet-enabled devices such as smartphones and tablets.

[1243] Messaging app: A software application for sending and receiving text and multimedia messages, such as LINE or WhatsApp.

[1244] Emotion analysis technology: Technology for analyzing the user's emotional state. For example, IBM Watson Tone Analyzer is used.

[1245] Schedule Database: A database system for storing schedule information. It uses the Google Calendar API.

[1246] Review site API: An API for obtaining store information. For example, the Yelp API is used.

[1247] Reservation API: An API for making restaurant reservations. For example, the OpenTable API is used.

[1248] As a specific example of operation, consider the following scenario.

[1249] User-entered keywords

[1250] When a user types a specific keyword, such as "Let's go for a drink!", through a messaging app, the device sends the input to a server, which receives the keyword through the messaging app's API and analyzes it.

[1251] The rise of virtual assistants

[1252] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device. The virtual assistant then appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1253] How emotion analysis technology works

[1254] The server receives the user's message and uses emotion analysis technology to analyze the user's emotional state. For example, if a user sends a message saying, "I've been busy and stressed lately," the server sends the message to emotion analysis technology and identifies the user as feeling stressed. The virtual assistant on the device adjusts the tone and content of the message to match the user's emotions, displaying, "I'll find you a great izakaya to relieve your stress!"

[1255] Schedule adjustment

[1256] When a user inputs a desired date and time into the virtual assistant, for example, "I'm free after 7 PM on May 10th," the server saves the date and time information in a schedule database and, if necessary, sends similar messages to other participants to collect their schedule information.

[1257] Store selection

[1258] The virtual assistant displays on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc." The user enters, "Izakaya is good." The server receives this information, searches for izakaya in the specified area via the Internet, and retrieves multiple candidates using the review site's API. The server selects several from the retrieved candidates and sends this information to the device. The message displayed is, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1259] Choosing a store and making a reservation

[1260] The user selects "2. Restaurant B is good." The server receives this selection and saves it in a database. The server uses the reservation API to send the necessary reservation information (date and time, number of people, names, etc.). If the reservation is successful, the server receives the reservation confirmation information and also performs error handling. The reservation confirmation information is saved in a database and sent to the terminal. The virtual assistant displays a message on the terminal saying, "Reservation for Restaurant B has been completed. Details are below."

[1261] Example prompts for generative AI models

[1262] "Please tell me the step-by-step process for hosting a drinking party at an izakaya after 7 PM on May 10th. Please include everything from coordinating the schedules of the participants to selecting the restaurant and making the reservation. Please also take into consideration the feelings of the participants."

[1263] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1264] Step 1: User enters specific keywords

[1265] A user types a specific keyword into a messaging app, such as "Let's go for drinks!"

[1266] The terminal sends the input to the application server.

[1267] Input: The keyword "Let's go for a drink!"

[1268] Output: The entered keyword is sent to the server

[1269] Specific actions: User A opens LINE on their smartphone and sends a message to a friend saying, "Let's go for drinks!"

[1270] Step 2: The server receives and parses the keyword

[1271] The server receives the keywords through the messaging app's API and analyzes the content. The analysis process is implemented in Python and JavaScript.

[1272] Input: Message data containing keywords

[1273] Output: Data that determines actions based on keywords

[1274] What it does: The server uses Amazon Web Services (AWS) Lambda to analyze the received message and detect the keyword "Let's go for a drink!"

[1275] Step 3: The virtual assistant arrives

[1276] When the server recognizes a specific keyword, it generates data to trigger the virtual assistant and sends it to the device.

[1277] A virtual assistant appears on the device and displays an initial message.

[1278] Input: Keyword analysis results

[1279] Output: Virtual assistant trigger data, initial message

[1280] Specific operation: The server generates data and displays the AI ​​assistant on User A's smartphone. The message displayed is, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants."

[1281] Step 4: Analyze user emotions using emotion analysis technology

[1282] The server receives the user's additional message and analyzes the user's emotional state using emotion analysis technology, such as IBM Watson Tone Analyzer.

[1283] Input: Message from the user (e.g., "I've been busy and stressed lately.")

[1284] Output: Sentiment analysis result (e.g. "I feel stressed")

[1285] What it does: The server sends an additional message to the emotion analysis technology, identifying it as stressed.

[1286] Step 5: Adjust the dialogue based on the sentiment results

[1287] The device receives the emotion analysis results, and the virtual assistant responds appropriately, adjusting the tone and message content according to the emotion.

[1288] Input: Sentiment analysis results

[1289] Output: Adjusted message content

[1290] Specific operation: The virtual assistant displays a message on the device saying, "I'll find you a great izakaya to relieve your stress!"

[1291] Step 6: Propose a schedule

[1292] The user inputs the desired date and time into the virtual assistant. For example, "I'm available after 7 PM on May 10th."

[1293] The server receives the schedule information and stores it in a schedule database (e.g., Google Calendar API). It also sends similar messages to other participants to collect their schedule information.

[1294] Input: Date information

[1295] Output: Date information stored in the database, messages sent to other participants

[1296] Specific behavior: User A enters "I'm free after 7pm on May 10th," and the server saves this to the Google Calendar API.

[1297] Step 7: Store Selection

[1298] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1299] The user types, "Izakaya is good."

[1300] The server receives the information and searches for izakayas in the specified area via the Internet, using the API of a review site (e.g., Yelp API) to obtain multiple candidates.

[1301] Input: Type of store (e.g. "Izakaya")

[1302] Output: Multiple store candidates

[1303] Specific operation: The user enters "izakaya" (Japanese pub) on their device, and the information is sent to the server. The server uses the Yelp API to search for izakayas, selects "Store A," "Store B," or "Store C," and sends the information back to the user's device.

[1304] Step 8: Virtual assistant suggests stores

[1305] The virtual assistant will display on the device, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1306] Input: Multiple store candidates

[1307] Output: List of store candidates

[1308] Specific operation: A list of "1. Store A," "2. Store B," and "3. Store C" is displayed on the user's device.

[1309] Step 9: User selects store

[1310] The user selects "2. Shop B is better."

[1311] The server receives this selection information and stores it in a database.

[1312] Input: Selected store information

[1313] Output: Selections stored in a database

[1314] Specific operation: The user enters "2. Shop B is good," and the server saves the data in the database.

[1315] Step 10: Server reserves the store

[1316] The server uses a reservation API (e.g., OpenTable API) to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[1317] If the reservation is successful, the server receives the reservation confirmation information, performs error handling, saves the reservation confirmation information in the database, and sends the information to the terminal.

[1318] Input: Selected store information, reservation information

[1319] Output: Booking confirmation information, error message (if necessary)

[1320] Specific operation: The server makes a reservation for Restaurant B using the OpenTable API, and the reservation is confirmed.

[1321] Step 11: Notify user of reservation confirmation

[1322] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[1323] Input: Confirmed reservation information

[1324] Output: Display a confirmation message

[1325] Specific operation: Detailed information such as "Reservation at Restaurant B completed. Date, time, location, reservation name" will be displayed on the user's device.

[1326] (Application example 2)

[1327] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1328] In modern society, where people are busy every day, coordinating group schedules and making restaurant reservations can be cumbersome and time-consuming. To solve these problems, there is a need for a system that allows users to easily coordinate schedules and make restaurant reservations. Furthermore, there is a need to provide more personalized and comfortable services by responding to each user's emotional state.

[1329] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for automatically causing a virtual agent to appear on the screen when a user inputs a specific keyword into a communication application; means for the virtual agent to arrange schedules for participants; means for the virtual agent to collect participants' preferences and search for stores via a digital network; means for the virtual agent to suggest candidate stores and automatically make reservations based on the user's selection; and means for the virtual agent to recognize the user's emotional state using an emotion analysis engine and provide appropriate dialogue. This allows users to easily arrange schedules and make store reservations without hassle, and to receive personalized services tailored to their emotions, simply by inputting a specific keyword.

[1330] A "communications application" is software that users use to exchange messages, such as text, voice, or images.

[1331] A "virtual agent" is a software agent that interacts with a user and automatically performs various tasks.

[1332] An "emotion analysis engine" is a system for analyzing and identifying a user's emotional state from text or voice.

[1333] A "digital network" refers to a digital communications network such as the Internet, which is a medium for transmitting and receiving data.

[1334] "User's Emotional State" means the momentary or ongoing psychological or emotional state of a User.

[1335] "User preferences" refers to the tastes and preferences that a user exhibits for a particular category or condition.

[1336] "Store candidates" is a list of multiple stores that may be suitable for the user's request.

[1337] "Reservation" refers to the advance procedure to reserve a specific service or location for a specified date and time.

[1338] The system of this invention allows a user to input a specific keyword into a communication application, and a virtual agent appears and automatically executes a series of subsequent tasks. It also uses an emotion analysis engine that recognizes the user's emotional state and adjusts the content of the dialogue accordingly.

[1339] System configuration

[1340] The system consists of three main components: the server, the terminal, and the user. The following is a detailed description of each component.

[1341] User

[1342] A user inputs a specific keyword through a communication application. For example, it is an everyday keyword such as "Let's go for drinks!" or "Want to go to dinner?". The user uses a device such as a smartphone or tablet.

[1343] Terminal

[1344] The terminal functions as an interface with the user, providing a screen for the virtual agent to interact with the user and collecting input from the user. Based on the user's input, the virtual agent then converses with the user to arrange a schedule or select a store.

[1345] server

[1346] The server performs the main processing of the virtual agent and the emotion analysis engine. It receives input from the user and analyzes their emotional state using the emotion analysis engine. This includes the following main processes:

[1347] 1. Keyword analysis: The server detects specific keywords in the user's input and triggers the virtual agent.

[1348] 2. Sentiment Analysis: Identify the user's emotional state using an emotion analysis engine (e.g., TextBlob). Adjust the virtual agent's dialogue accordingly.

[1349] 3. Schedule adjustment: Collects the user's schedule information and automatically selects the optimal schedule.

[1350] 4. Store Selection and Reservation: The virtual agent collects the user's preferences, searches for stores via the Internet, suggests suitable store candidates to the user, and makes a store reservation based on the user's selection.

[1351] Specific examples

[1352] For example, if User A types "Let's go for drinks!" into a communication application, the server recognizes the specific keyword and a virtual agent appears. The virtual agent displays the message, "Hello, let's go for drinks! This is AI. Let's first arrange the schedule for the participants." If User A types "May 10th after 7 PM," the server collects schedule information and determines the optimal date. The virtual agent then asks, "What are your preferences? For example, izakaya, bar, yakiniku, etc." The user responds, "Izakaya." The server searches for potential izakayas via the Internet, and the virtual agent suggests, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C." If the user selects "2. Restaurant B is good," the server makes the reservation, and once the reservation is complete, the virtual agent notifies the user, "Your reservation for Restaurant B has been completed. Details are below."

[1353] Example prompts for generative AI models

[1354] Below are some examples of prompts for generative AI models:

[1355] Analyze the following user messages and generate appropriate dialogue messages depending on the user's emotional state:

[1356] User message: "Tired after work, let's go for a drink!"

[1357] User data: {"date": "After 7 PM on May 10th", "preference": "Izakaya"}

[1358] Example of a generated conversation message:

[1359] """

[1360] Hello, let's go drinking! This is AI. Thank you for your hard work. I found a place where you can relax. How about going to an izakaya after 7pm on May 10th?

[1361] 1. Izakaya A (Rating: 4.3)

[1362] 2. Izakaya B (Rating: 4.5)

[1363] Which restaurant would you like to book?

[1364] """

[1365] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1366] Step 1:

[1367] Input: A user types a specific keyword (e.g., "Let's go for a drink!") into a communication application.

[1368] Operation: The terminal sends the user's input message to the server.

[1369] Output: Messages containing the keyword are sent to the server.

[1370] Step 2:

[1371] Input: The server receives the user's input message.

[1372] How it works: The server analyzes specific keywords (e.g., "Let's go for a drink!") and triggers the virtual agent.

[1373] Output: A virtual agent appears on the terminal and says, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1374] Step 3:

[1375] Input: The user who receives the message from the virtual agent inputs the desired date (e.g., "After 7 PM on May 10th").

[1376] Operation: The terminal sends the user's input schedule to the server.

[1377] Output: The schedule information is sent to the server.

[1378] Step 4:

[1379] Input: The server receives the user's schedule information.

[1380] How it works: The server parses the schedule information and stores it in its internal schedule database. It also sends similar messages to other participants as needed to collect schedule information.

[1381] Output: The optimal date is determined, and the virtual agent displays a message on the terminal saying, "Please tell us your preferences. For example, izakaya, bar, yakiniku, etc."

[1382] Step 5:

[1383] Input: The user inputs the type of store they want (e.g., "izakaya").

[1384] Operation: The terminal sends the user's input information to the server.

[1385] Output: Store preference information is sent to the server.

[1386] Step 6:

[1387] Input: The server receives the user's store preference information.

[1388] How it works: The server searches for establishments (e.g., "izakaya") in a specified area via the Internet, retrieves multiple candidates using the review site's API, analyzes the user's emotional state using a sentiment analysis engine, and selects the best candidate.

[1389] Output: Multiple store candidates (e.g., "Store A," "Store B," and "Store C") are obtained, and the virtual agent displays a message on the terminal saying, "Would you like to try one of the following izakayas? 1. Izakaya A 2. Izakaya B 3. Izakaya C."

[1390] Step 7:

[1391] Input: The user enters the number of the desired store (e.g., "2. Izakaya B").

[1392] Operation: The terminal sends the user's selection information to the server.

[1393] Output: The selection is sent to the server.

[1394] Step 8:

[1395] Input: The server receives the user's selection.

[1396] Operation: The server uses the reservation API to make a reservation at the specified store. It sends the necessary reservation information (e.g., date and time, number of people, names, etc.) to the reservation API. If the reservation is successful, it receives the reservation confirmation information and also performs error handling.

[1397] Output: The reservation confirmation information is saved on the server and sent to the device.

[1398] Step 9:

[1399] Input: The terminal that receives the reservation confirmation information displays a message to the user through the virtual agent.

[1400] Action: The virtual agent displays the message "Your reservation at Store B has been completed. Details below."

[1401] Output: The user is notified that the reservation has been completed.

[1402] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1403] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1404] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1405] [Fourth embodiment]

[1406] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1407] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1408] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1409] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1410] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1411] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1412] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1413] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1414] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1415] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1416] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1417] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1418] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1419] The present invention provides a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app, and automatically handles all steps from scheduling appointments to selecting a restaurant and making a reservation. This system operates as follows.

[1420] Explanation of program processing

[1421] User-entered keywords

[1422] A user types a specific keyword, such as "Let's go for a drink!", through a messaging app.

[1423] The server receives and analyzes this keyword through the messaging app's API.

[1424] The rise of virtual assistants

[1425] When the server recognizes a specific keyword, it triggers the appearance of a virtual assistant.

[1426] The server launches a virtual assistant on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules of the participants."

[1427] Schedule adjustment

[1428] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th."

[1429] The server receives this schedule information and stores it in a database.

[1430] The server sends similar messages to other participants to gather schedule information, for example, "Please tell me your free dates."

[1431] When the user (other participant) responds, the server also receives that information and stores it in the database.

[1432] The server processes the schedule information of all participants and determines the best time and date.

[1433] The server notifies the terminal of the optimal schedule.

[1434] Shop selection

[1435] The AI ​​will display on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1436] The user types, "Izakaya is good."

[1437] The server receives the information and searches for izakayas in the specified area via the Internet.

[1438] The server uses the review site's API to select multiple candidates and send that information to the device.

[1439] The AI ​​will display on the device, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1440] Choosing a restaurant and making a reservation

[1441] The user selects "2. Shop B is better."

[1442] The server receives this selection information and stores it in a database.

[1443] The server makes a reservation at store B using the reservation API.

[1444] The server sends a reservation completion message to the terminal and displays detailed information.

[1445] Specific examples

[1446] For example, if User A types "Let's go for drinks!" into a messaging app, the server responds immediately and a virtual assistant appears. The virtual assistant then displays, "Hello, let's go for drinks! This is AI. Let's first arrange the dates for the participants." User A enters a date, such as "May 10th after 7 PM," and the optimal date is determined. The virtual assistant then asks, "What are your preferences, such as izakaya, bar, or yakiniku?" and the user answers, "izakaya." The virtual assistant then searches for possible izakayas via the internet and automatically makes a reservation at the restaurant selected by the user.

[1447] In this way, users can set up a drinking party without going through any complicated procedures, simply by typing "Let's go drinking!"

[1448] The processing flow will be explained below.

[1449] Step 1:

[1450] A user types "Let's go for a drink!" into a messaging app. This message is sent to the server via the messaging app's API.

[1451] Step 2:

[1452] The server analyzes the received message and recognizes the specific keyword "Let's go for a drink!"

[1453] Step 3:

[1454] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device.

[1455] Step 4:

[1456] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1457] Step 5:

[1458] The user inputs the desired date and time into the virtual assistant. For example, they might input "I'm available after 7 PM on May 10th."

[1459] Step 6:

[1460] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[1461] Step 7:

[1462] The server receives the responses of other participants and also stores their itinerary information in the itinerary database. Once all participants' itineraries are collected, the server analyzes the data to determine the optimal itinerary.

[1463] Step 8:

[1464] The server determines the optimal schedule and sends that information to the terminal, which then notifies the terminal of the optimal schedule.

[1465] Step 9:

[1466] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1467] Step 10:

[1468] The user types, "Izakaya is good."

[1469] Step 11:

[1470] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[1471] Step 12:

[1472] The server selects some of the candidates it has obtained and sends that information to the terminal.

[1473] Step 13:

[1474] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1475] Step 14:

[1476] The user selects "2. Shop B is better."

[1477] Step 15:

[1478] The server receives this selection information and stores it in a database.

[1479] Step 16:

[1480] The server uses the reservation API to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[1481] Step 17:

[1482] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[1483] Step 18:

[1484] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[1485] Step 19:

[1486] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[1487] This allows users to easily set up drinking parties.

[1488] Example 1

[1489] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1490] The conventional process of scheduling and reserving a restaurant using a messaging app requires users to manually perform multiple steps, which is time-consuming and inefficient.There is a need for a system that can greatly improve user convenience by automating the entire process, from scheduling to selecting a restaurant and making a reservation, simply by entering specific keywords.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1492] In this invention, the server includes a means for a virtual assistant to automatically appear on the chat screen when a user inputs a specific keyword in a messaging app, a means for the virtual assistant to arrange schedules for participants, a means for the virtual assistant to hear the preferences of participants, search for stores via the Internet, and present multiple candidates, a means for the virtual assistant to automatically reserve a store based on the user's selection, and a means for the virtual assistant to send a reservation completion notification to the user's terminal. This automates a series of procedures from scheduling to reserving a store simply by inputting a specific keyword, making it possible to significantly reduce the user's effort and time.

[1493] A "user" is an entity that uses a messaging app to input specific keywords and make various reservations and adjustments through a virtual assistant.

[1494] A "messaging app" is a software application that enables users to send and receive text messages.

[1495] A "specific keyword" is a trigger word that a user can enter in the messaging app to activate the virtual assistant.

[1496] A "virtual assistant" is an artificial intelligence system that automates tasks such as scheduling, selecting stores, and making reservations based on user input.

[1497] The "talk screen" is the display area where messages are sent and received in a messaging app.

[1498] "Schedule adjustment" is the process of collecting the desired dates of multiple participants and determining the optimal date and time.

[1499] A "participant" is another individual who participates in a particular event (such as a drinking party) with the user.

[1500] The "Internet" is an information and communications network that connects computers and networks around the world.

[1501] "Store" refers to a commercial facility, restaurant, etc. that a user selects to visit.

[1502] "Candidates" are multiple store options that the virtual assistant suggests to the user.

[1503] "User selection" refers to the act of the user selecting a store from the candidates presented by the virtual assistant.

[1504] "Reservation" refers to securing a seat or service in advance at a store selected by the user.

[1505] "Notification" is the act of a virtual assistant sending information to a user's terminal.

[1506] A "terminal" is a device (e.g., a smartphone, tablet, or PC) on which a user uses a messaging app.

[1507] This invention is a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app, and automatically handles all steps from scheduling appointments to selecting a restaurant and making a reservation. This system is realized by a server, a user terminal, a messaging app, and virtual assistant software.

[1508] Explanation of program processing

[1509] User-entered keywords

[1510] A user enters a specific keyword, such as "Let's go for a drink!", through a messaging app (e.g., a general chat app). This message is sent to the server through the messaging app's API. The server receives and analyzes the keyword.

[1511] The rise of virtual assistants

[1512] When the server recognizes a specific keyword, it issues an instruction to trigger the appearance of the virtual assistant. The server requests the device to start the virtual assistant. Using virtual assistant software (e.g., Dialogflow), a message is displayed on the user's device saying, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants."

[1513] Schedule adjustment

[1514] The user inputs the desired date and time into the virtual assistant. For example, "I'm free after 7 PM on May 10th." The server receives this schedule information and stores it in a database (e.g., MySQL, PostgreSQL). The server then sends a similar message to other participants to collect their schedule information. For example, it sends a message saying, "Please tell me your free dates." When other participants respond, the server also receives that information and stores it in the database. The server aggregates the schedule information of all participants and determines the optimal date and time.

[1515] Shop selection

[1516] The server, through a virtual assistant, displays on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc." The user enters, "Izakaya is good." The server receives this information and searches for izakayas in the specified area using the Google Maps API, Yelp API, etc. The server uses review sites and store information APIs (such as Gurunavi and Hot Pepper) to generate a list of multiple candidates with high satisfaction rates, and sends this information to the user's device. The device then displays, "How about the following izakayas? 1. Store A 2. Store B 3. Store C."

[1517] Choosing a restaurant and making a reservation

[1518] The user selects "2. Restaurant B is good." The server saves this selection information in the database. The server uses the reservation API to reserve Restaurant B. The server sends a reservation completion message to the terminal and displays detailed information. This message includes the details, "Reservation for Restaurant B has been completed. The date and time is May 10th at 7 PM."

[1519] Specific examples

[1520] For example, if User A types "Let's go for a drink!" into a messaging app, the server receives and analyzes the keyword, and the virtual assistant launches. The process then automatically adjusts the date, selects a restaurant, and makes a reservation. This allows User A to get everything ready with just a few inputs.

[1521] Prompt Sentence Examples

[1522] You can simulate the functionality of a virtual assistant by inputting the following prompt sentences into a generative AI model (e.g., ChatGPT):

[1523] The user types "Let's go out for drinks!" into a messaging app. As a virtual assistant, you should display a message saying "Hello, this is Let's go out for drinks! AI. Let's first arrange the dates for the participants." After that, collect the participants' preferred dates and times, decide on the optimal date, and then proceed with selecting a restaurant and making a reservation.

[1524] This prompt can then be followed by a response or next step from the virtual assistant.

[1525] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1526] Step 1:

[1527] A user uses a messaging app to input a specific keyword, such as "Let's go for a drink!" The input keyword is sent to the server through the messaging app's API. The server receives the keyword and uses a text analysis algorithm to analyze it. As an output, the analysis results generate a flag that triggers the launch of the virtual assistant.

[1528] Step 2:

[1529] The server issues an instruction to start the virtual assistant. It sends message data to the device to display, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants." The device displays this message to the user. It uses virtual assistant software (e.g., Dialogflow) to output the message in text format.

[1530] Step 3:

[1531] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th." This date and time information is sent to the server and saved in a database. The server receives this as input and generates a message template to send similar messages to other participants. As output, it sends a message waiting for input from other participants.

[1532] Step 4:

[1533] The server receives the schedule information of other participants and stores it in a database. It aggregates the schedule information of all participants and runs an algorithm to calculate the optimal date and time. This algorithm takes participant response data as input and performs duplication and optimization. The determined optimal date and time is generated as output.

[1534] Step 5:

[1535] The server notifies the device of the optimal date and displays a message to the user through a virtual assistant saying, "Please tell us your preferences. For example, izakaya, bar, yakiniku, etc." The device displays this message to the user, who then inputs their preferences. The server receives this preference information as input.

[1536] Step 6:

[1537] The server searches for izakayas in a specified area via the Internet using the Google Maps API, Yelp API, etc. The user's preferences and local information are used as input. The server collects highly rated restaurant candidates through review sites and restaurant information APIs, generates a list of candidates, and sends it to the device. The output is a list containing multiple restaurant candidates.

[1538] Step 7:

[1539] The terminal displays "Would you like to try one of the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C," and the user selects "2. Restaurant B." The selection information is sent to the server and saved in the database. The server receives this selection information as input and begins the process of making a reservation for Restaurant B using the reservation API.

[1540] Step 8:

[1541] The server confirms the reservation for Store B through the reservation API, generates a reservation completion message, and sends it to the device. Detailed information such as "Reservation for Store B has been completed. The date and time is May 10th at 7 PM" is displayed on the device. This allows the user to confirm that the reservation has been confirmed.

[1542] Step 9:

[1543] The server sequentially verifies and updates the status of schedule adjustment, store selection, and reservation, and sends notifications to the user's device so that the user can check the progress.

[1544] (Application example 1)

[1545] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1546] Conventional food delivery services require users to go through several cumbersome procedures when using them. Specifically, users must launch the delivery service app, search for the food and restaurant that suits their preferences, and then order and pay separately. Furthermore, users are sometimes not notified of the order status or estimated delivery time, which can be inconvenient for them. Given this current situation, there is a demand for a system that allows users to use delivery services simply.

[1547] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1548] In this invention, the server includes: means for a virtual assistant to automatically appear on the chat screen when a user inputs a specific keyword in a messaging app; means for the virtual assistant to arrange schedules for participants; means for the virtual assistant to listen to participants' preferences and search for delivery locations via the Internet; means for the virtual assistant to suggest candidate delivery locations and automatically place orders and reservations based on the user's selection; means for the virtual assistant to prioritize highly rated delivery locations when searching for delivery locations in a specific area using an API of a review site; means for the virtual assistant to use an electronic payment service to process payment for the order; and means for the virtual assistant to notify the user of the completion of the order and the estimated delivery time. This enables users to easily use food delivery services from a messaging app, avoiding cumbersome procedures and keeping track of the situation in real time.

[1549] A "messaging app" is software that users use to send and receive text messages and other information.

[1550] "Specific keywords" are specific words or phrases that users type into messaging apps to indicate specific intent or requests.

[1551] A "virtual assistant" is a software agent that automatically responds to user input and performs specific tasks.

[1552] "Participant scheduling" is the process of checking the available times and days of multiple participants to determine the best date and time.

[1553] "Hearing participant preferences" is the process of asking and collecting the preferences and wishes of users and participants.

[1554] "Searching via the Internet" refers to the act of searching for information or stores through online networks.

[1555] "Search for locations" means searching for service locations such as restaurants and delivery services based on specified conditions.

[1556] "Proposing destination candidates" is a process of presenting multiple options to the user from the search results.

[1557] "Placing an order or reservation based on the user's selection" means automatically placing an order or reservation with the provider selected by the user.

[1558] "Using the API of a rating site" means using the application programming interface provided by other websites that provide ratings and reviews.

[1559] "Preferentially selecting providers with high ratings" means giving priority to selecting service providers with high user ratings and reviews.

[1560] "Using electronic payment services to process payments" means using digital payment methods such as credit cards or electronic money to make online payments.

[1561] "Notifying the user of order completion and estimated delivery time one by one" is a process in which the user is notified of information about the estimated delivery time one by one from the time the order is confirmed.

[1562] This invention is a system in which a virtual assistant automatically appears on the chat screen when a user inputs a specific keyword into a messaging app, and automatically handles everything from scheduling to selecting a delivery destination, placing an order, making a payment, and notifying the user of the estimated delivery time. The following describes in detail an embodiment of the invention.

[1563] Generating a Program

[1564] The system for realizing the present invention consists of a server, a terminal, and a user. The server receives specific keywords from the user and launches the virtual assistant. Specifically, it provides an API endpoint using a lightweight web framework called Flask. It also uses a library called Requests to call external APIs (for example, the API of a rating site or the API of an electronic payment service).

[1565] Natural language explanation of the process

[1566] The server consists of a means to:

[1567] 1. Message reception and analysis: When a user enters a specific keyword (e.g., "Delivery Master, I'm hungry") into the messaging app, the server receives and analyzes the message via API. Depending on the results of this analysis, it launches the virtual assistant.

[1568] 2. Launching a virtual assistant: When the server recognizes a specific keyword, a virtual assistant will automatically appear on the device and interact with the user. For example, if a user types "I want to eat pizza," the assistant will respond, "Hello, this is Delivery Master. Please tell us what you want to eat."

[1569] 3. Search and selection of restaurants: The server obtains the user's location information and searches for restaurants (for example, nearby pizza restaurants). At this time, it uses the API of rating sites such as Tabelog to prioritize restaurants with high ratings.

[1570] 4. Ordering and reservation procedures: The virtual assistant automatically places an order and makes a reservation for the delivery location selected by the user. For example, if the user selects "Pizza Hut Margherita," the server processes the order details and places the order with the delivery location.

[1571] 5. Electronic Payment: The server uses an electronic payment service (e.g., PayPal) to process payment for the order. The server processes the payment securely based on the user's credit card information.

[1572] 6. Status Notification: Once the order is completed, the virtual assistant will notify the user of important information such as the estimated delivery time. For example, "Your order is complete. The estimated delivery time is 7:00 PM."

[1573] Adding specific examples

[1574] For example, if a user types "Delivery Master, I'm hungry" into a messaging app, the following sequence of events will be processed by the server:

[1575] The server recognizes the user's input and the virtual assistant displays, "Hello, this is Delivery Master. Please tell us what you would like to eat."

[1576] If the user answers "pizza," the server searches for the best pizza place via the Internet based on the user's location information.

[1577] It then suggests multiple options to the user (e.g., "Pizza Hut A, Domino's Pizza B, Salvatore C").

[1578] When a user selects "Pizza Hut A, Margherita," the server places the order and processes the payment using PayPal.

[1579] Finally, the virtual assistant announces, "Your order is complete. Estimated delivery time is 7:00 PM."

[1580] Example prompt sentence:

[1581] "Delivery Master, I'm hungry."

[1582] This allows users to easily use food delivery services and eliminates complicated procedures.

[1583] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1584] Step 1:

[1585] The server receives a message in which the user enters a specific keyword, "Delivery Master, I'm hungry," into a messaging app. The server retrieves this message via an API and analyzes the keyword. The input data is the user's message, and the output is the recognition of the specific keyword. Based on this recognition, the launch of the virtual assistant is triggered.

[1586] Step 2:

[1587] The terminal receives instructions from the server, launches the virtual assistant, and displays a message to the user saying, "Hello, this is Delivery Master. Please tell us what you would like to eat." The input here is the instruction from the server, and the output is the message displayed on the terminal.

[1588] Step 3:

[1589] The user responds to the virtual assistant's message with "pizza." This user input is sent to the server via the terminal. The input data is the user's response message and is sent to the server.

[1590] Step 4:

[1591] The server obtains the user's location information based on the dish information received from the user. Using this location information, it searches for restaurants via the Internet. The server calls a rating site API to search for highly rated pizza restaurants in the specified area. The input data is the user's location information and preferred dishes, and the output data is a list of candidate restaurants.

[1592] Step 5:

[1593] The server proposes a list of providers obtained from the results of the rating site API to the user through a virtual assistant. For example, it displays a message such as, "Which would you like: Pizza Hut A, Domino's Pizza B, or Salvatore C?" The input here is a list of candidate providers, and the output is a proposal message to the user.

[1594] Step 6:

[1595] The user selects "Pizza Hut A, Margherita" from the presented list of destinations and inputs it into the virtual assistant. This information is sent to the server via the terminal. The input data is the user's selection information and is sent to the server.

[1596] Step 7:

[1597] The server obtains the user's selection information and places an order with the specified delivery destination. Using the order API, it executes an order for "Pizza Hut A, Margherita" with the delivery destination. The input data is the product information selected by the user, and the output data is the order completion status.

[1598] Step 8:

[1599] After the order is completed, the server processes the payment using an electronic payment service such as PayPal. When the payment is successfully completed, the result is obtained. The input data is the payment information, and the output data is the payment completion status.

[1600] Step 9:

[1601] After the server confirms that the order and payment procedures have been completed, it sends a message to the user through the virtual assistant saying, "The order has been completed. The estimated delivery time is 7:00 PM." The input here is the order and payment completion status, and the output is a notification message to the user.

[1602] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1603] This invention combines a system in which a virtual assistant automatically appears when a user inputs a specific keyword into a messaging app and automatically performs a series of processes such as scheduling, selecting a restaurant, and making a reservation, with an emotion engine. The emotion engine has the function of recognizing the user's emotional state and providing appropriate dialogue based on that. This system operates as follows.

[1604] Explanation of program processing

[1605] User-entered keywords

[1606] A user types a specific keyword, such as "Let's go for a drink!", through a messaging app.

[1607] The server receives and analyzes this keyword through the messaging app's API.

[1608] The rise of virtual assistants

[1609] When the server recognizes a specific keyword, it generates data to trigger the virtual assistant and sends it to the device.

[1610] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1611] Emotion Engine Operation

[1612] The server receives the user's message and analyzes the user's emotional state using an emotion engine.

[1613] The emotion engine analyzes the user's emotional state and provides the results to the virtual assistant, for example, identifying whether the user is happy or stressed.

[1614] The virtual assistant on the device will adjust the tone and content of messages to match the user's emotions. For example, if the user is feeling stressed, it will prioritize suggestions for places to relax.

[1615] Schedule adjustment

[1616] The user inputs the desired date and time into the virtual assistant. For example, they input "I'm free after 7 PM on May 10th."

[1617] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[1618] Shop selection

[1619] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1620] The user types, "Izakaya is good."

[1621] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[1622] The server selects some of the candidates it has obtained and sends that information to the terminal.

[1623] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1624] Choosing a restaurant and making a reservation

[1625] The user selects "2. Shop B is better."

[1626] The server receives this selection information and stores it in a database.

[1627] The server uses the reservation API to make a reservation at Restaurant B. The server sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[1628] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[1629] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[1630] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[1631] Specific examples

[1632] For example, if User A types "Let's go out for drinks!" into a messaging app, the server responds immediately and a virtual assistant appears. The virtual assistant then displays, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants." User A enters a date, such as "May 10th after 7 PM," and the optimal date is determined. The virtual assistant then asks, "What are your preferences, such as izakaya, bar, or yakiniku?" and the user answers, "izakaya." The virtual assistant then searches for potential izakayas via the internet and automatically makes a reservation at the restaurant selected by the user. The emotion engine analyzes the user's emotions and makes optimal suggestions, providing a more personalized service.

[1633] In this way, users can set up a drinking party without going through any complicated procedures, simply by typing "Let's go drinking!". In addition, the system makes suggestions and responds based on the user's emotions, resulting in a service that provides a higher level of satisfaction.

[1634] The processing flow will be explained below.

[1635] Step 1:

[1636] A user types "Let's go for a drink!" into a messaging app. This message is sent to the server via the messaging app's API.

[1637] Step 2:

[1638] The server analyzes the received message and recognizes the specific keyword "Let's go for a drink!"

[1639] Step 3:

[1640] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device.

[1641] Step 4:

[1642] A virtual assistant appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1643] Step 5:

[1644] The user inputs the desired date and time into the virtual assistant. For example, they might input "I'm available after 7 PM on May 10th."

[1645] Step 6:

[1646] The server receives this schedule information and stores it in a schedule database. If necessary, it sends a similar message to other participants to collect schedule information.

[1647] Step 7:

[1648] The server receives the responses of other participants and also stores their itinerary information in the itinerary database. Once all participants' itineraries are collected, the server analyzes the data to determine the optimal itinerary.

[1649] Step 8:

[1650] The server determines the optimal schedule and sends that information to the terminal, which then notifies the terminal of the optimal schedule.

[1651] Step 9:

[1652] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1653] Step 10:

[1654] The user types, "Izakaya is good."

[1655] Step 11:

[1656] The server receives the information and searches for izakayas in the specified area via the Internet, using the review site's API to retrieve multiple candidates.

[1657] Step 12:

[1658] The server selects some of the candidates it has obtained and sends that information to the terminal.

[1659] Step 13:

[1660] The virtual assistant will display a message on the device saying, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1661] Step 14:

[1662] The user selects "2. Shop B is better."

[1663] Step 15:

[1664] The server receives this selection information and stores it in a database.

[1665] Step 16:

[1666] The server uses the reservation API to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[1667] Step 17:

[1668] If the reservation is successful, the server receives the reservation confirmation information and also performs error handling.

[1669] Step 18:

[1670] The server stores the reservation confirmation information in a database and transmits the information to the terminal.

[1671] Step 19:

[1672] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[1673] Step 20:

[1674] The server activates an emotion engine to identify emotions from the user's messages and inputs.

[1675] Step 21:

[1676] The server uses an emotion engine to analyze the user's message and detect their emotional state (e.g., joy, stress, frustration, etc.).

[1677] Step 22:

[1678] The virtual assistant on the device will respond appropriately based on the results of emotion analysis. For example, it will suggest a store where a user can relax if they are feeling stressed.

[1679] Step 23:

[1680] The server tracks the user's emotional state and adjusts the tone and content of the message in real time.

[1681] Step 24:

[1682] The user receives suggestions and responses that correspond to their emotions, and the drinking party planning proceeds in an emotionally satisfied state.

[1683] Example 2

[1684] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1685] Currently, when using a messaging app to schedule a drinking party, the tasks of scheduling, selecting a restaurant, and making a reservation are extremely cumbersome and time-consuming. Furthermore, the system does not respond to the user's emotional state, which can lead to a decrease in participant satisfaction. Conventional systems do not offer a method for efficiently resolving these issues. Therefore, the present invention aims to solve these problems.

[1686] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1687] In this invention, the server includes: a means for automatically having a virtual assistant appear on the chat screen when a user inputs a specific keyword into a messaging app; a means for the virtual assistant to arrange schedules for participants; a means for the virtual assistant to listen to participants' preferences and search for restaurants via the Internet; a means for the virtual assistant to suggest candidate restaurants and automatically make reservations based on the user's selection; and a means for analyzing the user's emotional state using emotion analysis technology installed in the system and providing appropriate dialogue based on the analysis results. This allows users to easily and efficiently set up drinking parties using a messaging app and enables responses according to the user's emotional state.

[1688] "User" refers to an individual or group that uses this system.

[1689] "Messaging app" refers to a software application for sending and receiving text and multimedia messages over the Internet.

[1690] A "virtual assistant" refers to a computer program that automatically performs tasks through user interaction.

[1691] A "server" refers to a computer system that provides services to clients over a network.

[1692] "Emotion analysis technology" refers to technology for analyzing a user's emotional state from text or voice data.

[1693] "Schedule adjustment" refers to the process of checking the free times of multiple users and determining the optimal date and time.

[1694] "Store" refers to a place that a user visits, and in this case, it mainly refers to a restaurant.

[1695] "Hearing" refers to the act of collecting information from users.

[1696] "Internet" refers to the global system of computer networks and infrastructure that enables information sharing and communication.

[1697] "Reservation" refers to the act of a user making a reservation in advance to receive a specific service at a specific date and time.

[1698] "Candidates" refers to a list of options that a user can choose from.

[1699] "Analysis" refers to the process of examining data in detail to reveal its meaning and structure.

[1700] "Dialogue" refers to the exchange of information between a user and a virtual assistant.

[1701] This invention is a system in which, when a user inputs a specific keyword into a messaging app, a virtual assistant automatically appears and performs a series of steps such as scheduling, selecting a restaurant, making a reservation, etc. Furthermore, by incorporating emotion analysis technology, this system provides optimal dialogue according to the user's emotional state.

[1702] The system is implemented using the following hardware and software:

[1703] Server: A computer system that provides services to clients over a network, especially cloud services such as Amazon Web Services (AWS).

[1704] Device: Any device used by a user, including internet-enabled devices such as smartphones and tablets.

[1705] Messaging app: A software application for sending and receiving text and multimedia messages, such as LINE or WhatsApp.

[1706] Emotion analysis technology: Technology for analyzing the user's emotional state. For example, IBM Watson Tone Analyzer is used.

[1707] Schedule Database: A database system for storing schedule information. It uses the Google Calendar API.

[1708] Review site API: An API for obtaining store information. For example, the Yelp API is used.

[1709] Reservation API: An API for making restaurant reservations. For example, the OpenTable API is used.

[1710] As a specific example of operation, consider the following scenario.

[1711] User-entered keywords

[1712] When a user types a specific keyword, such as "Let's go for a drink!", through a messaging app, the device sends the input to a server, which receives the keyword through the messaging app's API and analyzes it.

[1713] The rise of virtual assistants

[1714] When the server recognizes the keyword, it generates data to trigger the virtual assistant and sends it to the device. The virtual assistant then appears on the device and displays the message, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1715] How emotion analysis technology works

[1716] The server receives the user's message and uses emotion analysis technology to analyze the user's emotional state. For example, if a user sends a message saying, "I've been busy and stressed lately," the server sends the message to emotion analysis technology and identifies the user as feeling stressed. The virtual assistant on the device adjusts the tone and content of the message to match the user's emotions, displaying, "I'll find you a great izakaya to relieve your stress!"

[1717] Schedule adjustment

[1718] When a user inputs a desired date and time into the virtual assistant, for example, "I'm free after 7 PM on May 10th," the server saves the date and time information in a schedule database and, if necessary, sends similar messages to other participants to collect their schedule information.

[1719] Store selection

[1720] The virtual assistant displays on the device, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc." The user enters, "Izakaya is good." The server receives this information, searches for izakaya in the specified area via the Internet, and retrieves multiple candidates using the review site's API. The server selects several from the retrieved candidates and sends this information to the device. The message displayed is, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1721] Choosing a store and making a reservation

[1722] The user selects "2. Restaurant B is good." The server receives this selection and saves it in a database. The server uses the reservation API to send the necessary reservation information (date and time, number of people, names, etc.). If the reservation is successful, the server receives the reservation confirmation information and also performs error handling. The reservation confirmation information is saved in a database and sent to the terminal. The virtual assistant displays a message on the terminal saying, "Reservation for Restaurant B has been completed. Details are below."

[1723] Example prompts for generative AI models

[1724] "Please tell me the step-by-step process for hosting a drinking party at an izakaya after 7 PM on May 10th. Please include everything from coordinating the schedules of the participants to selecting the restaurant and making the reservation. Please also take into consideration the feelings of the participants."

[1725] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1726] Step 1: User enters specific keywords

[1727] A user types a specific keyword into a messaging app, such as "Let's go for drinks!"

[1728] The terminal sends the input to the application server.

[1729] Input: The keyword "Let's go for a drink!"

[1730] Output: The entered keyword is sent to the server

[1731] Specific actions: User A opens LINE on their smartphone and sends a message to a friend saying, "Let's go for drinks!"

[1732] Step 2: The server receives and parses the keyword

[1733] The server receives the keywords through the messaging app's API and analyzes the content. The analysis process is implemented in Python and JavaScript.

[1734] Input: Message data containing keywords

[1735] Output: Data that determines actions based on keywords

[1736] What it does: The server uses Amazon Web Services (AWS) Lambda to analyze the received message and detect the keyword "Let's go for a drink!"

[1737] Step 3: The virtual assistant arrives

[1738] When the server recognizes a specific keyword, it generates data to trigger the virtual assistant and sends it to the device.

[1739] A virtual assistant appears on the device and displays an initial message.

[1740] Input: Keyword analysis results

[1741] Output: Virtual assistant trigger data, initial message

[1742] Specific operation: The server generates data and displays the AI ​​assistant on User A's smartphone. The message displayed is, "Hello, let's go out for drinks! This is AI. Let's first arrange the schedules for the participants."

[1743] Step 4: Analyze user emotions using emotion analysis technology

[1744] The server receives the user's additional message and analyzes the user's emotional state using emotion analysis technology, such as IBM Watson Tone Analyzer.

[1745] Input: Message from the user (e.g., "I've been busy and stressed lately.")

[1746] Output: Sentiment analysis result (e.g. "I feel stressed")

[1747] What it does: The server sends an additional message to the emotion analysis technology, identifying it as stressed.

[1748] Step 5: Adjust the dialogue based on the sentiment results

[1749] The device receives the emotion analysis results, and the virtual assistant responds appropriately, adjusting the tone and message content according to the emotion.

[1750] Input: Sentiment analysis results

[1751] Output: Adjusted message content

[1752] Specific operation: The virtual assistant displays a message on the device saying, "I'll find you a great izakaya to relieve your stress!"

[1753] Step 6: Propose a schedule

[1754] The user inputs the desired date and time into the virtual assistant. For example, "I'm available after 7 PM on May 10th."

[1755] The server receives the schedule information and stores it in a schedule database (e.g., Google Calendar API). It also sends similar messages to other participants to collect their schedule information.

[1756] Input: Date information

[1757] Output: Date information stored in the database, messages sent to other participants

[1758] Specific behavior: User A enters "I'm free after 7pm on May 10th," and the server saves this to the Google Calendar API.

[1759] Step 7: Store Selection

[1760] A virtual assistant will appear on the device and ask, "Please tell us your preferences, participants. For example, izakaya, bar, yakiniku, etc."

[1761] The user types, "Izakaya is good."

[1762] The server receives the information and searches for izakayas in the specified area via the Internet, using the API of a review site (e.g., Yelp API) to obtain multiple candidates.

[1763] Input: Type of store (e.g. "Izakaya")

[1764] Output: Multiple store candidates

[1765] Specific operation: The user enters "izakaya" (Japanese pub) on their device, and the information is sent to the server. The server uses the Yelp API to search for izakayas, selects "Store A," "Store B," or "Store C," and sends the information back to the user's device.

[1766] Step 8: Virtual assistant suggests stores

[1767] The virtual assistant will display on the device, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C."

[1768] Input: Multiple store candidates

[1769] Output: List of store candidates

[1770] Specific operation: A list of "1. Store A," "2. Store B," and "3. Store C" is displayed on the user's device.

[1771] Step 9: User selects store

[1772] The user selects "2. Shop B is better."

[1773] The server receives this selection information and stores it in a database.

[1774] Input: Selected store information

[1775] Output: Selections stored in a database

[1776] Specific operation: The user enters "2. Shop B is good," and the server saves the data in the database.

[1777] Step 10: Server reserves the store

[1778] The server uses a reservation API (e.g., OpenTable API) to make a reservation at Restaurant B. It sends the necessary reservation information (date, time, number of people, names, etc.) to the reservation API.

[1779] If the reservation is successful, the server receives the reservation confirmation information, performs error handling, saves the reservation confirmation information in the database, and sends the information to the terminal.

[1780] Input: Selected store information, reservation information

[1781] Output: Booking confirmation information, error message (if necessary)

[1782] Specific operation: The server makes a reservation for Restaurant B using the OpenTable API, and the reservation is confirmed.

[1783] Step 11: Notify user of reservation confirmation

[1784] The virtual assistant displays the message on the device: "Your reservation for Restaurant B has been completed. Details are below."

[1785] Input: Confirmed reservation information

[1786] Output: Display a confirmation message

[1787] Specific operation: Detailed information such as "Reservation at Restaurant B completed. Date, time, location, reservation name" will be displayed on the user's device.

[1788] (Application example 2)

[1789] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1790] In modern society, where people are busy every day, coordinating group schedules and making restaurant reservations can be cumbersome and time-consuming. To solve these problems, there is a need for a system that allows users to easily coordinate schedules and make restaurant reservations. Furthermore, there is a need to provide more personalized and comfortable services by responding to each user's emotional state.

[1791] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for automatically causing a virtual agent to appear on the screen when a user inputs a specific keyword into a communication application; means for the virtual agent to arrange schedules for participants; means for the virtual agent to collect participants' preferences and search for stores via a digital network; means for the virtual agent to suggest candidate stores and automatically make reservations based on the user's selection; and means for the virtual agent to recognize the user's emotional state using an emotion analysis engine and provide appropriate dialogue. This allows users to easily arrange schedules and make store reservations without hassle, and to receive personalized services tailored to their emotions, simply by inputting a specific keyword.

[1792] A "communications application" is software that users use to exchange messages, such as text, voice, or images.

[1793] A "virtual agent" is a software agent that interacts with a user and automatically performs various tasks.

[1794] An "emotion analysis engine" is a system for analyzing and identifying a user's emotional state from text or voice.

[1795] A "digital network" refers to a digital communications network such as the Internet, which is a medium for transmitting and receiving data.

[1796] "User's Emotional State" means the momentary or ongoing psychological or emotional state of a User.

[1797] "User preferences" refers to the tastes and preferences that a user exhibits for a particular category or condition.

[1798] "Store candidates" is a list of multiple stores that may be suitable for the user's request.

[1799] "Reservation" refers to the advance procedure to reserve a specific service or location for a specified date and time.

[1800] The system of this invention allows a user to input a specific keyword into a communication application, and a virtual agent appears and automatically executes a series of subsequent tasks. It also uses an emotion analysis engine that recognizes the user's emotional state and adjusts the content of the dialogue accordingly.

[1801] System configuration

[1802] The system consists of three main components: the server, the terminal, and the user. The following is a detailed description of each component.

[1803] User

[1804] A user inputs a specific keyword through a communication application. For example, it is an everyday keyword such as "Let's go for drinks!" or "Want to go to dinner?". The user uses a device such as a smartphone or tablet.

[1805] Terminal

[1806] The terminal functions as an interface with the user, providing a screen for the virtual agent to interact with the user and collecting input from the user. Based on the user's input, the virtual agent then converses with the user to arrange a schedule or select a store.

[1807] server

[1808] The server performs the main processing of the virtual agent and the emotion analysis engine. It receives input from the user and analyzes their emotional state using the emotion analysis engine. This includes the following main processes:

[1809] 1. Keyword analysis: The server detects specific keywords in the user's input and triggers the virtual agent.

[1810] 2. Sentiment Analysis: Identify the user's emotional state using an emotion analysis engine (e.g., TextBlob). Adjust the virtual agent's dialogue accordingly.

[1811] 3. Schedule adjustment: Collects the user's schedule information and automatically selects the optimal schedule.

[1812] 4. Store Selection and Reservation: The virtual agent collects the user's preferences, searches for stores via the Internet, suggests suitable store candidates to the user, and makes a store reservation based on the user's selection.

[1813] Specific examples

[1814] For example, if User A types "Let's go for drinks!" into a communication application, the server recognizes the specific keyword and a virtual agent appears. The virtual agent displays the message, "Hello, let's go for drinks! This is AI. Let's first arrange the schedule for the participants." If User A types "May 10th after 7 PM," the server collects schedule information and determines the optimal date. The virtual agent then asks, "What are your preferences? For example, izakaya, bar, yakiniku, etc." The user responds, "Izakaya." The server searches for potential izakayas via the Internet, and the virtual agent suggests, "How about the following izakayas? 1. Restaurant A 2. Restaurant B 3. Restaurant C." If the user selects "2. Restaurant B is good," the server makes the reservation, and once the reservation is complete, the virtual agent notifies the user, "Your reservation for Restaurant B has been completed. Details are below."

[1815] Example prompts for generative AI models

[1816] Below are some examples of prompts for generative AI models:

[1817] Analyze the following user messages and generate appropriate dialogue messages depending on the user's emotional state:

[1818] User message: "Tired after work, let's go for a drink!"

[1819] User data: {"date": "After 7 PM on May 10th", "preference": "Izakaya"}

[1820] Example of a generated conversation message:

[1821] """

[1822] Hello, let's go drinking! This is AI. Thank you for your hard work. I found a place where you can relax. How about going to an izakaya after 7pm on May 10th?

[1823] 1. Izakaya A (Rating: 4.3)

[1824] 2. Izakaya B (Rating: 4.5)

[1825] Which restaurant would you like to book?

[1826] """

[1827] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1828] Step 1:

[1829] Input: A user types a specific keyword (e.g., "Let's go for a drink!") into a communication application.

[1830] Operation: The terminal sends the user's input message to the server.

[1831] Output: Messages containing the keyword are sent to the server.

[1832] Step 2:

[1833] Input: The server receives the user's input message.

[1834] How it works: The server analyzes specific keywords (e.g., "Let's go for a drink!") and triggers the virtual agent.

[1835] Output: A virtual agent appears on the terminal and says, "Hello, let's go out for drinks! This is AI. Let's first arrange the dates for the participants."

[1836] Step 3:

[1837] Input: The user who receives the message from the virtual agent inputs the desired date (e.g., "After 7 PM on May 10th").

[1838] Operation: The terminal sends the user's input schedule to the server.

[1839] Output: The schedule information is sent to the server.

[1840] Step 4:

[1841] Input: The server receives the user's schedule information.

[1842] How it works: The server parses the schedule information and stores it in its internal schedule database. It also sends similar messages to other participants as needed to collect schedule information.

[1843] Output: The optimal date is determined, and the virtual agent displays a message on the terminal saying, "Please tell us your preferences. For example, izakaya, bar, yakiniku, etc."

[1844] Step 5:

[1845] Input: The user inputs the type of store they want (e.g., "izakaya").

[1846] Operation: The terminal sends the user's input information to the server.

[1847] Output: Store preference information is sent to the server.

[1848] Step 6:

[1849] Input: The server receives the user's store preference information.

[1850] How it works: The server searches for establishments (e.g., "izakaya") in a specified area via the Internet, retrieves multiple candidates using the review site's API, analyzes the user's emotional state using a sentiment analysis engine, and selects the best candidate.

[1851] Output: Multiple store candidates (e.g., "Store A," "Store B," and "Store C") are obtained, and the virtual agent displays a message on the terminal saying, "Would you like to try one of the following izakayas? 1. Izakaya A 2. Izakaya B 3. Izakaya C."

[1852] Step 7:

[1853] Input: The user enters the number of the desired store (e.g., "2. Izakaya B").

[1854] Operation: The terminal sends the user's selection information to the server.

[1855] Output: The selection is sent to the server.

[1856] Step 8:

[1857] Input: The server receives the user's selection.

[1858] Operation: The server uses the reservation API to make a reservation at the specified store. It sends the necessary reservation information (e.g., date and time, number of people, names, etc.) to the reservation API. If the reservation is successful, it receives the reservation confirmation information and also performs error handling.

[1859] Output: The reservation confirmation information is saved on the server and sent to the device.

[1860] Step 9:

[1861] Input: The terminal that receives the reservation confirmation information displays a message to the user through the virtual agent.

[1862] Action: The virtual agent displays the message "Your reservation at Store B has been completed. Details below."

[1863] Output: The user is notified that the reservation has been completed.

[1864] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1865] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1866] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1867] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1868] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1869] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1870] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1871] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1872] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1873] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1874] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1875] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1876] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1877] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1878] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1879] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1880] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1881] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1882] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1883] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1884] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1885] The following is further disclosed regarding the above embodiment.

[1886] (Claim 1)

[1887] When users enter a specific keyword in a messaging app, a virtual assistant will automatically appear on the chat screen.

[1888] A means for the virtual assistant to schedule participants;

[1889] A means for the virtual assistant to hear the preferences of the participants and search for stores via the Internet;

[1890] A means for the virtual assistant to suggest candidate stores and automatically make a reservation based on the user's selection;

[1891] A system including:

[1892] (Claim 2)

[1893] 2. The system of claim 1, wherein the specific keyword is a suggestion for a drinking party.

[1894] (Claim 3)

[1895] The system of claim 1, wherein the virtual assistant has a means for informing the user of the status of schedule adjustment, store selection, and reservation completion.

[1896] "Example 1"

[1897] (Claim 1)

[1898] When users enter a specific keyword in a messaging app, a virtual assistant will automatically appear on the chat screen.

[1899] A means for the virtual assistant to schedule participants;

[1900] A means for the virtual assistant to hear the preferences of the participant, search for stores via the Internet, and present multiple candidates;

[1901] A means for the virtual assistant to automatically reserve a store based on the user's selection;

[1902] A means for the virtual assistant to send a notification of reservation completion to the user's terminal;

[1903] A system including:

[1904] (Claim 2)

[1905] 2. The system of claim 1, wherein the specific keyword is a suggestion for a drinking party.

[1906] (Claim 3)

[1907] The system of claim 1, wherein the virtual assistant has a means for informing the user of the status of scheduling, store selection, and reservation completion.

[1908] "Application Example 1"

[1909] (Claim 1)

[1910] When users enter a specific keyword in a messaging app, a virtual assistant will automatically appear on the chat screen.

[1911] A means for the virtual assistant to schedule participants;

[1912] A means for the virtual assistant to hear the preferences of the participants and search for providers via the Internet;

[1913] A means for the virtual assistant to suggest possible destinations and automatically place orders and reservations based on the user's selection;

[1914] A means for using an API of a rating site to preferentially select providers with high ratings when the virtual assistant searches for providers in a specific area;

[1915] means for the virtual assistant to use an electronic payment service to process payment for the order;

[1916] A means for the virtual assistant to notify the user of order completion and estimated delivery time;

[1917] A system including:

[1918] (Claim 2)

[1919] 2. The system of claim 1, wherein the specific keywords are meal or food delivery suggestions.

[1920] (Claim 3)

[1921] 10. The system of claim 1, wherein the virtual assistant has means for informing the user of the status of scheduling, destination selection, order payment, and completion of the scheduled delivery time.

[1922] "Example 2: Combining Emotion Engines"

[1923] (Claim 1)

[1924] When users enter a specific keyword in a messaging app, a virtual assistant will automatically appear on the chat screen.

[1925] A means for the virtual assistant to schedule participants;

[1926] A means for the virtual assistant to hear the preferences of the participants and search for stores via the Internet;

[1927] A means for the virtual assistant to suggest candidate stores and automatically make a reservation based on the user's selection;

[1928] means for analyzing the emotional state of a user using emotion analysis technology installed in the system and providing appropriate dialogue based on the analysis results;

[1929] A system including:

[1930] (Claim 2)

[1931] 2. The system of claim 1, wherein the specific keyword is a suggestion for a drinking party.

[1932] (Claim 3)

[1933] The system of claim 1, wherein the virtual assistant has a means for informing the user of the status of schedule adjustment, store selection, and reservation completion.

[1934] (Claim 4)

[1935] 2. The system of claim 1, wherein the emotion analysis technology includes means for analyzing a user's input message to identify the user's emotional state and adjusting the virtual assistant's dialogue content based on the results.

[1936] (Claim 5)

[1937] The system according to claim 1, wherein the virtual assistant has a means for sending messages to other participants when scheduling or selecting a store, and for collecting information about each participant.

[1938] "Application example 2 when combining emotion engines"

[1939] (Claim 1)

[1940] A virtual agent will automatically appear on the screen when the user enters a specific keyword in a communication application.

[1941] means for the virtual agent to schedule participants;

[1942] means for the virtual agent to collect participant preferences and search for stores via a digital network;

[1943] A means for the virtual agent to propose candidate stores and automatically make a reservation based on the user's selection;

[1944] a means for the virtual agent to recognize the emotional state of the user using an emotion analysis engine and provide appropriate dialogue;

[1945] A system including:

[1946] (Claim 2)

[1947] The system of claim 1 , wherein the specific keyword is a suggestion for going out.

[1948] (Claim 3)

[1949] 2. The system according to claim 1, wherein the virtual agent has means for informing the user of the status of schedule adjustment, store selection, and reservation completion. [Explanation of symbols]

[1950] ...

Claims

1. When users enter a specific keyword in a messaging app, a virtual assistant will automatically appear on the chat screen. A means for the virtual assistant to schedule participants; A means for the virtual assistant to hear the preferences of participants and search for stores via the Internet; A means for the virtual assistant to suggest candidate stores and automatically make a reservation based on the user's selection; A system including:

2. The system of claim 1 , wherein the specific keyword is a suggestion for a drinking party.

3. The system according to claim 1, wherein the virtual assistant has a means for informing the user of the status of schedule adjustment, store selection, and reservation completion.

Images