system

Abstract

We provide a system that streamlines reservation processes and reduces the burden on users. [Solution] A system including means for a user to input a request, means for a terminal to send the request to a server, means for the server to receive and analyze the request, means for the server to check the available dates and times for booking, means for the server to generate a response message and send it to the terminal, means for the terminal to display the response message to the user, means for a user to input a confirmation request, means for the terminal to send the confirmation request to the server, means for the server to receive the confirmation request and confirm the booking, means for the server to generate a booking confirmation message and send it to the terminal, and means for the terminal to display the booking confirmation message to the user.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003] [[ID=2)3]]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, when a user makes reservations for various services such as beauty salons, gyms, and hospitals, it has become a problem that it requires a lot of effort and time. In particular, since it is necessary to make a reservation by oneself through a phone call or a website, the procedure is complicated and burdensome for the user. Furthermore, when it is necessary to confirm or change a reservation, it is necessary to contact again, so there is a lack of convenience. There is a need to solve such problems, streamline the reservation business, and reduce the burden on users.

Means for Solving the Problems

[0005] The present invention solves the above problems by providing a system that includes means for a user to input a request, means for a terminal to send the request to a server, means for the server to receive and analyze the request, means for the server to check available dates and times for booking, means for the server to generate a response message and send it to the terminal, means for the terminal to display the response message to the user, means for a user to input a confirmation request, means for the terminal to send the confirmation request to the server, means for the server to receive the confirmation request and confirm the booking, means for the server to generate a booking confirmation message and send it to the terminal, and means for the terminal to display the booking confirmation message to the user. As a result, users can easily input requests from their terminals and the booking process is automated, significantly reducing the effort and time required. In addition, responses to booking confirmations and changes can be handled quickly, improving user convenience.

[0006] A "user" refers to an individual or group that uses the system to request a reservation.

[0007] A "terminal" refers to an electronic device, such as a computer or smartphone, that a user uses to input and receive requests.

[0008] A "server" refers to a computer system that receives, analyzes, and processes requests sent from a terminal.

[0009] A "request" refers to reservation request information that a user sends from their device to the server.

[0010] "Analysis" refers to the process by which a server understands the content of a received request and determines the appropriate response.

[0011] "Available dates and times for booking" refers to the time slots and dates within the time slots provided by the service provider that are available for booking.

[0012] A "response message" refers to a message related to a reservation that is generated by the server and sent to the user via the terminal.

[0013] A "confirmation request" refers to a request that a user sends from their device to the server to confirm their reservation.

[0014] "Reservation confirmation" refers to the process where the server receives a confirmation request and formally registers the reservation from the service provider.

[0015] A "reservation confirmation message" refers to a confirmation message generated by the server and sent to the user via their device after the reservation has been officially registered. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention relates to a system for automating and efficiently processing reservations requested by users. This system consists of a user, a terminal, and a server, with each program fulfilling its respective role.

[0038] System Overview

[0039] User: Enter your reservation request and send it to the server via your device.

[0040] Terminal: Receives user input, sends requests to the server, receives responses from the server, and displays them to the user.

[0041] Server: Receives user requests, parses them, checks reservation status, and generates response messages.

[0042] Program processing

[0043] 1. Entering and submitting the user's request.

[0044] A user wants to make a reservation at a hair salon and enters "I want to make a reservation at a hair salon" into the terminal. The terminal sends this input to the server.

[0045] 2. Receiving and parsing server requests

[0046] The server receives a request from the terminal and analyzes its content, such as "hair salon reservation." During this process, it extracts appropriate keywords.

[0047] 3. Check available dates and times for reservations.

[0048] The server queries the beauty salon's API or database to check for available appointment times. For example, the server might get a result like "Appointment is available at XX / XX."

[0049] 4. Generating and sending response messages

[0050] The server generates a response message based on the available dates and times, such as "You can make a reservation at XX / XX. Do you want to make a reservation?", and sends it to the terminal.

[0051] 5. Display of the terminal's response message

[0052] The terminal displays the received response message to the user. The user confirms the message and responds, "Yes, book it."

[0053] 6. Sending a user verification request

[0054] The device resends the user's confirmation request to the server.

[0055] 7. Server reservation confirmation process

[0056] The server receives the confirmation request and accesses the salon's reservation system to confirm the reservation. If the reservation is confirmed, the server generates a message saying "Reservation confirmed".

[0057] 8. Sending and displaying the reservation confirmation message.

[0058] The server sends a reservation confirmation message to the terminal, which then displays it to the user.

[0059] Specific example

[0060] The following is a specific scenario for a user who wants to make a reservation at a hair salon.

[0061] 1. User request input

[0062] User: "I want to make a reservation at a hair salon."

[0063] Terminal: Sends a request to the server

[0064] 2. Server analysis and reservation confirmation

[0065] Server: Analyzes "hair salon reservations"

[0066] Server: Check the availability of the hair salon (at XX / XX)

[0067] 3. Server response generation and transmission

[0068] Server: "You can make a reservation at [time]. Would you like to make a reservation?"

[0069] Server: Sends a response message to the terminal.

[0070] 4. Displaying messages on the device and verifying the user.

[0071] Terminal: Message display

[0072] User: "Yes, please make a reservation."

[0073] Terminal: Sends a confirmation request to the server.

[0074] 5. Confirm server reservation and send message

[0075] Server: Confirm reservation

[0076] Server: "Your reservation has been confirmed."

[0077] Server: Sends a reservation confirmation message to the terminal.

[0078] 6. Display of the device reservation confirmation message

[0079] Terminal: Reservation confirmation message displayed

[0080] Thus, the present invention allows users to easily make reservations, simplifying and streamlining the process. Furthermore, reservations can be quickly confirmed and modified, greatly improving user convenience.

[0081] The following describes the processing flow.

[0082] Step 1:

[0083] The user enters a request using the terminal, such as "I want to make a reservation at a hair salon." This input is triggered by the terminal's program.

[0084] Step 2:

[0085] The device sends the user's request as a string to the server. It uses protocols such as HTTP requests to send data to the specified endpoint.

[0086] Step 3:

[0087] The server receives a request sent from the terminal. It analyzes the content of the request and extracts the specific keyword "hair salon reservation".

[0088] Step 4:

[0089] The server accesses the hair salon's reservation system (API or database) to search for available dates and times. For example, it might query the reservation system's API endpoint to retrieve availability data.

[0090] Step 5:

[0091] The server generates a response message based on the available reservation dates and times. For example, it generates a message in the format of "You can make a reservation on [Month] [Day] at [Time]. Would you like to make a reservation?".

[0092] Step 6:

[0093] The server sends the generated response message to the terminal. Then, it sends the response data again using an HTTP request or similar method.

[0094] Step 7:

[0095] The terminal receives a response message from the server and displays it to the user. The user interface (UI) is updated to display the received message on the screen.

[0096] Step 8:

[0097] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their intention to confirm the reservation.

[0098] Step 9:

[0099] The device sends a user verification request to the server. The verification request is sent as an HTTP request, just like the initial request.

[0100] Step 10:

[0101] The server receives the confirmation request. It then accesses the reservation system (API or database) again to formally confirm the reservation.

[0102] Step 11:

[0103] The server generates a reservation confirmation message. For example, it might create a message in the format "Your reservation has been confirmed."

[0104] Step 12:

[0105] The server sends a reservation confirmation message to the terminal. It sends data to notify the user of the reservation confirmation.

[0106] Step 13:

[0107] The terminal receives a reservation confirmation message from the server and displays it to the user. The user interface is then refreshed to notify the user that the reservation has been confirmed.

[0108] The above outlines the specific program processing flow when a user makes a reservation at a hair salon.

[0109] (Example 1)

[0110] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0111] In recent years, there has been a growing demand for efficient reservation systems. However, traditional systems have suffered from cumbersome reservation procedures and low user convenience. In particular, users often had to manually enter and verify information when requesting a reservation, and the process of confirming a reservation was often time-consuming. This resulted in a poor user experience and decreased operational efficiency.

[0112] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0113] In this invention, the server includes means for a user to input a reservation request, means for a terminal to send the reservation request to the server, means for the server to receive the reservation request and parse it using a natural language processing algorithm, means for the server to query an external reservation system to confirm available dates and times, means for the server to generate a response message based on the available dates and times and send it to the terminal, means for the terminal to display the response message to the user, means for the user to input a confirmation request in response to the response message, means for the terminal to resend the confirmation request to the server, means for the server to receive the confirmation request, access the external reservation system and confirm the reservation, means for the server to generate a reservation confirmation message and send it to the terminal, and means for the terminal to display the reservation confirmation message to the user. This allows the user to easily perform the process from request input to reservation confirmation, enabling simplification and efficiency of the procedure.

[0114] A "user" is an individual who uses the reservation system to enter a request and completes the process until the reservation is confirmed.

[0115] A "terminal" is a device used by a user to input requests and communicate with a server, and includes smartphones, tablets, and personal computers.

[0116] A "server" is a computer system that receives requests from users, analyzes them, checks available dates and times for reservations, and generates and sends response messages.

[0117] A "request" is information entered by a user that expresses their desired reservation details.

[0118] A "natural language processing algorithm" is a computational method for analyzing and understanding text entered by a user and extracting appropriate keywords.

[0119] A "reservation system" is a system that uses an external database or API to check reservation availability and process reservations.

[0120] A "response message" is a message generated by the server and sent to the terminal, which includes available dates and times for booking and a request for confirmation.

[0121] A "confirmation request" is a message that a user enters in response to a reply message to confirm their reservation.

[0122] A "reservation confirmation message" is a message generated by the server to notify the user that the reservation has been confirmed.

[0123] This invention relates to a system that automates and efficiently processes reservations requested by users. The system consists of three elements: the user, the terminal, and the server, each element playing its own role.

[0124] The user enters the details of their reservation request into the device. Specifically, they enter a request in text format, such as "I would like to make a reservation at a hair salon." Users can use devices such as smartphones, tablets, or personal computers.

[0125] The terminal receives requests entered by the user and sends them to the server. The terminal obtains user requests using an input interface (e.g., keyboard, touchscreen, voice input system) and sends them to the server using a communication module (e.g., Wi-Fi, mobile data communication). This ensures that the user's request reaches the server.

[0126] When the server analyzes requests received from terminals, it uses natural language processing algorithms (e.g., OpenAI®, GPT-4®) to understand the request content and extract appropriate keywords. After analyzing the request, the server queries an external reservation system (e.g., reservation management API or database) to check available dates and times.

[0127] The server generates a response message based on the confirmed available dates and times and sends a confirmation message to the terminal asking, "Can I make a reservation?". The server's message generation logic is used to generate the message.

[0128] The terminal displays the received response message to the user. The user confirms the displayed message and enters a confirmation request, such as "Yes, book it," into the terminal. The terminal then uses the communication module to send the confirmation request to the server again.

[0129] Upon receiving the confirmation request, the server accesses the external reservation system again to formally confirm the reservation. Once the reservation is confirmed, it generates a notification message "Reservation Confirmed" and sends it to the terminal.

[0130] The device displays a reservation confirmation message to the user, and the reservation process is complete.

[0131] Specific example

[0132] The following is a specific scenario for a user who wants to make a reservation at a hair salon.

[0133] User request input

[0134] User: Types "I want to make a hair salon appointment" on their smartphone.

[0135] Terminal: Sends the entered string to the server as an HTTP request.

[0136] Server analysis and reservation confirmation

[0137] Server: Parses the JSON data received as an HTTP request ({"request": "I want to make a reservation at a hair salon"}).

[0138] Server: Uses OpenAI GPT-4 to extract keywords such as "hair salon" and "reservation".

[0139] Server: Query the beauty salon reservation system's API to check available dates and times (e.g., "October 1st, 2:00 PM").

[0140] Server response generation and transmission

[0141] Server: Based on the available dates and times, it generates the message, "You can make a reservation for October 1st at 2:00 PM. Would you like to make a reservation?"

[0142] Server: Sends the generated message to the terminal as an HTTP response.

[0143] Displaying messages on the device and verifying the user.

[0144] Device: The received message is displayed on the screen. A pop-up appears saying, "You can make a reservation for October 1st at 2:00 PM. Do you want to make a reservation?"

[0145] User: In response to the displayed message, type "Yes, book it" into the terminal.

[0146] Terminal: Sends the confirmation request back to the server as an HTTP request.

[0147] Server reservation confirmation and message sending

[0148] Server: Accesses the hair salon reservation system and processes the reservation confirmation.

[0149] Server: If the reservation confirmation is successful, it generates the message "Reservation confirmed".

[0150] Server: Sends a confirmation message to the terminal.

[0151] Display of the device reservation confirmation message

[0152] Terminal: Receives a confirmation message and displays to the user that "Your reservation has been confirmed."

[0153] Example of a prompt

[0154] The following is an example of a prompt message to input into a generative AI model.

[0155] Describe the steps a user takes from requesting and submitting a hair salon appointment to confirming the booking. Include all hardware and software used in detail.

[0156] This invention allows users to easily complete the process from request entry to reservation confirmation, simplifying and streamlining the procedure. Furthermore, reservations can be quickly confirmed and modified, greatly improving user convenience.

[0157] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0158] Step 1:

[0159] The user enters a reservation request and sends it to their device. Specifically, the user uses the device's input interface (e.g., keyboard, touchscreen) to enter "I want to make a reservation at a hair salon." The input is the user's request, and the output is sending that request to the server as an HTTP request. The user performs this operation on a device such as a smartphone or tablet.

[0160] Step 2:

[0161] The terminal receives user input and sends it to the server as an HTTP request. The terminal converts the input request into JSON format and sends it to the server. A communication module (e.g., Wi-Fi or mobile data communication) is used in this process. The input is the request content entered by the user, and the output is the HTTP request sent to the server.

[0162] Step 3:

[0163] The server parses the HTTP request received from the terminal. The server parses the request content into JSON and extracts it as text data. Then, it performs natural language processing using a generative AI model (e.g., OpenAI GPT-4) to extract keywords such as "hair salon reservation". The input is the JSON data of the HTTP request, and the output is the parsed keywords.

[0164] Step 4:

[0165] The server queries an external reservation system (e.g., a hair salon's reservation API) based on the analysis results to check available dates and times. The reservation system returns availability information via the API query, which the server receives. The input is the analyzed keywords, and the output is information about available dates and times. A web API is used for this query.

[0166] Step 5:

[0167] The server generates a response message based on the received information about available dates and times for booking. Specifically, it generates a message that includes the available date and time for booking (e.g., "You can book on October 1st at 2:00 PM. Do you want to book?"). The generated response message is then sent back to the terminal as an HTTP response. The input is the information about available dates and times for booking, and the output is the response message.

[0168] Step 6:

[0169] The terminal displays the response message received from the server to the user. The terminal parses the received message and displays it through the user interface. The input is the response message from the server, and the output is the message that the user sees.

[0170] Step 7:

[0171] The user enters a confirmation request in response to the displayed response message. They enter a confirmation request, such as "Yes, book it," into the terminal and send it. The input is the displayed response message, and the output is the confirmation request.

[0172] Step 8:

[0173] The terminal sends the user's confirmation request to the server. This request is then compiled again into JSON format data as an HTTP request and sent to the server. The input is the confirmation request, and the output is the HTTP request sent to the server.

[0174] Step 9:

[0175] The server receives the confirmation request and processes the reservation confirmation. The server accesses the external reservation system again and sends a confirmation request. If the reservation is successfully confirmed, the server receives that information. The input is the confirmation request, and the output is the reservation confirmation information.

[0176] Step 10:

[0177] The server generates a reservation confirmation message, "Your reservation has been confirmed," based on the reservation confirmation information, and sends it to the terminal. The generated message is sent to the terminal as an HTTP response. The input is the reservation confirmation information, and the output is the reservation confirmation message.

[0178] Step 11:

[0179] The terminal displays a reservation confirmation message to the user. The terminal parses the received message and displays it again through the user interface. The input is the reservation confirmation message, and the output is the confirmation message displayed to the user.

[0180] As a result, users can easily make reservations at hair salons, and the system efficiently automates the reservation process.

[0181] (Application Example 1)

[0182] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0183] The problem this invention aims to solve is to streamline the process by which users can easily reserve and confirm food delivery orders. Conventional systems require numerous manual steps for users to make reservations, which places a significant burden on them. Furthermore, the time required for reservation confirmation and finalization is inconvenient for food delivery services where immediacy is essential.

[0184] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0185] In this invention, the server includes means for querying the API or database of the provider to be booked and confirming available dates and times; means for parsing the user's request in natural language and generating a response using a generative AI model; and means for utilizing a database or API for booking confirmation and booking confirmation. This makes it possible for users to book food delivery quickly and accurately simply by entering a request, reducing the burden on the user and streamlining the entire process.

[0186] A "user" is a person or entity that wishes to make a reservation and enters a request.

[0187] A "terminal" is an electronic device used by a user to input and send requests to a server; it is a device that handles request input and displays response messages.

[0188] A "server" is a central processing unit that receives user requests and performs processing such as analysis, reservation confirmation, and reservation confirmation.

[0189] A "request" is a request that a user enters into their terminal with the details of their desired reservation and sends to the server.

[0190] A "reservation target provider" refers to the provider of the service or product that the user wishes to reserve.

[0191] "API" stands for Application Programming Interface, which is an interface that allows different systems and programs to communicate with each other.

[0192] A "database" is a collection of data that systematically stores information and allows it to be searched and manipulated.

[0193] "Available dates and times for booking" refers to the dates and times when the provider of the booking can provide the service.

[0194] A "response message" is a message generated by a server to respond to a user after analyzing the user's request.

[0195] "Natural language analysis" refers to a technology that allows a computer to mechanically understand the language entered by a user, meaning that the computer interprets the language that humans speak naturally.

[0196] A "generative AI model" is an artificial intelligence model generated based on machine learning, and it is an algorithm that generates the optimal response from input data.

[0197] This invention relates to a system for automating and efficiently processing food delivery reservations requested by users. This system consists of users, terminals, and a server, with each program fulfilling its respective role. Embodiments of this system are described in detail below.

[0198] System Configuration

[0199] 1. User:

[0200] The user is someone who wants to order food delivery and enters their request via a device such as a smartphone. For example, they might make a request like, "I want a pizza delivered at 7 PM," using voice input or text input.

[0201] 2. Terminal:

[0202] The terminal is a smartphone, tablet, or other mobile device that receives requests entered by the user and sends them to the server. The terminal also displays response messages and confirmation messages received from the server to the user.

[0203] 3. Server:

[0204] The server receives requests from users, parses them, checks reservation status, and generates response messages. The server queries the API or database of food delivery service providers to check available dates and times. It also parses user requests in natural language and generates responses using generative AI models.

[0205] Hardware and software to be used

[0206] Smartphone: Used as a user interface and a device for sending requests.

[0207] Server: Cloud-based or on-premises central processing unit. Programming languages ​​used are Python and Flask.

[0208] API: Used for communication with food delivery providers.

[0209] Generative AI models: Machine learning models used for natural language processing and response generation.

[0210] Data processing and data calculation

[0211] The server analyzes the user's natural language request and extracts keywords such as "pizza" and "7 PM." This analysis is performed using a generative AI model. The server then queries the API of a food delivery service provider to check available delivery times. Based on the results, the server generates a response message such as, "We can deliver a pizza at 7 PM. Would you like to order?" This message is sent to the device, which then displays it to the user.

[0212] Specific example of processing

[0213] For example, if a user enters a request saying, "I want a pizza delivered at 7 PM," the following exchange will take place:

[0214] 1. User: "I'd like the pizza delivered at 7 PM."

[0215] 2. Terminal: Sends the request to the server.

[0216] 3. Server: Analyzes "pizza" and "7 PM"

[0217] 4. Server: Query the API of the food delivery provider to check available reservation times.

[0218] 5. Server: Generates a response message, "We can deliver a pizza at 7 PM. Would you like to order?", and sends it to the terminal.

[0219] 6. Terminal: Display message

[0220] 7. User: "Yes, place the order."

[0221] 8. Terminal: Sends a confirmation request to the server.

[0222] 9. Server: Confirms the order, generates a message "Order Confirmed," and sends it to the terminal.

[0223] 10. Terminal: Display a reservation confirmation message to the user.

[0224] This allows users to easily book food delivery orders, streamlining the entire process.

[0225] Example of a prompt:

[0226] User: "I want the pizza delivered at 7 PM."

[0227] System: "We can deliver a pizza at 7 PM. Would you like to order?"

[0228] User: "Yes, place the order."

[0229] System: "Your order has been confirmed."

[0230] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0231] Step 1:

[0232] The user enters a request into a smartphone application saying, "I want a pizza delivered at 7 PM."

[0233] Input: User's natural language request ("I want a pizza delivered at 7 PM")

[0234] Output: Input text data

[0235] Step 2:

[0236] The terminal sends the entered request to the server.

[0237] Input: User input text data

[0238] Output: Text request sent to the server

[0239] Step 3:

[0240] The server analyzes the received request and uses a generative AI model to extract the necessary information for the reservation, such as "pizza" and "7 PM."

[0241] Input: Text request sent from the terminal

[0242] Data processing: Natural language analysis using generative AI models

[0243] Output: Analysis results regarding "pizza" and "7 PM"

[0244] Step 4:

[0245] Based on the analysis results, the server queries the API or database of food delivery service providers to check available dates and times for reservations.

[0246] Input: Analysis results for "pizza" and "7 PM"

[0247] Data processing: API queries and retrieval of available dates and times.

[0248] Output: Available dates and times for booking (e.g., "7 PM")

[0249] Step 5:

[0250] Based on the query results, the server generates a response message saying, "We can deliver a pizza at 7 PM. Would you like to order?" and sends it to the terminal.

[0251] Input: Available dates and times

[0252] Data processing: Generating response messages

[0253] Output: Response message (Example: "We can deliver a pizza at 7 PM. Would you like to order?")

[0254] Step 6:

[0255] The terminal displays the response message received from the server to the user.

[0256] Input: Response message from the server

[0257] Output: Message displayed to the user

[0258] Step 7:

[0259] The user enters "Yes, place order" into the terminal as a confirmation request.

[0260] Input: User confirmation request ("Yes, place order")

[0261] Output: Input text data

[0262] Step 8:

[0263] The device sends a user verification request to the server.

[0264] Input: User input text data

[0265] Output: Confirmation request to send to the server

[0266] Step 9:

[0267] The server receives the confirmation request and uses the food delivery service provider's API to confirm the reservation.

[0268] Input: Verification request sent from the device

[0269] Data processing: Execution of reservation confirmation process (API call)

[0270] Output: Booking Confirmation Status

[0271] Step 10:

[0272] The server generates a message indicating that the reservation has been confirmed and sends it to the terminal.

[0273] Input: Booking Confirmation Status

[0274] Data processing: Generating reservation confirmation messages

[0275] Output: Reservation confirmation message (e.g., "Your order has been confirmed")

[0276] Step 11:

[0277] The terminal displays the reservation confirmation message received from the server to the user.

[0278] Input: Reservation confirmation message from the server

[0279] Output: Message displayed to the user

[0280] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0281] The present invention relates to a system that combines an emotion engine with a system for automatically processing user requests, and provides optimal responses and services based on the user's emotional state. This system consists of a user, a terminal, a server, and an emotion engine, and operates as follows.

[0282] System Overview

[0283] User: Inputs a reservation request and receives recommendations based on the emotional state.

[0284] Terminal: Receives the user's input, sends it to the server together with the emotional information, and displays the response from the server to the user.

[0285] Server: Receives and analyzes the user's request and emotional information, checks the reservation status, and generates a response message.

[0286] Emotion Engine: Analyzes the emotional state from the user's input, voice, expression, etc., and provides the result to the server.

[0287] Program Processing

[0288] 1. Input of User Request and Emotion Recognition

[0289] The user wishes to make a reservation at a beauty salon and enters "I want to make a reservation at a beauty salon" into the terminal. At the same time, the emotion engine analyzes the emotional state (e.g., joy, stress, anxiety, etc.) from the user's expression and voice tone.

[0290] 2. Transmission of Terminal Request and Emotional Information

[0291] The terminal sends the user's request data and the emotional information obtained from the emotion engine to the server. The data is sent to the designated endpoint using an HTTP request or the like.

[0292] 3. Server Request Reception and Analysis

[0293] The server receives requests and sentiment information sent from the terminal. It analyzes the request content and extracts specific keywords such as "hair salon reservation." Sentiment information is also analyzed simultaneously.

[0294] 4. Check available dates and times for booking.

[0295] The server accesses the hair salon's reservation system (API or database) to search for available dates and times. During this process, the number and content of the displayed options are adjusted based on the user's emotional state.

[0296] 5. Generating a response message

[0297] Based on the available dates and times retrieved by the server, a response message optimized for the user's emotional state is generated. For example, a message such as "You can make a reservation on [Month] [Day] at [Time]. Would you like to make a reservation?" is modified to include emotionally appropriate expressions and recommendations.

[0298] 6. Sending a response message

[0299] The server sends the generated response message to the terminal. Then, it sends the response data again using an HTTP request or similar method.

[0300] 7. Display of the terminal's response message

[0301] The terminal receives a response message from the server and displays it to the user. The user interface (UI) is updated to display the received message on the screen.

[0302] 8. Enter the user's confirmation request.

[0303] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their intention to confirm the reservation.

[0304] 9. Sending a confirmation request

[0305] The terminal sends a user's confirmation request to the server. The confirmation request is sent as an HTTP request, similar to the initial request.

[0306] 10. Server reservation confirmation process

[0307] The server receives the confirmation request, accesses the beauty salon reservation system (API or database), and confirms the reservation. If the reservation is confirmed, the server generates a message saying "The reservation has been confirmed."

[0308] 11. Sending and displaying the reservation confirmation message

[0309] The server sends the reservation confirmation message it generated to the terminal, and the terminal displays it to the user.

[0310] Specific example

[0311] The following shows a specific scenario of a user who wishes to make a reservation at a beauty salon.

[0312] 1. User request input and emotion recognition

[0313] User: "I want to make a reservation at a beauty salon."

[0314] Emotion engine: Analyzes the user's voice and determines that the user is relaxed.

[0315] 2. Terminal request and emotion information transmission

[0316] The terminal sends the request and emotion information to the server.

[0317] 3. Server analysis and reservation confirmation

[0318] Server: Analyzes "Reservation at a beauty salon" and "Relaxed."

[0319] Server: Checks the availability of the beauty salon (〇 / 〇〇 o'clock)

[0320] 4. Server response generation and transmission

[0321] Server: Generates the response message: "You can make a reservation at XX / XX. Don't be nervous, relax and enjoy yourself!"

[0322] Server: Sends a response message to the terminal.

[0323] 5. Displaying messages on the device and verifying the user.

[0324] Terminal: Message display

[0325] User: "Yes, please make a reservation."

[0326] Terminal: Sends a confirmation request to the server.

[0327] 6. Confirm server reservation and send message

[0328] Server: Confirm reservation

[0329] Server: "Your reservation has been confirmed."

[0330] Server: Sends a reservation confirmation message to the terminal.

[0331] 7. Display of the device reservation confirmation message.

[0332] Terminal: Reservation confirmation message displayed

[0333] Thus, the present invention allows users to easily make reservations and enjoy an even more comfortable service experience by receiving support tailored to their emotional state.

[0334] The following describes the processing flow.

[0335] Step 1:

[0336] The user enters a request using the device, such as "I want to make a reservation at a hair salon." Simultaneously, the emotion engine built into the device analyzes the user's voice and facial expressions to identify their emotional state (e.g., relaxed, stressed, happy).

[0337] Step 2:

[0338] The device combines the user's request with the sentiment information obtained from the sentiment engine and sends it to the server as an HTTP request. The request includes the reservation request details and sentiment information.

[0339] Step 3:

[0340] The server receives the request sent from the terminal. Here, it analyzes the content of the request and sentiment information, and extracts the keyword "hair salon reservation" and its associated sentiment.

[0341] Step 4:

[0342] The server accesses the hair salon's reservation system (e.g., via an API or database) to search for available dates and times. During this process, it selects reservation options to present based on the user's sentiment information.

[0343] Step 5:

[0344] The server generates a response message based on the available reservation dates and times it has obtained. For example, if the user is relaxed, it will generate a response message such as, "You can make a reservation at XX o'clock. Please relax and come."

[0345] Step 6:

[0346] The server generates a response message and sends it to the terminal as an HTTP response. This message includes available dates and times for booking and additional messages depending on the sentiment.

[0347] Step 7:

[0348] The terminal receives a response message from the server and displays it to the user. The user interface is updated so that the user can see the message.

[0349] Step 8:

[0350] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their desire to confirm the reservation.

[0351] Step 9:

[0352] The device sends the user's confirmation request back to the server as an HTTP request. This request includes the intention to confirm the reservation and the original request information.

[0353] Step 10:

[0354] The server receives the confirmation request. Access the salon's reservation system again to formally confirm the reservation.

[0355] Step 11:

[0356] The server generates a reservation confirmation message. For example, it might create a message in the format, "Your reservation has been confirmed. We look forward to seeing you."

[0357] Step 12:

[0358] The server generates a reservation confirmation message and sends it to the terminal as an HTTP response. This notifies the user that their reservation has been confirmed.

[0359] Step 13:

[0360] The terminal receives a reservation confirmation message from the server and displays it to the user. The user interface is then refreshed to notify the user of the reservation confirmation.

[0361] The above describes the specific program processing flow of the present invention, which incorporates an emotion engine. This system allows users to enjoy a reservation experience that takes their own emotional state into consideration.

[0362] (Example 2)

[0363] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0364] Traditional reservation systems simply presented reservation options uniformly, without considering the user's emotional state. This sometimes led to user stress and decreased satisfaction. Furthermore, reservation confirmation messages were not tailored to the user's emotional state, resulting in an unoptimized user experience.

[0365] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0366] In this invention, the server includes means for recognizing the user's emotional state, means for selecting the optimal candidate based on the emotional state, and means for generating a response message with expressions corresponding to the emotion. This enables the provision of optimal services and the generation of response messages based on the user's emotional state.

[0367] A "user" is someone who uses a system to enter a request and receive a service.

[0368] A "terminal" is an electronic device used by a user to input requests and communicate with a server.

[0369] A "server" is a central device that receives and analyzes requests from users and executes the specified service (e.g., confirming a reservation).

[0370] A "request" is a request for a service that a user sends through their device.

[0371] "Emotional state" refers to the user's psychological state at that time, and includes feelings such as joy, relaxation, stress, and anxiety.

[0372] An "emotion engine" is a software or hardware mechanism that analyzes a user's emotional state based on their facial expressions, tone of voice, text input, and other factors.

[0373] "Analysis" is the process by which a server understands requests and sentiment information and determines the appropriate response.

[0374] "Available dates and times" refers to the dates and times when the service specified by the user (e.g., a hair salon reservation) is available.

[0375] A "response message" is a message sent from the server to the terminal that contains the results or information regarding the request.

[0376] A "confirmation request" is a request that a user submits to finalize a reservation or other service.

[0377] A "reservation confirmation message" is a message sent by the server to the user informing them that the server has processed the request and that the reservation has finally been confirmed.

[0378] A "database" is a system used to store reservation information and user data.

[0379] "API" stands for Application Programming Interface, and it is an interface for communicating with external systems and exchanging information.

[0380] This invention is a system that combines an emotion engine with a system that automatically processes user requests, thereby providing optimal responses and services based on the user's emotional state. The system consists of a user, a terminal, a server, and an emotion engine.

[0381] System Configuration

[0382] User: A person who uses a system to enter requests and receive services.

[0383] Terminal: An electronic device that receives user input, sends it to a server along with emotional information, and displays the server's response to the user. Examples include smartphones and personal computers.

[0384] Server: A central device that receives and analyzes user requests and sentiment information, checks reservation status, and generates response messages.

[0385] Emotion engine: A software or hardware mechanism that analyzes the user's emotional state from user input, voice, facial expressions, etc., and provides the results to a server.

[0386] Detailed explanation of the process

[0387] 1. User request input:

[0388] The user enters "I want to make a reservation at a hair salon" into their device (such as a smartphone or computer). Input can be done via text or voice, and if the user uses voice input, their voice is captured using a microphone.

[0389] 2. Emotion recognition:

[0390] The text and voice input by the user are sent to the emotion engine via the device. The emotion engine analyzes the user's facial expressions and tone of voice to determine their emotional state. For example, if a user says, "I want to make a reservation at the hair salon," the emotion engine analyzes the user's tone of voice and assesses whether they are relaxed.

[0391] 3. Sending requests and sentiment information:

[0392] The device combines the user's request data and sentiment information obtained from the sentiment engine into a single JSON file, and sends this data to the server using an HTTP POST request. Network communication is used for this process.

[0393] 4. Request reception and parsing:

[0394] The server receives request data and emotional information sent from the terminal. The server analyzes the request content and extracts specific keywords such as "hair salon reservation." At the same time, the server analyzes the user's emotional state based on the received emotional information.

[0395] 5. Check available dates and times:

[0396] The server accesses an external salon booking API or an internal database to check for available dates and times. During this process, the server adjusts the number and content of the options displayed based on the user's emotional state. For example, if the user is very relaxed, it will present many options; if they are in a hurry, it will present only a limited number.

[0397] 6. Generating a response message:

[0398] The server generates a response message optimized for the user's emotional state based on the available reservation date and time. For example, it might generate a message like, "You can make a reservation on [Month] [Day] at [Time]. Please relax and enjoy yourself."

[0399] 7. Sending a response message:

[0400] The server sends the generated response message back to the terminal as an HTTP POST request, and the terminal receives it.

[0401] 8. Display of response message:

[0402] The terminal displays the response message received from the server in the user interface. The user reviews it and decides on the next action.

[0403] 9. Enter your confirmation request:

[0404] The user types "Yes, book it" to confirm the reservation. They can do this by typing into a text box or by giving a voice command. This indicates that they wish to confirm the reservation.

[0405] 10. Submit a confirmation request:

[0406] The device then sends the confirmation request received from the user back to the server as an HTTP POST request.

[0407] 11. Booking confirmation process:

[0408] The server receives the confirmation request and accesses the salon's reservation system API to confirm the reservation. If the reservation is successfully confirmed, the server generates a message saying "Reservation confirmed".

[0409] 12. Sending and displaying booking confirmation messages:

[0410] The server sends the generated reservation confirmation message to the terminal, and the terminal updates its user interface to display the reservation confirmation message.

[0411] Specific example

[0412] The following is a specific scenario for a user who wishes to make a reservation at a hair salon.

[0413] 1. User request input and sentiment recognition

[0414] User: "I want to make a reservation at a hair salon."

[0415] Emotion Engine: Analyzes the user's voice and determines that they are relaxed.

[0416] 2. Sending requests and sentiment information from the device.

[0417] The device sends the request and sentiment information to the server.

[0418] 3. Server analysis and reservation confirmation

[0419] Server: Analyzes "hair salon reservation" and "relaxed"

[0420] Server: Check the availability of the hair salon (e.g., Month XX, Day XX, Time XX)

[0421] 4. Server response generation and transmission

[0422] Server: Generates a response message saying, "Your reservation is available on [Month] [Day] at [Time]. Relax and enjoy!"

[0423] Server: Sends a response message to the terminal.

[0424] 5. Displaying messages on the device and verifying the user.

[0425] Terminal: Message display

[0426] User: "Yes, please make a reservation."

[0427] Terminal: Sends a confirmation request to the server.

[0428] 6. Confirm server reservation and send message

[0429] Server: Confirm reservation

[0430] Server: "Your reservation has been confirmed."

[0431] Server: Sends a reservation confirmation message to the terminal.

[0432] 7. Display of the device reservation confirmation message.

[0433] Terminal: Reservation confirmation message displayed

[0434] Example of a prompt

[0435] User input example: "I want to make a reservation at a hair salon."

[0436] Input to the response generation AI: "Process the following request and generate the most appropriate response message based on the user's emotional state. Request: 'Make a hair salon appointment', Emotional state: 'Relaxed'"

[0437] This system allows users to easily make reservations and enjoy an even more comfortable service experience by receiving support tailored to their emotional state.

[0438] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0439] Step 1:

[0440] User request input and sentiment recognition

[0441] The user enters "I want to make a reservation at a hair salon" into the device. Input can be done via text or voice. If the user uses voice input, the device's microphone captures the audio. Simultaneously, the device's camera captures the user's facial expressions. The emotion engine receives the audio and video data and analyzes the emotional state using natural language processing and image analysis. This generates request data and emotion information.

[0442] Input: User text or voice input, camera video.

[0443] Output: Request data (hair salon reservation), emotional information (e.g., relaxed)

[0444] Step 2:

[0445] Sending requests and sentiment information

[0446] The device combines the user's request data and sentiment information obtained from the sentiment engine into a single JSON file. This data is then sent to the server's specified endpoint using an HTTP POST request. Specifically, the data transmission is performed using network communication capabilities.

[0447] Input: Request data, sentiment information

[0448] Output: Data sent to the server as an HTTP request

[0449] Step 3:

[0450] Server request reception and analysis

[0451] The server receives request data and sentiment information sent from the terminal. The server parses the JSON data and extracts the request content. For example, it extracts and analyzes the keyword "hair salon reservation" and sentiment information. This allows the server to understand the user's desired service and emotional state.

[0452] Input: HTTP request data (request content, sentiment information)

[0453] Output: Analysis results (service details, hair salon reservation, emotional state)

[0454] Step 4:

[0455] Check available dates and times for booking

[0456] The server accesses the salon's reservation system API or internal database to check for available dates and times. This process narrows down the options according to specific rules based on the user's emotional state. For example, it presents more reservation options to a relaxed user and fewer options to a user in a hurry.

[0457] Input: Analyzed request (service details, sentiment information)

[0458] Output: List of available dates and times

[0459] Step 5:

[0460] Generating a response message

[0461] Based on the available dates and times retrieved by the server, a response message optimized for the user's emotional state is generated. Expressions are added according to the emotional state. For example, a message such as "You can make a reservation on [Month] [Day] at [Time]. Relax and enjoy!" is generated.

[0462] Input: List of available dates and times, sentiment information

[0463] Output: Optimized response message

[0464] Step 6:

[0465] Sending a response message

[0466] The server generates a response message, which is then compiled into JSON format and sent again to the specified endpoint on the device using an HTTP POST request.

[0467] Input: Response message

[0468] Output: Response message sent to the terminal as an HTTP request

[0469] Step 7:

[0470] Display of terminal response messages

[0471] The terminal analyzes the response message received from the server and updates the user interface. The message is displayed on the screen, prompting the user to confirm.

[0472] Input: Response message sent to the terminal

[0473] Output: Message displayed in the updated user interface

[0474] Step 8:

[0475] Entering a user verification request

[0476] If the user wants to confirm the reservation, they type "Yes, book it." They can type this into the text box or use a voice command. This will generate a confirmation request.

[0477] Input: User confirmation input (text or voice)

[0478] Output: Confirmation Request

[0479] Step 9:

[0480] Sending a confirmation request

[0481] The device compiles the confirmation request into JSON format and sends it again as an HTTP POST request to the specified endpoint on the server.

[0482] Input: Confirmation Request

[0483] Output: Confirmation request sent to the server as an HTTP request

[0484] Step 10:

[0485] Server reservation confirmation process

[0486] The server receives the confirmation request and accesses the salon's reservation system API to confirm the reservation. If the reservation is successfully confirmed as a result of this process, the server generates a message saying "Reservation confirmed".

[0487] Input: Confirmation Request

[0488] Output: Booking confirmation message

[0489] Step 11:

[0490] Sending and displaying reservation confirmation messages

[0491] The server generates a reservation confirmation message and sends it to the terminal. The terminal receives this message, updates the user interface, and displays the reservation confirmation message on the screen.

[0492] Input: Booking confirmation message

[0493] Output: Booking confirmation message displayed in the updated user interface

[0494] (Application Example 2)

[0495] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0496] Traditional reservation systems simply respond to user requests and lack the flexibility to adapt to user emotional states or request content. Therefore, to improve the user experience in service delivery, a system is needed that provides optimal responses and services based on the user's emotional state.

[0497] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0498] In this invention, the server includes means for analyzing the user's facial expressions and tone of voice to recognize their emotional state, means for optimizing requests based on the emotional state, and means for the server to generate a response message and send it to the terminal. This enables flexible and optimal responses and service provision based on the user's emotional state.

[0499] "User" refers to an individual or group that uses the system.

[0500] A "request" refers to an operation or request that a user enters.

[0501] A "terminal" is a device used by a user to input requests and to receive and display responses from a server.

[0502] A "server" is a component that receives and analyzes requests sent from a terminal and generates the optimal response.

[0503] "Analysis" refers to the process of breaking down and understanding request and emotional state data.

[0504] "Available dates and times" refers to the dates and times when the service can be provided, and is presented as an option in response to the user's request.

[0505] A "response message" refers to information generated by the server and sent to the user via the terminal.

[0506] "Facial expressions" refer to the expressions on a user's face and are a source of information for determining their emotions.

[0507] "Voice tone" refers to the pitch and volume of a user's voice and is a source of information for analyzing their emotions.

[0508] "Emotional state" refers to the user's psychological state and is obtained from analyzed facial expressions and tone of voice.

[0509] "Optimization" refers to the process of adjusting requests and responses based on emotional states.

[0510] A "confirmation request" refers to an action taken by a user to provide additional information or confirm a request that was initially submitted.

[0511] A "reservation confirmation message" refers to the final response message generated by the server to notify the user that the reservation was successful.

[0512] "System" refers to the entire set of components, including users, terminals, servers, sentiment analysis engines, and their interfaces.

[0513] This invention is a system that automatically processes user requests and provides optimal responses and services based on the user's emotional state. The system consists of a user, a terminal, a server, and an emotion analysis engine.

[0514] Hardware and software to be used

[0515] Device: Smartphone with built-in camera and microphone

[0516] Server: Cloud-based web server

[0517] Sentiment analysis engine: Microsoft® Azure® Face API, Google® Cloud Vision API

[0518] Video streaming platform APIs: YouTube® Data API, Netflix API

[0519] Data processing and computation

[0520] 1. Recognizing the user's emotional state

[0521] The device uses its built-in camera and microphone to capture the user's facial expressions and voice tone.

[0522] An emotion analysis engine (such as Microsoft Azure Face API or Google Cloud Vision API) is used to analyze the acquired data and recognize the user's emotional state.

[0523] 2. Sending emotional data

[0524] The analyzed emotional data (e.g., joy, sadness, stress) is sent to the server using an HTTP request.

[0525] 3. Emotion-based content recommendations

[0526] The server analyzes the received emotion data and uses video streaming platform APIs (e.g., YouTube Data API, Netflix API) to search for videos that are appropriate for the user's emotional state.

[0527] 4. Generating and sending video recommendations

[0528] The server selects the most suitable video recommendation and sends the recommendation result to the user's device using an HTTP request.

[0529] The terminal receives a response message from the server and displays it to the user.

[0530] Specific Scenario

[0531] 1. User emotion recognition

[0532] When a user opens the app, their facial expressions and voice are captured using the smartphone's camera and microphone.

[0533] The emotion analysis engine analyzes facial expressions and voice tone to determine that "the user is relaxed."

[0534] 2. Sending emotional data

[0535] Send emotional data (relaxed) to the server.

[0536] 3. Generating video recommendations

[0537] The server uses emotional data to search for videos suitable for relaxation (for example, videos with nature sounds or relaxing music videos).

[0538] 4. Sending and displaying video recommendations

[0539] The server sends a list of related videos to the user's smartphone with the comment, "These videos are perfect for relaxing."

[0540] Users can view a list of videos on their smartphones, select their favorite video, and play it.

[0541] Example of a prompt

[0542] "Please recommend videos that match the user's current emotional state. Based on the user's facial expressions and tone of voice, their emotional state is relaxed."

[0543] Thus, the present invention allows users to quickly obtain videos that are suitable for their emotional state, resulting in a personalized and excellent user experience.

[0544] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0545] Step 1:

[0546] The user opens the smartphone app. The device uses its built-in camera and microphone to capture the user's facial expressions and voice tone. This data is input into the emotion analysis engine. The emotion analysis engine uses the Microsoft Azure Face API or Google Cloud Vision API to output emotion data (e.g., relaxed, stressed).

[0547] Step 2:

[0548] The device sends analyzed sentiment data to the server. The server receives the HTTP request and verifies the user's sentiment state. Based on the input sentiment data, it requests suitable video recommendations from video streaming platform APIs (e.g., YouTube Data API, Netflix API).

[0549] Step 3:

[0550] The server retrieves a list of relevant videos from the video streaming platform API. The server filters the retrieved video information based on sentiment data to generate an optimal recommendation list. The generated list is sent to the terminal along with an optimized response message.

[0551] Step 4:

[0552] The device receives a response message from the server and displays it to the user. A list of videos is displayed, including comments tailored to the user's emotional state. For example, the video list might be displayed with the message, "These videos are perfect for relaxing."

[0553] Step 5:

[0554] The user selects a video from the displayed video list and starts playback. The device passes the information of the selected video to the video player and begins streaming the video.

[0555] This allows users to easily find and watch videos that match their emotional state.

[0556] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0557] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0558] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0559] [Second Embodiment]

[0560] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0561] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0562] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0563] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0564] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0565] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0566] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0567] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0568] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0569] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0570] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0571] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0572] This invention relates to a system for automating and efficiently processing reservations requested by users. This system consists of a user, a terminal, and a server, with each program fulfilling its respective role.

[0573] System Overview

[0574] User: Enter your reservation request and send it to the server via your device.

[0575] Terminal: Receives user input, sends requests to the server, receives responses from the server, and displays them to the user.

[0576] Server: Receives user requests, parses them, checks reservation status, and generates response messages.

[0577] Program processing

[0578] 1. Entering and submitting the user's request.

[0579] A user wants to make a reservation at a hair salon and enters "I want to make a reservation at a hair salon" into the terminal. The terminal sends this input to the server.

[0580] 2. Receiving and parsing server requests

[0581] The server receives a request from the terminal and analyzes its content, such as "hair salon reservation." During this process, it extracts appropriate keywords.

[0582] 3. Check available dates and times for reservations.

[0583] The server queries the beauty salon's API or database to check for available appointment times. For example, the server might get a result like "Appointment is available at XX / XX."

[0584] 4. Generating and sending response messages

[0585] The server generates a response message based on the available dates and times, such as "You can make a reservation at XX / XX. Do you want to make a reservation?", and sends it to the terminal.

[0586] 5. Display of the terminal's response message

[0587] The terminal displays the received response message to the user. The user confirms the message and responds, "Yes, book it."

[0588] 6. Sending a user verification request

[0589] The device resends the user's confirmation request to the server.

[0590] 7. Server reservation confirmation process

[0591] The server receives the confirmation request and accesses the salon's reservation system to confirm the reservation. If the reservation is confirmed, the server generates a message saying "Reservation confirmed".

[0592] 8. Sending and displaying the reservation confirmation message.

[0593] The server sends a reservation confirmation message to the terminal, which then displays it to the user.

[0594] Specific example

[0595] The following is a specific scenario for a user who wants to make a reservation at a hair salon.

[0596] 1. User request input

[0597] User: "I want to make a reservation at a hair salon."

[0598] Terminal: Sends a request to the server

[0599] 2. Server analysis and reservation confirmation

[0600] Server: Analyzes "hair salon reservations"

[0601] Server: Check the availability of the hair salon (at XX / XX)

[0602] 3. Server response generation and transmission

[0603] Server: "You can make a reservation at [time]. Would you like to make a reservation?"

[0604] Server: Sends a response message to the terminal.

[0605] 4. Displaying messages on the device and verifying the user.

[0606] Terminal: Message display

[0607] User: "Yes, please make a reservation."

[0608] Terminal: Sends a confirmation request to the server.

[0609] 5. Confirm server reservation and send message

[0610] Server: Confirm reservation

[0611] Server: "Your reservation has been confirmed."

[0612] Server: Sends a reservation confirmation message to the terminal.

[0613] 6. Display of the device reservation confirmation message

[0614] Terminal: Reservation confirmation message displayed

[0615] Thus, the present invention allows users to easily make reservations, simplifying and streamlining the process. Furthermore, reservations can be quickly confirmed and modified, greatly improving user convenience.

[0616] The following describes the processing flow.

[0617] Step 1:

[0618] The user enters a request using the terminal, such as "I want to make a reservation at a hair salon." This input is triggered by the terminal's program.

[0619] Step 2:

[0620] The device sends the user's request as a string to the server. It uses protocols such as HTTP requests to send data to the specified endpoint.

[0621] Step 3:

[0622] The server receives a request sent from the terminal. It analyzes the content of the request and extracts the specific keyword "hair salon reservation".

[0623] Step 4:

[0624] The server accesses the hair salon's reservation system (API or database) to search for available dates and times. For example, it might query the reservation system's API endpoint to retrieve availability data.

[0625] Step 5:

[0626] The server generates a response message based on the available reservation dates and times. For example, it generates a message in the format of "You can make a reservation on [Month] [Day] at [Time]. Would you like to make a reservation?".

[0627] Step 6:

[0628] The server sends the generated response message to the terminal. Then, it sends the response data again using an HTTP request or similar method.

[0629] Step 7:

[0630] The terminal receives a response message from the server and displays it to the user. The user interface (UI) is updated to display the received message on the screen.

[0631] Step 8:

[0632] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their intention to confirm the reservation.

[0633] Step 9:

[0634] The device sends a user verification request to the server. The verification request is sent as an HTTP request, just like the initial request.

[0635] Step 10:

[0636] The server receives the confirmation request. It then accesses the reservation system (API or database) again to formally confirm the reservation.

[0637] Step 11:

[0638] The server generates a reservation confirmation message. For example, it might create a message in the format "Your reservation has been confirmed."

[0639] Step 12:

[0640] The server sends a reservation confirmation message to the terminal. It sends data to notify the user of the reservation confirmation.

[0641] Step 13:

[0642] The terminal receives a reservation confirmation message from the server and displays it to the user. The user interface is then refreshed to notify the user that the reservation has been confirmed.

[0643] The above outlines the specific program processing flow when a user makes a reservation at a hair salon.

[0644] (Example 1)

[0645] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0646] In recent years, there has been a growing demand for efficient reservation systems. However, traditional systems have suffered from cumbersome reservation procedures and low user convenience. In particular, users often had to manually enter and verify information when requesting a reservation, and the process of confirming a reservation was often time-consuming. This resulted in a poor user experience and decreased operational efficiency.

[0647] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0648] In this invention, the server includes means for a user to input a reservation request, means for a terminal to send the reservation request to the server, means for the server to receive the reservation request and parse it using a natural language processing algorithm, means for the server to query an external reservation system to confirm available dates and times, means for the server to generate a response message based on the available dates and times and send it to the terminal, means for the terminal to display the response message to the user, means for the user to input a confirmation request in response to the response message, means for the terminal to resend the confirmation request to the server, means for the server to receive the confirmation request, access the external reservation system and confirm the reservation, means for the server to generate a reservation confirmation message and send it to the terminal, and means for the terminal to display the reservation confirmation message to the user. This allows the user to easily perform the process from request input to reservation confirmation, enabling simplification and efficiency of the procedure.

[0649] A "user" is an individual who uses the reservation system to enter a request and completes the process until the reservation is confirmed.

[0650] A "terminal" is a device used by a user to input requests and communicate with a server, and includes smartphones, tablets, and personal computers.

[0651] A "server" is a computer system that receives requests from users, analyzes them, checks available dates and times for reservations, and generates and sends response messages.

[0652] A "request" is information entered by a user that expresses their desired reservation details.

[0653] A "natural language processing algorithm" is a computational method for analyzing and understanding text entered by a user and extracting appropriate keywords.

[0654] A "reservation system" is a system that uses an external database or API to check reservation availability and process reservations.

[0655] A "response message" is a message generated by the server and sent to the terminal, which includes available dates and times for booking and a request for confirmation.

[0656] A "confirmation request" is a message that a user enters in response to a reply message to confirm their reservation.

[0657] A "reservation confirmation message" is a message generated by the server to notify the user that the reservation has been confirmed.

[0658] This invention relates to a system that automates and efficiently processes reservations requested by users. The system consists of three elements: the user, the terminal, and the server, each element playing its own role.

[0659] The user enters the details of their reservation request into the device. Specifically, they enter a request in text format, such as "I would like to make a reservation at a hair salon." Users can use devices such as smartphones, tablets, or personal computers.

[0660] The terminal receives requests entered by the user and sends them to the server. The terminal obtains user requests using an input interface (e.g., keyboard, touchscreen, voice input system) and sends them to the server using a communication module (e.g., Wi-Fi, mobile data communication). This ensures that the user's request reaches the server.

[0661] When the server analyzes a request received from a terminal, it uses a natural language processing algorithm (e.g., OpenAI GPT-4) to understand the request content and extract appropriate keywords. After analyzing the request, the server queries an external reservation system (e.g., a reservation management API or database) to check for available dates and times.

[0662] The server generates a response message based on the confirmed available dates and times and sends a confirmation message to the terminal asking, "Can I make a reservation?". The server's message generation logic is used to generate the message.

[0663] The terminal displays the received response message to the user. The user confirms the displayed message and enters a confirmation request, such as "Yes, book it," into the terminal. The terminal then uses the communication module to send the confirmation request to the server again.

[0664] Upon receiving the confirmation request, the server accesses the external reservation system again to formally confirm the reservation. Once the reservation is confirmed, it generates a notification message "Reservation Confirmed" and sends it to the terminal.

[0665] The device displays a reservation confirmation message to the user, and the reservation process is complete.

[0666] Specific example

[0667] The following is a specific scenario for a user who wants to make a reservation at a hair salon.

[0668] User request input

[0669] User: Types "I want to make a hair salon appointment" on their smartphone.

[0670] Terminal: Sends the entered string to the server as an HTTP request.

[0671] Server analysis and reservation confirmation

[0672] Server: Parses the JSON data received as an HTTP request ({"request": "I want to make a reservation at a hair salon"}).

[0673] Server: Uses OpenAI GPT-4 to extract keywords such as "hair salon" and "reservation".

[0674] Server: Query the beauty salon reservation system's API to check available dates and times (e.g., "October 1st, 2:00 PM").

[0675] Server response generation and transmission

[0676] Server: Based on the available dates and times, it generates the message, "You can make a reservation for October 1st at 2:00 PM. Would you like to make a reservation?"

[0677] Server: Sends the generated message to the terminal as an HTTP response.

[0678] Displaying messages on the device and verifying the user.

[0679] Device: The received message is displayed on the screen. A pop-up appears saying, "You can make a reservation for October 1st at 2:00 PM. Do you want to make a reservation?"

[0680] User: In response to the displayed message, type "Yes, book it" into the terminal.

[0681] Terminal: Sends the confirmation request back to the server as an HTTP request.

[0682] Server reservation confirmation and message sending

[0683] Server: Accesses the hair salon reservation system and processes the reservation confirmation.

[0684] Server: If the reservation confirmation is successful, it generates the message "Reservation confirmed".

[0685] Server: Sends a confirmation message to the terminal.

[0686] Display of the device reservation confirmation message

[0687] Terminal: Receives a confirmation message and displays to the user that "Your reservation has been confirmed."

[0688] Example of a prompt

[0689] The following is an example of a prompt message to input into a generative AI model.

[0690] Describe the steps a user takes from requesting and submitting a hair salon appointment to confirming the booking. Include all hardware and software used in detail.

[0691] This invention allows users to easily complete the process from request entry to reservation confirmation, simplifying and streamlining the procedure. Furthermore, reservations can be quickly confirmed and modified, greatly improving user convenience.

[0692] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0693] Step 1:

[0694] The user enters a reservation request and sends it to their device. Specifically, the user uses the device's input interface (e.g., keyboard, touchscreen) to enter "I want to make a reservation at a hair salon." The input is the user's request, and the output is sending that request to the server as an HTTP request. The user performs this operation on a device such as a smartphone or tablet.

[0695] Step 2:

[0696] The terminal receives user input and sends it to the server as an HTTP request. The terminal converts the input request into JSON format and sends it to the server. A communication module (e.g., Wi-Fi or mobile data communication) is used in this process. The input is the request content entered by the user, and the output is the HTTP request sent to the server.

[0697] Step 3:

[0698] The server parses the HTTP request received from the terminal. The server parses the request content into JSON and extracts it as text data. Then, it performs natural language processing using a generative AI model (e.g., OpenAI GPT-4) to extract keywords such as "hair salon reservation". The input is the JSON data of the HTTP request, and the output is the parsed keywords.

[0699] Step 4:

[0700] The server queries an external reservation system (e.g., a hair salon's reservation API) based on the analysis results to check available dates and times. The reservation system returns availability information via the API query, which the server receives. The input is the analyzed keywords, and the output is information about available dates and times. A web API is used for this query.

[0701] Step 5:

[0702] The server generates a response message based on the received information about available dates and times for booking. Specifically, it generates a message that includes the available date and time for booking (e.g., "You can book on October 1st at 2:00 PM. Do you want to book?"). The generated response message is then sent back to the terminal as an HTTP response. The input is the information about available dates and times for booking, and the output is the response message.

[0703] Step 6:

[0704] The terminal displays the response message received from the server to the user. The terminal parses the received message and displays it through the user interface. The input is the response message from the server, and the output is the message that the user sees.

[0705] Step 7:

[0706] The user enters a confirmation request in response to the displayed response message. They enter a confirmation request, such as "Yes, book it," into the terminal and send it. The input is the displayed response message, and the output is the confirmation request.

[0707] Step 8:

[0708] The terminal sends the user's confirmation request to the server. This request is then compiled again into JSON format data as an HTTP request and sent to the server. The input is the confirmation request, and the output is the HTTP request sent to the server.

[0709] Step 9:

[0710] The server receives the confirmation request and processes the reservation confirmation. The server accesses the external reservation system again and sends a confirmation request. If the reservation is successfully confirmed, the server receives that information. The input is the confirmation request, and the output is the reservation confirmation information.

[0711] Step 10:

[0712] The server generates a reservation confirmation message, "Your reservation has been confirmed," based on the reservation confirmation information, and sends it to the terminal. The generated message is sent to the terminal as an HTTP response. The input is the reservation confirmation information, and the output is the reservation confirmation message.

[0713] Step 11:

[0714] The terminal displays a reservation confirmation message to the user. The terminal parses the received message and displays it again through the user interface. The input is the reservation confirmation message, and the output is the confirmation message displayed to the user.

[0715] As a result, users can easily make reservations at hair salons, and the system efficiently automates the reservation process.

[0716] (Application Example 1)

[0717] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0718] The problem this invention aims to solve is to streamline the process by which users can easily reserve and confirm food delivery orders. Conventional systems require numerous manual steps for users to make reservations, which places a significant burden on them. Furthermore, the time required for reservation confirmation and finalization is inconvenient for food delivery services where immediacy is essential.

[0719] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0720] In this invention, the server includes means for querying the API or database of the provider to be booked and confirming available dates and times; means for parsing the user's request in natural language and generating a response using a generative AI model; and means for utilizing a database or API for booking confirmation and booking confirmation. This makes it possible for users to book food delivery quickly and accurately simply by entering a request, reducing the burden on the user and streamlining the entire process.

[0721] A "user" is a person or entity that wishes to make a reservation and enters a request.

[0722] A "terminal" is an electronic device used by a user to input and send requests to a server; it is a device that handles request input and displays response messages.

[0723] A "server" is a central processing unit that receives user requests and performs processing such as analysis, reservation confirmation, and reservation confirmation.

[0724] A "request" is a request that a user enters into their terminal with the details of their desired reservation and sends to the server.

[0725] A "reservation target provider" refers to the provider of the service or product that the user wishes to reserve.

[0726] "API" stands for Application Programming Interface, which is an interface that allows different systems and programs to communicate with each other.

[0727] A "database" is a collection of data that systematically stores information and allows it to be searched and manipulated.

[0728] "Available dates and times for booking" refers to the dates and times when the provider of the booking can provide the service.

[0729] A "response message" is a message generated by a server to respond to a user after analyzing the user's request.

[0730] "Natural language analysis" refers to a technology that allows a computer to mechanically understand the language entered by a user, meaning that the computer interprets the language that humans speak naturally.

[0731] A "generative AI model" is an artificial intelligence model generated based on machine learning, and it is an algorithm that generates the optimal response from input data.

[0732] This invention relates to a system for automating and efficiently processing food delivery reservations requested by users. This system consists of users, terminals, and a server, with each program fulfilling its respective role. Embodiments of this system are described in detail below.

[0733] System Configuration

[0734] 1. User:

[0735] The user is someone who wants to order food delivery and enters their request via a device such as a smartphone. For example, they might make a request like, "I want a pizza delivered at 7 PM," using voice input or text input.

[0736] 2. Terminal:

[0737] The terminal is a smartphone, tablet, or other mobile device that receives requests entered by the user and sends them to the server. The terminal also displays response messages and confirmation messages received from the server to the user.

[0738] 3. Server:

[0739] The server receives requests from users, parses them, checks reservation status, and generates response messages. The server queries the API or database of food delivery service providers to check available dates and times. It also parses user requests in natural language and generates responses using generative AI models.

[0740] Hardware and software to be used

[0741] Smartphone: Used as a user interface and a device for sending requests.

[0742] Server: Cloud-based or on-premises central processing unit. Programming languages ​​used are Python and Flask.

[0743] API: Used for communication with food delivery providers.

[0744] Generative AI models: Machine learning models used for natural language processing and response generation.

[0745] Data processing and data calculation

[0746] The server analyzes the user's natural language request and extracts keywords such as "pizza" and "7 PM." This analysis is performed using a generative AI model. The server then queries the API of a food delivery service provider to check available delivery times. Based on the results, the server generates a response message such as, "We can deliver a pizza at 7 PM. Would you like to order?" This message is sent to the device, which then displays it to the user.

[0747] Specific example of processing

[0748] For example, if a user enters a request saying, "I want a pizza delivered at 7 PM," the following exchange will take place:

[0749] 1. User: "I'd like the pizza delivered at 7 PM."

[0750] 2. Terminal: Sends the request to the server.

[0751] 3. Server: Analyzes "pizza" and "7 PM"

[0752] 4. Server: Query the API of the food delivery provider to check available reservation times.

[0753] 5. Server: Generates a response message, "We can deliver a pizza at 7 PM. Would you like to order?", and sends it to the terminal.

[0754] 6. Terminal: Display message

[0755] 7. User: "Yes, place the order."

[0756] 8. Terminal: Sends a confirmation request to the server.

[0757] 9. Server: Confirms the order, generates a message "Order Confirmed," and sends it to the terminal.

[0758] 10. Terminal: Display a reservation confirmation message to the user.

[0759] This allows users to easily book food delivery orders, streamlining the entire process.

[0760] Example of a prompt:

[0761] User: "I want the pizza delivered at 7 PM."

[0762] System: "We can deliver a pizza at 7 PM. Would you like to order?"

[0763] User: "Yes, place the order."

[0764] System: "Your order has been confirmed."

[0765] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0766] Step 1:

[0767] The user enters a request into a smartphone application saying, "I want a pizza delivered at 7 PM."

[0768] Input: User's natural language request ("I want a pizza delivered at 7 PM")

[0769] Output: Input text data

[0770] Step 2:

[0771] The terminal sends the entered request to the server.

[0772] Input: User input text data

[0773] Output: Text request sent to the server

[0774] Step 3:

[0775] The server analyzes the received request and uses a generative AI model to extract the necessary information for the reservation, such as "pizza" and "7 PM."

[0776] Input: Text request sent from the terminal

[0777] Data processing: Natural language analysis using generative AI models

[0778] Output: Analysis results regarding "pizza" and "7 PM"

[0779] Step 4:

[0780] Based on the analysis results, the server queries the API or database of food delivery service providers to check available dates and times for reservations.

[0781] Input: Analysis results for "pizza" and "7 PM"

[0782] Data processing: API queries and retrieval of available dates and times.

[0783] Output: Available dates and times for booking (e.g., "7 PM")

[0784] Step 5:

[0785] Based on the query results, the server generates a response message saying, "We can deliver a pizza at 7 PM. Would you like to order?" and sends it to the terminal.

[0786] Input: Available dates and times

[0787] Data processing: Generating response messages

[0788] Output: Response message (Example: "We can deliver a pizza at 7 PM. Would you like to order?")

[0789] Step 6:

[0790] The terminal displays the response message received from the server to the user.

[0791] Input: Response message from the server

[0792] Output: Message displayed to the user

[0793] Step 7:

[0794] The user enters "Yes, place order" into the terminal as a confirmation request.

[0795] Input: User confirmation request ("Yes, place order")

[0796] Output: Input text data

[0797] Step 8:

[0798] The device sends a user verification request to the server.

[0799] Input: User input text data

[0800] Output: Confirmation request to send to the server

[0801] Step 9:

[0802] The server receives the confirmation request and uses the food delivery service provider's API to confirm the reservation.

[0803] Input: Verification request sent from the device

[0804] Data processing: Execution of reservation confirmation process (API call)

[0805] Output: Booking Confirmation Status

[0806] Step 10:

[0807] The server generates a message indicating that the reservation has been confirmed and sends it to the terminal.

[0808] Input: Booking Confirmation Status

[0809] Data processing: Generating reservation confirmation messages

[0810] Output: Reservation confirmation message (e.g., "Your order has been confirmed")

[0811] Step 11:

[0812] The terminal displays the reservation confirmation message received from the server to the user.

[0813] Input: Reservation confirmation message from the server

[0814] Output: Message displayed to the user

[0815] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0816] This invention relates to a system that combines an emotion engine with a system that automatically processes user requests to provide optimal responses and services based on the user's emotional state. This system consists of a user, a terminal, a server, and an emotion engine, and operates as follows.

[0817] System Overview

[0818] User: Enter a reservation request and receive recommendations based on their emotional state.

[0819] Terminal: Receives user input, sends it to the server along with sentiment information, and displays the server's response to the user.

[0820] Server: Receives and analyzes user requests and sentiment information, checks reservation status, and generates response messages.

[0821] Emotion Engine: Analyzes the user's emotional state from their input, voice, facial expressions, etc., and provides the results to the server.

[0822] Program processing

[0823] 1. User request input and sentiment recognition

[0824] A user wants to make a reservation at a hair salon and enters "I want to make a reservation at a hair salon" into the terminal. Simultaneously, the emotion engine analyzes the user's facial expressions and tone of voice to determine their emotional state (e.g., joy, stress, anxiety, etc.).

[0825] 2. Sending requests and sentiment information from the device.

[0826] The device sends the user's request data and the emotion information obtained from the emotion engine to the server. The data is sent to the specified endpoint using HTTP requests or similar methods.

[0827] 3. Receiving and parsing server requests

[0828] The server receives requests and sentiment information sent from the terminal. It analyzes the request content and extracts specific keywords such as "hair salon reservation." Sentiment information is also analyzed simultaneously.

[0829] 4. Check available dates and times for booking.

[0830] The server accesses the hair salon's reservation system (API or database) to search for available dates and times. During this process, the number and content of the displayed options are adjusted based on the user's emotional state.

[0831] 5. Generating a response message

[0832] Based on the available dates and times retrieved by the server, a response message optimized for the user's emotional state is generated. For example, a message such as "You can make a reservation on [Month] [Day] at [Time]. Would you like to make a reservation?" is modified to include emotionally appropriate expressions and recommendations.

[0833] 6. Sending a response message

[0834] The server sends the generated response message to the terminal. Then, it sends the response data again using an HTTP request or similar method.

[0835] 7. Display of the terminal's response message

[0836] The terminal receives a response message from the server and displays it to the user. The user interface (UI) is updated to display the received message on the screen.

[0837] 8. Enter the user's confirmation request.

[0838] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their intention to confirm the reservation.

[0839] 9. Sending a confirmation request

[0840] The device sends a user verification request to the server. The verification request is sent as an HTTP request, just like the initial request.

[0841] 10. Server reservation confirmation process

[0842] The server receives the confirmation request and accesses the salon's reservation system (API or database) to confirm the reservation. If the reservation is confirmed, the server generates a message saying "Reservation confirmed".

[0843] 11. Sending and displaying the reservation confirmation message.

[0844] The server generates a reservation confirmation message and sends it to the terminal, which then displays it to the user.

[0845] Specific example

[0846] The following is a specific scenario for a user who wishes to make a reservation at a hair salon.

[0847] 1. User request input and sentiment recognition

[0848] User: "I want to make a reservation at a hair salon."

[0849] Emotion Engine: Analyzes the user's voice and determines that they are relaxed.

[0850] 2. Sending requests and sentiment information from the device.

[0851] The device sends the request and sentiment information to the server.

[0852] 3. Server analysis and reservation confirmation

[0853] Server: Analyzes "hair salon reservation" and "relaxed"

[0854] Server: Check the availability of the hair salon (at XX / XX)

[0855] 4. Server response generation and transmission

[0856] Server: Generates the response message: "You can make a reservation at XX / XX. Don't be nervous, relax and enjoy yourself!"

[0857] Server: Sends a response message to the terminal.

[0858] 5. Displaying messages on the device and verifying the user.

[0859] Terminal: Message display

[0860] User: "Yes, please make a reservation."

[0861] Terminal: Sends a confirmation request to the server.

[0862] 6. Confirm server reservation and send message

[0863] Server: Confirm reservation

[0864] Server: "Your reservation has been confirmed."

[0865] Server: Sends a reservation confirmation message to the terminal.

[0866] 7. Display of the device reservation confirmation message.

[0867] Terminal: Reservation confirmation message displayed

[0868] Thus, the present invention allows users to easily make reservations and enjoy an even more comfortable service experience by receiving support tailored to their emotional state.

[0869] The following describes the processing flow.

[0870] Step 1:

[0871] The user enters a request using the device, such as "I want to make a reservation at a hair salon." Simultaneously, the emotion engine built into the device analyzes the user's voice and facial expressions to identify their emotional state (e.g., relaxed, stressed, happy).

[0872] Step 2:

[0873] The device combines the user's request with the sentiment information obtained from the sentiment engine and sends it to the server as an HTTP request. The request includes the reservation request details and sentiment information.

[0874] Step 3:

[0875] The server receives the request sent from the terminal. Here, it analyzes the content of the request and sentiment information, and extracts the keyword "hair salon reservation" and its associated sentiment.

[0876] Step 4:

[0877] The server accesses the hair salon's reservation system (e.g., via an API or database) to search for available dates and times. During this process, it selects reservation options to present based on the user's sentiment information.

[0878] Step 5:

[0879] The server generates a response message based on the available reservation dates and times it has obtained. For example, if the user is relaxed, it will generate a response message such as, "You can make a reservation at XX o'clock. Please relax and come."

[0880] Step 6:

[0881] The server generates a response message and sends it to the terminal as an HTTP response. This message includes available dates and times for booking and additional messages depending on the sentiment.

[0882] Step 7:

[0883] The terminal receives a response message from the server and displays it to the user. The user interface is updated so that the user can see the message.

[0884] Step 8:

[0885] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their desire to confirm the reservation.

[0886] Step 9:

[0887] The device sends the user's confirmation request back to the server as an HTTP request. This request includes the intention to confirm the reservation and the original request information.

[0888] Step 10:

[0889] The server receives the confirmation request. Access the salon's reservation system again to formally confirm the reservation.

[0890] Step 11:

[0891] The server generates a reservation confirmation message. For example, it might create a message in the format, "Your reservation has been confirmed. We look forward to seeing you."

[0892] Step 12:

[0893] The server generates a reservation confirmation message and sends it to the terminal as an HTTP response. This notifies the user that their reservation has been confirmed.

[0894] Step 13:

[0895] The terminal receives a reservation confirmation message from the server and displays it to the user. The user interface is then refreshed to notify the user of the reservation confirmation.

[0896] The above describes the specific program processing flow of the present invention, which incorporates an emotion engine. This system allows users to enjoy a reservation experience that takes their own emotional state into consideration.

[0897] (Example 2)

[0898] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0899] Traditional reservation systems simply presented reservation options uniformly, without considering the user's emotional state. This sometimes led to user stress and decreased satisfaction. Furthermore, reservation confirmation messages were not tailored to the user's emotional state, resulting in an unoptimized user experience.

[0900] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0901] In this invention, the server includes means for recognizing the user's emotional state, means for selecting the optimal candidate based on the emotional state, and means for generating a response message with expressions corresponding to the emotion. This enables the provision of optimal services and the generation of response messages based on the user's emotional state.

[0902] A "user" is someone who uses a system to enter a request and receive a service.

[0903] A "terminal" is an electronic device used by a user to input requests and communicate with a server.

[0904] A "server" is a central device that receives and analyzes requests from users and executes the specified service (e.g., confirming a reservation).

[0905] A "request" is a request for a service that a user sends through their device.

[0906] "Emotional state" refers to the user's psychological state at that time, and includes feelings such as joy, relaxation, stress, and anxiety.

[0907] An "emotion engine" is a software or hardware mechanism that analyzes a user's emotional state based on their facial expressions, tone of voice, text input, and other factors.

[0908] "Analysis" is the process by which a server understands requests and sentiment information and determines the appropriate response.

[0909] "Available dates and times" refers to the dates and times when the service specified by the user (e.g., a hair salon reservation) is available.

[0910] A "response message" is a message sent from the server to the terminal that contains the results or information regarding the request.

[0911] A "confirmation request" is a request that a user submits to finalize a reservation or other service.

[0912] A "reservation confirmation message" is a message sent by the server to the user informing them that the server has processed the request and that the reservation has finally been confirmed.

[0913] A "database" is a system used to store reservation information and user data.

[0914] "API" stands for Application Programming Interface, and it is an interface for communicating with external systems and exchanging information.

[0915] This invention is a system that combines an emotion engine with a system that automatically processes user requests, thereby providing optimal responses and services based on the user's emotional state. The system consists of a user, a terminal, a server, and an emotion engine.

[0916] System Configuration

[0917] User: A person who uses a system to enter requests and receive services.

[0918] Terminal: An electronic device that receives user input, sends it to a server along with emotional information, and displays the server's response to the user. Examples include smartphones and personal computers.

[0919] Server: A central device that receives and analyzes user requests and sentiment information, checks reservation status, and generates response messages.

[0920] Emotion engine: A software or hardware mechanism that analyzes the user's emotional state from user input, voice, facial expressions, etc., and provides the results to a server.

[0921] Detailed explanation of the process

[0922] 1. User request input:

[0923] The user enters "I want to make a reservation at a hair salon" into their device (such as a smartphone or computer). Input can be done via text or voice, and if the user uses voice input, their voice is captured using a microphone.

[0924] 2. Emotion recognition:

[0925] The text and voice input by the user are sent to the emotion engine via the device. The emotion engine analyzes the user's facial expressions and tone of voice to determine their emotional state. For example, if a user says, "I want to make a reservation at the hair salon," the emotion engine analyzes the user's tone of voice and assesses whether they are relaxed.

[0926] 3. Sending requests and sentiment information:

[0927] The device combines the user's request data and sentiment information obtained from the sentiment engine into a single JSON file, and sends this data to the server using an HTTP POST request. Network communication is used for this process.

[0928] 4. Request reception and parsing:

[0929] The server receives request data and emotional information sent from the terminal. The server analyzes the request content and extracts specific keywords such as "hair salon reservation." At the same time, the server analyzes the user's emotional state based on the received emotional information.

[0930] 5. Check available dates and times:

[0931] The server accesses an external salon booking API or an internal database to check for available dates and times. During this process, the server adjusts the number and content of the options displayed based on the user's emotional state. For example, if the user is very relaxed, it will present many options; if they are in a hurry, it will present only a limited number.

[0932] 6. Generating a response message:

[0933] The server generates a response message optimized for the user's emotional state based on the available reservation date and time. For example, it might generate a message like, "You can make a reservation on [Month] [Day] at [Time]. Please relax and enjoy yourself."

[0934] 7. Sending a response message:

[0935] The server sends the generated response message back to the terminal as an HTTP POST request, and the terminal receives it.

[0936] 8. Display of response message:

[0937] The terminal displays the response message received from the server in the user interface. The user reviews it and decides on the next action.

[0938] 9. Enter your confirmation request:

[0939] The user types "Yes, book it" to confirm the reservation. They can do this by typing into a text box or by giving a voice command. This indicates that they wish to confirm the reservation.

[0940] 10. Submit a confirmation request:

[0941] The device then sends the confirmation request received from the user back to the server as an HTTP POST request.

[0942] 11. Booking confirmation process:

[0943] The server receives the confirmation request and accesses the salon's reservation system API to confirm the reservation. If the reservation is successfully confirmed, the server generates a message saying "Reservation confirmed".

[0944] 12. Sending and displaying booking confirmation messages:

[0945] The server sends the generated reservation confirmation message to the terminal, and the terminal updates its user interface to display the reservation confirmation message.

[0946] Specific example

[0947] The following is a specific scenario for a user who wishes to make a reservation at a hair salon.

[0948] 1. User request input and sentiment recognition

[0949] User: "I want to make a reservation at a hair salon."

[0950] Emotion Engine: Analyzes the user's voice and determines that they are relaxed.

[0951] 2. Sending requests and sentiment information from the device.

[0952] The device sends the request and sentiment information to the server.

[0953] 3. Server analysis and reservation confirmation

[0954] Server: Analyzes "hair salon reservation" and "relaxed"

[0955] Server: Check the availability of the hair salon (e.g., Month XX, Day XX, Time XX)

[0956] 4. Server response generation and transmission

[0957] Server: Generates a response message saying, "Your reservation is available on [Month] [Day] at [Time]. Relax and enjoy!"

[0958] Server: Sends a response message to the terminal.

[0959] 5. Displaying messages on the device and verifying the user.

[0960] Terminal: Message display

[0961] User: "Yes, please make a reservation."

[0962] Terminal: Sends a confirmation request to the server.

[0963] 6. Confirm server reservation and send message

[0964] Server: Confirm reservation

[0965] Server: "Your reservation has been confirmed."

[0966] Server: Sends a reservation confirmation message to the terminal.

[0967] 7. Display of the device reservation confirmation message.

[0968] Terminal: Reservation confirmation message displayed

[0969] Example of a prompt

[0970] User input example: "I want to make a reservation at a hair salon."

[0971] Input to the response generation AI: "Process the following request and generate the most appropriate response message based on the user's emotional state. Request: 'Make a hair salon appointment', Emotional state: 'Relaxed'"

[0972] This system allows users to easily make reservations and enjoy an even more comfortable service experience by receiving support tailored to their emotional state.

[0973] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0974] Step 1:

[0975] User request input and sentiment recognition

[0976] The user enters "I want to make a reservation at a hair salon" into the device. Input can be done via text or voice. If the user uses voice input, the device's microphone captures the audio. Simultaneously, the device's camera captures the user's facial expressions. The emotion engine receives the audio and video data and analyzes the emotional state using natural language processing and image analysis. This generates request data and emotion information.

[0977] Input: User text or voice input, camera video.

[0978] Output: Request data (hair salon reservation), emotional information (e.g., relaxed)

[0979] Step 2:

[0980] Sending requests and sentiment information

[0981] The device combines the user's request data and sentiment information obtained from the sentiment engine into a single JSON file. This data is then sent to the server's specified endpoint using an HTTP POST request. Specifically, the data transmission is performed using network communication capabilities.

[0982] Input: Request data, sentiment information

[0983] Output: Data sent to the server as an HTTP request

[0984] Step 3:

[0985] Server request reception and analysis

[0986] The server receives request data and sentiment information sent from the terminal. The server parses the JSON data and extracts the request content. For example, it extracts and analyzes the keyword "hair salon reservation" and sentiment information. This allows the server to understand the user's desired service and emotional state.

[0987] Input: HTTP request data (request content, sentiment information)

[0988] Output: Analysis results (service details, hair salon reservation, emotional state)

[0989] Step 4:

[0990] Check available dates and times for booking

[0991] The server accesses the salon's reservation system API or internal database to check for available dates and times. This process narrows down the options according to specific rules based on the user's emotional state. For example, it presents more reservation options to a relaxed user and fewer options to a user in a hurry.

[0992] Input: Analyzed request (service details, sentiment information)

[0993] Output: List of available dates and times

[0994] Step 5:

[0995] Generating a response message

[0996] Based on the available dates and times retrieved by the server, a response message optimized for the user's emotional state is generated. Expressions are added according to the emotional state. For example, a message such as "You can make a reservation on [Month] [Day] at [Time]. Relax and enjoy!" is generated.

[0997] Input: List of available dates and times, sentiment information

[0998] Output: Optimized response message

[0999] Step 6:

[1000] Sending a response message

[1001] The server generates a response message, which is then compiled into JSON format and sent again to the specified endpoint on the device using an HTTP POST request.

[1002] Input: Response message

[1003] Output: Response message sent to the terminal as an HTTP request

[1004] Step 7:

[1005] Display of terminal response messages

[1006] The terminal analyzes the response message received from the server and updates the user interface. The message is displayed on the screen, prompting the user to confirm.

[1007] Input: Response message sent to the terminal

[1008] Output: Message displayed in the updated user interface

[1009] Step 8:

[1010] Entering a user verification request

[1011] If the user wants to confirm the reservation, they type "Yes, book it." They can type this into the text box or use a voice command. This will generate a confirmation request.

[1012] Input: User confirmation input (text or voice)

[1013] Output: Confirmation Request

[1014] Step 9:

[1015] Sending a confirmation request

[1016] The device compiles the confirmation request into JSON format and sends it again as an HTTP POST request to the specified endpoint on the server.

[1017] Input: Confirmation Request

[1018] Output: Confirmation request sent to the server as an HTTP request

[1019] Step 10:

[1020] Server reservation confirmation process

[1021] The server receives the confirmation request and accesses the salon's reservation system API to confirm the reservation. If the reservation is successfully confirmed as a result of this process, the server generates a message saying "Reservation confirmed".

[1022] Input: Confirmation Request

[1023] Output: Booking confirmation message

[1024] Step 11:

[1025] Sending and displaying reservation confirmation messages

[1026] The server generates a reservation confirmation message and sends it to the terminal. The terminal receives this message, updates the user interface, and displays the reservation confirmation message on the screen.

[1027] Input: Booking confirmation message

[1028] Output: Booking confirmation message displayed in the updated user interface

[1029] (Application Example 2)

[1030] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1031] Traditional reservation systems simply respond to user requests and lack the flexibility to adapt to user emotional states or request content. Therefore, to improve the user experience in service delivery, a system is needed that provides optimal responses and services based on the user's emotional state.

[1032] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1033] In this invention, the server includes means for analyzing the user's facial expressions and tone of voice to recognize their emotional state, means for optimizing requests based on the emotional state, and means for the server to generate a response message and send it to the terminal. This enables flexible and optimal responses and service provision based on the user's emotional state.

[1034] "User" refers to an individual or group that uses the system.

[1035] A "request" refers to an operation or request that a user enters.

[1036] A "terminal" is a device used by a user to input requests and to receive and display responses from a server.

[1037] A "server" is a component that receives and analyzes requests sent from a terminal and generates the optimal response.

[1038] "Analysis" refers to the process of breaking down and understanding request and emotional state data.

[1039] "Available dates and times" refers to the dates and times when the service can be provided, and is presented as an option in response to the user's request.

[1040] A "response message" refers to information generated by the server and sent to the user via the terminal.

[1041] "Facial expressions" refer to the expressions on a user's face and are a source of information for determining their emotions.

[1042] "Voice tone" refers to the pitch and volume of a user's voice and is a source of information for analyzing their emotions.

[1043] "Emotional state" refers to the user's psychological state and is obtained from analyzed facial expressions and tone of voice.

[1044] "Optimization" refers to the process of adjusting requests and responses based on emotional states.

[1045] A "confirmation request" refers to an action taken by a user to provide additional information or confirm a request that was initially submitted.

[1046] A "reservation confirmation message" refers to the final response message generated by the server to notify the user that the reservation was successful.

[1047] "System" refers to the entire set of components, including users, terminals, servers, sentiment analysis engines, and their interfaces.

[1048] This invention is a system that automatically processes user requests and provides optimal responses and services based on the user's emotional state. The system consists of a user, a terminal, a server, and an emotion analysis engine.

[1049] Hardware and software to be used

[1050] Device: Smartphone with built-in camera and microphone

[1051] Server: Cloud-based web server

[1052] Sentiment analysis engine: Microsoft Azure Face API, Google Cloud Vision API

[1053] Video streaming platform APIs: YouTube Data API, Netflix API

[1054] Data processing and computation

[1055] 1. Recognizing the user's emotional state

[1056] The device uses its built-in camera and microphone to capture the user's facial expressions and voice tone.

[1057] An emotion analysis engine (such as Microsoft Azure Face API or Google Cloud Vision API) is used to analyze the acquired data and recognize the user's emotional state.

[1058] 2. Sending emotional data

[1059] The analyzed emotional data (e.g., joy, sadness, stress) is sent to the server using an HTTP request.

[1060] 3. Emotion-based content recommendations

[1061] The server analyzes the received emotion data and uses video streaming platform APIs (e.g., YouTube Data API, Netflix API) to search for videos that are appropriate for the user's emotional state.

[1062] 4. Generating and sending video recommendations

[1063] The server selects the most suitable video recommendation and sends the recommendation result to the user's device using an HTTP request.

[1064] The terminal receives a response message from the server and displays it to the user.

[1065] Specific Scenario

[1066] 1. User emotion recognition

[1067] When a user opens the app, their facial expressions and voice are captured using the smartphone's camera and microphone.

[1068] The emotion analysis engine analyzes facial expressions and voice tone to determine that "the user is relaxed."

[1069] 2. Sending emotional data

[1070] Send emotional data (relaxed) to the server.

[1071] 3. Generating video recommendations

[1072] The server uses emotional data to search for videos suitable for relaxation (for example, videos with nature sounds or relaxing music videos).

[1073] 4. Sending and displaying video recommendations

[1074] The server sends a list of related videos to the user's smartphone with the comment, "These videos are perfect for relaxing."

[1075] Users can view a list of videos on their smartphones, select their favorite video, and play it.

[1076] Example of a prompt

[1077] "Please recommend videos that match the user's current emotional state. Based on the user's facial expressions and tone of voice, their emotional state is relaxed."

[1078] Thus, the present invention allows users to quickly obtain videos that are suitable for their emotional state, resulting in a personalized and excellent user experience.

[1079] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1080] Step 1:

[1081] The user opens the smartphone app. The device uses its built-in camera and microphone to capture the user's facial expressions and voice tone. This data is input into the emotion analysis engine. The emotion analysis engine uses the Microsoft Azure Face API or Google Cloud Vision API to output emotion data (e.g., relaxed, stressed).

[1082] Step 2:

[1083] The device sends analyzed sentiment data to the server. The server receives the HTTP request and verifies the user's sentiment state. Based on the input sentiment data, it requests suitable video recommendations from video streaming platform APIs (e.g., YouTube Data API, Netflix API).

[1084] Step 3:

[1085] The server retrieves a list of relevant videos from the video streaming platform API. The server filters the retrieved video information based on sentiment data to generate an optimal recommendation list. The generated list is sent to the terminal along with an optimized response message.

[1086] Step 4:

[1087] The device receives a response message from the server and displays it to the user. A list of videos is displayed, including comments tailored to the user's emotional state. For example, the video list might be displayed with the message, "These videos are perfect for relaxing."

[1088] Step 5:

[1089] The user selects a video from the displayed video list and starts playback. The device passes the information of the selected video to the video player and begins streaming the video.

[1090] This allows users to easily find and watch videos that match their emotional state.

[1091] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1092] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1093] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1094] [Third Embodiment]

[1095] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1096] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1097] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1098] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1099] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1100] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1101] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1102] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1103] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1104] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1105] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1106] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1107] This invention relates to a system for automating and efficiently processing reservations requested by users. This system consists of a user, a terminal, and a server, with each program fulfilling its respective role.

[1108] System Overview

[1109] User: Enter your reservation request and send it to the server via your device.

[1110] Terminal: Receives user input, sends requests to the server, receives responses from the server, and displays them to the user.

[1111] Server: Receives user requests, parses them, checks reservation status, and generates response messages.

[1112] Program processing

[1113] 1. Entering and submitting the user's request.

[1114] A user wants to make a reservation at a hair salon and enters "I want to make a reservation at a hair salon" into the terminal. The terminal sends this input to the server.

[1115] 2. Receiving and parsing server requests

[1116] The server receives a request from the terminal and analyzes its content, such as "hair salon reservation." During this process, it extracts appropriate keywords.

[1117] 3. Check available dates and times for reservations.

[1118] The server queries the beauty salon's API or database to check for available appointment times. For example, the server might get a result like "Appointment is available at XX / XX."

[1119] 4. Generating and sending response messages

[1120] The server generates a response message based on the available dates and times, such as "You can make a reservation at XX / XX. Do you want to make a reservation?", and sends it to the terminal.

[1121] 5. Display of the terminal's response message

[1122] The terminal displays the received response message to the user. The user confirms the message and responds, "Yes, book it."

[1123] 6. Sending a user verification request

[1124] The device resends the user's confirmation request to the server.

[1125] 7. Server reservation confirmation process

[1126] The server receives the confirmation request and accesses the salon's reservation system to confirm the reservation. If the reservation is confirmed, the server generates a message saying "Reservation confirmed".

[1127] 8. Sending and displaying the reservation confirmation message.

[1128] The server sends a reservation confirmation message to the terminal, which then displays it to the user.

[1129] Specific example

[1130] The following is a specific scenario for a user who wants to make a reservation at a hair salon.

[1131] 1. User request input

[1132] User: "I want to make a reservation at a hair salon."

[1133] Terminal: Sends a request to the server

[1134] 2. Server analysis and reservation confirmation

[1135] Server: Analyzes "hair salon reservations"

[1136] Server: Check the availability of the hair salon (at XX / XX)

[1137] 3. Server response generation and transmission

[1138] Server: "You can make a reservation at [time]. Would you like to make a reservation?"

[1139] Server: Sends a response message to the terminal.

[1140] 4. Displaying messages on the device and verifying the user.

[1141] Terminal: Message display

[1142] User: "Yes, please make a reservation."

[1143] Terminal: Sends a confirmation request to the server.

[1144] 5. Confirm server reservation and send message

[1145] Server: Confirm reservation

[1146] Server: "Your reservation has been confirmed."

[1147] Server: Sends a reservation confirmation message to the terminal.

[1148] 6. Display of the device reservation confirmation message

[1149] Terminal: Reservation confirmation message displayed

[1150] Thus, the present invention allows users to easily make reservations, simplifying and streamlining the process. Furthermore, reservations can be quickly confirmed and modified, greatly improving user convenience.

[1151] The following describes the processing flow.

[1152] Step 1:

[1153] The user enters a request using the terminal, such as "I want to make a reservation at a hair salon." This input is triggered by the terminal's program.

[1154] Step 2:

[1155] The device sends the user's request as a string to the server. It uses protocols such as HTTP requests to send data to the specified endpoint.

[1156] Step 3:

[1157] The server receives a request sent from the terminal. It analyzes the content of the request and extracts the specific keyword "hair salon reservation".

[1158] Step 4:

[1159] The server accesses the hair salon's reservation system (API or database) to search for available dates and times. For example, it might query the reservation system's API endpoint to retrieve availability data.

[1160] Step 5:

[1161] The server generates a response message based on the available reservation dates and times. For example, it generates a message in the format of "You can make a reservation on [Month] [Day] at [Time]. Would you like to make a reservation?".

[1162] Step 6:

[1163] The server sends the generated response message to the terminal. Then, it sends the response data again using an HTTP request or similar method.

[1164] Step 7:

[1165] The terminal receives a response message from the server and displays it to the user. The user interface (UI) is updated to display the received message on the screen.

[1166] Step 8:

[1167] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their intention to confirm the reservation.

[1168] Step 9:

[1169] The device sends a user verification request to the server. The verification request is sent as an HTTP request, just like the initial request.

[1170] Step 10:

[1171] The server receives the confirmation request. It then accesses the reservation system (API or database) again to formally confirm the reservation.

[1172] Step 11:

[1173] The server generates a reservation confirmation message. For example, it might create a message in the format "Your reservation has been confirmed."

[1174] Step 12:

[1175] The server sends a reservation confirmation message to the terminal. It sends data to notify the user of the reservation confirmation.

[1176] Step 13:

[1177] The terminal receives a reservation confirmation message from the server and displays it to the user. The user interface is then refreshed to notify the user that the reservation has been confirmed.

[1178] The above outlines the specific program processing flow when a user makes a reservation at a hair salon.

[1179] (Example 1)

[1180] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1181] In recent years, there has been a growing demand for efficient reservation systems. However, traditional systems have suffered from cumbersome reservation procedures and low user convenience. In particular, users often had to manually enter and verify information when requesting a reservation, and the process of confirming a reservation was often time-consuming. This resulted in a poor user experience and decreased operational efficiency.

[1182] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1183] In this invention, the server includes means for a user to input a reservation request, means for a terminal to send the reservation request to the server, means for the server to receive the reservation request and parse it using a natural language processing algorithm, means for the server to query an external reservation system to confirm available dates and times, means for the server to generate a response message based on the available dates and times and send it to the terminal, means for the terminal to display the response message to the user, means for the user to input a confirmation request in response to the response message, means for the terminal to resend the confirmation request to the server, means for the server to receive the confirmation request, access the external reservation system and confirm the reservation, means for the server to generate a reservation confirmation message and send it to the terminal, and means for the terminal to display the reservation confirmation message to the user. This allows the user to easily perform the process from request input to reservation confirmation, enabling simplification and efficiency of the procedure.

[1184] A "user" is an individual who uses the reservation system to enter a request and completes the process until the reservation is confirmed.

[1185] A "terminal" is a device used by a user to input requests and communicate with a server, and includes smartphones, tablets, and personal computers.

[1186] A "server" is a computer system that receives requests from users, analyzes them, checks available dates and times for reservations, and generates and sends response messages.

[1187] A "request" is information entered by a user that expresses their desired reservation details.

[1188] A "natural language processing algorithm" is a computational method for analyzing and understanding text entered by a user and extracting appropriate keywords.

[1189] A "reservation system" is a system that uses an external database or API to check reservation availability and process reservations.

[1190] A "response message" is a message generated by the server and sent to the terminal, which includes available dates and times for booking and a request for confirmation.

[1191] A "confirmation request" is a message that a user enters in response to a reply message to confirm their reservation.

[1192] A "reservation confirmation message" is a message generated by the server to notify the user that the reservation has been confirmed.

[1193] This invention relates to a system that automates and efficiently processes reservations requested by users. The system consists of three elements: the user, the terminal, and the server, each element playing its own role.

[1194] The user enters the details of their reservation request into the device. Specifically, they enter a request in text format, such as "I would like to make a reservation at a hair salon." Users can use devices such as smartphones, tablets, or personal computers.

[1195] The terminal receives requests entered by the user and sends them to the server. The terminal obtains user requests using an input interface (e.g., keyboard, touchscreen, voice input system) and sends them to the server using a communication module (e.g., Wi-Fi, mobile data communication). This ensures that the user's request reaches the server.

[1196] When the server analyzes a request received from a terminal, it uses a natural language processing algorithm (e.g., OpenAI GPT-4) to understand the request content and extract appropriate keywords. After analyzing the request, the server queries an external reservation system (e.g., a reservation management API or database) to check for available dates and times.

[1197] The server generates a response message based on the confirmed available dates and times and sends a confirmation message to the terminal asking, "Can I make a reservation?". The server's message generation logic is used to generate the message.

[1198] The terminal displays the received response message to the user. The user confirms the displayed message and enters a confirmation request, such as "Yes, book it," into the terminal. The terminal then uses the communication module to send the confirmation request to the server again.

[1199] Upon receiving the confirmation request, the server accesses the external reservation system again to formally confirm the reservation. Once the reservation is confirmed, it generates a notification message "Reservation Confirmed" and sends it to the terminal.

[1200] The device displays a reservation confirmation message to the user, and the reservation process is complete.

[1201] Specific example

[1202] The following is a specific scenario for a user who wants to make a reservation at a hair salon.

[1203] User request input

[1204] User: Types "I want to make a hair salon appointment" on their smartphone.

[1205] Terminal: Sends the entered string to the server as an HTTP request.

[1206] Server analysis and reservation confirmation

[1207] Server: Parses the JSON data received as an HTTP request ({"request": "I want to make a reservation at a hair salon"}).

[1208] Server: Uses OpenAI GPT-4 to extract keywords such as "hair salon" and "reservation".

[1209] Server: Query the beauty salon reservation system's API to check available dates and times (e.g., "October 1st, 2:00 PM").

[1210] Server response generation and transmission

[1211] Server: Based on the available dates and times, it generates the message, "You can make a reservation for October 1st at 2:00 PM. Would you like to make a reservation?"

[1212] Server: Sends the generated message to the terminal as an HTTP response.

[1213] Displaying messages on the device and verifying the user.

[1214] Device: The received message is displayed on the screen. A pop-up appears saying, "You can make a reservation for October 1st at 2:00 PM. Do you want to make a reservation?"

[1215] User: In response to the displayed message, type "Yes, book it" into the terminal.

[1216] Terminal: Sends the confirmation request back to the server as an HTTP request.

[1217] Server reservation confirmation and message sending

[1218] Server: Accesses the hair salon reservation system and processes the reservation confirmation.

[1219] Server: If the reservation confirmation is successful, it generates the message "Reservation confirmed".

[1220] Server: Sends a confirmation message to the terminal.

[1221] Display of the device reservation confirmation message

[1222] Terminal: Receives a confirmation message and displays to the user that "Your reservation has been confirmed."

[1223] Example of a prompt

[1224] The following is an example of a prompt message to input into a generative AI model.

[1225] Describe the steps a user takes from requesting and submitting a hair salon appointment to confirming the booking. Include all hardware and software used in detail.

[1226] This invention allows users to easily complete the process from request entry to reservation confirmation, simplifying and streamlining the procedure. Furthermore, reservations can be quickly confirmed and modified, greatly improving user convenience.

[1227] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1228] Step 1:

[1229] The user enters a reservation request and sends it to their device. Specifically, the user uses the device's input interface (e.g., keyboard, touchscreen) to enter "I want to make a reservation at a hair salon." The input is the user's request, and the output is sending that request to the server as an HTTP request. The user performs this operation on a device such as a smartphone or tablet.

[1230] Step 2:

[1231] The terminal receives user input and sends it to the server as an HTTP request. The terminal converts the input request into JSON format and sends it to the server. A communication module (e.g., Wi-Fi or mobile data communication) is used in this process. The input is the request content entered by the user, and the output is the HTTP request sent to the server.

[1232] Step 3:

[1233] The server parses the HTTP request received from the terminal. The server parses the request content into JSON and extracts it as text data. Then, it performs natural language processing using a generative AI model (e.g., OpenAI GPT-4) to extract keywords such as "hair salon reservation". The input is the JSON data of the HTTP request, and the output is the parsed keywords.

[1234] Step 4:

[1235] The server queries an external reservation system (e.g., a hair salon's reservation API) based on the analysis results to check available dates and times. The reservation system returns availability information via the API query, which the server receives. The input is the analyzed keywords, and the output is information about available dates and times. A web API is used for this query.

[1236] Step 5:

[1237] The server generates a response message based on the received information about available dates and times for booking. Specifically, it generates a message that includes the available date and time for booking (e.g., "You can book on October 1st at 2:00 PM. Do you want to book?"). The generated response message is then sent back to the terminal as an HTTP response. The input is the information about available dates and times for booking, and the output is the response message.

[1238] Step 6:

[1239] The terminal displays the response message received from the server to the user. The terminal parses the received message and displays it through the user interface. The input is the response message from the server, and the output is the message that the user sees.

[1240] Step 7:

[1241] The user enters a confirmation request in response to the displayed response message. They enter a confirmation request, such as "Yes, book it," into the terminal and send it. The input is the displayed response message, and the output is the confirmation request.

[1242] Step 8:

[1243] The terminal sends the user's confirmation request to the server. This request is then compiled again into JSON format data as an HTTP request and sent to the server. The input is the confirmation request, and the output is the HTTP request sent to the server.

[1244] Step 9:

[1245] The server receives the confirmation request and processes the reservation confirmation. The server accesses the external reservation system again and sends a confirmation request. If the reservation is successfully confirmed, the server receives that information. The input is the confirmation request, and the output is the reservation confirmation information.

[1246] Step 10:

[1247] The server generates a reservation confirmation message, "Your reservation has been confirmed," based on the reservation confirmation information, and sends it to the terminal. The generated message is sent to the terminal as an HTTP response. The input is the reservation confirmation information, and the output is the reservation confirmation message.

[1248] Step 11:

[1249] The terminal displays a reservation confirmation message to the user. The terminal parses the received message and displays it again through the user interface. The input is the reservation confirmation message, and the output is the confirmation message displayed to the user.

[1250] As a result, users can easily make reservations at hair salons, and the system efficiently automates the reservation process.

[1251] (Application Example 1)

[1252] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1253] The problem this invention aims to solve is to streamline the process by which users can easily reserve and confirm food delivery orders. Conventional systems require numerous manual steps for users to make reservations, which places a significant burden on them. Furthermore, the time required for reservation confirmation and finalization is inconvenient for food delivery services where immediacy is essential.

[1254] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1255] In this invention, the server includes means for querying the API or database of the provider to be booked and confirming available dates and times; means for parsing the user's request in natural language and generating a response using a generative AI model; and means for utilizing a database or API for booking confirmation and booking confirmation. This makes it possible for users to book food delivery quickly and accurately simply by entering a request, reducing the burden on the user and streamlining the entire process.

[1256] A "user" is a person or entity that wishes to make a reservation and enters a request.

[1257] A "terminal" is an electronic device used by a user to input and send requests to a server; it is a device that handles request input and displays response messages.

[1258] A "server" is a central processing unit that receives user requests and performs processing such as analysis, reservation confirmation, and reservation confirmation.

[1259] A "request" is a request that a user enters into their terminal with the details of their desired reservation and sends to the server.

[1260] A "reservation target provider" refers to the provider of the service or product that the user wishes to reserve.

[1261] "API" stands for Application Programming Interface, which is an interface that allows different systems and programs to communicate with each other.

[1262] A "database" is a collection of data that systematically stores information and allows it to be searched and manipulated.

[1263] "Available dates and times for booking" refers to the dates and times when the provider of the booking can provide the service.

[1264] A "response message" is a message generated by a server to respond to a user after analyzing the user's request.

[1265] "Natural language analysis" refers to a technology that allows a computer to mechanically understand the language entered by a user, meaning that the computer interprets the language that humans speak naturally.

[1266] A "generative AI model" is an artificial intelligence model generated based on machine learning, and it is an algorithm that generates the optimal response from input data.

[1267] This invention relates to a system for automating and efficiently processing food delivery reservations requested by users. This system consists of users, terminals, and a server, with each program fulfilling its respective role. Embodiments of this system are described in detail below.

[1268] System Configuration

[1269] 1. User:

[1270] The user is someone who wants to order food delivery and enters their request via a device such as a smartphone. For example, they might make a request like, "I want a pizza delivered at 7 PM," using voice input or text input.

[1271] 2. Terminal:

[1272] The terminal is a smartphone, tablet, or other mobile device that receives requests entered by the user and sends them to the server. The terminal also displays response messages and confirmation messages received from the server to the user.

[1273] 3. Server:

[1274] The server receives requests from users, parses them, checks reservation status, and generates response messages. The server queries the API or database of food delivery service providers to check available dates and times. It also parses user requests in natural language and generates responses using generative AI models.

[1275] Hardware and software to be used

[1276] Smartphone: Used as a user interface and a device for sending requests.

[1277] Server: Cloud-based or on-premises central processing unit. Programming languages ​​used are Python and Flask.

[1278] API: Used for communication with food delivery providers.

[1279] Generative AI models: Machine learning models used for natural language processing and response generation.

[1280] Data processing and data calculation

[1281] The server analyzes the user's natural language request and extracts keywords such as "pizza" and "7 PM." This analysis is performed using a generative AI model. The server then queries the API of a food delivery service provider to check available delivery times. Based on the results, the server generates a response message such as, "We can deliver a pizza at 7 PM. Would you like to order?" This message is sent to the device, which then displays it to the user.

[1282] Specific example of processing

[1283] For example, if a user enters a request saying, "I want a pizza delivered at 7 PM," the following exchange will take place:

[1284] 1. User: "I'd like the pizza delivered at 7 PM."

[1285] 2. Terminal: Sends the request to the server.

[1286] 3. Server: Analyzes "pizza" and "7 PM"

[1287] 4. Server: Query the API of the food delivery provider to check available reservation times.

[1288] 5. Server: Generates a response message, "We can deliver a pizza at 7 PM. Would you like to order?", and sends it to the terminal.

[1289] 6. Terminal: Display message

[1290] 7. User: "Yes, place the order."

[1291] 8. Terminal: Sends a confirmation request to the server.

[1292] 9. Server: Confirms the order, generates a message "Order Confirmed," and sends it to the terminal.

[1293] 10. Terminal: Display a reservation confirmation message to the user.

[1294] This allows users to easily book food delivery orders, streamlining the entire process.

[1295] Example of a prompt:

[1296] User: "I want the pizza delivered at 7 PM."

[1297] System: "We can deliver a pizza at 7 PM. Would you like to order?"

[1298] User: "Yes, place the order."

[1299] System: "Your order has been confirmed."

[1300] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1301] Step 1:

[1302] The user enters a request into a smartphone application saying, "I want a pizza delivered at 7 PM."

[1303] Input: User's natural language request ("I want a pizza delivered at 7 PM")

[1304] Output: Input text data

[1305] Step 2:

[1306] The terminal sends the entered request to the server.

[1307] Input: User input text data

[1308] Output: Text request sent to the server

[1309] Step 3:

[1310] The server analyzes the received request and uses a generative AI model to extract the necessary information for the reservation, such as "pizza" and "7 PM."

[1311] Input: Text request sent from the terminal

[1312] Data processing: Natural language analysis using generative AI models

[1313] Output: Analysis results regarding "pizza" and "7 PM"

[1314] Step 4:

[1315] Based on the analysis results, the server queries the API or database of food delivery service providers to check available dates and times for reservations.

[1316] Input: Analysis results for "pizza" and "7 PM"

[1317] Data processing: API queries and retrieval of available dates and times.

[1318] Output: Available dates and times for booking (e.g., "7 PM")

[1319] Step 5:

[1320] Based on the query results, the server generates a response message saying, "We can deliver a pizza at 7 PM. Would you like to order?" and sends it to the terminal.

[1321] Input: Available dates and times

[1322] Data processing: Generating response messages

[1323] Output: Response message (Example: "We can deliver a pizza at 7 PM. Would you like to order?")

[1324] Step 6:

[1325] The terminal displays the response message received from the server to the user.

[1326] Input: Response message from the server

[1327] Output: Message displayed to the user

[1328] Step 7:

[1329] The user enters "Yes, place order" into the terminal as a confirmation request.

[1330] Input: User confirmation request ("Yes, place order")

[1331] Output: Input text data

[1332] Step 8:

[1333] The device sends a user verification request to the server.

[1334] Input: User input text data

[1335] Output: Confirmation request to send to the server

[1336] Step 9:

[1337] The server receives the confirmation request and uses the food delivery service provider's API to confirm the reservation.

[1338] Input: Verification request sent from the device

[1339] Data processing: Execution of reservation confirmation process (API call)

[1340] Output: Booking Confirmation Status

[1341] Step 10:

[1342] The server generates a message indicating that the reservation has been confirmed and sends it to the terminal.

[1343] Input: Booking Confirmation Status

[1344] Data processing: Generating reservation confirmation messages

[1345] Output: Reservation confirmation message (e.g., "Your order has been confirmed")

[1346] Step 11:

[1347] The terminal displays the reservation confirmation message received from the server to the user.

[1348] Input: Reservation confirmation message from the server

[1349] Output: Message displayed to the user

[1350] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1351] This invention relates to a system that combines an emotion engine with a system that automatically processes user requests to provide optimal responses and services based on the user's emotional state. This system consists of a user, a terminal, a server, and an emotion engine, and operates as follows.

[1352] System Overview

[1353] User: Enter a reservation request and receive recommendations based on their emotional state.

[1354] Terminal: Receives user input, sends it to the server along with sentiment information, and displays the server's response to the user.

[1355] Server: Receives and analyzes user requests and sentiment information, checks reservation status, and generates response messages.

[1356] Emotion Engine: Analyzes the user's emotional state from their input, voice, facial expressions, etc., and provides the results to the server.

[1357] Program processing

[1358] 1. User request input and sentiment recognition

[1359] A user wants to make a reservation at a hair salon and enters "I want to make a reservation at a hair salon" into the terminal. Simultaneously, the emotion engine analyzes the user's facial expressions and tone of voice to determine their emotional state (e.g., joy, stress, anxiety, etc.).

[1360] 2. Sending requests and sentiment information from the device.

[1361] The device sends the user's request data and the emotion information obtained from the emotion engine to the server. The data is sent to the specified endpoint using HTTP requests or similar methods.

[1362] 3. Receiving and parsing server requests

[1363] The server receives requests and sentiment information sent from the terminal. It analyzes the request content and extracts specific keywords such as "hair salon reservation." Sentiment information is also analyzed simultaneously.

[1364] 4. Check available dates and times for booking.

[1365] The server accesses the hair salon's reservation system (API or database) to search for available dates and times. During this process, the number and content of the displayed options are adjusted based on the user's emotional state.

[1366] 5. Generating a response message

[1367] Based on the available dates and times retrieved by the server, a response message optimized for the user's emotional state is generated. For example, a message such as "You can make a reservation on [Month] [Day] at [Time]. Would you like to make a reservation?" is modified to include emotionally appropriate expressions and recommendations.

[1368] 6. Sending a response message

[1369] The server sends the generated response message to the terminal. Then, it sends the response data again using an HTTP request or similar method.

[1370] 7. Display of the terminal's response message

[1371] The terminal receives a response message from the server and displays it to the user. The user interface (UI) is updated to display the received message on the screen.

[1372] 8. Enter the user's confirmation request.

[1373] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their intention to confirm the reservation.

[1374] 9. Sending a confirmation request

[1375] The device sends a user verification request to the server. The verification request is sent as an HTTP request, just like the initial request.

[1376] 10. Server reservation confirmation process

[1377] The server receives the confirmation request and accesses the salon's reservation system (API or database) to confirm the reservation. If the reservation is confirmed, the server generates a message saying "Reservation confirmed".

[1378] 11. Sending and displaying the reservation confirmation message.

[1379] The server generates a reservation confirmation message and sends it to the terminal, which then displays it to the user.

[1380] Specific example

[1381] The following is a specific scenario for a user who wishes to make a reservation at a hair salon.

[1382] 1. User request input and sentiment recognition

[1383] User: "I want to make a reservation at a hair salon."

[1384] Emotion Engine: Analyzes the user's voice and determines that they are relaxed.

[1385] 2. Sending requests and sentiment information from the device.

[1386] The device sends the request and sentiment information to the server.

[1387] 3. Server analysis and reservation confirmation

[1388] Server: Analyzes "hair salon reservation" and "relaxed"

[1389] Server: Check the availability of the hair salon (at XX / XX)

[1390] 4. Server response generation and transmission

[1391] Server: Generates the response message: "You can make a reservation at XX / XX. Don't be nervous, relax and enjoy yourself!"

[1392] Server: Sends a response message to the terminal.

[1393] 5. Displaying messages on the device and verifying the user.

[1394] Terminal: Message display

[1395] User: "Yes, please make a reservation."

[1396] Terminal: Sends a confirmation request to the server.

[1397] 6. Confirm server reservation and send message

[1398] Server: Confirm reservation

[1399] Server: "Your reservation has been confirmed."

[1400] Server: Sends a reservation confirmation message to the terminal.

[1401] 7. Display of the device reservation confirmation message.

[1402] Terminal: Reservation confirmation message displayed

[1403] Thus, the present invention allows users to easily make reservations and enjoy an even more comfortable service experience by receiving support tailored to their emotional state.

[1404] The following describes the processing flow.

[1405] Step 1:

[1406] The user enters a request using the device, such as "I want to make a reservation at a hair salon." Simultaneously, the emotion engine built into the device analyzes the user's voice and facial expressions to identify their emotional state (e.g., relaxed, stressed, happy).

[1407] Step 2:

[1408] The device combines the user's request with the sentiment information obtained from the sentiment engine and sends it to the server as an HTTP request. The request includes the reservation request details and sentiment information.

[1409] Step 3:

[1410] The server receives the request sent from the terminal. Here, it analyzes the content of the request and sentiment information, and extracts the keyword "hair salon reservation" and its associated sentiment.

[1411] Step 4:

[1412] The server accesses the hair salon's reservation system (e.g., via an API or database) to search for available dates and times. During this process, it selects reservation options to present based on the user's sentiment information.

[1413] Step 5:

[1414] The server generates a response message based on the available reservation dates and times it has obtained. For example, if the user is relaxed, it will generate a response message such as, "You can make a reservation at XX o'clock. Please relax and come."

[1415] Step 6:

[1416] The server generates a response message and sends it to the terminal as an HTTP response. This message includes available dates and times for booking and additional messages depending on the sentiment.

[1417] Step 7:

[1418] The terminal receives a response message from the server and displays it to the user. The user interface is updated so that the user can see the message.

[1419] Step 8:

[1420] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their desire to confirm the reservation.

[1421] Step 9:

[1422] The device sends the user's confirmation request back to the server as an HTTP request. This request includes the intention to confirm the reservation and the original request information.

[1423] Step 10:

[1424] The server receives the confirmation request. Access the salon's reservation system again to formally confirm the reservation.

[1425] Step 11:

[1426] The server generates a reservation confirmation message. For example, it might create a message in the format, "Your reservation has been confirmed. We look forward to seeing you."

[1427] Step 12:

[1428] The server generates a reservation confirmation message and sends it to the terminal as an HTTP response. This notifies the user that their reservation has been confirmed.

[1429] Step 13:

[1430] The terminal receives a reservation confirmation message from the server and displays it to the user. The user interface is then refreshed to notify the user of the reservation confirmation.

[1431] The above describes the specific program processing flow of the present invention, which incorporates an emotion engine. This system allows users to enjoy a reservation experience that takes their own emotional state into consideration.

[1432] (Example 2)

[1433] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1434] Traditional reservation systems simply presented reservation options uniformly, without considering the user's emotional state. This sometimes led to user stress and decreased satisfaction. Furthermore, reservation confirmation messages were not tailored to the user's emotional state, resulting in an unoptimized user experience.

[1435] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1436] In this invention, the server includes means for recognizing the user's emotional state, means for selecting the optimal candidate based on the emotional state, and means for generating a response message with expressions corresponding to the emotion. This enables the provision of optimal services and the generation of response messages based on the user's emotional state.

[1437] A "user" is someone who uses a system to enter a request and receive a service.

[1438] A "terminal" is an electronic device used by a user to input requests and communicate with a server.

[1439] A "server" is a central device that receives and analyzes requests from users and executes the specified service (e.g., confirming a reservation).

[1440] A "request" is a request for a service that a user sends through their device.

[1441] "Emotional state" refers to the user's psychological state at that time, and includes feelings such as joy, relaxation, stress, and anxiety.

[1442] An "emotion engine" is a software or hardware mechanism that analyzes a user's emotional state based on their facial expressions, tone of voice, text input, and other factors.

[1443] "Analysis" is the process by which a server understands requests and sentiment information and determines the appropriate response.

[1444] "Available dates and times" refers to the dates and times when the service specified by the user (e.g., a hair salon reservation) is available.

[1445] A "response message" is a message sent from the server to the terminal that contains the results or information regarding the request.

[1446] A "confirmation request" is a request that a user submits to finalize a reservation or other service.

[1447] A "reservation confirmation message" is a message sent by the server to the user informing them that the server has processed the request and that the reservation has finally been confirmed.

[1448] A "database" is a system used to store reservation information and user data.

[1449] "API" stands for Application Programming Interface, and it is an interface for communicating with external systems and exchanging information.

[1450] This invention is a system that combines an emotion engine with a system that automatically processes user requests, thereby providing optimal responses and services based on the user's emotional state. The system consists of a user, a terminal, a server, and an emotion engine.

[1451] System Configuration

[1452] User: A person who uses a system to enter requests and receive services.

[1453] Terminal: An electronic device that receives user input, sends it to a server along with emotional information, and displays the server's response to the user. Examples include smartphones and personal computers.

[1454] Server: A central device that receives and analyzes user requests and sentiment information, checks reservation status, and generates response messages.

[1455] Emotion engine: A software or hardware mechanism that analyzes the user's emotional state from user input, voice, facial expressions, etc., and provides the results to a server.

[1456] Detailed explanation of the process

[1457] 1. User request input:

[1458] The user enters "I want to make a reservation at a hair salon" into their device (such as a smartphone or computer). Input can be done via text or voice, and if the user uses voice input, their voice is captured using a microphone.

[1459] 2. Emotion recognition:

[1460] The text and voice input by the user are sent to the emotion engine via the device. The emotion engine analyzes the user's facial expressions and tone of voice to determine their emotional state. For example, if a user says, "I want to make a reservation at the hair salon," the emotion engine analyzes the user's tone of voice and assesses whether they are relaxed.

[1461] 3. Sending requests and sentiment information:

[1462] The device combines the user's request data and sentiment information obtained from the sentiment engine into a single JSON file, and sends this data to the server using an HTTP POST request. Network communication is used for this process.

[1463] 4. Request reception and parsing:

[1464] The server receives request data and emotional information sent from the terminal. The server analyzes the request content and extracts specific keywords such as "hair salon reservation." At the same time, the server analyzes the user's emotional state based on the received emotional information.

[1465] 5. Check available dates and times:

[1466] The server accesses an external salon booking API or an internal database to check for available dates and times. During this process, the server adjusts the number and content of the options displayed based on the user's emotional state. For example, if the user is very relaxed, it will present many options; if they are in a hurry, it will present only a limited number.

[1467] 6. Generating a response message:

[1468] The server generates a response message optimized for the user's emotional state based on the available reservation date and time. For example, it might generate a message like, "You can make a reservation on [Month] [Day] at [Time]. Please relax and enjoy yourself."

[1469] 7. Sending a response message:

[1470] The server sends the generated response message back to the terminal as an HTTP POST request, and the terminal receives it.

[1471] 8. Display of response message:

[1472] The terminal displays the response message received from the server in the user interface. The user reviews it and decides on the next action.

[1473] 9. Enter your confirmation request:

[1474] The user types "Yes, book it" to confirm the reservation. They can do this by typing into a text box or by giving a voice command. This indicates that they wish to confirm the reservation.

[1475] 10. Submit a confirmation request:

[1476] The device then sends the confirmation request received from the user back to the server as an HTTP POST request.

[1477] 11. Booking confirmation process:

[1478] The server receives the confirmation request and accesses the salon's reservation system API to confirm the reservation. If the reservation is successfully confirmed, the server generates a message saying "Reservation confirmed".

[1479] 12. Sending and displaying booking confirmation messages:

[1480] The server sends the generated reservation confirmation message to the terminal, and the terminal updates its user interface to display the reservation confirmation message.

[1481] Specific example

[1482] The following is a specific scenario for a user who wishes to make a reservation at a hair salon.

[1483] 1. User request input and sentiment recognition

[1484] User: "I want to make a reservation at a hair salon."

[1485] Emotion Engine: Analyzes the user's voice and determines that they are relaxed.

[1486] 2. Sending requests and sentiment information from the device.

[1487] The device sends the request and sentiment information to the server.

[1488] 3. Server analysis and reservation confirmation

[1489] Server: Analyzes "hair salon reservation" and "relaxed"

[1490] Server: Check the availability of the hair salon (e.g., Month XX, Day XX, Time XX)

[1491] 4. Server response generation and transmission

[1492] Server: Generates a response message saying, "Your reservation is available on [Month] [Day] at [Time]. Relax and enjoy!"

[1493] Server: Sends a response message to the terminal.

[1494] 5. Displaying messages on the device and verifying the user.

[1495] Terminal: Message display

[1496] User: "Yes, please make a reservation."

[1497] Terminal: Sends a confirmation request to the server.

[1498] 6. Confirm server reservation and send message

[1499] Server: Confirm reservation

[1500] Server: "Your reservation has been confirmed."

[1501] Server: Sends a reservation confirmation message to the terminal.

[1502] 7. Display of the device reservation confirmation message.

[1503] Terminal: Reservation confirmation message displayed

[1504] Example of a prompt

[1505] User input example: "I want to make a reservation at a hair salon."

[1506] Input to the response generation AI: "Process the following request and generate the most appropriate response message based on the user's emotional state. Request: 'Make a hair salon appointment', Emotional state: 'Relaxed'"

[1507] This system allows users to easily make reservations and enjoy an even more comfortable service experience by receiving support tailored to their emotional state.

[1508] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1509] Step 1:

[1510] User request input and sentiment recognition

[1511] The user enters "I want to make a reservation at a hair salon" into the device. Input can be done via text or voice. If the user uses voice input, the device's microphone captures the audio. Simultaneously, the device's camera captures the user's facial expressions. The emotion engine receives the audio and video data and analyzes the emotional state using natural language processing and image analysis. This generates request data and emotion information.

[1512] Input: User text or voice input, camera video.

[1513] Output: Request data (hair salon reservation), emotional information (e.g., relaxed)

[1514] Step 2:

[1515] Sending requests and sentiment information

[1516] The device combines the user's request data and sentiment information obtained from the sentiment engine into a single JSON file. This data is then sent to the server's specified endpoint using an HTTP POST request. Specifically, the data transmission is performed using network communication capabilities.

[1517] Input: Request data, sentiment information

[1518] Output: Data sent to the server as an HTTP request

[1519] Step 3:

[1520] Server request reception and analysis

[1521] The server receives request data and sentiment information sent from the terminal. The server parses the JSON data and extracts the request content. For example, it extracts and analyzes the keyword "hair salon reservation" and sentiment information. This allows the server to understand the user's desired service and emotional state.

[1522] Input: HTTP request data (request content, sentiment information)

[1523] Output: Analysis results (service details, hair salon reservation, emotional state)

[1524] Step 4:

[1525] Check available dates and times for booking

[1526] The server accesses the salon's reservation system API or internal database to check for available dates and times. This process narrows down the options according to specific rules based on the user's emotional state. For example, it presents more reservation options to a relaxed user and fewer options to a user in a hurry.

[1527] Input: Analyzed request (service details, sentiment information)

[1528] Output: List of available dates and times

[1529] Step 5:

[1530] Generating a response message

[1531] Based on the available dates and times retrieved by the server, a response message optimized for the user's emotional state is generated. Expressions are added according to the emotional state. For example, a message such as "You can make a reservation on [Month] [Day] at [Time]. Relax and enjoy!" is generated.

[1532] Input: List of available dates and times, sentiment information

[1533] Output: Optimized response message

[1534] Step 6:

[1535] Sending a response message

[1536] The server generates a response message, which is then compiled into JSON format and sent again to the specified endpoint on the device using an HTTP POST request.

[1537] Input: Response message

[1538] Output: Response message sent to the terminal as an HTTP request

[1539] Step 7:

[1540] Display of terminal response messages

[1541] The terminal analyzes the response message received from the server and updates the user interface. The message is displayed on the screen, prompting the user to confirm.

[1542] Input: Response message sent to the terminal

[1543] Output: Message displayed in the updated user interface

[1544] Step 8:

[1545] Entering a user verification request

[1546] If the user wants to confirm the reservation, they type "Yes, book it." They can type this into the text box or use a voice command. This will generate a confirmation request.

[1547] Input: User confirmation input (text or voice)

[1548] Output: Confirmation Request

[1549] Step 9:

[1550] Sending a confirmation request

[1551] The device compiles the confirmation request into JSON format and sends it again as an HTTP POST request to the specified endpoint on the server.

[1552] Input: Confirmation Request

[1553] Output: Confirmation request sent to the server as an HTTP request

[1554] Step 10:

[1555] Server reservation confirmation process

[1556] The server receives the confirmation request and accesses the salon's reservation system API to confirm the reservation. If the reservation is successfully confirmed as a result of this process, the server generates a message saying "Reservation confirmed".

[1557] Input: Confirmation Request

[1558] Output: Booking confirmation message

[1559] Step 11:

[1560] Sending and displaying reservation confirmation messages

[1561] The server generates a reservation confirmation message and sends it to the terminal. The terminal receives this message, updates the user interface, and displays the reservation confirmation message on the screen.

[1562] Input: Booking confirmation message

[1563] Output: Booking confirmation message displayed in the updated user interface

[1564] (Application Example 2)

[1565] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1566] Traditional reservation systems simply respond to user requests and lack the flexibility to adapt to user emotional states or request content. Therefore, to improve the user experience in service delivery, a system is needed that provides optimal responses and services based on the user's emotional state.

[1567] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1568] In this invention, the server includes means for analyzing the user's facial expressions and tone of voice to recognize their emotional state, means for optimizing requests based on the emotional state, and means for the server to generate a response message and send it to the terminal. This enables flexible and optimal responses and service provision based on the user's emotional state.

[1569] "User" refers to an individual or group that uses the system.

[1570] A "request" refers to an operation or request that a user enters.

[1571] A "terminal" is a device used by a user to input requests and to receive and display responses from a server.

[1572] A "server" is a component that receives and analyzes requests sent from a terminal and generates the optimal response.

[1573] "Analysis" refers to the process of breaking down and understanding request and emotional state data.

[1574] "Available dates and times" refers to the dates and times when the service can be provided, and is presented as an option in response to the user's request.

[1575] A "response message" refers to information generated by the server and sent to the user via the terminal.

[1576] "Facial expressions" refer to the expressions on a user's face and are a source of information for determining their emotions.

[1577] "Voice tone" refers to the pitch and volume of a user's voice and is a source of information for analyzing their emotions.

[1578] "Emotional state" refers to the user's psychological state and is obtained from analyzed facial expressions and tone of voice.

[1579] "Optimization" refers to the process of adjusting requests and responses based on emotional states.

[1580] A "confirmation request" refers to an action taken by a user to provide additional information or confirm a request that was initially submitted.

[1581] A "reservation confirmation message" refers to the final response message generated by the server to notify the user that the reservation was successful.

[1582] "System" refers to the entire set of components, including users, terminals, servers, sentiment analysis engines, and their interfaces.

[1583] This invention is a system that automatically processes user requests and provides optimal responses and services based on the user's emotional state. The system consists of a user, a terminal, a server, and an emotion analysis engine.

[1584] Hardware and software to be used

[1585] Device: Smartphone with built-in camera and microphone

[1586] Server: Cloud-based web server

[1587] Sentiment analysis engine: Microsoft Azure Face API, Google Cloud Vision API

[1588] Video streaming platform APIs: YouTube Data API, Netflix API

[1589] Data processing and computation

[1590] 1. Recognizing the user's emotional state

[1591] The device uses its built-in camera and microphone to capture the user's facial expressions and voice tone.

[1592] An emotion analysis engine (such as Microsoft Azure Face API or Google Cloud Vision API) is used to analyze the acquired data and recognize the user's emotional state.

[1593] 2. Sending emotional data

[1594] The analyzed emotional data (e.g., joy, sadness, stress) is sent to the server using an HTTP request.

[1595] 3. Emotion-based content recommendations

[1596] The server analyzes the received emotion data and uses video streaming platform APIs (e.g., YouTube Data API, Netflix API) to search for videos that are appropriate for the user's emotional state.

[1597] 4. Generating and sending video recommendations

[1598] The server selects the most suitable video recommendation and sends the recommendation result to the user's device using an HTTP request.

[1599] The terminal receives a response message from the server and displays it to the user.

[1600] Specific Scenario

[1601] 1. User emotion recognition

[1602] When a user opens the app, their facial expressions and voice are captured using the smartphone's camera and microphone.

[1603] The emotion analysis engine analyzes facial expressions and voice tone to determine that "the user is relaxed."

[1604] 2. Sending emotional data

[1605] Send emotional data (relaxed) to the server.

[1606] 3. Generating video recommendations

[1607] The server uses emotional data to search for videos suitable for relaxation (for example, videos with nature sounds or relaxing music videos).

[1608] 4. Sending and displaying video recommendations

[1609] The server sends a list of related videos to the user's smartphone with the comment, "These videos are perfect for relaxing."

[1610] Users can view a list of videos on their smartphones, select their favorite video, and play it.

[1611] Example of a prompt

[1612] "Please recommend videos that match the user's current emotional state. Based on the user's facial expressions and tone of voice, their emotional state is relaxed."

[1613] Thus, the present invention allows users to quickly obtain videos that are suitable for their emotional state, resulting in a personalized and excellent user experience.

[1614] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1615] Step 1:

[1616] The user opens the smartphone app. The device uses its built-in camera and microphone to capture the user's facial expressions and voice tone. This data is input into the emotion analysis engine. The emotion analysis engine uses the Microsoft Azure Face API or Google Cloud Vision API to output emotion data (e.g., relaxed, stressed).

[1617] Step 2:

[1618] The device sends analyzed sentiment data to the server. The server receives the HTTP request and verifies the user's sentiment state. Based on the input sentiment data, it requests suitable video recommendations from video streaming platform APIs (e.g., YouTube Data API, Netflix API).

[1619] Step 3:

[1620] The server retrieves a list of relevant videos from the video streaming platform API. The server filters the retrieved video information based on sentiment data to generate an optimal recommendation list. The generated list is sent to the terminal along with an optimized response message.

[1621] Step 4:

[1622] The device receives a response message from the server and displays it to the user. A list of videos is displayed, including comments tailored to the user's emotional state. For example, the video list might be displayed with the message, "These videos are perfect for relaxing."

[1623] Step 5:

[1624] The user selects a video from the displayed video list and starts playback. The device passes the information of the selected video to the video player and begins streaming the video.

[1625] This allows users to easily find and watch videos that match their emotional state.

[1626] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1627] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1628] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1629] [Fourth Embodiment]

[1630] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1631] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1632] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1633] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1634] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1635] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1636] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1637] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1638] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1639] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1640] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1641] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1642] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1643] This invention relates to a system for automating and efficiently processing reservations requested by users. This system consists of a user, a terminal, and a server, with each program fulfilling its respective role.

[1644] System Overview

[1645] User: Enter your reservation request and send it to the server via your device.

[1646] Terminal: Receives user input, sends requests to the server, receives responses from the server, and displays them to the user.

[1647] Server: Receives user requests, parses them, checks reservation status, and generates response messages.

[1648] Program processing

[1649] 1. Entering and submitting the user's request.

[1650] A user wants to make a reservation at a hair salon and enters "I want to make a reservation at a hair salon" into the terminal. The terminal sends this input to the server.

[1651] 2. Receiving and parsing server requests

[1652] The server receives a request from the terminal and analyzes its content, such as "hair salon reservation." During this process, it extracts appropriate keywords.

[1653] 3. Check available dates and times for reservations.

[1654] The server queries the beauty salon's API or database to check for available appointment times. For example, the server might get a result like "Appointment is available at XX / XX."

[1655] 4. Generating and sending response messages

[1656] The server generates a response message based on the available dates and times, such as "You can make a reservation at XX / XX. Do you want to make a reservation?", and sends it to the terminal.

[1657] 5. Display of the terminal's response message

[1658] The terminal displays the received response message to the user. The user confirms the message and responds, "Yes, book it."

[1659] 6. Sending a user verification request

[1660] The device resends the user's confirmation request to the server.

[1661] 7. Server reservation confirmation process

[1662] The server receives the confirmation request and accesses the salon's reservation system to confirm the reservation. If the reservation is confirmed, the server generates a message saying "Reservation confirmed".

[1663] 8. Sending and displaying the reservation confirmation message.

[1664] The server sends a reservation confirmation message to the terminal, which then displays it to the user.

[1665] Specific example

[1666] The following is a specific scenario for a user who wants to make a reservation at a hair salon.

[1667] 1. User request input

[1668] User: "I want to make a reservation at a hair salon."

[1669] Terminal: Sends a request to the server

[1670] 2. Server analysis and reservation confirmation

[1671] Server: Analyzes "hair salon reservations"

[1672] Server: Check the availability of the hair salon (at XX / XX)

[1673] 3. Server response generation and transmission

[1674] Server: "You can make a reservation at [time]. Would you like to make a reservation?"

[1675] Server: Sends a response message to the terminal.

[1676] 4. Displaying messages on the device and verifying the user.

[1677] Terminal: Message display

[1678] User: "Yes, please make a reservation."

[1679] Terminal: Sends a confirmation request to the server.

[1680] 5. Confirm server reservation and send message

[1681] Server: Confirm reservation

[1682] Server: "Your reservation has been confirmed."

[1683] Server: Sends a reservation confirmation message to the terminal.

[1684] 6. Display of the device reservation confirmation message

[1685] Terminal: Reservation confirmation message displayed

[1686] Thus, the present invention allows users to easily make reservations, simplifying and streamlining the process. Furthermore, reservations can be quickly confirmed and modified, greatly improving user convenience.

[1687] The following describes the processing flow.

[1688] Step 1:

[1689] The user enters a request using the terminal, such as "I want to make a reservation at a hair salon." This input is triggered by the terminal's program.

[1690] Step 2:

[1691] The device sends the user's request as a string to the server. It uses protocols such as HTTP requests to send data to the specified endpoint.

[1692] Step 3:

[1693] The server receives a request sent from the terminal. It analyzes the content of the request and extracts the specific keyword "hair salon reservation".

[1694] Step 4:

[1695] The server accesses the hair salon's reservation system (API or database) to search for available dates and times. For example, it might query the reservation system's API endpoint to retrieve availability data.

[1696] Step 5:

[1697] The server generates a response message based on the available reservation dates and times. For example, it generates a message in the format of "You can make a reservation on [Month] [Day] at [Time]. Would you like to make a reservation?".

[1698] Step 6:

[1699] The server sends the generated response message to the terminal. Then, it sends the response data again using an HTTP request or similar method.

[1700] Step 7:

[1701] The terminal receives a response message from the server and displays it to the user. The user interface (UI) is updated to display the received message on the screen.

[1702] Step 8:

[1703] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their intention to confirm the reservation.

[1704] Step 9:

[1705] The device sends a user verification request to the server. The verification request is sent as an HTTP request, just like the initial request.

[1706] Step 10:

[1707] The server receives the confirmation request. It then accesses the reservation system (API or database) again to formally confirm the reservation.

[1708] Step 11:

[1709] The server generates a reservation confirmation message. For example, it might create a message in the format "Your reservation has been confirmed."

[1710] Step 12:

[1711] The server sends a reservation confirmation message to the terminal. It sends data to notify the user of the reservation confirmation.

[1712] Step 13:

[1713] The terminal receives a reservation confirmation message from the server and displays it to the user. The user interface is then refreshed to notify the user that the reservation has been confirmed.

[1714] The above outlines the specific program processing flow when a user makes a reservation at a hair salon.

[1715] (Example 1)

[1716] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1717] In recent years, there has been a growing demand for efficient reservation systems. However, traditional systems have suffered from cumbersome reservation procedures and low user convenience. In particular, users often had to manually enter and verify information when requesting a reservation, and the process of confirming a reservation was often time-consuming. This resulted in a poor user experience and decreased operational efficiency.

[1718] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1719] In this invention, the server includes means for a user to input a reservation request, means for a terminal to send the reservation request to the server, means for the server to receive the reservation request and parse it using a natural language processing algorithm, means for the server to query an external reservation system to confirm available dates and times, means for the server to generate a response message based on the available dates and times and send it to the terminal, means for the terminal to display the response message to the user, means for the user to input a confirmation request in response to the response message, means for the terminal to resend the confirmation request to the server, means for the server to receive the confirmation request, access the external reservation system and confirm the reservation, means for the server to generate a reservation confirmation message and send it to the terminal, and means for the terminal to display the reservation confirmation message to the user. This allows the user to easily perform the process from request input to reservation confirmation, enabling simplification and efficiency of the procedure.

[1720] A "user" is an individual who uses the reservation system to enter a request and completes the process until the reservation is confirmed.

[1721] A "terminal" is a device used by a user to input requests and communicate with a server, and includes smartphones, tablets, and personal computers.

[1722] A "server" is a computer system that receives requests from users, analyzes them, checks available dates and times for reservations, and generates and sends response messages.

[1723] A "request" is information entered by a user that expresses their desired reservation details.

[1724] A "natural language processing algorithm" is a computational method for analyzing and understanding text entered by a user and extracting appropriate keywords.

[1725] A "reservation system" is a system that uses an external database or API to check reservation availability and process reservations.

[1726] A "response message" is a message generated by the server and sent to the terminal, which includes available dates and times for booking and a request for confirmation.

[1727] A "confirmation request" is a message that a user enters in response to a reply message to confirm their reservation.

[1728] A "reservation confirmation message" is a message generated by the server to notify the user that the reservation has been confirmed.

[1729] This invention relates to a system that automates and efficiently processes reservations requested by users. The system consists of three elements: the user, the terminal, and the server, each element playing its own role.

[1730] The user enters the details of their reservation request into the device. Specifically, they enter a request in text format, such as "I would like to make a reservation at a hair salon." Users can use devices such as smartphones, tablets, or personal computers.

[1731] The terminal receives requests entered by the user and sends them to the server. The terminal obtains user requests using an input interface (e.g., keyboard, touchscreen, voice input system) and sends them to the server using a communication module (e.g., Wi-Fi, mobile data communication). This ensures that the user's request reaches the server.

[1732] When the server analyzes a request received from a terminal, it uses a natural language processing algorithm (e.g., OpenAI GPT-4) to understand the request content and extract appropriate keywords. After analyzing the request, the server queries an external reservation system (e.g., a reservation management API or database) to check for available dates and times.

[1733] The server generates a response message based on the confirmed available dates and times and sends a confirmation message to the terminal asking, "Can I make a reservation?". The server's message generation logic is used to generate the message.

[1734] The terminal displays the received response message to the user. The user confirms the displayed message and enters a confirmation request, such as "Yes, book it," into the terminal. The terminal then uses the communication module to send the confirmation request to the server again.

[1735] Upon receiving the confirmation request, the server accesses the external reservation system again to formally confirm the reservation. Once the reservation is confirmed, it generates a notification message "Reservation Confirmed" and sends it to the terminal.

[1736] The device displays a reservation confirmation message to the user, and the reservation process is complete.

[1737] Specific example

[1738] The following is a specific scenario for a user who wants to make a reservation at a hair salon.

[1739] User request input

[1740] User: Types "I want to make a hair salon appointment" on their smartphone.

[1741] Terminal: Sends the entered string to the server as an HTTP request.

[1742] Server analysis and reservation confirmation

[1743] Server: Parses the JSON data received as an HTTP request ({"request": "I want to make a reservation at a hair salon"}).

[1744] Server: Uses OpenAI GPT-4 to extract keywords such as "hair salon" and "reservation".

[1745] Server: Query the beauty salon reservation system's API to check available dates and times (e.g., "October 1st, 2:00 PM").

[1746] Server response generation and transmission

[1747] Server: Based on the available dates and times, it generates the message, "You can make a reservation for October 1st at 2:00 PM. Would you like to make a reservation?"

[1748] Server: Sends the generated message to the terminal as an HTTP response.

[1749] Displaying messages on the device and verifying the user.

[1750] Device: The received message is displayed on the screen. A pop-up appears saying, "You can make a reservation for October 1st at 2:00 PM. Do you want to make a reservation?"

[1751] User: In response to the displayed message, type "Yes, book it" into the terminal.

[1752] Terminal: Sends the confirmation request back to the server as an HTTP request.

[1753] Server reservation confirmation and message sending

[1754] Server: Accesses the hair salon reservation system and processes the reservation confirmation.

[1755] Server: If the reservation confirmation is successful, it generates the message "Reservation confirmed".

[1756] Server: Sends a confirmation message to the terminal.

[1757] Display of the device reservation confirmation message

[1758] Terminal: Receives a confirmation message and displays to the user that "Your reservation has been confirmed."

[1759] Example of a prompt

[1760] The following is an example of a prompt message to input into a generative AI model.

[1761] Describe the steps a user takes from requesting and submitting a hair salon appointment to confirming the booking. Include all hardware and software used in detail.

[1762] This invention allows users to easily complete the process from request entry to reservation confirmation, simplifying and streamlining the procedure. Furthermore, reservations can be quickly confirmed and modified, greatly improving user convenience.

[1763] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1764] Step 1:

[1765] The user enters a reservation request and sends it to their device. Specifically, the user uses the device's input interface (e.g., keyboard, touchscreen) to enter "I want to make a reservation at a hair salon." The input is the user's request, and the output is sending that request to the server as an HTTP request. The user performs this operation on a device such as a smartphone or tablet.

[1766] Step 2:

[1767] The terminal receives user input and sends it to the server as an HTTP request. The terminal converts the input request into JSON format and sends it to the server. A communication module (e.g., Wi-Fi or mobile data communication) is used in this process. The input is the request content entered by the user, and the output is the HTTP request sent to the server.

[1768] Step 3:

[1769] The server parses the HTTP request received from the terminal. The server parses the request content into JSON and extracts it as text data. Then, it performs natural language processing using a generative AI model (e.g., OpenAI GPT-4) to extract keywords such as "hair salon reservation". The input is the JSON data of the HTTP request, and the output is the parsed keywords.

[1770] Step 4:

[1771] The server queries an external reservation system (e.g., a hair salon's reservation API) based on the analysis results to check available dates and times. The reservation system returns availability information via the API query, which the server receives. The input is the analyzed keywords, and the output is information about available dates and times. A web API is used for this query.

[1772] Step 5:

[1773] The server generates a response message based on the received information about available dates and times for booking. Specifically, it generates a message that includes the available date and time for booking (e.g., "You can book on October 1st at 2:00 PM. Do you want to book?"). The generated response message is then sent back to the terminal as an HTTP response. The input is the information about available dates and times for booking, and the output is the response message.

[1774] Step 6:

[1775] The terminal displays the response message received from the server to the user. The terminal parses the received message and displays it through the user interface. The input is the response message from the server, and the output is the message that the user sees.

[1776] Step 7:

[1777] The user enters a confirmation request in response to the displayed response message. They enter a confirmation request, such as "Yes, book it," into the terminal and send it. The input is the displayed response message, and the output is the confirmation request.

[1778] Step 8:

[1779] The terminal sends the user's confirmation request to the server. This request is then compiled again into JSON format data as an HTTP request and sent to the server. The input is the confirmation request, and the output is the HTTP request sent to the server.

[1780] Step 9:

[1781] The server receives the confirmation request and processes the reservation confirmation. The server accesses the external reservation system again and sends a confirmation request. If the reservation is successfully confirmed, the server receives that information. The input is the confirmation request, and the output is the reservation confirmation information.

[1782] Step 10:

[1783] The server generates a reservation confirmation message, "Your reservation has been confirmed," based on the reservation confirmation information, and sends it to the terminal. The generated message is sent to the terminal as an HTTP response. The input is the reservation confirmation information, and the output is the reservation confirmation message.

[1784] Step 11:

[1785] The terminal displays a reservation confirmation message to the user. The terminal parses the received message and displays it again through the user interface. The input is the reservation confirmation message, and the output is the confirmation message displayed to the user.

[1786] As a result, users can easily make reservations at hair salons, and the system efficiently automates the reservation process.

[1787] (Application Example 1)

[1788] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1789] The problem this invention aims to solve is to streamline the process by which users can easily reserve and confirm food delivery orders. Conventional systems require numerous manual steps for users to make reservations, which places a significant burden on them. Furthermore, the time required for reservation confirmation and finalization is inconvenient for food delivery services where immediacy is essential.

[1790] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1791] In this invention, the server includes means for querying the API or database of the provider to be booked and confirming available dates and times; means for parsing the user's request in natural language and generating a response using a generative AI model; and means for utilizing a database or API for booking confirmation and booking confirmation. This makes it possible for users to book food delivery quickly and accurately simply by entering a request, reducing the burden on the user and streamlining the entire process.

[1792] A "user" is a person or entity that wishes to make a reservation and enters a request.

[1793] A "terminal" is an electronic device used by a user to input and send requests to a server; it is a device that handles request input and displays response messages.

[1794] A "server" is a central processing unit that receives user requests and performs processing such as analysis, reservation confirmation, and reservation confirmation.

[1795] A "request" is a request that a user enters into their terminal with the details of their desired reservation and sends to the server.

[1796] A "reservation target provider" refers to the provider of the service or product that the user wishes to reserve.

[1797] "API" stands for Application Programming Interface, which is an interface that allows different systems and programs to communicate with each other.

[1798] A "database" is a collection of data that systematically stores information and allows it to be searched and manipulated.

[1799] "Available dates and times for booking" refers to the dates and times when the provider of the booking can provide the service.

[1800] A "response message" is a message generated by a server to respond to a user after analyzing the user's request.

[1801] "Natural language analysis" refers to a technology that allows a computer to mechanically understand the language entered by a user, meaning that the computer interprets the language that humans speak naturally.

[1802] A "generative AI model" is an artificial intelligence model generated based on machine learning, and it is an algorithm that generates the optimal response from input data.

[1803] This invention relates to a system for automating and efficiently processing food delivery reservations requested by users. This system consists of users, terminals, and a server, with each program fulfilling its respective role. Embodiments of this system are described in detail below.

[1804] System Configuration

[1805] 1. User:

[1806] The user is someone who wants to order food delivery and enters their request via a device such as a smartphone. For example, they might make a request like, "I want a pizza delivered at 7 PM," using voice input or text input.

[1807] 2. Terminal:

[1808] The terminal is a smartphone, tablet, or other mobile device that receives requests entered by the user and sends them to the server. The terminal also displays response messages and confirmation messages received from the server to the user.

[1809] 3. Server:

[1810] The server receives requests from users, parses them, checks reservation status, and generates response messages. The server queries the API or database of food delivery service providers to check available dates and times. It also parses user requests in natural language and generates responses using generative AI models.

[1811] Hardware and software to be used

[1812] Smartphone: Used as a user interface and a device for sending requests.

[1813] Server: Cloud-based or on-premises central processing unit. Programming languages ​​used are Python and Flask.

[1814] API: Used for communication with food delivery providers.

[1815] Generative AI models: Machine learning models used for natural language processing and response generation.

[1816] Data processing and data calculation

[1817] The server analyzes the user's natural language request and extracts keywords such as "pizza" and "7 PM." This analysis is performed using a generative AI model. The server then queries the API of a food delivery service provider to check available delivery times. Based on the results, the server generates a response message such as, "We can deliver a pizza at 7 PM. Would you like to order?" This message is sent to the device, which then displays it to the user.

[1818] Specific example of processing

[1819] For example, if a user enters a request saying, "I want a pizza delivered at 7 PM," the following exchange will take place:

[1820] 1. User: "I'd like the pizza delivered at 7 PM."

[1821] 2. Terminal: Sends the request to the server.

[1822] 3. Server: Analyzes "pizza" and "7 PM"

[1823] 4. Server: Query the API of the food delivery provider to check available reservation times.

[1824] 5. Server: Generates a response message, "We can deliver a pizza at 7 PM. Would you like to order?", and sends it to the terminal.

[1825] 6. Terminal: Display message

[1826] 7. User: "Yes, place the order."

[1827] 8. Terminal: Sends a confirmation request to the server.

[1828] 9. Server: Confirms the order, generates a message "Order Confirmed," and sends it to the terminal.

[1829] 10. Terminal: Display a reservation confirmation message to the user.

[1830] This allows users to easily book food delivery orders, streamlining the entire process.

[1831] Example of a prompt:

[1832] User: "I want the pizza delivered at 7 PM."

[1833] System: "We can deliver a pizza at 7 PM. Would you like to order?"

[1834] User: "Yes, place the order."

[1835] System: "Your order has been confirmed."

[1836] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1837] Step 1:

[1838] The user enters a request into a smartphone application saying, "I want a pizza delivered at 7 PM."

[1839] Input: User's natural language request ("I want a pizza delivered at 7 PM")

[1840] Output: Input text data

[1841] Step 2:

[1842] The terminal sends the entered request to the server.

[1843] Input: User input text data

[1844] Output: Text request sent to the server

[1845] Step 3:

[1846] The server analyzes the received request and uses a generative AI model to extract the necessary information for the reservation, such as "pizza" and "7 PM."

[1847] Input: Text request sent from the terminal

[1848] Data processing: Natural language analysis using generative AI models

[1849] Output: Analysis results regarding "pizza" and "7 PM"

[1850] Step 4:

[1851] Based on the analysis results, the server queries the API or database of food delivery service providers to check available dates and times for reservations.

[1852] Input: Analysis results for "pizza" and "7 PM"

[1853] Data processing: API queries and retrieval of available dates and times.

[1854] Output: Available dates and times for booking (e.g., "7 PM")

[1855] Step 5:

[1856] Based on the query results, the server generates a response message saying, "We can deliver a pizza at 7 PM. Would you like to order?" and sends it to the terminal.

[1857] Input: Available dates and times

[1858] Data processing: Generating response messages

[1859] Output: Response message (Example: "We can deliver a pizza at 7 PM. Would you like to order?")

[1860] Step 6:

[1861] The terminal displays the response message received from the server to the user.

[1862] Input: Response message from the server

[1863] Output: Message displayed to the user

[1864] Step 7:

[1865] The user enters "Yes, place order" into the terminal as a confirmation request.

[1866] Input: User confirmation request ("Yes, place order")

[1867] Output: Input text data

[1868] Step 8:

[1869] The device sends a user verification request to the server.

[1870] Input: User input text data

[1871] Output: Confirmation request to send to the server

[1872] Step 9:

[1873] The server receives the confirmation request and uses the food delivery service provider's API to confirm the reservation.

[1874] Input: Verification request sent from the device

[1875] Data processing: Execution of reservation confirmation process (API call)

[1876] Output: Booking Confirmation Status

[1877] Step 10:

[1878] The server generates a message indicating that the reservation has been confirmed and sends it to the terminal.

[1879] Input: Booking Confirmation Status

[1880] Data processing: Generating reservation confirmation messages

[1881] Output: Reservation confirmation message (e.g., "Your order has been confirmed")

[1882] Step 11:

[1883] The terminal displays the reservation confirmation message received from the server to the user.

[1884] Input: Reservation confirmation message from the server

[1885] Output: Message displayed to the user

[1886] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1887] This invention relates to a system that combines an emotion engine with a system that automatically processes user requests to provide optimal responses and services based on the user's emotional state. This system consists of a user, a terminal, a server, and an emotion engine, and operates as follows.

[1888] System Overview

[1889] User: Enter a reservation request and receive recommendations based on their emotional state.

[1890] Terminal: Receives user input, sends it to the server along with sentiment information, and displays the server's response to the user.

[1891] Server: Receives and analyzes user requests and sentiment information, checks reservation status, and generates response messages.

[1892] Emotion Engine: Analyzes the user's emotional state from their input, voice, facial expressions, etc., and provides the results to the server.

[1893] Program processing

[1894] 1. User request input and sentiment recognition

[1895] A user wants to make a reservation at a hair salon and enters "I want to make a reservation at a hair salon" into the terminal. Simultaneously, the emotion engine analyzes the user's facial expressions and tone of voice to determine their emotional state (e.g., joy, stress, anxiety, etc.).

[1896] 2. Sending requests and sentiment information from the device.

[1897] The device sends the user's request data and the emotion information obtained from the emotion engine to the server. The data is sent to the specified endpoint using HTTP requests or similar methods.

[1898] 3. Receiving and parsing server requests

[1899] The server receives requests and sentiment information sent from the terminal. It analyzes the request content and extracts specific keywords such as "hair salon reservation." Sentiment information is also analyzed simultaneously.

[1900] 4. Check available dates and times for booking.

[1901] The server accesses the hair salon's reservation system (API or database) to search for available dates and times. During this process, the number and content of the displayed options are adjusted based on the user's emotional state.

[1902] 5. Generating a response message

[1903] Based on the available dates and times retrieved by the server, a response message optimized for the user's emotional state is generated. For example, a message such as "You can make a reservation on [Month] [Day] at [Time]. Would you like to make a reservation?" is modified to include emotionally appropriate expressions and recommendations.

[1904] 6. Sending a response message

[1905] The server sends the generated response message to the terminal. Then, it sends the response data again using an HTTP request or similar method.

[1906] 7. Display of the terminal's response message

[1907] The terminal receives a response message from the server and displays it to the user. The user interface (UI) is updated to display the received message on the screen.

[1908] 8. Enter the user's confirmation request.

[1909] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their intention to confirm the reservation.

[1910] 9. Sending a confirmation request

[1911] The device sends a user verification request to the server. The verification request is sent as an HTTP request, just like the initial request.

[1912] 10. Server reservation confirmation process

[1913] The server receives the confirmation request and accesses the salon's reservation system (API or database) to confirm the reservation. If the reservation is confirmed, the server generates a message saying "Reservation confirmed".

[1914] 11. Sending and displaying the reservation confirmation message.

[1915] The server generates a reservation confirmation message and sends it to the terminal, which then displays it to the user.

[1916] Specific example

[1917] The following is a specific scenario for a user who wishes to make a reservation at a hair salon.

[1918] 1. User request input and sentiment recognition

[1919] User: "I want to make a reservation at a hair salon."

[1920] Emotion Engine: Analyzes the user's voice and determines that they are relaxed.

[1921] 2. Sending requests and sentiment information from the device.

[1922] The device sends the request and sentiment information to the server.

[1923] 3. Server analysis and reservation confirmation

[1924] Server: Analyzes "hair salon reservation" and "relaxed"

[1925] Server: Check the availability of the hair salon (at XX / XX)

[1926] 4. Server response generation and transmission

[1927] Server: Generates the response message: "You can make a reservation at XX / XX. Don't be nervous, relax and enjoy yourself!"

[1928] Server: Sends a response message to the terminal.

[1929] 5. Displaying messages on the device and verifying the user.

[1930] Terminal: Message display

[1931] User: "Yes, please make a reservation."

[1932] Terminal: Sends a confirmation request to the server.

[1933] 6. Confirm server reservation and send message

[1934] Server: Confirm reservation

[1935] Server: "Your reservation has been confirmed."

[1936] Server: Sends a reservation confirmation message to the terminal.

[1937] 7. Display of the device reservation confirmation message.

[1938] Terminal: Reservation confirmation message displayed

[1939] Thus, the present invention allows users to easily make reservations and enjoy an even more comfortable service experience by receiving support tailored to their emotional state.

[1940] The following describes the processing flow.

[1941] Step 1:

[1942] The user enters a request using the device, such as "I want to make a reservation at a hair salon." Simultaneously, the emotion engine built into the device analyzes the user's voice and facial expressions to identify their emotional state (e.g., relaxed, stressed, happy).

[1943] Step 2:

[1944] The device combines the user's request with the sentiment information obtained from the sentiment engine and sends it to the server as an HTTP request. The request includes the reservation request details and sentiment information.

[1945] Step 3:

[1946] The server receives the request sent from the terminal. Here, it analyzes the content of the request and sentiment information, and extracts the keyword "hair salon reservation" and its associated sentiment.

[1947] Step 4:

[1948] The server accesses the hair salon's reservation system (e.g., via an API or database) to search for available dates and times. During this process, it selects reservation options to present based on the user's sentiment information.

[1949] Step 5:

[1950] The server generates a response message based on the available reservation dates and times it has obtained. For example, if the user is relaxed, it will generate a response message such as, "You can make a reservation at XX o'clock. Please relax and come."

[1951] Step 6:

[1952] The server generates a response message and sends it to the terminal as an HTTP response. This message includes available dates and times for booking and additional messages depending on the sentiment.

[1953] Step 7:

[1954] The terminal receives a response message from the server and displays it to the user. The user interface is updated so that the user can see the message.

[1955] Step 8:

[1956] The user enters a confirmation request. For example, they might type "Yes, book it" to indicate their desire to confirm the reservation.

[1957] Step 9:

[1958] The device sends the user's confirmation request back to the server as an HTTP request. This request includes the intention to confirm the reservation and the original request information.

[1959] Step 10:

[1960] The server receives the confirmation request. Access the salon's reservation system again to formally confirm the reservation.

[1961] Step 11:

[1962] The server generates a reservation confirmation message. For example, it might create a message in the format, "Your reservation has been confirmed. We look forward to seeing you."

[1963] Step 12:

[1964] The server generates a reservation confirmation message and sends it to the terminal as an HTTP response. This notifies the user that their reservation has been confirmed.

[1965] Step 13:

[1966] The terminal receives a reservation confirmation message from the server and displays it to the user. The user interface is then refreshed to notify the user of the reservation confirmation.

[1967] The above describes the specific program processing flow of the present invention, which incorporates an emotion engine. This system allows users to enjoy a reservation experience that takes their own emotional state into consideration.

[1968] (Example 2)

[1969] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1970] Traditional reservation systems simply presented reservation options uniformly, without considering the user's emotional state. This sometimes led to user stress and decreased satisfaction. Furthermore, reservation confirmation messages were not tailored to the user's emotional state, resulting in an unoptimized user experience.

[1971] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1972] In this invention, the server includes means for recognizing the user's emotional state, means for selecting the optimal candidate based on the emotional state, and means for generating a response message with expressions corresponding to the emotion. This enables the provision of optimal services and the generation of response messages based on the user's emotional state.

[1973] A "user" is someone who uses a system to enter a request and receive a service.

[1974] A "terminal" is an electronic device used by a user to input requests and communicate with a server.

[1975] A "server" is a central device that receives and analyzes requests from users and executes the specified service (e.g., confirming a reservation).

[1976] A "request" is a request for a service that a user sends through their device.

[1977] "Emotional state" refers to the user's psychological state at that time, and includes feelings such as joy, relaxation, stress, and anxiety.

[1978] An "emotion engine" is a software or hardware mechanism that analyzes a user's emotional state based on their facial expressions, tone of voice, text input, and other factors.

[1979] "Analysis" is the process by which a server understands requests and sentiment information and determines the appropriate response.

[1980] "Available dates and times" refers to the dates and times when the service specified by the user (e.g., a hair salon reservation) is available.

[1981] A "response message" is a message sent from the server to the terminal that contains the results or information regarding the request.

[1982] A "confirmation request" is a request that a user submits to finalize a reservation or other service.

[1983] A "reservation confirmation message" is a message sent by the server to the user informing them that the server has processed the request and that the reservation has finally been confirmed.

[1984] A "database" is a system used to store reservation information and user data.

[1985] "API" stands for Application Programming Interface, and it is an interface for communicating with external systems and exchanging information.

[1986] This invention is a system that combines an emotion engine with a system that automatically processes user requests, thereby providing optimal responses and services based on the user's emotional state. The system consists of a user, a terminal, a server, and an emotion engine.

[1987] System Configuration

[1988] User: A person who uses a system to enter requests and receive services.

[1989] Terminal: An electronic device that receives user input, sends it to a server along with emotional information, and displays the server's response to the user. Examples include smartphones and personal computers.

[1990] Server: A central device that receives and analyzes user requests and sentiment information, checks reservation status, and generates response messages.

[1991] Emotion engine: A software or hardware mechanism that analyzes the user's emotional state from user input, voice, facial expressions, etc., and provides the results to a server.

[1992] Detailed explanation of the process

[1993] 1. User request input:

[1994] The user enters "I want to make a reservation at a hair salon" into their device (such as a smartphone or computer). Input can be done via text or voice, and if the user uses voice input, their voice is captured using a microphone.

[1995] 2. Emotion recognition:

[1996] The text and voice input by the user are sent to the emotion engine via the device. The emotion engine analyzes the user's facial expressions and tone of voice to determine their emotional state. For example, if a user says, "I want to make a reservation at the hair salon," the emotion engine analyzes the user's tone of voice and assesses whether they are relaxed.

[1997] 3. Sending requests and sentiment information:

[1998] The device combines the user's request data and sentiment information obtained from the sentiment engine into a single JSON file, and sends this data to the server using an HTTP POST request. Network communication is used for this process.

[1999] 4. Request reception and parsing:

[2000] The server receives request data and emotional information sent from the terminal. The server analyzes the request content and extracts specific keywords such as "hair salon reservation." At the same time, the server analyzes the user's emotional state based on the received emotional information.

[2001] 5. Check available dates and times:

[2002] The server accesses an external salon booking API or an internal database to check for available dates and times. During this process, the server adjusts the number and content of the options displayed based on the user's emotional state. For example, if the user is very relaxed, it will present many options; if they are in a hurry, it will present only a limited number.

[2003] 6. Generating a response message:

[2004] The server generates a response message optimized for the user's emotional state based on the available reservation date and time. For example, it might generate a message like, "You can make a reservation on [Month] [Day] at [Time]. Please relax and enjoy yourself."

[2005] 7. Sending a response message:

[2006] The server sends the generated response message back to the terminal as an HTTP POST request, and the terminal receives it.

[2007] 8. Display of response message:

[2008] The terminal displays the response message received from the server in the user interface. The user reviews it and decides on the next action.

[2009] 9. Enter your confirmation request:

[2010] The user types "Yes, book it" to confirm the reservation. They can do this by typing into a text box or by giving a voice command. This indicates that they wish to confirm the reservation.

[2011] 10. Submit a confirmation request:

[2012] The device then sends the confirmation request received from the user back to the server as an HTTP POST request.

[2013] 11. Booking confirmation process:

[2014] The server receives the confirmation request and accesses the salon's reservation system API to confirm the reservation. If the reservation is successfully confirmed, the server generates a message saying "Reservation confirmed".

[2015] 12. Sending and displaying booking confirmation messages:

[2016] The server sends the generated reservation confirmation message to the terminal, and the terminal updates its user interface to display the reservation confirmation message.

[2017] Specific example

[2018] The following is a specific scenario for a user who wishes to make a reservation at a hair salon.

[2019] 1. User request input and sentiment recognition

[2020] User: "I want to make a reservation at a hair salon."

[2021] Emotion Engine: Analyzes the user's voice and determines that they are relaxed.

[2022] 2. Sending requests and sentiment information from the device.

[2023] The device sends the request and sentiment information to the server.

[2024] 3. Server analysis and reservation confirmation

[2025] Server: Analyzes "hair salon reservation" and "relaxed"

[2026] Server: Check the availability of the hair salon (e.g., Month XX, Day XX, Time XX)

[2027] 4. Server response generation and transmission

[2028] Server: Generates a response message saying, "Your reservation is available on [Month] [Day] at [Time]. Relax and enjoy!"

[2029] Server: Sends a response message to the terminal.

[2030] 5. Displaying messages on the device and verifying the user.

[2031] Terminal: Message display

[2032] User: "Yes, please make a reservation."

[2033] Terminal: Sends a confirmation request to the server.

[2034] 6. Confirm server reservation and send message

[2035] Server: Confirm reservation

[2036] Server: "Your reservation has been confirmed."

[2037] Server: Sends a reservation confirmation message to the terminal.

[2038] 7. Display of the device reservation confirmation message.

[2039] Terminal: Reservation confirmation message displayed

[2040] Example of a prompt

[2041] User input example: "I want to make a reservation at a hair salon."

[2042] Input to the response generation AI: "Process the following request and generate the most appropriate response message based on the user's emotional state. Request: 'Make a hair salon appointment', Emotional state: 'Relaxed'"

[2043] This system allows users to easily make reservations and enjoy an even more comfortable service experience by receiving support tailored to their emotional state.

[2044] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2045] Step 1:

[2046] User request input and sentiment recognition

[2047] The user enters "I want to make a reservation at a hair salon" into the device. Input can be done via text or voice. If the user uses voice input, the device's microphone captures the audio. Simultaneously, the device's camera captures the user's facial expressions. The emotion engine receives the audio and video data and analyzes the emotional state using natural language processing and image analysis. This generates request data and emotion information.

[2048] Input: User text or voice input, camera video.

[2049] Output: Request data (hair salon reservation), emotional information (e.g., relaxed)

[2050] Step 2:

[2051] Sending requests and sentiment information

[2052] The device combines the user's request data and sentiment information obtained from the sentiment engine into a single JSON file. This data is then sent to the server's specified endpoint using an HTTP POST request. Specifically, the data transmission is performed using network communication capabilities.

[2053] Input: Request data, sentiment information

[2054] Output: Data sent to the server as an HTTP request

[2055] Step 3:

[2056] Server request reception and analysis

[2057] The server receives request data and sentiment information sent from the terminal. The server parses the JSON data and extracts the request content. For example, it extracts and analyzes the keyword "hair salon reservation" and sentiment information. This allows the server to understand the user's desired service and emotional state.

[2058] Input: HTTP request data (request content, sentiment information)

[2059] Output: Analysis results (service details, hair salon reservation, emotional state)

[2060] Step 4:

[2061] Check available dates and times for booking

[2062] The server accesses the salon's reservation system API or internal database to check for available dates and times. This process narrows down the options according to specific rules based on the user's emotional state. For example, it presents more reservation options to a relaxed user and fewer options to a user in a hurry.

[2063] Input: Analyzed request (service details, sentiment information)

[2064] Output: List of available dates and times

[2065] Step 5:

[2066] Generating a response message

[2067] Based on the available dates and times retrieved by the server, a response message optimized for the user's emotional state is generated. Expressions are added according to the emotional state. For example, a message such as "You can make a reservation on [Month] [Day] at [Time]. Relax and enjoy!" is generated.

[2068] Input: List of available dates and times, sentiment information

[2069] Output: Optimized response message

[2070] Step 6:

[2071] Sending a response message

[2072] The server generates a response message, which is then compiled into JSON format and sent again to the specified endpoint on the device using an HTTP POST request.

[2073] Input: Response message

[2074] Output: Response message sent to the terminal as an HTTP request

[2075] Step 7:

[2076] Display of terminal response messages

[2077] The terminal analyzes the response message received from the server and updates the user interface. The message is displayed on the screen, prompting the user to confirm.

[2078] Input: Response message sent to the terminal

[2079] Output: Message displayed in the updated user interface

[2080] Step 8:

[2081] Entering a user verification request

[2082] If the user wants to confirm the reservation, they type "Yes, book it." They can type this into the text box or use a voice command. This will generate a confirmation request.

[2083] Input: User confirmation input (text or voice)

[2084] Output: Confirmation Request

[2085] Step 9:

[2086] Sending a confirmation request

[2087] The device compiles the confirmation request into JSON format and sends it again as an HTTP POST request to the specified endpoint on the server.

[2088] Input: Confirmation Request

[2089] Output: Confirmation request sent to the server as an HTTP request

[2090] Step 10:

[2091] Server reservation confirmation process

[2092] The server receives the confirmation request and accesses the salon's reservation system API to confirm the reservation. If the reservation is successfully confirmed as a result of this process, the server generates a message saying "Reservation confirmed".

[2093] Input: Confirmation Request

[2094] Output: Booking confirmation message

[2095] Step 11:

[2096] Sending and displaying reservation confirmation messages

[2097] The server generates a reservation confirmation message and sends it to the terminal. The terminal receives this message, updates the user interface, and displays the reservation confirmation message on the screen.

[2098] Input: Booking confirmation message

[2099] Output: Booking confirmation message displayed in the updated user interface

[2100] (Application Example 2)

[2101] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2102] Traditional reservation systems simply respond to user requests and lack the flexibility to adapt to user emotional states or request content. Therefore, to improve the user experience in service delivery, a system is needed that provides optimal responses and services based on the user's emotional state.

[2103] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2104] In this invention, the server includes means for analyzing the user's facial expressions and tone of voice to recognize their emotional state, means for optimizing requests based on the emotional state, and means for the server to generate a response message and send it to the terminal. This enables flexible and optimal responses and service provision based on the user's emotional state.

[2105] "User" refers to an individual or group that uses the system.

[2106] A "request" refers to an operation or request that a user enters.

[2107] A "terminal" is a device used by a user to input requests and to receive and display responses from a server.

[2108] A "server" is a component that receives and analyzes requests sent from a terminal and generates the optimal response.

[2109] "Analysis" refers to the process of breaking down and understanding request and emotional state data.

[2110] "Available dates and times" refers to the dates and times when the service can be provided, and is presented as an option in response to the user's request.

[2111] A "response message" refers to information generated by the server and sent to the user via the terminal.

[2112] "Facial expressions" refer to the expressions on a user's face and are a source of information for determining their emotions.

[2113] "Voice tone" refers to the pitch and volume of a user's voice and is a source of information for analyzing their emotions.

[2114] "Emotional state" refers to the user's psychological state and is obtained from analyzed facial expressions and tone of voice.

[2115] "Optimization" refers to the process of adjusting requests and responses based on emotional states.

[2116] A "confirmation request" refers to an action taken by a user to provide additional information or confirm a request that was initially submitted.

[2117] A "reservation confirmation message" refers to the final response message generated by the server to notify the user that the reservation was successful.

[2118] "System" refers to the entire set of components, including users, terminals, servers, sentiment analysis engines, and their interfaces.

[2119] This invention is a system that automatically processes user requests and provides optimal responses and services based on the user's emotional state. The system consists of a user, a terminal, a server, and an emotion analysis engine.

[2120] Hardware and software to be used

[2121] Device: Smartphone with built-in camera and microphone

[2122] Server: Cloud-based web server

[2123] Sentiment analysis engine: Microsoft Azure Face API, Google Cloud Vision API

[2124] Video streaming platform APIs: YouTube Data API, Netflix API

[2125] Data processing and computation

[2126] 1. Recognizing the user's emotional state

[2127] The device uses its built-in camera and microphone to capture the user's facial expressions and voice tone.

[2128] An emotion analysis engine (such as Microsoft Azure Face API or Google Cloud Vision API) is used to analyze the acquired data and recognize the user's emotional state.

[2129] 2. Sending emotional data

[2130] The analyzed emotional data (e.g., joy, sadness, stress) is sent to the server using an HTTP request.

[2131] 3. Emotion-based content recommendations

[2132] The server analyzes the received emotion data and uses video streaming platform APIs (e.g., YouTube Data API, Netflix API) to search for videos that are appropriate for the user's emotional state.

[2133] 4. Generating and sending video recommendations

[2134] The server selects the most suitable video recommendation and sends the recommendation result to the user's device using an HTTP request.

[2135] The terminal receives a response message from the server and displays it to the user.

[2136] Specific Scenario

[2137] 1. User emotion recognition

[2138] When a user opens the app, their facial expressions and voice are captured using the smartphone's camera and microphone.

[2139] The emotion analysis engine analyzes facial expressions and voice tone to determine that "the user is relaxed."

[2140] 2. Sending emotional data

[2141] Send emotional data (relaxed) to the server.

[2142] 3. Generating video recommendations

[2143] The server uses emotional data to search for videos suitable for relaxation (for example, videos with nature sounds or relaxing music videos).

[2144] 4. Sending and displaying video recommendations

[2145] The server sends a list of related videos to the user's smartphone with the comment, "These videos are perfect for relaxing."

[2146] Users can view a list of videos on their smartphones, select their favorite video, and play it.

[2147] Example of a prompt

[2148] "Please recommend videos that match the user's current emotional state. Based on the user's facial expressions and tone of voice, their emotional state is relaxed."

[2149] Thus, the present invention allows users to quickly obtain videos that are suitable for their emotional state, resulting in a personalized and excellent user experience.

[2150] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2151] Step 1:

[2152] The user opens the smartphone app. The device uses its built-in camera and microphone to capture the user's facial expressions and voice tone. This data is input into the emotion analysis engine. The emotion analysis engine uses the Microsoft Azure Face API or Google Cloud Vision API to output emotion data (e.g., relaxed, stressed).

[2153] Step 2:

[2154] The device sends analyzed sentiment data to the server. The server receives the HTTP request and verifies the user's sentiment state. Based on the input sentiment data, it requests suitable video recommendations from video streaming platform APIs (e.g., YouTube Data API, Netflix API).

[2155] Step 3:

[2156] The server retrieves a list of relevant videos from the video streaming platform API. The server filters the retrieved video information based on sentiment data to generate an optimal recommendation list. The generated list is sent to the terminal along with an optimized response message.

[2157] Step 4:

[2158] The device receives a response message from the server and displays it to the user. A list of videos is displayed, including comments tailored to the user's emotional state. For example, the video list might be displayed with the message, "These videos are perfect for relaxing."

[2159] Step 5:

[2160] The user selects a video from the displayed video list and starts playback. The device passes the information of the selected video to the video player and begins streaming the video.

[2161] This allows users to easily find and watch videos that match their emotional state.

[2162] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2163] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2164] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2165] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2166] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2167] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2168] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2169] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2170] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2171] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2172] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2173] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2174] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2175] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2176] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2177] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2178] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2179] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[2180] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[2181] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[2182] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[2183] The following is further disclosed regarding the embodiments described above.

[2184] (Claim 1)

[2185] The means by which the user enters a request,

[2186] Means for the terminal to send the aforementioned request to the server,

[2187] A means for the server to receive and analyze the aforementioned request,

[2188] A way to check the available dates and times for the server to be reserved,

[2189] A means for the server to generate a response message and send it to the terminal,

[2190] The terminal provides means for displaying the response message to the user,

[2191] A means for the user to enter a confirmation request,

[2192] A means by which the terminal sends the aforementioned confirmation request to the server,

[2193] A means by which the server receives the aforementioned confirmation request and confirms the reservation,

[2194] A means by which the server generates a reservation confirmation message and sends it to the terminal,

[2195] The terminal provides means for displaying the reservation confirmation message to the user,

[2196] A system that includes this.

[2197] (Claim 2)

[2198] The system according to claim 1, further comprising means for the server to identify the subject of the reservation based on the analysis of the request.

[2199] (Claim 3)

[2200] The system according to claim 1, further comprising means by which the server uses a database or API for checking and confirming reservations.

[2201] "Example 1"

[2202] (Claim 1)

[2203] A means for users to enter reservation requests,

[2204] A means by which the terminal sends the reservation request to the server,

[2205] A server receives the reservation request and parses it using a natural language processing algorithm,

[2206] A means of querying an external reservation system to check the available dates and times for the server to make a reservation,

[2207] A means of generating a response message based on the available date and time on the server and sending it to the terminal,

[2208] The terminal provides means for displaying the response message to the user,

[2209] A means for the user to enter a confirmation request in response to the response message,

[2210] A means by which the terminal resends the aforementioned confirmation request to the server,

[2211] The server receives the aforementioned confirmation request and accesses an external reservation system to confirm the reservation;

[2212] A means by which the server generates a reservation confirmation message and sends it to the terminal,

[2213] The terminal provides means for displaying the reservation confirmation message to the user,

[2214] A system that includes this.

[2215] (Claim 2)

[2216] The system according to claim 1, further comprising means for the server to extract appropriate keywords using a natural language processing algorithm based on the analysis of the request and to identify the subject of the reservation.

[2217] (Claim 3)

[2218] The system according to claim 1, further comprising means by which the server uses a database or API to check and confirm reservations.

[2219] "Application Example 1"

[2220] (Claim 1)

[2221] The means by which the user enters a request,

[2222] Means for the terminal to send the aforementioned request to the server,

[2223] A means for the server to receive and analyze the aforementioned request,

[2224] The server has a means of querying the API or database of the provider to be booked to confirm available dates and times.

[2225] A means for the server to generate a response message and send it to the terminal,

[2226] The terminal provides means for displaying the response message to the user,

[2227] A means for the user to enter a confirmation request,

[2228] A means by which the terminal sends the aforementioned confirmation request to the server,

[2229] A means by which the server receives the aforementioned confirmation request and confirms the reservation,

[2230] A means by which the server generates a reservation confirmation message and sends it to the terminal,

[2231] The terminal provides means for displaying the reservation confirmation message to the user,

[2232] A means for analyzing user requests in natural language and generating responses using a generative AI model,

[2233] A system that includes this.

[2234] (Claim 2)

[2235] The system according to claim 1, further comprising means for the server to identify the subject of the reservation based on the analysis of the request.

[2236] (Claim 3)

[2237] The system according to claim 1, further comprising means by which the server uses a database or API for checking and confirming reservations.

[2238] "Example 2 of combining an emotion engine"

[2239] (Claim 1)

[2240] The means by which the user enters a request,

[2241] Means for the terminal to send the aforementioned request to the server,

[2242] A means for the server to receive and analyze the aforementioned request,

[2243] A way to check the available dates and times for the server to be reserved,

[2244] A means for the server to generate a response message and send it to the terminal,

[2245] The terminal provides means for displaying the response message to the user,

[2246] A means of recognizing the user's emotional state,

[2247] A method for selecting the optimal candidate based on emotional state,

[2248] A means by which the server generates response messages by adding emotionally appropriate expressions,

[2249] A means for the user to enter a confirmation request,

[2250] A means by which the terminal sends the aforementioned confirmation request to the server,

[2251] A means by which the server receives the aforementioned confirmation request and confirms the reservation,

[2252] A means by which the server generates a reservation confirmation message and sends it to the terminal,

[2253] The terminal provides means for displaying the reservation confirmation message to the user,

[2254] A system that includes this.

[2255] (Claim 2)

[2256] The system according to claim 1, further comprising means for the server to identify the subject of the reservation based on the analysis of the request.

[2257] (Claim 3)

[2258] The system according to claim 1, further comprising means by which the server uses a database or API for checking and confirming reservations.

[2259] "Application example 2 when combining with an emotional engine"

[2260] (Claim 1)

[2261] The means by which the user enters a request,

[2262] Means for the terminal to send the aforementioned request to the server,

[2263] A means for the server to receive and analyze the aforementioned request,

[2264] A way to check the available dates and times for the server to be reserved,

[2265] A means for the server to generate a response message and send it to the terminal,

[2266] The terminal provides means for displaying the response message to the user,

[2267] A means of recognizing the emotional state by analyzing the user's facial expressions and tone of voice,

[2268] The server provides means for optimizing requests based on the aforementioned emotional state,

[2269] A means for the user to enter a confirmation request,

[2270] A means by which the terminal sends the aforementioned confirmation request to the server,

[2271] A means by which the server receives the aforementioned confirmation request and confirms the reservation,

[2272] A means by which the server generates a reservation confirmation message and sends it to the terminal,

[2273] ...

Claims

1. The means by which the user enters a request, Means for the terminal to send the aforementioned request to the server, A means for the server to receive and analyze the aforementioned request, A way to check the available dates and times for the server to be reserved, A means for the server to generate a response message and send it to the terminal, The terminal provides means for displaying the response message to the user, A means for the user to enter a confirmation request, A means by which the terminal sends the aforementioned confirmation request to the server, A means by which the server receives the aforementioned confirmation request and confirms the reservation, A means by which the server generates a reservation confirmation message and sends it to the terminal, The terminal provides means for displaying the reservation confirmation message to the user, A system that includes this.

2. The system according to claim 1, further comprising means for the server to identify the target of the reservation based on the analysis of the request.

3. The system according to claim 1, further comprising means by which the server uses a database or API for checking and confirming reservations.

Images