system

Abstract

We provide the system. [Solution] A means of receiving requests from users, A means for analyzing the aforementioned request using natural language processing and extracting relevant keywords and intent, A means of collecting information from multiple external sources, analyzing the acquired information, and making suggestions to the user, A means of accepting user selections based on the above proposal, A means of making reservations and arranging necessary services based on the aforementioned selection, A system including means for notifying the user of the completion of the aforementioned reservation and arrangement.

Description

Technical Field

[0004] , , , ,

[0005] , , , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Modern consumers attempt to satisfy their desires by using many information sources and services, but it requires a great deal of time and effort to use each platform and service individually. For example, to decide on a place to eat, it is necessary to investigate restaurant information, check reviews, make a reservation, and then arrange transportation means. This series of processes is cumbersome and impairs the convenience of consumers. Therefore, an integrated system that can seamlessly execute a series of tasks based on user requests is required.

Means for Solving the Problems

[0005] This invention provides a means for receiving user requests and analyzing them using natural language processing. It then collects relevant information from multiple external sources (restaurant information, reviews, reservation services, etc.), analyzes that information, and makes suggestions to the user. Next, it accepts the user's selection and, based on that selection, makes a reservation and provides necessary services (e.g., transportation arrangements). Once this series of processes is complete, the system notifies the user of the completion of the reservation and arrangements. This allows users to seamlessly perform information gathering, reservations, and arrangements on a single platform, significantly improving consumer convenience.

[0006] A "request" is a request made by a user to a system for information gathering or service provision.

[0007] "Natural language processing" is a technology that enables computers to understand and analyze human language.

[0008] "Keywords" are specific words or phrases extracted from user requests and are used to search for related information.

[0009] "Intention" refers to the purpose or request that the user hopes to achieve through the request.

[0010] "External information sources" refer to other services or databases that a system accesses to collect information.

[0011] "Analysis" is the process of organizing collected information and evaluating and classifying it in a way that meets the user's needs.

[0012] A "suggestion" refers to a system displaying available information or options based on a user's request.

[0013] "Selection" is the act of a user choosing a specific item from the information or options suggested by the system.

[0014] "Reservation" refers to the procedure for securing proposed services or products in advance.

[0015] "Arrangement" refers to the process of preparing and providing necessary services or products based on the user's selection.

[0016] "Notification" refers to the action of the system communicating information or completion status to the user.

Brief Description of Drawings

[0017] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] Modes for carrying out the invention

[0039] This invention is a system that handles everything from information gathering and proposals to reservations and necessary arrangements, primarily based on user requests. The program of this system mainly consists of the following components, and its detailed processing is described below.

[0040] System components

[0041] 1. User terminal: Responsible for inputting requests and displaying results.

[0042] 2. Server: Responsible for parsing requests, collecting and analyzing information, generating suggestions, arranging reservations, and notifying results.

[0043] 3. External APIs: Provide restaurant information, reviews, reservation services, and transportation information.

[0044] Explanation of program processing

[0045] 1. Receiving user requests

[0046] The user enters a request into their device and sends it. For example, they might enter, "I want to eat delicious udon in Tokyo."

[0047] The terminal receives the request and sends it to the server. The server logs the received request.

[0048] 2. Intent Analysis using Natural Language Processing (NLP)

[0049] The server passes the received request to the natural language processing engine for analysis.

[0050] The NLP engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the request and analyzes the user's intent.

[0051] The server retrieves the analysis results and prepares the data for subsequent processing.

[0052] 3. Information Gathering and Analysis

[0053] The server collects relevant information through external APIs. Specifically, it retrieves data from sources such as restaurant information, customer reviews, and reservation services.

[0054] Example: The server collects information on udon restaurants in Tokyo from a restaurant API and obtains ratings for each restaurant from a review API. It also obtains availability and waiting times for each restaurant from a reservation API.

[0055] The server organizes the collected information and creates a list to present to the user.

[0056] 4. Suggestions for users

[0057] The server generates multiple suggestions for the user based on the organized information.

[0058] For example, the server might suggest options like "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" or "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[0059] The device receives the suggestion and displays it to the user.

[0060] 5. User Selection and Optimization

[0061] The user makes a selection from the presented suggestions. For example, "Select store A."

[0062] The device sends the user's selection to the server.

[0063] The server receives the selection and begins the reservation and necessary arrangements. Specifically, it makes a reservation at the selected store and, if necessary, arranges transportation (such as a taxi).

[0064] 6. Confirmation and notification of reservations and arrangements

[0065] The server confirms the completion of the reservation and arrangements and organizes the information.

[0066] Example: The server creates a notification such as "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and sends it to the device.

[0067] The device receives a completion notification and provides the user with final confirmation information (e.g., push notification or SMS).

[0068] Specific example

[0069] If a user requests "I want to eat delicious udon in Tokyo," the server will process it as follows:

[0070] The server analyzes the request and extracts keywords such as "Tokyo area," "udon," and "delicious."

[0071] The server collects information on target stores from external APIs and analyzes customer reviews and seating availability.

[0072] The server suggests multiple store options to the user.

[0073] The user selects store A from the suggested stores.

[0074] The server makes a reservation for store A and arranges a taxi if necessary.

[0075] The server notifies the terminal of the reservation completion and arrangement details, and the terminal informs the user of this.

[0076] In this way, users can seamlessly complete everything from information gathering to booking and arrangements through a single platform.

[0077] The following describes the processing flow.

[0078] Step 1:

[0079] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo." The device receives this request and sends it to the server.

[0080] Step 2:

[0081] The server sends the received request to the natural language processing engine. The natural language processing engine analyzes the request and extracts keywords and the user's intent (e.g., "in Tokyo", "udon", "delicious"). The server retrieves the analysis results and prepares the data for use in the next processing step.

[0082] Step 3:

[0083] The server uses multiple external APIs to collect relevant information. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. Furthermore, it obtains the availability and waiting times for each restaurant from a reservation API.

[0084] Step 4:

[0085] The server performs analysis based on the collected data and generates a list of suggestions to present to the user. For example, it creates a list that includes information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes".

[0086] Step 5:

[0087] The server sends the generated suggestion list to the terminal in JSON format. The terminal then displays the received suggestion list in an easy-to-understand format for the user.

[0088] Step 6:

[0089] The user selects their preferred store from the displayed suggestions. For example, "Select Store A." The terminal receives the user's selection and sends it to the server.

[0090] Step 7:

[0091] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. The server uses the reservation API to make reservations for the selected stores and, if necessary, arranges transportation (e.g., taxis) via the taxi API.

[0092] Step 8:

[0093] The server confirms that the reservation and arrangements are complete and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes." The server then sends this notification message to the terminal.

[0094] Step 9:

[0095] The device receives notifications sent from the server and provides the user with final confirmation information (e.g., push notifications or SMS). This allows the user to confirm that the reservation and arrangements are complete.

[0096] This series of processes allows users to seamlessly navigate the entire process, from making a request and gathering information to selecting suggestions, booking and arranging, and finally confirming their booking.

[0097] (Example 1)

[0098] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] Traditional systems require users to use multiple platforms when making requests, resulting in a lack of consistency from information gathering to booking. Furthermore, the lack of coordination between platforms leads to significant time and effort. Additionally, the system lacks the ability to accurately understand user intent and provide appropriate suggestions, potentially resulting in lower user satisfaction.

[0100] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0101] In this invention, the server includes means for receiving requests from users, means for analyzing the requests using natural language processing and extracting relevant keywords and intentions, means for collecting information from multiple external sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary transportation based on the selections, means for notifying the user of the completion of the reservations and arrangements, and means for logging the requests, extracted keywords, and intentions. This enables users to consistently and seamlessly perform tasks from information gathering to reservation arrangements.

[0102] "Means for receiving user requests" refers to the means by which a user enters requests or demands into the system and initiates processing within the system.

[0103] A "natural language processing tool" is a means of analyzing a user's input request and extracting relevant keywords and intent from the text.

[0104] "Information gathering means" refers to the means of collecting relevant information from multiple external sources and converting it into a format usable within the system.

[0105] A "proposal generation method" is a means of generating appropriate suggestions for the user based on collected and analyzed information, and presenting the results to the user.

[0106] A "selection acceptance mechanism" is a means for a user to make a selection from the presented suggestions, accept that selection, and reflect it in the next process.

[0107] A "reservation method" refers to a means of reserving a desired store or service based on the user's choice.

[0108] "Means of arranging transportation" refers to the means of arranging the necessary means of travel in conjunction with a reservation.

[0109] "Notification means" refers to the means by which the system transmits information to inform the user that the reservation and arrangements have been completed.

[0110] A "logging method" is a means of recording user requests, extracted keywords, intentions, etc., within the system for later analysis and auditing.

[0111] This invention provides a system that allows users to seamlessly access a series of services. This system consistently handles information gathering, analysis, suggestions, reservations, and necessary arrangements, all centered around the user's request. Detailed embodiments of this system are described below.

[0112] Hardware and software to be used

[0113] 1. User terminal:

[0114] User devices can include smartphones, tablets, and personal computers. These devices receive requests from users and send them to the server. They also display suggestions and notifications to the user.

[0115] 2. Server:

[0116] The server oversees a series of processes, from analyzing user requests and gathering information to generating suggestions, arranging reservations, and sending notifications. Specifically, it uses the following software:

[0117] Natural Language Processing Engine (NLP): Utilizes Google's Cloud Natural Language API and other tools to understand user requests and extract relevant keywords and intents.

[0118] External APIs: Restaurant information, ratings, availability, reservation status, and transportation information are obtained via APIs such as the Yelp API and Google Places API.

[0119] Process Overview

[0120] 1. Receiving user requests:

[0121] The user enters a request through their device. For example, they might request, "I want to eat delicious udon in Tokyo." This request is sent from the device to the server.

[0122] 2. Intent analysis using natural language processing:

[0123] After receiving a request, the server passes the utterance to a natural language processing engine. The engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the request and analyzes the user's intent. The server retrieves the analysis results and prepares queries for the next data collection.

[0124] 3. Information gathering and analysis:

[0125] The server sends queries to external APIs to collect and analyze necessary information. Specifically, it uses the Yelp API to obtain a list of udon restaurants in Tokyo and the Google Places API to obtain information on the availability of seats and customer reviews for each restaurant.

[0126] Based on the information obtained, a list of suggestions for the user is generated.

[0127] 4. Suggestions for users:

[0128] The server generates several suggestions based on the collected information and sends them to the terminal. An example suggestion might be, "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes." The terminal then displays this to the user.

[0129] 5. User Selection and Optimization:

[0130] When a user makes a selection from a list of suggestions, for example, by choosing "Store A," the terminal sends the selection to the server. The server receives the selection and proceeds with the reservation process.

[0131] 6. Confirmation and notification of reservations and arrangements:

[0132] Once the server completes the reservation and necessary arrangements (e.g., taxi booking), it compiles the reservation confirmation information and sends it to the device. The device then notifies the user of the reservation completion details (e.g., via push notification or SMS).

[0133] Specific example

[0134] The specific processing flow when a user enters a request such as "I want to eat delicious udon in Tokyo" is as follows:

[0135] The server analyzes the request using an NLP engine and extracts keywords such as "Tokyo," "udon," and "delicious."

[0136] The server collects and analyzes information about the target stores from the Yelp API and Google Places API.

[0137] Based on the information collected by the server, it suggests potential stores to the user, such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes."

[0138] The user selects "Store A," the server makes a reservation for that store, and arranges a taxi if necessary.

[0139] The server notifies the user's terminal with reservation confirmation and taxi arrangement details.

[0140] Example of a prompt

[0141] Examples of prompts that users might enter include the following:

[0142] "I want to eat delicious udon in Tokyo."

[0143] "Can you recommend a good cafe for the weekend?"

[0144] "I'm looking for a yakiniku restaurant that's open tonight."

[0145] In this way, this system can efficiently and seamlessly handle everything from information gathering to arrangement in response to user requests.

[0146] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0147] Program processing steps

[0148] Step 1: Receiving User Requests

[0149] Specific actions:

[0150] The user enters the prompt "I want to eat delicious udon in Tokyo" into the terminal and sends it. The terminal sends this input to the server as an HTTP request. The server receives the HTTP request and logs its contents. The log includes the request content, a timestamp, and the user's IP address.

[0151] input:

[0152] User prompt (e.g., "I want to eat delicious udon in Tokyo")

[0153] output:

[0154] The prompt message received by the server, and the request content recorded in the log.

[0155] Step 2: Intent Analysis using Natural Language Processing

[0156] Specific actions:

[0157] The server sends the received prompt text to a natural language processing engine (e.g., Google Cloud Natural Language API). The NLP engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the prompt text. The server retrieves the analysis results from the NLP engine and prepares queries containing these keywords for subsequent data collection.

[0158] input:

[0159] Received prompt message

[0160] output:

[0161] Extracted keywords (e.g., "Tokyo area", "udon", "delicious")

[0162] Step 3: Information Gathering and Analysis

[0163] Specific actions:

[0164] The server sends queries to external information source APIs using the extracted keywords. For example, it might retrieve a list of udon restaurants in Tokyo and their ratings via the Yelp API, and obtain the availability and waiting times for each restaurant via the Google Places API. The server receives this information, organizes and analyzes it, and generates a list to present to the user.

[0165] input:

[0166] Extracted keywords

[0167] output:

[0168] Store information and analyzed data (e.g., store name, customer reviews, seating availability, waiting time)

[0169] Step 4: Proposal to the user

[0170] Specific actions:

[0171] The server generates multiple suggestions based on the collected and analyzed store information. The generated suggestions are sent to the terminal in JSON format. The terminal parses the received JSON data and displays the suggestions in a user-friendly format. For example, it might display information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes."

[0172] input:

[0173] Organized and analyzed store information

[0174] output:

[0175] A list of suggestions for users (Example: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes")

[0176] Step 5: User Selection and Optimization

[0177] Specific actions:

[0178] The user makes a selection from a list of suggestions, for example, by selecting "Store A." The terminal sends the user's selection to the server. The server receives the selection and begins arranging a reservation for the selected store. Specifically, it sends a request to the store's reservation API and makes the reservation.

[0179] input:

[0180] User selection (e.g., "Store A")

[0181] output:

[0182] Submitting a booking request and booking confirmation information

[0183] Step 6: Confirmation and notification of booking / arrangement

[0184] Specific actions:

[0185] The server confirms the completion of the reservation and arrangement and organizes the information. The server sends the organized information to the terminal in JSON format. The terminal parses the received JSON data and notifies the user of the reservation and arrangement completion (e.g., push notification or SMS). The reservation confirmation information includes the reservation number, estimated taxi arrival time, etc.

[0186] input:

[0187] Reservation confirmation information

[0188] output:

[0189] Notifications and detailed information to the user (e.g., "Reservation complete. Reservation number 12345. Taxi is expected to arrive in 15 minutes.")

[0190] In this way, the system can efficiently and seamlessly handle everything from information gathering to arrangement in response to user requests.

[0191] (Application Example 1)

[0192] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0193] In recent years, there has been a growing demand for systems that allow users to easily order and reserve meals. However, current food delivery platforms require the use of multiple applications and services, making the process cumbersome. Therefore, there is a need to develop a system that allows users to handle everything from requests and orders to delivery arrangements and status notifications on a single platform.

[0194] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0195] In this invention, the server includes means for receiving requests from users, means for analyzing requests using natural language processing and extracting relevant keywords and intents, means for collecting information from multiple external sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations, orders, and arrangements for necessary services based on the selections, means for notifying the user of the completion of the reservations and arrangements, and means for notifying the user of delivery status and order status in real time. This enables users to seamlessly perform everything from making requests to ordering, making arrangements, and checking the status on a single platform.

[0196] "Means for receiving requests from users" refers to the part of a system that sends requests entered by users via devices such as smartphones and computers to a server and receives those requests.

[0197] "Natural language processing" is a technology that analyzes text data entered by users to extract keywords and intent.

[0198] "External information sources" refer to data sources provided from outside the system, such as restaurant information, customer reviews, reservation services, and delivery services.

[0199] "Information gathering means" refers to the part that has the function of acquiring and analyzing data from external information sources.

[0200] The "means of suggestion" refers to the function that provides users with multiple options based on the information collected and analyzed.

[0201] "Means of making reservations and arrangements" refers to the part of the system that has the functionality to make restaurant reservations and arrange delivery based on the options selected by the user.

[0202] "Notification means" refers to the function that informs users in real time about the completion of reservations and arrangements, as well as the delivery status.

[0203] "Restaurant information" refers to information about food and beverage establishments such as restaurants and cafes, and specifically includes the name of the establishment, address, business hours, menu, prices, and ratings.

[0204] "Word-of-mouth information" refers to feedback such as evaluations and impressions provided by users and customers.

[0205] "Ordering service" refers to the function or service that allows customers to order food from restaurants online.

[0206] "Delivery method" refers to the arrangements and means of transportation used to deliver the ordered meal to the location specified by the user.

[0207] This invention is a system that automatically suggests, reserves, orders, and arranges delivery of the most suitable restaurant based on a user's meal request. This system consists of a user terminal, a server, and multiple external information sources.

[0208] 1. System Program Overview

[0209] The server has the following functions:

[0210] Natural language processing engine (spaCy library)

[0211] Data collection from external sources (Google Places API, Yelp API, etc.)

[0212] Data analysis and proposal generation.

[0213] Reservation and order arrangement

[0214] Real-time notifications (Firebase Cloud Messaging)

[0215] User devices (primarily smartphones) have the following functions:

[0216] User Request Input Form

[0217] Display the proposal list

[0218] Receiving real-time notifications

[0219] 2. Program Processing

[0220] The server receives requests sent by users and analyzes them using a natural language processing engine. This analysis extracts keywords (e.g., "spicy food") from the request and understands its intent.

[0221] Based on analyzed keywords, the system collects restaurant information, reviews, and reservation data from external sources (such as the Google Places API and Yelp API). Based on this collected information, data analysis is performed to generate a list of restaurants best suited to the user.

[0222] When a user selects a restaurant from the suggested list, the server arranges the reservation and order with the selected restaurant. Furthermore, if delivery is required, it also arranges the delivery. Once these arrangements are complete, the server uses Firebase Cloud Messaging to notify the user in real time of the delivery and order status.

[0223] 3. Specific examples

[0224] When a user enters "I want to eat spicy Indian food" into a smartphone app, the server extracts the keywords "spicy" and "Indian food," and gathers and suggests the most suitable restaurant information. If the user selects "Indian Restaurant A" from the suggestions, the order is confirmed and delivery is arranged. This allows the user to track the status of their order in real time until delivery is complete.

[0225] Example of a prompt

[0226] "I'd like to eat something spicy nearby."

[0227] "I'd like to order Indian food for delivery."

[0228] "What Asian dishes are popular today?"

[0229] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0230] Step 1:

[0231] The process by which the user enters a request.

[0232] Input: The user enters a request into the smartphone app saying, "I want to eat spicy Indian food."

[0233] Specific action: Enter text into the input form on the user's terminal and press the submit button.

[0234] Output: The user terminal sends this request data to the server.

[0235] Step 2:

[0236] A process that receives a request and performs natural language processing.

[0237] Input: Request data sent from the user's terminal.

[0238] Specific operation: The server receives the request and passes the request text to the natural language processing engine (spaCy).

[0239] Output: Keywords extracted by the natural language processing engine (e.g., "spicy", "Indian food") and intent are returned.

[0240] Step 3:

[0241] Information gathering process

[0242] Input: Keywords extracted by the natural language processing engine.

[0243] Specific operation: The server calls external information sources (such as the Google Places API and Yelp API) to collect restaurant information, reviews, and reservation status related to keywords.

[0244] Output: Collected restaurant information, customer reviews, and reservation status data.

[0245] Step 4:

[0246] Process to generate a list of suggestions

[0247] Input: Collected restaurant information, customer reviews, and reservation status data.

[0248] Specific operation: The server analyzes the collected data, selects the most suitable restaurants for the user, and generates a list of suggestions.

[0249] Output: Suggestion data listing multiple restaurant options.

[0250] Step 5:

[0251] Process to display a list of suggestions to the user.

[0252] Input: Proposal list data.

[0253] Specific operation: The server sends the suggestion list data to the user's terminal, and the terminal displays the list.

[0254] Output: A list of suggested restaurants displayed on the user's terminal.

[0255] Step 6:

[0256] The process of a user selecting a restaurant from suggestions.

[0257] Input: The displayed list of suggestions.

[0258] Specific action: The user selects a restaurant from the suggested list and presses the select button.

[0259] Output: The user terminal sends information about the selected restaurant to the server.

[0260] Step 7:

[0261] Processing to arrange reservations and orders.

[0262] Input: Information about the restaurant selected by the user.

[0263] Specific operation: The server makes reservations and orders for selected restaurants through an online reservation and ordering system (external API).

[0264] Output: Reservation confirmation and order processing completion data.

[0265] Step 8:

[0266] Process to notify the user of the completion of the reservation and order.

[0267] Input: Reservation confirmation and order processing completion data.

[0268] Specific operation: The server sends a notification to the user in real time via Firebase Cloud Messaging (FCM). The device receives the notification and displays it to the user.

[0269] Output: Reservation and order completion notifications displayed on the user's terminal.

[0270] Step 9:

[0271] A process to notify delivery status in real time.

[0272] Input: Delivery arrangement progress information.

[0273] Specific operation: The server updates the delivery status in real time based on progress information obtained from the delivery system and notifies the user using Firebase Cloud Messaging.

[0274] Output: Delivery status notification displayed in real time on the user's device.

[0275] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0276] Modes for carrying out the invention

[0277] This invention is a system that receives user requests, recognizes and analyzes the user's emotional state using an emotion engine, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that analysis. The program of this system consists of the following components, and its detailed processing is described below.

[0278] System components

[0279] 1. User terminal: Responsible for inputting requests and displaying results.

[0280] 2. Server: Responsible for request analysis, sentiment recognition, information gathering and analysis, suggestion generation, reservation arrangement, and result notification.

[0281] 3. External APIs: Provide restaurant information, reviews, reservation services, and transportation information.

[0282] 4. Emotion Engine: Extracts emotions from user requests and optimizes suggestions.

[0283] Explanation of program processing

[0284] 1. Receiving User Requests

[0285] The user inputs and sends a request using the terminal. For example, the user inputs "I want to eat delicious udon in Tokyo".

[0286] The terminal receives this request and sends it to the server. The server records the received request in the log.

[0287] 2. Intent Analysis and Sentiment Recognition by Natural Language Processing (NLP)

[0288] The server sends the received request to the natural language processing engine for analysis.

[0289] The natural language processing engine extracts keywords and intent such as "Tokyo", "udon", and "delicious" from the request.

[0290] At the same time, the sentiment engine recognizes the user's sentiment state (e.g., joy, expectation, fatigue) from the words and context included in the request.

[0291] The server obtains these analysis results and sentiment data, and prepares data for use in the next process.

[0292] 3. Information Collection and Analysis

[0293] The server collects relevant information using multiple external APIs. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from the restaurant API, collects evaluations and review counts of each store from the review API, and obtains the vacancy status and waiting time of each store from the reservation API.

[0294] The server conducts analysis based on the collected data and generates a list of proposals for presentation to the user. Here, the proposals are optimized based on the analysis results of the sentiment engine. For example, when the user is feeling tired, stores with short waiting times are prioritized for proposal.

[0295] 4. Suggestions for users

[0296] The server sends the generated suggestion list to the terminal in JSON format. The terminal then displays the received suggestion list in an easy-to-understand format for the user.

[0297] For example, the server might suggest "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes." The system also optimizes the suggestions based on the user's emotional state.

[0298] 5. User Selection and Optimization

[0299] The user selects their preferred store from the displayed suggestions. For example, "Select Store A."

[0300] The terminal receives the user's selection and sends it to the server.

[0301] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. Specifically, it makes reservations at the selected establishments and, if necessary, arranges transportation (such as taxis).

[0302] 6. Confirmation and notification of reservations and arrangements

[0303] The server confirms the completion of the reservation and arrangement and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is scheduled to arrive in 15 minutes."

[0304] The server sends this notification to the device. The device receives the completion notification and provides the user with final confirmation information (e.g., push notification or SMS). The user can then confirm that the reservation and arrangements are complete.

[0305] Specific example

[0306] When the user requests "want to eat delicious udon in Tokyo", the server processes as follows:

[0307] The server analyzes the request and extracts keywords such as "Tokyo", "udon", and "delicious". At the same time, the emotion engine recognizes the emotional state from the user's request as "expectation".

[0308] The server collects the target store information from the external API and analyzes the word-of-mouth evaluation and vacancy status. Based on the emotion of "expectation", the emotion engine gives priority to proposing highly rated stores.

[0309] The server generates a proposal list containing information such as "Store A: word-of-mouth evaluation 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: word-of-mouth evaluation 4.0, average price 900 yen, waiting time 5 minutes" and presents it to the user.

[0310] The user selects Store A from the proposed stores. The server makes a reservation for Store A and arranges a taxi if necessary.

[0311] The server notifies the terminal of the reservation completion and the details of the arrangement, and the terminal informs the user of it.

[0312] In this way, the user can receive a more personalized and optimal service considering their own emotional state.

[0313] The following explains the processing flow.

[0314] Step 1:

[0315] The user uses the terminal to input and send a request. For example, input "want to eat delicious udon in Tokyo". The terminal receives this request and sends it to the server. The server records the received request in the log.

[0316] Step 2:

[0317] The server sends the received request to the natural language processing engine for analysis. The natural language processing engine extracts keywords and intent from the request, such as "in Tokyo," "udon," and "delicious." Simultaneously, the sentiment engine recognizes the user's emotional state from the words and context included in the request. For example, it extracts emotions such as "expectation" or "fatigue." The server obtains these analysis results and sentiment data and prepares the data for use in the next processing step.

[0318] Step 3:

[0319] The server collects relevant information using multiple external APIs. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. It also uses a reservation API to obtain the availability and waiting times for each restaurant. The server organizes and integrates this information to form recommendations for users.

[0320] Step 4:

[0321] The server generates a list of suggestions to present to the user based on the collected and organized information. In this process, it optimizes the suggestions using the results of the emotion engine's analysis. For example, if the user's emotional state is "fatigued," it prioritizes suggesting stores with short waiting times. Also, if the user expresses feelings of "anticipation," it prioritizes highly-rated stores.

[0322] Step 5:

[0323] The server sends the generated list of suggestions to the terminal in JSON format. The terminal then displays the received list of suggestions in an easy-to-understand format for the user. Specifically, suggestions are presented in the following format: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[0324] Step 6:

[0325] The user selects their preferred store from the displayed suggestions. For example, "Select Store A." The terminal receives the user's selection and sends it to the server.

[0326] Step 7:

[0327] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. Specifically, it calls the reservation API to make a reservation at the selected store. It also uses the taxi API to arrange transportation (such as a taxi) as needed. For example, it might use the reservation API to secure a reservation at store A and the taxi API to arrange transportation for the user.

[0328] Step 8:

[0329] The server confirms that the reservation and arrangements are complete and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes." The server then sends this notification message to the terminal.

[0330] Step 9:

[0331] The device receives notifications sent from the server and provides the user with final confirmation information. Specifically, users are notified via push notifications or SMS. This allows users to confirm that their reservation and arrangements are complete.

[0332] Through this series of steps, users can seamlessly receive more personalized and optimal services that even take their emotional state into consideration.

[0333] (Example 2)

[0334] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0335] Traditional information gathering and suggestion systems have a problem in that they provide simple information without considering the user's emotions, making it difficult to improve user satisfaction. Furthermore, there was a lack of methods to integrate information from multiple external sources and provide optimal suggestions based on the user's specific needs. As a result, it was difficult for users to obtain the necessary information quickly and accurately, leading to a decline in the quality of the user experience.

[0336] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0337] In this invention, the server includes means for receiving requests from users, means for analyzing requests using natural language processing and extracting relevant keywords and intentions, means for an emotion engine that analyzes the user's emotional state, means for collecting information from multiple external information sources, analyzing the acquired information and making suggestions to the user, means for optimizing the suggestions based on the emotional state, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary services based on the selection, and means for notifying the user of the completion of the reservations and arrangements. This enables more personalized and optimal information provision and service arrangements that take the user's emotions into consideration.

[0338] "Means of receiving requests" refers to devices or software that receive input from users and transfer it to a server.

[0339] "Natural language processing" is the process of analyzing incoming requests and extracting keywords and intent.

[0340] "Means for extracting relevant keywords and intent" refers to engines or software that identify important words and user intent from requests based on natural language processing.

[0341] An "emotional engine" refers to algorithms or software used to recognize and analyze a user's emotional state based on their request content and context.

[0342] "Means of collecting information, analyzing the acquired information, and making suggestions to users" refers to the process or engine for acquiring data from external sources, analyzing it, and creating appropriate suggestions for users.

[0343] "Methods for optimizing suggestions based on emotional state" refer to methods or software that take the user's emotional state into consideration and select the most appropriate suggestion from the collected information.

[0344] "Means for receiving user selections" refers to the interface or software used to receive a user's selection from the provided suggestions and communicate that selection to the server.

[0345] "Means for making reservations and arranging necessary services" refers to systems or software that automatically handle reservation procedures and the arrangement of related services based on the user's selection.

[0346] "Means of notifying users of the completion of reservations and arrangements" refers to notification systems or software that inform users when reservations or arrangements have been completed.

[0347] "External information sources" refer to external APIs and databases such as restaurant information, customer review information, reservation services, and transportation information.

[0348] A "system" is a set of hardware and software configurations that operate by integrating the above-mentioned means and processes.

[0349] Modes for carrying out the invention

[0350] This invention is a system that receives user requests, recognizes and analyzes the user's emotional state using an emotion engine, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that analysis. This system consists of the following components, and its detailed processing is described below.

[0351] System components

[0352] 1. User terminal: Responsible for inputting requests and displaying results.

[0353] 2. Server: Responsible for request analysis, sentiment recognition, information gathering and analysis, suggestion generation, reservation arrangement, and result notification.

[0354] 3. External information sources: Provide information on restaurants, customer reviews, reservation services, and transportation options.

[0355] 4. Emotion Engine: Extracts emotions from user requests and optimizes suggestions.

[0356] Specific operation of the system

[0357] Receiving user requests

[0358] When a user enters a request into their device and sends it, the device sends that request to the server. For example, if a user enters "I want to eat delicious udon in Tokyo," the device generates an HTTP request and sends it to the server. The server receives this request and logs it.

[0359] Intent analysis and emotion recognition using Natural Language Processing (NLP)

[0360] The server sends received requests to a natural language processing engine (e.g., SpaCy or Google Cloud NLP) for analysis. The NLP engine extracts keywords such as "Tokyo," "udon," and "delicious," analyzes the context, and understands the intent. An emotion engine (e.g., IBM Watson® Tone Analyzer) recognizes the user's emotional state (e.g., "expectation") from the words and context included in the request.

[0361] Information gathering and analysis

[0362] The server collects relevant information using multiple external sources (e.g., restaurant API, review API, reservation API). Specifically, it obtains a list of udon restaurants in Tokyo and basic information from the restaurant API, ratings and review counts for each restaurant from the review API, and availability and waiting times for restaurants from the reservation API. Based on this data, the server performs analysis and generates a list of suggestions to present to the user. Here, the suggestions are optimized based on the results of the emotion engine's analysis. For example, if the user is feeling tired, the server prioritizes suggesting restaurants with short waiting times.

[0363] Suggestions for users

[0364] The server encodes the generated suggestion list in JSON format and sends it to the terminal. The terminal parses the received data and displays it in the user interface. For example, suggestions such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes" may be displayed.

[0365] User selection and optimization

[0366] When a user selects a desired store from the device's suggestion list, the device sends that selection to the server. The server makes a reservation for the selected store and arranges transportation (e.g., a taxi) if necessary. Specifically, the server accesses a reservation API, makes a reservation, and uses a transportation API (e.g., Uber API) to arrange a taxi.

[0367] Reservation and arrangement confirmation and notification

[0368] The server retrieves information about completed reservations and arrangements and generates a notification for the user. For example, it might generate a notification such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and send it to the device. The device receives this notification and sends it to the user via push notification or SMS.

[0369] Examples of prompt statements

[0370] You can obtain a detailed explanation of the system by inputting the following prompt sentences into the generating AI model:

[0371] Please describe in detail the system's processing steps from the moment a user enters a request such as "I want to eat delicious udon in Tokyo," through which the emotion engine analyzes the user's emotions, collects and analyzes information from external sources, generates optimal suggestions, makes reservations and arrangements, and notifies the user.

[0372] This will enable a system that provides more personalized and optimal services, taking into account the user's emotional state.

[0373] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0374] Step 1: Receiving User Requests

[0375] The user enters a request into their device saying, "I want to eat delicious udon in Tokyo."

[0376] The terminal receives input, generates an HTTP request, and sends it to the server. The input is the user's text input, and the output is the data sent to the server.

[0377] The server receives this request at the API endpoint and records it in a log file or database. The input is an HTTP request from the terminal, and the output is the log entry.

[0378] Step 2: Intent analysis and emotion recognition using Natural Language Processing (NLP)

[0379] The server sends the received request to the natural language processing engine for analysis. The input is the user's request, and the output is the NLP analysis result.

[0380] A natural language processing engine (e.g., SpaCy or Google Cloud NLP) extracts keywords and intent from a request, such as "in Tokyo," "udon," and "delicious." The input is the text data of the request, and the output is the extracted keywords and intent.

[0381] An emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotional state (e.g., "expectation") from the words and context included in the request. The input is the text data of the request, and the output is the emotional state data.

[0382] Step 3: Information Gathering and Analysis

[0383] The server collects relevant information from multiple external sources. Specifically, it utilizes restaurant APIs, review APIs, and reservation APIs. The input consists of analyzed keywords and intent, while the output is the collected information.

[0384] This system retrieves a list of udon restaurants in Tokyo and their basic information from a restaurant API, and collects each restaurant's rating and review count from a review API. The input is the requested conditions, and the output is a list of restaurants and rating data.

[0385] This system uses a reservation API to retrieve seat availability and waiting times for each store. Input is store information, and output is seat availability and waiting time data.

[0386] The server integrates this data and generates a list of suggestions to present to the user, taking into account the results of the sentiment engine. Input is data from multiple APIs, and output is a list of suggestions.

[0387] Step 4: Proposal to the user

[0388] The server sends the generated proposal list to the terminal in JSON format. The input is the proposal list, and the output is the transmitted data.

[0389] The terminal analyzes the received data and displays it on the user interface. The input is a list of suggestions from the server, and the output is what is displayed to the user.

[0390] For example, "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes" will be displayed.

[0391] Step 5: User Selection and Optimization

[0392] The user selects their preferred store from the suggested stores. The input is the suggested list, and the output is the user's selection.

[0393] The terminal receives the user's selection and sends it to the server. The input is the user's selection, and the output is the data sent to the server.

[0394] The server makes reservations for the selected restaurant and arranges a taxi if necessary. Specifically, it accesses a reservation API, completes the reservation process, and uses a transportation API (e.g., Uber API) to arrange a taxi. Input is the user's selection information, and output is reservation and arrangement data.

[0395] Step 6: Confirmation and notification of booking / arrangement

[0396] The server confirms the completion of reservations and arrangements and organizes the information. Inputs are response data from the reservation API and the transportation API, and outputs are completion notifications.

[0397] The server generates a notification message such as "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and sends it to the terminal. The input is the booking status data, and the output is the notification data.

[0398] The device receives a completion notification and provides the user with final confirmation information. Specifically, this is done via push notification or SMS. The input is notification data from the server, and the output is the notification to the user.

[0399] The above is a detailed explanation of the system's program processing flow.

[0400] (Application Example 2)

[0401] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0402] In recent years, providing services tailored to individual user needs has become increasingly important. However, conventional request processing systems have been unable to consider users' emotional states when making suggestions, limiting the improvement of user satisfaction. Similarly, in food delivery services, the lack of optimal suggestions and procedures that align with users' emotions has been a problem.

[0403] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving requests from users, means for analyzing the requests using natural language processing and extracting relevant keywords and intentions, means for recognizing the user's emotional state from the requests using an emotion analysis engine, means for collecting information from multiple external information sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary services based on the selections, and means for notifying the user of the completion of the reservations and arrangements. This makes it possible to provide optimal services and food delivery procedures that take into account the user's emotional state.

[0404] "Means for receiving user requests" refers to interfaces and systems for receiving requests and questions entered by users through their devices.

[0405] "Means for analyzing using natural language processing and extracting relevant keywords and intents" refers to technologies that analyze user requests, understand the context and meaning of words, and find appropriate keywords and intents.

[0406] "A means of recognizing a user's emotional state from a request using an emotion analysis engine" refers to a technology that analyzes the emotional aspects of a user's request and identifies emotional states such as joy, anticipation, and fatigue.

[0407] "A means of collecting information from multiple external sources, analyzing the acquired information, and making suggestions to the user" refers to a technology that collects information from external sources through APIs and databases, analyzes it, and makes the most suitable suggestions to the user.

[0408] "Means for receiving user selections based on the aforementioned proposals" refers to an interface and system for presenting proposals to the user and receiving the user's selection from among them.

[0409] "Means for making reservations and arranging necessary services based on the aforementioned selection" refers to systems and technologies that automatically perform reservation procedures and arrange related services according to the user's selection.

[0410] "Means for notifying the user of the completion of the reservation and arrangement" refers to a system and interface for notifying the user of the completion information of the reservation and arrangement.

[0411] "Restaurant information" refers to basic information such as the restaurant's location, menu, business hours, and contact information.

[0412] "Rating information" refers to reviews and ratings of restaurants and services provided by users and reviewers.

[0413] A "delivery service" is a service that aims to deliver meals or goods ordered by users to a specified location.

[0414] "Transportation" includes means of enabling the movement of users and goods, such as taxis, buses, and sharing services.

[0415] This invention is a system that receives requests from users, recognizes the user's emotional state through sentiment analysis, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that. Specific embodiments for realizing this system will be described below.

[0416] System Configuration

[0417] This system consists of the following components:

[0418] 1. User terminal

[0419] User devices include smartphones and smart glasses. Users use these devices to enter requests and view and select suggested information.

[0420] 2. Server

[0421] The server receives requests from users and is responsible for their analysis, information gathering, suggestion generation, booking arrangements, and result notifications. The server interacts with the following engines and external APIs.

[0422] 3. Emotion Analysis Engine

[0423] The sentiment analysis engine recognizes the emotional state from the user's request. For example, if the user includes the phrase "I expect," the sentiment analysis engine recognizes the emotion "expectation."

[0424] 4. Natural Language Processing Engine

[0425] A natural language processing (NLP) engine analyzes a user's request and extracts relevant keywords and intent. For example, if the user enters "I want to eat delicious udon in Tokyo," the NLP engine will extract the keywords "Tokyo," "udon," and "delicious."

[0426] 5. External API

[0427] External APIs provide restaurant information, ratings, delivery services, and transportation options. For example, the Google Places API is used to retrieve information on nearby restaurants and to refer to reviews and ratings.

[0428] Explanation of the process

[0429] Request reception and analysis

[0430] The user enters a request via their smartphone or smart glasses and sends it to the server. The server then passes the request to a natural language processing engine, which analyzes the keywords and intent.

[0431] Recognition of emotions

[0432] The server inputs the analyzed request into its sentiment analysis engine to recognize the user's emotional state. For example, if the emotion "expectation" is extracted from the request, the server optimizes its suggestions based on that.

[0433] Information gathering and proposals

[0434] The server uses external APIs to collect relevant information. For example, it retrieves information about restaurants, ratings, and delivery status. It analyzes the collected information and generates optimal suggestions based on the user's emotional state.

[0435] User notifications and choices

[0436] The server notifies the user terminal of the generated suggestions and assists the user in the process of selecting from them. The user's selection information is sent to the server, and reservations and delivery arrangements are made.

[0437] Completion notification

[0438] The server confirms the completion of the reservation and arrangement and notifies the user of the result. The user's terminal receives the notification and displays the result to the user.

[0439] Specific example

[0440] If a user enters a request saying, "I want to eat delicious udon in Tokyo":

[0441] 1. The server analyzes the request using a natural language processing engine and extracts the keywords "Tokyo," "udon," and "delicious."

[0442] 2. The emotion analysis engine recognizes the emotion of "expectation."

[0443] 3. The server generates suggestions tailored to the user based on information obtained from external APIs.

[0444] 4. Notify users of the following suggestions: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes".

[0445] 5. The user selects store A, and the server makes the reservation and delivery arrangements.

[0446] 6. The server sends a notification to the user that the arrangement is complete, and the user confirms it.

[0447] Example of a prompt

[0448] "I want to eat delicious udon in Tokyo."

[0449] "I'd like sushi delivered for the dinner I'm looking forward to."

[0450] "I'd like to order a pizza for lunch, but I'd like to check if it's being delivered quickly."

[0451] Thus, the system based on the present invention enables the provision of optimal services that take user emotions into consideration.

[0452] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0453] Step 1:

[0454] The device receives a request from the user. This request is entered in text format. An example of such a request might be, "I want to eat delicious udon in Tokyo." The device then sends this request to the server.

[0455] Step 2:

[0456] The server sends the received request to the natural language processing engine for analysis. The NLP engine extracts keywords and intent from the request, such as "in Tokyo," "udon," and "delicious." The analysis results are returned to the server and used in the next processing step.

[0457] Step 3:

[0458] The server sends the analysis results to the sentiment analysis engine, which recognizes and analyzes the emotional state included in the request. The sentiment analysis engine extracts emotions such as "expectation" from the context and returns the results to the server. The server then prepares to optimize the suggestions based on this emotional state.

[0459] Step 4:

[0460] The server uses multiple external APIs to collect relevant information. For example, it retrieves a list and basic information of "udon restaurants in Tokyo" from a restaurant information API, and collects customer reviews and the number of reviews for each restaurant from a rating information API. This collected data is then integrated on the server side.

[0461] Step 5:

[0462] The server generates a suggestion list based on the collected information. Here, based on the emotional state "expectation" provided by the sentiment analysis engine, stores with high customer reviews are prioritized and included in the list. The generated suggestion list is sent to the terminal in JSON format.

[0463] Step 6:

[0464] The device displays a list of received suggestions to the user. Examples of suggestions displayed include: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[0465] Step 7:

[0466] The user makes a selection from the presented suggestions. For example, "Select store A." The terminal sends this selection to the server.

[0467] Step 8:

[0468] The server makes reservations and necessary arrangements based on the user's selection. Specifically, it makes reservations for the selected store via an online reservation API and arranges delivery methods as needed. The results of the reservations and arrangements are stored on the server.

[0469] Step 9:

[0470] The server confirms the completion of the reservation and arrangement and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. Delivery is expected to arrive in 15 minutes."

[0471] Step 10:

[0472] The server sends this final confirmation information to the device. The device receives a completion notification and provides the user with the final confirmation information via push notification or display. This allows the user to confirm the completion of their reservation and arrangements.

[0473] In this way, users can receive the optimal food delivery service tailored to their emotional state.

[0474] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0475] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0476] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0477] [Second Embodiment]

[0478] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0479] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0480] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0481] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0482] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0483] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0484] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0485] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0486] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0487] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0488] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0489] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0490] Modes for carrying out the invention

[0491] This invention is a system that handles everything from information gathering and proposals to reservations and necessary arrangements, primarily based on user requests. The program of this system mainly consists of the following components, and its detailed processing is described below.

[0492] System components

[0493] 1. User terminal: Responsible for inputting requests and displaying results.

[0494] 2. Server: Responsible for parsing requests, collecting and analyzing information, generating suggestions, arranging reservations, and notifying results.

[0495] 3. External APIs: Provide restaurant information, reviews, reservation services, and transportation information.

[0496] Explanation of program processing

[0497] 1. Receiving user requests

[0498] The user enters a request into their device and sends it. For example, they might enter, "I want to eat delicious udon in Tokyo."

[0499] The terminal receives the request and sends it to the server. The server logs the received request.

[0500] 2. Intent Analysis using Natural Language Processing (NLP)

[0501] The server passes the received request to the natural language processing engine for analysis.

[0502] The NLP engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the request and analyzes the user's intent.

[0503] The server retrieves the analysis results and prepares the data for subsequent processing.

[0504] 3. Information Gathering and Analysis

[0505] The server collects relevant information through external APIs. Specifically, it retrieves data from sources such as restaurant information, customer reviews, and reservation services.

[0506] Example: The server collects information on udon restaurants in Tokyo from a restaurant API and obtains ratings for each restaurant from a review API. It also obtains availability and waiting times for each restaurant from a reservation API.

[0507] The server organizes the collected information and creates a list to present to the user.

[0508] 4. Suggestions for users

[0509] The server generates multiple suggestions for the user based on the organized information.

[0510] For example, the server might suggest options like "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" or "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[0511] The device receives the suggestion and displays it to the user.

[0512] 5. User Selection and Optimization

[0513] The user makes a selection from the presented suggestions. For example, "Select store A."

[0514] The device sends the user's selection to the server.

[0515] The server receives the selection and begins the reservation and necessary arrangements. Specifically, it makes a reservation at the selected store and, if necessary, arranges transportation (such as a taxi).

[0516] 6. Confirmation and notification of reservations and arrangements

[0517] The server confirms the completion of the reservation and arrangements and organizes the information.

[0518] Example: The server creates a notification such as "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and sends it to the device.

[0519] The device receives a completion notification and provides the user with final confirmation information (e.g., push notification or SMS).

[0520] Specific example

[0521] If a user requests "I want to eat delicious udon in Tokyo," the server will process it as follows:

[0522] The server analyzes the request and extracts keywords such as "Tokyo area," "udon," and "delicious."

[0523] The server collects information on target stores from external APIs and analyzes customer reviews and seating availability.

[0524] The server suggests multiple store options to the user.

[0525] The user selects store A from the suggested stores.

[0526] The server makes a reservation for store A and arranges a taxi if necessary.

[0527] The server notifies the terminal of the reservation completion and arrangement details, and the terminal informs the user of this.

[0528] In this way, users can seamlessly complete everything from information gathering to booking and arrangements through a single platform.

[0529] The following describes the processing flow.

[0530] Step 1:

[0531] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo." The device receives this request and sends it to the server.

[0532] Step 2:

[0533] The server sends the received request to the natural language processing engine. The natural language processing engine analyzes the request and extracts keywords and the user's intent (e.g., "in Tokyo", "udon", "delicious"). The server retrieves the analysis results and prepares the data for use in the next processing step.

[0534] Step 3:

[0535] The server uses multiple external APIs to collect relevant information. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. Furthermore, it obtains the availability and waiting times for each restaurant from a reservation API.

[0536] Step 4:

[0537] The server performs analysis based on the collected data and generates a list of suggestions to present to the user. For example, it creates a list that includes information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes".

[0538] Step 5:

[0539] The server sends the generated suggestion list to the terminal in JSON format. The terminal then displays the received suggestion list in an easy-to-understand format for the user.

[0540] Step 6:

[0541] The user selects their preferred store from the displayed suggestions. For example, "Select Store A." The terminal receives the user's selection and sends it to the server.

[0542] Step 7:

[0543] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. The server uses the reservation API to make reservations for the selected stores and, if necessary, arranges transportation (e.g., taxis) via the taxi API.

[0544] Step 8:

[0545] The server confirms that the reservation and arrangements are complete and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes." The server then sends this notification message to the terminal.

[0546] Step 9:

[0547] The device receives notifications sent from the server and provides the user with final confirmation information (e.g., push notifications or SMS). This allows the user to confirm that the reservation and arrangements are complete.

[0548] This series of processes allows users to seamlessly navigate the entire process, from making a request and gathering information to selecting suggestions, booking and arranging, and finally confirming their booking.

[0549] (Example 1)

[0550] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0551] Traditional systems require users to use multiple platforms when making requests, resulting in a lack of consistency from information gathering to booking. Furthermore, the lack of coordination between platforms leads to significant time and effort. Additionally, the system lacks the ability to accurately understand user intent and provide appropriate suggestions, potentially resulting in lower user satisfaction.

[0552] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0553] In this invention, the server includes means for receiving requests from users, means for analyzing the requests using natural language processing and extracting relevant keywords and intentions, means for collecting information from multiple external sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary transportation based on the selections, means for notifying the user of the completion of the reservations and arrangements, and means for logging the requests, extracted keywords, and intentions. This enables users to consistently and seamlessly perform tasks from information gathering to reservation arrangements.

[0554] "Means for receiving user requests" refers to the means by which a user enters requests or demands into the system and initiates processing within the system.

[0555] A "natural language processing tool" is a means of analyzing a user's input request and extracting relevant keywords and intent from the text.

[0556] "Information gathering means" refers to the means of collecting relevant information from multiple external sources and converting it into a format usable within the system.

[0557] A "proposal generation method" is a means of generating appropriate suggestions for the user based on collected and analyzed information, and presenting the results to the user.

[0558] A "selection acceptance mechanism" is a means for a user to make a selection from the presented suggestions, accept that selection, and reflect it in the next process.

[0559] A "reservation method" refers to a means of reserving a desired store or service based on the user's choice.

[0560] "Means of arranging transportation" refers to the means of arranging the necessary means of travel in conjunction with a reservation.

[0561] "Notification means" refers to the means by which the system transmits information to inform the user that the reservation and arrangements have been completed.

[0562] A "logging method" is a means of recording user requests, extracted keywords, intentions, etc., within the system for later analysis and auditing.

[0563] This invention provides a system that allows users to seamlessly access a series of services. This system consistently handles information gathering, analysis, suggestions, reservations, and necessary arrangements, all centered around the user's request. Detailed embodiments of this system are described below.

[0564] Hardware and software to be used

[0565] 1. User terminal:

[0566] User devices can include smartphones, tablets, and personal computers. These devices receive requests from users and send them to the server. They also display suggestions and notifications to the user.

[0567] 2. Server:

[0568] The server oversees a series of processes, from analyzing user requests and gathering information to generating suggestions, arranging reservations, and sending notifications. Specifically, it uses the following software:

[0569] Natural Language Processing Engine (NLP): Utilizes tools such as the Google Cloud Natural Language API to understand user requests and extract relevant keywords and intents.

[0570] External APIs: Restaurant information, ratings, availability, reservation status, and transportation information are obtained via APIs such as the Yelp API and Google Places API.

[0571] Process Overview

[0572] 1. Receiving user requests:

[0573] The user enters a request through their device. For example, they might request, "I want to eat delicious udon in Tokyo." This request is sent from the device to the server.

[0574] 2. Intent analysis using natural language processing:

[0575] After receiving a request, the server passes the utterance to a natural language processing engine. The engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the request and analyzes the user's intent. The server retrieves the analysis results and prepares queries for the next data collection.

[0576] 3. Information gathering and analysis:

[0577] The server sends queries to external APIs to collect and analyze necessary information. Specifically, it uses the Yelp API to obtain a list of udon restaurants in Tokyo and the Google Places API to obtain information on the availability of seats and customer reviews for each restaurant.

[0578] Based on the information obtained, a list of suggestions for the user is generated.

[0579] 4. Suggestions for users:

[0580] The server generates several suggestions based on the collected information and sends them to the terminal. An example suggestion might be, "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes." The terminal then displays this to the user.

[0581] 5. User Selection and Optimization:

[0582] When a user makes a selection from a list of suggestions, for example, by choosing "Store A," the terminal sends the selection to the server. The server receives the selection and proceeds with the reservation process.

[0583] 6. Confirmation and notification of reservations and arrangements:

[0584] Once the server completes the reservation and necessary arrangements (e.g., taxi booking), it compiles the reservation confirmation information and sends it to the device. The device then notifies the user of the reservation completion details (e.g., via push notification or SMS).

[0585] Specific example

[0586] The specific processing flow when a user enters a request such as "I want to eat delicious udon in Tokyo" is as follows:

[0587] The server analyzes the request using an NLP engine and extracts keywords such as "Tokyo," "udon," and "delicious."

[0588] The server collects and analyzes information about the target stores from the Yelp API and Google Places API.

[0589] Based on the information collected by the server, it suggests potential stores to the user, such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes."

[0590] The user selects "Store A," the server makes a reservation for that store, and arranges a taxi if necessary.

[0591] The server notifies the user's terminal with reservation confirmation and taxi arrangement details.

[0592] Example of a prompt

[0593] Examples of prompts that users might enter include the following:

[0594] "I want to eat delicious udon in Tokyo."

[0595] "Can you recommend a good cafe for the weekend?"

[0596] "I'm looking for a yakiniku restaurant that's open tonight."

[0597] In this way, this system can efficiently and seamlessly handle everything from information gathering to arrangement in response to user requests.

[0598] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0599] Program processing steps

[0600] Step 1: Receiving User Requests

[0601] Specific actions:

[0602] The user enters the prompt "I want to eat delicious udon in Tokyo" into the terminal and sends it. The terminal sends this input to the server as an HTTP request. The server receives the HTTP request and logs its contents. The log includes the request content, a timestamp, and the user's IP address.

[0603] input:

[0604] User prompt (e.g., "I want to eat delicious udon in Tokyo")

[0605] output:

[0606] The prompt message received by the server, and the request content recorded in the log.

[0607] Step 2: Intent Analysis using Natural Language Processing

[0608] Specific actions:

[0609] The server sends the received prompt text to a natural language processing engine (e.g., Google Cloud Natural Language API). The NLP engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the prompt text. The server retrieves the analysis results from the NLP engine and prepares queries containing these keywords for subsequent data collection.

[0610] input:

[0611] Received prompt message

[0612] output:

[0613] Extracted keywords (e.g., "Tokyo area", "udon", "delicious")

[0614] Step 3: Information Gathering and Analysis

[0615] Specific actions:

[0616] The server sends queries to external information source APIs using the extracted keywords. For example, it might retrieve a list of udon restaurants in Tokyo and their ratings via the Yelp API, and obtain the availability and waiting times for each restaurant via the Google Places API. The server receives this information, organizes and analyzes it, and generates a list to present to the user.

[0617] input:

[0618] Extracted keywords

[0619] output:

[0620] Store information and analyzed data (e.g., store name, customer reviews, seating availability, waiting time)

[0621] Step 4: Proposal to the user

[0622] Specific actions:

[0623] The server generates multiple suggestions based on the collected and analyzed store information. The generated suggestions are sent to the terminal in JSON format. The terminal parses the received JSON data and displays the suggestions in a user-friendly format. For example, it might display information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes."

[0624] input:

[0625] Organized and analyzed store information

[0626] output:

[0627] A list of suggestions for users (Example: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes")

[0628] Step 5: User Selection and Optimization

[0629] Specific actions:

[0630] The user makes a selection from a list of suggestions, for example, by selecting "Store A." The terminal sends the user's selection to the server. The server receives the selection and begins arranging a reservation for the selected store. Specifically, it sends a request to the store's reservation API and makes the reservation.

[0631] input:

[0632] User selection (e.g., "Store A")

[0633] output:

[0634] Submitting a booking request and booking confirmation information

[0635] Step 6: Confirmation and notification of booking / arrangement

[0636] Specific actions:

[0637] The server confirms the completion of the reservation and arrangement and organizes the information. The server sends the organized information to the terminal in JSON format. The terminal parses the received JSON data and notifies the user of the reservation and arrangement completion (e.g., push notification or SMS). The reservation confirmation information includes the reservation number, estimated taxi arrival time, etc.

[0638] input:

[0639] Reservation confirmation information

[0640] output:

[0641] Notifications and detailed information to the user (e.g., "Reservation complete. Reservation number 12345. Taxi is expected to arrive in 15 minutes.")

[0642] In this way, the system can efficiently and seamlessly handle everything from information gathering to arrangement in response to user requests.

[0643] (Application Example 1)

[0644] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0645] In recent years, there has been a growing demand for systems that allow users to easily order and reserve meals. However, current food delivery platforms require the use of multiple applications and services, making the process cumbersome. Therefore, there is a need to develop a system that allows users to handle everything from requests and orders to delivery arrangements and status notifications on a single platform.

[0646] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0647] In this invention, the server includes means for receiving requests from users, means for analyzing requests using natural language processing and extracting relevant keywords and intents, means for collecting information from multiple external sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations, orders, and arrangements for necessary services based on the selections, means for notifying the user of the completion of the reservations and arrangements, and means for notifying the user of delivery status and order status in real time. This enables users to seamlessly perform everything from making requests to ordering, making arrangements, and checking the status on a single platform.

[0648] "Means for receiving requests from users" refers to the part of a system that sends requests entered by users via devices such as smartphones and computers to a server and receives those requests.

[0649] "Natural language processing" is a technology that analyzes text data entered by users to extract keywords and intent.

[0650] "External information sources" refer to data sources provided from outside the system, such as restaurant information, customer reviews, reservation services, and delivery services.

[0651] "Information gathering means" refers to the part that has the function of acquiring and analyzing data from external information sources.

[0652] The "means of suggestion" refers to the function that provides users with multiple options based on the information collected and analyzed.

[0653] "Means of making reservations and arrangements" refers to the part of the system that has the functionality to make restaurant reservations and arrange delivery based on the options selected by the user.

[0654] "Notification means" refers to the function that informs users in real time about the completion of reservations and arrangements, as well as the delivery status.

[0655] "Restaurant information" refers to information about food and beverage establishments such as restaurants and cafes, and specifically includes the name of the establishment, address, business hours, menu, prices, and ratings.

[0656] "Word-of-mouth information" refers to feedback such as evaluations and impressions provided by users and customers.

[0657] "Ordering service" refers to the function or service that allows customers to order food from restaurants online.

[0658] "Delivery method" refers to the arrangements and means of transportation used to deliver the ordered meal to the location specified by the user.

[0659] This invention is a system that automatically suggests, reserves, orders, and arranges delivery of the most suitable restaurant based on a user's meal request. This system consists of a user terminal, a server, and multiple external information sources.

[0660] 1. System Program Overview

[0661] The server has the following functions:

[0662] Natural language processing engine (spaCy library)

[0663] Data collection from external sources (Google Places API, Yelp API, etc.)

[0664] Data analysis and proposal generation.

[0665] Reservation and order arrangement

[0666] Real-time notifications (Firebase Cloud Messaging)

[0667] User devices (primarily smartphones) have the following functions:

[0668] User Request Input Form

[0669] Display the proposal list

[0670] Receiving real-time notifications

[0671] 2. Program Processing

[0672] The server receives requests sent by users and analyzes them using a natural language processing engine. This analysis extracts keywords (e.g., "spicy food") from the request and understands its intent.

[0673] Based on analyzed keywords, the system collects restaurant information, reviews, and reservation data from external sources (such as the Google Places API and Yelp API). Based on this collected information, data analysis is performed to generate a list of restaurants best suited to the user.

[0674] When a user selects a restaurant from the suggested list, the server arranges the reservation and order with the selected restaurant. Furthermore, if delivery is required, it also arranges the delivery. Once these arrangements are complete, the server uses Firebase Cloud Messaging to notify the user in real time of the delivery and order status.

[0675] 3. Specific examples

[0676] When a user enters "I want to eat spicy Indian food" into a smartphone app, the server extracts the keywords "spicy" and "Indian food," and gathers and suggests the most suitable restaurant information. If the user selects "Indian Restaurant A" from the suggestions, the order is confirmed and delivery is arranged. This allows the user to track the status of their order in real time until delivery is complete.

[0677] Example of a prompt

[0678] "I'd like to eat something spicy nearby."

[0679] "I'd like to order Indian food for delivery."

[0680] "What Asian dishes are popular today?"

[0681] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0682] Step 1:

[0683] The process by which the user enters a request.

[0684] Input: The user enters a request into the smartphone app saying, "I want to eat spicy Indian food."

[0685] Specific action: Enter text into the input form on the user's terminal and press the submit button.

[0686] Output: The user terminal sends this request data to the server.

[0687] Step 2:

[0688] A process that receives a request and performs natural language processing.

[0689] Input: Request data sent from the user's terminal.

[0690] Specific operation: The server receives the request and passes the request text to the natural language processing engine (spaCy).

[0691] Output: Keywords extracted by the natural language processing engine (e.g., "spicy", "Indian food") and intent are returned.

[0692] Step 3:

[0693] Information gathering process

[0694] Input: Keywords extracted by the natural language processing engine.

[0695] Specific operation: The server calls external information sources (such as the Google Places API and Yelp API) to collect restaurant information, reviews, and reservation status related to keywords.

[0696] Output: Collected restaurant information, customer reviews, and reservation status data.

[0697] Step 4:

[0698] Process to generate a list of suggestions

[0699] Input: Collected restaurant information, customer reviews, and reservation status data.

[0700] Specific operation: The server analyzes the collected data, selects the most suitable restaurants for the user, and generates a list of suggestions.

[0701] Output: Suggestion data listing multiple restaurant options.

[0702] Step 5:

[0703] Process to display a list of suggestions to the user.

[0704] Input: Proposal list data.

[0705] Specific operation: The server sends the suggestion list data to the user's terminal, and the terminal displays the list.

[0706] Output: A list of suggested restaurants displayed on the user's terminal.

[0707] Step 6:

[0708] The process of a user selecting a restaurant from suggestions.

[0709] Input: The displayed list of suggestions.

[0710] Specific action: The user selects a restaurant from the suggested list and presses the select button.

[0711] Output: The user terminal sends information about the selected restaurant to the server.

[0712] Step 7:

[0713] Processing to arrange reservations and orders.

[0714] Input: Information about the restaurant selected by the user.

[0715] Specific operation: The server makes reservations and orders for selected restaurants through an online reservation and ordering system (external API).

[0716] Output: Reservation confirmation and order processing completion data.

[0717] Step 8:

[0718] Process to notify the user of the completion of the reservation and order.

[0719] Input: Reservation confirmation and order processing completion data.

[0720] Specific operation: The server sends a notification to the user in real time via Firebase Cloud Messaging (FCM). The device receives the notification and displays it to the user.

[0721] Output: Reservation and order completion notifications displayed on the user's terminal.

[0722] Step 9:

[0723] A process to notify delivery status in real time.

[0724] Input: Delivery arrangement progress information.

[0725] Specific operation: The server updates the delivery status in real time based on progress information obtained from the delivery system and notifies the user using Firebase Cloud Messaging.

[0726] Output: Delivery status notification displayed in real time on the user's device.

[0727] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0728] Modes for carrying out the invention

[0729] This invention is a system that receives user requests, recognizes and analyzes the user's emotional state using an emotion engine, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that analysis. The program of this system consists of the following components, and its detailed processing is described below.

[0730] System components

[0731] 1. User terminal: Responsible for inputting requests and displaying results.

[0732] 2. Server: Responsible for request analysis, sentiment recognition, information gathering and analysis, suggestion generation, reservation arrangement, and result notification.

[0733] 3. External APIs: Provide restaurant information, reviews, reservation services, and transportation information.

[0734] 4. Emotion Engine: Extracts emotions from user requests and optimizes suggestions.

[0735] Explanation of program processing

[0736] 1. Receiving user requests

[0737] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo."

[0738] The terminal receives this request and sends it to the server. The server logs the received request.

[0739] 2. Intention analysis and sentiment recognition using natural language processing (NLP)

[0740] The server sends the received request to the natural language processing engine for analysis.

[0741] The natural language processing engine extracts keywords and intents from the request, such as "in Tokyo," "udon," and "delicious."

[0742] At the same time, the emotion engine recognizes the user's emotional state (e.g., joy, anticipation, fatigue) from the words and context included in the request.

[0743] The server retrieves these analysis results and sentiment data, and prepares the data for use in the next processing step.

[0744] 3. Information Gathering and Analysis

[0745] The server uses multiple external APIs to collect relevant information. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. Furthermore, it obtains the availability and waiting times for each restaurant from a reservation API.

[0746] The server performs analysis based on the collected data and generates a list of suggestions to present to the user. Here, the suggestions are optimized based on the results of the emotion engine's analysis. For example, if the user is feeling fatigued, the server prioritizes suggesting stores with short waiting times.

[0747] 4. Suggestions for users

[0748] The server sends the generated suggestion list to the terminal in JSON format. The terminal then displays the received suggestion list in an easy-to-understand format for the user.

[0749] For example, the server might suggest "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes." The system also optimizes the suggestions based on the user's emotional state.

[0750] 5. User Selection and Optimization

[0751] The user selects their preferred store from the displayed suggestions. For example, "Select Store A."

[0752] The terminal receives the user's selection and sends it to the server.

[0753] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. Specifically, it makes reservations at the selected establishments and, if necessary, arranges transportation (such as taxis).

[0754] 6. Confirmation and notification of reservations and arrangements

[0755] The server confirms the completion of the reservation and arrangement and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is scheduled to arrive in 15 minutes."

[0756] The server sends this notification to the device. The device receives the completion notification and provides the user with final confirmation information (e.g., push notification or SMS). The user can then confirm that the reservation and arrangements are complete.

[0757] Specific example

[0758] If a user requests "I want to eat delicious udon in Tokyo," the server will process it as follows:

[0759] The server analyzes the request and extracts keywords such as "Tokyo," "udon," and "delicious." At the same time, the emotion engine recognizes the user's emotional state from the request as "expectation."

[0760] The server collects target store information from an external API and analyzes customer reviews and seating availability. The emotion engine prioritizes and suggests highly-rated stores based on the emotion of "expectation."

[0761] The server generates a list of suggestions that include information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes," and presents them to the user.

[0762] The user selects store A from the suggested stores. The server makes a reservation for store A and arranges a taxi if necessary.

[0763] The server notifies the terminal of the reservation completion and arrangement details, and the terminal informs the user of this.

[0764] In this way, users can receive more personalized and optimal services that take their own emotional state into consideration.

[0765] The following describes the processing flow.

[0766] Step 1:

[0767] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo." The device receives this request and sends it to the server. The server logs the received request.

[0768] Step 2:

[0769] The server sends the received request to the natural language processing engine for analysis. The natural language processing engine extracts keywords and intent from the request, such as "in Tokyo," "udon," and "delicious." Simultaneously, the sentiment engine recognizes the user's emotional state from the words and context included in the request. For example, it extracts emotions such as "expectation" or "fatigue." The server obtains these analysis results and sentiment data and prepares the data for use in the next processing step.

[0770] Step 3:

[0771] The server collects relevant information using multiple external APIs. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. It also uses a reservation API to obtain the availability and waiting times for each restaurant. The server organizes and integrates this information to form recommendations for users.

[0772] Step 4:

[0773] The server generates a list of suggestions to present to the user based on the collected and organized information. In this process, it optimizes the suggestions using the results of the emotion engine's analysis. For example, if the user's emotional state is "fatigued," it prioritizes suggesting stores with short waiting times. Also, if the user expresses feelings of "anticipation," it prioritizes highly-rated stores.

[0774] Step 5:

[0775] The server sends the generated list of suggestions to the terminal in JSON format. The terminal then displays the received list of suggestions in an easy-to-understand format for the user. Specifically, suggestions are presented in the following format: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[0776] Step 6:

[0777] The user selects their preferred store from the displayed suggestions. For example, "Select Store A." The terminal receives the user's selection and sends it to the server.

[0778] Step 7:

[0779] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. Specifically, it calls the reservation API to make a reservation at the selected store. It also uses the taxi API to arrange transportation (such as a taxi) as needed. For example, it might use the reservation API to secure a reservation at store A and the taxi API to arrange transportation for the user.

[0780] Step 8:

[0781] The server confirms that the reservation and arrangements are complete and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes." The server then sends this notification message to the terminal.

[0782] Step 9:

[0783] The device receives notifications sent from the server and provides the user with final confirmation information. Specifically, users are notified via push notifications or SMS. This allows users to confirm that their reservation and arrangements are complete.

[0784] Through this series of steps, users can seamlessly receive more personalized and optimal services that even take their emotional state into consideration.

[0785] (Example 2)

[0786] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0787] Traditional information gathering and suggestion systems have a problem in that they provide simple information without considering the user's emotions, making it difficult to improve user satisfaction. Furthermore, there was a lack of methods to integrate information from multiple external sources and provide optimal suggestions based on the user's specific needs. As a result, it was difficult for users to obtain the necessary information quickly and accurately, leading to a decline in the quality of the user experience.

[0788] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0789] In this invention, the server includes means for receiving requests from users, means for analyzing requests using natural language processing and extracting relevant keywords and intentions, means for an emotion engine that analyzes the user's emotional state, means for collecting information from multiple external information sources, analyzing the acquired information and making suggestions to the user, means for optimizing the suggestions based on the emotional state, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary services based on the selection, and means for notifying the user of the completion of the reservations and arrangements. This enables more personalized and optimal information provision and service arrangements that take the user's emotions into consideration.

[0790] "Means of receiving requests" refers to devices or software that receive input from users and transfer it to a server.

[0791] "Natural language processing" is the process of analyzing incoming requests and extracting keywords and intent.

[0792] "Means for extracting relevant keywords and intent" refers to engines or software that identify important words and user intent from requests based on natural language processing.

[0793] An "emotional engine" refers to algorithms or software used to recognize and analyze a user's emotional state based on their request content and context.

[0794] "Means of collecting information, analyzing the acquired information, and making suggestions to users" refers to the process or engine for acquiring data from external sources, analyzing it, and creating appropriate suggestions for users.

[0795] "Methods for optimizing suggestions based on emotional state" refer to methods or software that take the user's emotional state into consideration and select the most appropriate suggestion from the collected information.

[0796] "Means for receiving user selections" refers to the interface or software used to receive a user's selection from the provided suggestions and communicate that selection to the server.

[0797] "Means for making reservations and arranging necessary services" refers to systems or software that automatically handle reservation procedures and the arrangement of related services based on the user's selection.

[0798] "Means of notifying users of the completion of reservations and arrangements" refers to notification systems or software that inform users when reservations or arrangements have been completed.

[0799] "External information sources" refer to external APIs and databases such as restaurant information, customer review information, reservation services, and transportation information.

[0800] A "system" is a set of hardware and software configurations that operate by integrating the above-mentioned means and processes.

[0801] Modes for carrying out the invention

[0802] This invention is a system that receives user requests, recognizes and analyzes the user's emotional state using an emotion engine, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that analysis. This system consists of the following components, and its detailed processing is described below.

[0803] System components

[0804] 1. User terminal: Responsible for inputting requests and displaying results.

[0805] 2. Server: Responsible for request analysis, sentiment recognition, information gathering and analysis, suggestion generation, reservation arrangement, and result notification.

[0806] 3. External information sources: Provide information on restaurants, customer reviews, reservation services, and transportation options.

[0807] 4. Emotion Engine: Extracts emotions from user requests and optimizes suggestions.

[0808] Specific operation of the system

[0809] Receiving user requests

[0810] When a user enters a request into their device and sends it, the device sends that request to the server. For example, if a user enters "I want to eat delicious udon in Tokyo," the device generates an HTTP request and sends it to the server. The server receives this request and logs it.

[0811] Intent analysis and emotion recognition using Natural Language Processing (NLP)

[0812] The server sends received requests to a natural language processing engine (e.g., SpaCy or Google Cloud NLP) for analysis. The NLP engine extracts keywords such as "Tokyo," "udon," and "delicious," analyzes the context, and understands the intent. An emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotional state (e.g., "expectation") from the words and context included in the request.

[0813] Information gathering and analysis

[0814] The server collects relevant information using multiple external sources (e.g., restaurant API, review API, reservation API). Specifically, it obtains a list of udon restaurants in Tokyo and basic information from the restaurant API, ratings and review counts for each restaurant from the review API, and availability and waiting times for restaurants from the reservation API. Based on this data, the server performs analysis and generates a list of suggestions to present to the user. Here, the suggestions are optimized based on the results of the emotion engine's analysis. For example, if the user is feeling tired, the server prioritizes suggesting restaurants with short waiting times.

[0815] Suggestions for users

[0816] The server encodes the generated suggestion list in JSON format and sends it to the terminal. The terminal parses the received data and displays it in the user interface. For example, suggestions such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes" may be displayed.

[0817] User selection and optimization

[0818] When a user selects a desired store from the device's suggestion list, the device sends that selection to the server. The server makes a reservation for the selected store and arranges transportation (e.g., a taxi) if necessary. Specifically, the server accesses a reservation API, makes a reservation, and uses a transportation API (e.g., Uber API) to arrange a taxi.

[0819] Reservation and arrangement confirmation and notification

[0820] The server retrieves information about completed reservations and arrangements and generates a notification for the user. For example, it might generate a notification such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and send it to the device. The device receives this notification and sends it to the user via push notification or SMS.

[0821] Examples of prompt statements

[0822] You can obtain a detailed explanation of the system by inputting the following prompt sentences into the generating AI model:

[0823] Please describe in detail the system's processing steps from the moment a user enters a request such as "I want to eat delicious udon in Tokyo," through which the emotion engine analyzes the user's emotions, collects and analyzes information from external sources, generates optimal suggestions, makes reservations and arrangements, and notifies the user.

[0824] This will enable a system that provides more personalized and optimal services, taking into account the user's emotional state.

[0825] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0826] Step 1: Receiving User Requests

[0827] The user enters a request into their device saying, "I want to eat delicious udon in Tokyo."

[0828] The terminal receives input, generates an HTTP request, and sends it to the server. The input is the user's text input, and the output is the data sent to the server.

[0829] The server receives this request at the API endpoint and records it in a log file or database. The input is an HTTP request from the terminal, and the output is the log entry.

[0830] Step 2: Intent analysis and emotion recognition using Natural Language Processing (NLP)

[0831] The server sends the received request to the natural language processing engine for analysis. The input is the user's request, and the output is the NLP analysis result.

[0832] A natural language processing engine (e.g., SpaCy or Google Cloud NLP) extracts keywords and intent from a request, such as "in Tokyo," "udon," and "delicious." The input is the text data of the request, and the output is the extracted keywords and intent.

[0833] An emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotional state (e.g., "expectation") from the words and context included in the request. The input is the text data of the request, and the output is the emotional state data.

[0834] Step 3: Information Gathering and Analysis

[0835] The server collects relevant information from multiple external sources. Specifically, it utilizes restaurant APIs, review APIs, and reservation APIs. The input consists of analyzed keywords and intent, while the output is the collected information.

[0836] This system retrieves a list of udon restaurants in Tokyo and their basic information from a restaurant API, and collects each restaurant's rating and review count from a review API. The input is the requested conditions, and the output is a list of restaurants and rating data.

[0837] This system uses a reservation API to retrieve seat availability and waiting times for each store. Input is store information, and output is seat availability and waiting time data.

[0838] The server integrates this data and generates a list of suggestions to present to the user, taking into account the results of the sentiment engine. Input is data from multiple APIs, and output is a list of suggestions.

[0839] Step 4: Proposal to the user

[0840] The server sends the generated proposal list to the terminal in JSON format. The input is the proposal list, and the output is the transmitted data.

[0841] The terminal analyzes the received data and displays it on the user interface. The input is a list of suggestions from the server, and the output is what is displayed to the user.

[0842] For example, "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes" will be displayed.

[0843] Step 5: User Selection and Optimization

[0844] The user selects their preferred store from the suggested stores. The input is the suggested list, and the output is the user's selection.

[0845] The terminal receives the user's selection and sends it to the server. The input is the user's selection, and the output is the data sent to the server.

[0846] The server makes reservations for the selected restaurant and arranges a taxi if necessary. Specifically, it accesses a reservation API, completes the reservation process, and uses a transportation API (e.g., Uber API) to arrange a taxi. Input is the user's selection information, and output is reservation and arrangement data.

[0847] Step 6: Confirmation and notification of booking / arrangement

[0848] The server confirms the completion of reservations and arrangements and organizes the information. Inputs are response data from the reservation API and the transportation API, and outputs are completion notifications.

[0849] The server generates a notification message such as "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and sends it to the terminal. The input is the booking status data, and the output is the notification data.

[0850] The device receives a completion notification and provides the user with final confirmation information. Specifically, this is done via push notification or SMS. The input is notification data from the server, and the output is the notification to the user.

[0851] The above is a detailed explanation of the system's program processing flow.

[0852] (Application Example 2)

[0853] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0854] In recent years, providing services tailored to individual user needs has become increasingly important. However, conventional request processing systems have been unable to consider users' emotional states when making suggestions, limiting the improvement of user satisfaction. Similarly, in food delivery services, the lack of optimal suggestions and procedures that align with users' emotions has been a problem.

[0855] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving requests from users, means for analyzing the requests using natural language processing and extracting relevant keywords and intentions, means for recognizing the user's emotional state from the requests using an emotion analysis engine, means for collecting information from multiple external information sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary services based on the selections, and means for notifying the user of the completion of the reservations and arrangements. This makes it possible to provide optimal services and food delivery procedures that take into account the user's emotional state.

[0856] "Means for receiving user requests" refers to interfaces and systems for receiving requests and questions entered by users through their devices.

[0857] "Means for analyzing using natural language processing and extracting relevant keywords and intents" refers to technologies that analyze user requests, understand the context and meaning of words, and find appropriate keywords and intents.

[0858] "A means of recognizing a user's emotional state from a request using an emotion analysis engine" refers to a technology that analyzes the emotional aspects of a user's request and identifies emotional states such as joy, anticipation, and fatigue.

[0859] "A means of collecting information from multiple external sources, analyzing the acquired information, and making suggestions to the user" refers to a technology that collects information from external sources through APIs and databases, analyzes it, and makes the most suitable suggestions to the user.

[0860] "Means for receiving user selections based on the aforementioned proposals" refers to an interface and system for presenting proposals to the user and receiving the user's selection from among them.

[0861] "Means for making reservations and arranging necessary services based on the aforementioned selection" refers to systems and technologies that automatically perform reservation procedures and arrange related services according to the user's selection.

[0862] "Means for notifying the user of the completion of the reservation and arrangement" refers to a system and interface for notifying the user of the completion information of the reservation and arrangement.

[0863] "Restaurant information" refers to basic information such as the restaurant's location, menu, business hours, and contact information.

[0864] "Rating information" refers to reviews and ratings of restaurants and services provided by users and reviewers.

[0865] A "delivery service" is a service that aims to deliver meals or goods ordered by users to a specified location.

[0866] "Transportation" includes means of enabling the movement of users and goods, such as taxis, buses, and sharing services.

[0867] This invention is a system that receives requests from users, recognizes the user's emotional state through sentiment analysis, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that. Specific embodiments for realizing this system will be described below.

[0868] System Configuration

[0869] This system consists of the following components:

[0870] 1. User terminal

[0871] User devices include smartphones and smart glasses. Users use these devices to enter requests and view and select suggested information.

[0872] 2. Server

[0873] The server receives requests from users and is responsible for their analysis, information gathering, suggestion generation, booking arrangements, and result notifications. The server interacts with the following engines and external APIs.

[0874] 3. Emotion Analysis Engine

[0875] The sentiment analysis engine recognizes the emotional state from the user's request. For example, if the user includes the phrase "I expect," the sentiment analysis engine recognizes the emotion "expectation."

[0876] 4. Natural Language Processing Engine

[0877] A natural language processing (NLP) engine analyzes a user's request and extracts relevant keywords and intent. For example, if the user enters "I want to eat delicious udon in Tokyo," the NLP engine will extract the keywords "Tokyo," "udon," and "delicious."

[0878] 5. External API

[0879] External APIs provide restaurant information, ratings, delivery services, and transportation options. For example, the Google Places API is used to retrieve information on nearby restaurants and to refer to reviews and ratings.

[0880] Explanation of the process

[0881] Request reception and analysis

[0882] The user enters a request via their smartphone or smart glasses and sends it to the server. The server then passes the request to a natural language processing engine, which analyzes the keywords and intent.

[0883] Recognition of emotions

[0884] The server inputs the analyzed request into its sentiment analysis engine to recognize the user's emotional state. For example, if the emotion "expectation" is extracted from the request, the server optimizes its suggestions based on that.

[0885] Information gathering and proposals

[0886] The server uses external APIs to collect relevant information. For example, it retrieves information about restaurants, ratings, and delivery status. It analyzes the collected information and generates optimal suggestions based on the user's emotional state.

[0887] User notifications and choices

[0888] The server notifies the user terminal of the generated suggestions and assists the user in the process of selecting from them. The user's selection information is sent to the server, and reservations and delivery arrangements are made.

[0889] Completion notification

[0890] The server confirms the completion of the reservation and arrangement and notifies the user of the result. The user's terminal receives the notification and displays the result to the user.

[0891] Specific example

[0892] If a user enters a request saying, "I want to eat delicious udon in Tokyo":

[0893] 1. The server analyzes the request using a natural language processing engine and extracts the keywords "Tokyo," "udon," and "delicious."

[0894] 2. The emotion analysis engine recognizes the emotion of "expectation."

[0895] 3. The server generates suggestions tailored to the user based on information obtained from external APIs.

[0896] 4. Notify users of the following suggestions: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes".

[0897] 5. The user selects store A, and the server makes the reservation and delivery arrangements.

[0898] 6. The server sends a notification to the user that the arrangement is complete, and the user confirms it.

[0899] Example of a prompt

[0900] "I want to eat delicious udon in Tokyo."

[0901] "I'd like sushi delivered for the dinner I'm looking forward to."

[0902] "I'd like to order a pizza for lunch, but I'd like to check if it's being delivered quickly."

[0903] Thus, the system based on the present invention enables the provision of optimal services that take user emotions into consideration.

[0904] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0905] Step 1:

[0906] The device receives a request from the user. This request is entered in text format. An example of such a request might be, "I want to eat delicious udon in Tokyo." The device then sends this request to the server.

[0907] Step 2:

[0908] The server sends the received request to the natural language processing engine for analysis. The NLP engine extracts keywords and intent from the request, such as "in Tokyo," "udon," and "delicious." The analysis results are returned to the server and used in the next processing step.

[0909] Step 3:

[0910] The server sends the analysis results to the sentiment analysis engine, which recognizes and analyzes the emotional state included in the request. The sentiment analysis engine extracts emotions such as "expectation" from the context and returns the results to the server. The server then prepares to optimize the suggestions based on this emotional state.

[0911] Step 4:

[0912] The server uses multiple external APIs to collect relevant information. For example, it retrieves a list and basic information of "udon restaurants in Tokyo" from a restaurant information API, and collects customer reviews and the number of reviews for each restaurant from a rating information API. This collected data is then integrated on the server side.

[0913] Step 5:

[0914] The server generates a suggestion list based on the collected information. Here, based on the emotional state "expectation" provided by the sentiment analysis engine, stores with high customer reviews are prioritized and included in the list. The generated suggestion list is sent to the terminal in JSON format.

[0915] Step 6:

[0916] The device displays a list of received suggestions to the user. Examples of suggestions displayed include: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[0917] Step 7:

[0918] The user makes a selection from the presented suggestions. For example, "Select store A." The terminal sends this selection to the server.

[0919] Step 8:

[0920] The server makes reservations and necessary arrangements based on the user's selection. Specifically, it makes reservations for the selected store via an online reservation API and arranges delivery methods as needed. The results of the reservations and arrangements are stored on the server.

[0921] Step 9:

[0922] The server confirms the completion of the reservation and arrangement and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. Delivery is expected to arrive in 15 minutes."

[0923] Step 10:

[0924] The server sends this final confirmation information to the device. The device receives a completion notification and provides the user with the final confirmation information via push notification or display. This allows the user to confirm the completion of their reservation and arrangements.

[0925] In this way, users can receive the optimal food delivery service tailored to their emotional state.

[0926] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0927] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0928] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0929] [Third Embodiment]

[0930] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0931] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0932] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0933] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0934] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0935] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0936] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0937] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0938] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0939] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0940] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0941] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0942] Modes for carrying out the invention

[0943] This invention is a system that handles everything from information gathering and proposals to reservations and necessary arrangements, primarily based on user requests. The program of this system mainly consists of the following components, and its detailed processing is described below.

[0944] System components

[0945] 1. User terminal: Responsible for inputting requests and displaying results.

[0946] 2. Server: Responsible for parsing requests, collecting and analyzing information, generating suggestions, arranging reservations, and notifying results.

[0947] 3. External APIs: Provide restaurant information, reviews, reservation services, and transportation information.

[0948] Explanation of program processing

[0949] 1. Receiving user requests

[0950] The user enters a request into their device and sends it. For example, they might enter, "I want to eat delicious udon in Tokyo."

[0951] The terminal receives the request and sends it to the server. The server logs the received request.

[0952] 2. Intent Analysis using Natural Language Processing (NLP)

[0953] The server passes the received request to the natural language processing engine for analysis.

[0954] The NLP engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the request and analyzes the user's intent.

[0955] The server retrieves the analysis results and prepares the data for subsequent processing.

[0956] 3. Information Gathering and Analysis

[0957] The server collects relevant information through external APIs. Specifically, it retrieves data from sources such as restaurant information, customer reviews, and reservation services.

[0958] Example: The server collects information on udon restaurants in Tokyo from a restaurant API and obtains ratings for each restaurant from a review API. It also obtains availability and waiting times for each restaurant from a reservation API.

[0959] The server organizes the collected information and creates a list to present to the user.

[0960] 4. Suggestions for users

[0961] The server generates multiple suggestions for the user based on the organized information.

[0962] For example, the server might suggest options like "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" or "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[0963] The device receives the suggestion and displays it to the user.

[0964] 5. User Selection and Optimization

[0965] The user makes a selection from the presented suggestions. For example, "Select store A."

[0966] The device sends the user's selection to the server.

[0967] The server receives the selection and begins the reservation and necessary arrangements. Specifically, it makes a reservation at the selected store and, if necessary, arranges transportation (such as a taxi).

[0968] 6. Confirmation and notification of reservations and arrangements

[0969] The server confirms the completion of the reservation and arrangements and organizes the information.

[0970] Example: The server creates a notification such as "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and sends it to the device.

[0971] The device receives a completion notification and provides the user with final confirmation information (e.g., push notification or SMS).

[0972] Specific example

[0973] If a user requests "I want to eat delicious udon in Tokyo," the server will process it as follows:

[0974] The server analyzes the request and extracts keywords such as "Tokyo area," "udon," and "delicious."

[0975] The server collects information on target stores from external APIs and analyzes customer reviews and seating availability.

[0976] The server suggests multiple store options to the user.

[0977] The user selects store A from the suggested stores.

[0978] The server makes a reservation for store A and arranges a taxi if necessary.

[0979] The server notifies the terminal of the reservation completion and arrangement details, and the terminal informs the user of this.

[0980] In this way, users can seamlessly complete everything from information gathering to booking and arrangements through a single platform.

[0981] The following describes the processing flow.

[0982] Step 1:

[0983] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo." The device receives this request and sends it to the server.

[0984] Step 2:

[0985] The server sends the received request to the natural language processing engine. The natural language processing engine analyzes the request and extracts keywords and the user's intent (e.g., "in Tokyo", "udon", "delicious"). The server retrieves the analysis results and prepares the data for use in the next processing step.

[0986] Step 3:

[0987] The server uses multiple external APIs to collect relevant information. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. Furthermore, it obtains the availability and waiting times for each restaurant from a reservation API.

[0988] Step 4:

[0989] The server performs analysis based on the collected data and generates a list of suggestions to present to the user. For example, it creates a list that includes information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes".

[0990] Step 5:

[0991] The server sends the generated suggestion list to the terminal in JSON format. The terminal then displays the received suggestion list in an easy-to-understand format for the user.

[0992] Step 6:

[0993] The user selects their preferred store from the displayed suggestions. For example, "Select Store A." The terminal receives the user's selection and sends it to the server.

[0994] Step 7:

[0995] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. The server uses the reservation API to make reservations for the selected stores and, if necessary, arranges transportation (e.g., taxis) via the taxi API.

[0996] Step 8:

[0997] The server confirms that the reservation and arrangements are complete and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes." The server then sends this notification message to the terminal.

[0998] Step 9:

[0999] The device receives notifications sent from the server and provides the user with final confirmation information (e.g., push notifications or SMS). This allows the user to confirm that the reservation and arrangements are complete.

[1000] This series of processes allows users to seamlessly navigate the entire process, from making a request and gathering information to selecting suggestions, booking and arranging, and finally confirming their booking.

[1001] (Example 1)

[1002] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1003] Traditional systems require users to use multiple platforms when making requests, resulting in a lack of consistency from information gathering to booking. Furthermore, the lack of coordination between platforms leads to significant time and effort. Additionally, the system lacks the ability to accurately understand user intent and provide appropriate suggestions, potentially resulting in lower user satisfaction.

[1004] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1005] In this invention, the server includes means for receiving requests from users, means for analyzing the requests using natural language processing and extracting relevant keywords and intentions, means for collecting information from multiple external sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary transportation based on the selections, means for notifying the user of the completion of the reservations and arrangements, and means for logging the requests, extracted keywords, and intentions. This enables users to consistently and seamlessly perform tasks from information gathering to reservation arrangements.

[1006] "Means for receiving user requests" refers to the means by which a user enters requests or demands into the system and initiates processing within the system.

[1007] A "natural language processing tool" is a means of analyzing a user's input request and extracting relevant keywords and intent from the text.

[1008] "Information gathering means" refers to the means of collecting relevant information from multiple external sources and converting it into a format usable within the system.

[1009] A "proposal generation method" is a means of generating appropriate suggestions for the user based on collected and analyzed information, and presenting the results to the user.

[1010] A "selection acceptance mechanism" is a means for a user to make a selection from the presented suggestions, accept that selection, and reflect it in the next process.

[1011] A "reservation method" refers to a means of reserving a desired store or service based on the user's choice.

[1012] "Means of arranging transportation" refers to the means of arranging the necessary means of travel in conjunction with a reservation.

[1013] "Notification means" refers to the means by which the system transmits information to inform the user that the reservation and arrangements have been completed.

[1014] A "logging method" is a means of recording user requests, extracted keywords, intentions, etc., within the system for later analysis and auditing.

[1015] This invention provides a system that allows users to seamlessly access a series of services. This system consistently handles information gathering, analysis, suggestions, reservations, and necessary arrangements, all centered around the user's request. Detailed embodiments of this system are described below.

[1016] Hardware and software to be used

[1017] 1. User terminal:

[1018] User devices can include smartphones, tablets, and personal computers. These devices receive requests from users and send them to the server. They also display suggestions and notifications to the user.

[1019] 2. Server:

[1020] The server oversees a series of processes, from analyzing user requests and gathering information to generating suggestions, arranging reservations, and sending notifications. Specifically, it uses the following software:

[1021] Natural Language Processing Engine (NLP): Utilizes tools such as the Google Cloud Natural Language API to understand user requests and extract relevant keywords and intents.

[1022] External APIs: Restaurant information, ratings, availability, reservation status, and transportation information are obtained via APIs such as the Yelp API and Google Places API.

[1023] Process Overview

[1024] 1. Receiving user requests:

[1025] The user enters a request through their device. For example, they might request, "I want to eat delicious udon in Tokyo." This request is sent from the device to the server.

[1026] 2. Intent analysis using natural language processing:

[1027] After receiving a request, the server passes the utterance to a natural language processing engine. The engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the request and analyzes the user's intent. The server retrieves the analysis results and prepares queries for the next data collection.

[1028] 3. Information gathering and analysis:

[1029] The server sends queries to external APIs to collect and analyze necessary information. Specifically, it uses the Yelp API to obtain a list of udon restaurants in Tokyo and the Google Places API to obtain information on the availability of seats and customer reviews for each restaurant.

[1030] Based on the information obtained, a list of suggestions for the user is generated.

[1031] 4. Suggestions for users:

[1032] The server generates several suggestions based on the collected information and sends them to the terminal. An example suggestion might be, "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes." The terminal then displays this to the user.

[1033] 5. User Selection and Optimization:

[1034] When a user makes a selection from a list of suggestions, for example, by choosing "Store A," the terminal sends the selection to the server. The server receives the selection and proceeds with the reservation process.

[1035] 6. Confirmation and notification of reservations and arrangements:

[1036] Once the server completes the reservation and necessary arrangements (e.g., taxi booking), it compiles the reservation confirmation information and sends it to the device. The device then notifies the user of the reservation completion details (e.g., via push notification or SMS).

[1037] Specific example

[1038] The specific processing flow when a user enters a request such as "I want to eat delicious udon in Tokyo" is as follows:

[1039] The server analyzes the request using an NLP engine and extracts keywords such as "Tokyo," "udon," and "delicious."

[1040] The server collects and analyzes information about the target stores from the Yelp API and Google Places API.

[1041] Based on the information collected by the server, it suggests potential stores to the user, such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes."

[1042] The user selects "Store A," the server makes a reservation for that store, and arranges a taxi if necessary.

[1043] The server notifies the user's terminal with reservation confirmation and taxi arrangement details.

[1044] Example of a prompt

[1045] Examples of prompts that users might enter include the following:

[1046] "I want to eat delicious udon in Tokyo."

[1047] "Can you recommend a good cafe for the weekend?"

[1048] "I'm looking for a yakiniku restaurant that's open tonight."

[1049] In this way, this system can efficiently and seamlessly handle everything from information gathering to arrangement in response to user requests.

[1050] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1051] Program processing steps

[1052] Step 1: Receiving User Requests

[1053] Specific actions:

[1054] The user enters the prompt "I want to eat delicious udon in Tokyo" into the terminal and sends it. The terminal sends this input to the server as an HTTP request. The server receives the HTTP request and logs its contents. The log includes the request content, a timestamp, and the user's IP address.

[1055] input:

[1056] User prompt (e.g., "I want to eat delicious udon in Tokyo")

[1057] output:

[1058] The prompt message received by the server, and the request content recorded in the log.

[1059] Step 2: Intent Analysis using Natural Language Processing

[1060] Specific actions:

[1061] The server sends the received prompt text to a natural language processing engine (e.g., Google Cloud Natural Language API). The NLP engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the prompt text. The server retrieves the analysis results from the NLP engine and prepares queries containing these keywords for subsequent data collection.

[1062] input:

[1063] Received prompt message

[1064] output:

[1065] Extracted keywords (e.g., "Tokyo area", "udon", "delicious")

[1066] Step 3: Information Gathering and Analysis

[1067] Specific actions:

[1068] The server sends queries to external information source APIs using the extracted keywords. For example, it might retrieve a list of udon restaurants in Tokyo and their ratings via the Yelp API, and obtain the availability and waiting times for each restaurant via the Google Places API. The server receives this information, organizes and analyzes it, and generates a list to present to the user.

[1069] input:

[1070] Extracted keywords

[1071] output:

[1072] Store information and analyzed data (e.g., store name, customer reviews, seating availability, waiting time)

[1073] Step 4: Proposal to the user

[1074] Specific actions:

[1075] The server generates multiple suggestions based on the collected and analyzed store information. The generated suggestions are sent to the terminal in JSON format. The terminal parses the received JSON data and displays the suggestions in a user-friendly format. For example, it might display information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes."

[1076] input:

[1077] Organized and analyzed store information

[1078] output:

[1079] A list of suggestions for users (Example: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes")

[1080] Step 5: User Selection and Optimization

[1081] Specific actions:

[1082] The user makes a selection from a list of suggestions, for example, by selecting "Store A." The terminal sends the user's selection to the server. The server receives the selection and begins arranging a reservation for the selected store. Specifically, it sends a request to the store's reservation API and makes the reservation.

[1083] input:

[1084] User selection (e.g., "Store A")

[1085] output:

[1086] Submitting a booking request and booking confirmation information

[1087] Step 6: Confirmation and notification of booking / arrangement

[1088] Specific actions:

[1089] The server confirms the completion of the reservation and arrangement and organizes the information. The server sends the organized information to the terminal in JSON format. The terminal parses the received JSON data and notifies the user of the reservation and arrangement completion (e.g., push notification or SMS). The reservation confirmation information includes the reservation number, estimated taxi arrival time, etc.

[1090] input:

[1091] Reservation confirmation information

[1092] output:

[1093] Notifications and detailed information to the user (e.g., "Reservation complete. Reservation number 12345. Taxi is expected to arrive in 15 minutes.")

[1094] In this way, the system can efficiently and seamlessly handle everything from information gathering to arrangement in response to user requests.

[1095] (Application Example 1)

[1096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1097] In recent years, there has been a growing demand for systems that allow users to easily order and reserve meals. However, current food delivery platforms require the use of multiple applications and services, making the process cumbersome. Therefore, there is a need to develop a system that allows users to handle everything from requests and orders to delivery arrangements and status notifications on a single platform.

[1098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1099] In this invention, the server includes means for receiving requests from users, means for analyzing requests using natural language processing and extracting relevant keywords and intents, means for collecting information from multiple external sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations, orders, and arrangements for necessary services based on the selections, means for notifying the user of the completion of the reservations and arrangements, and means for notifying the user of delivery status and order status in real time. This enables users to seamlessly perform everything from making requests to ordering, making arrangements, and checking the status on a single platform.

[1100] "Means for receiving requests from users" refers to the part of a system that sends requests entered by users via devices such as smartphones and computers to a server and receives those requests.

[1101] "Natural language processing" is a technology that analyzes text data entered by users to extract keywords and intent.

[1102] "External information sources" refer to data sources provided from outside the system, such as restaurant information, customer reviews, reservation services, and delivery services.

[1103] "Information gathering means" refers to the part that has the function of acquiring and analyzing data from external information sources.

[1104] The "means of suggestion" refers to the function that provides users with multiple options based on the information collected and analyzed.

[1105] "Means of making reservations and arrangements" refers to the part of the system that has the functionality to make restaurant reservations and arrange delivery based on the options selected by the user.

[1106] "Notification means" refers to the function that informs users in real time about the completion of reservations and arrangements, as well as the delivery status.

[1107] "Restaurant information" refers to information about food and beverage establishments such as restaurants and cafes, and specifically includes the name of the establishment, address, business hours, menu, prices, and ratings.

[1108] "Word-of-mouth information" refers to feedback such as evaluations and impressions provided by users and customers.

[1109] "Ordering service" refers to the function or service that allows customers to order food from restaurants online.

[1110] "Delivery method" refers to the arrangements and means of transportation used to deliver the ordered meal to the location specified by the user.

[1111] This invention is a system that automatically suggests, reserves, orders, and arranges delivery of the most suitable restaurant based on a user's meal request. This system consists of a user terminal, a server, and multiple external information sources.

[1112] 1. System Program Overview

[1113] The server has the following functions:

[1114] Natural language processing engine (spaCy library)

[1115] Data collection from external sources (Google Places API, Yelp API, etc.)

[1116] Data analysis and proposal generation.

[1117] Reservation and order arrangement

[1118] Real-time notifications (Firebase Cloud Messaging)

[1119] User devices (primarily smartphones) have the following functions:

[1120] User Request Input Form

[1121] Display the proposal list

[1122] Receiving real-time notifications

[1123] 2. Program Processing

[1124] The server receives requests sent by users and analyzes them using a natural language processing engine. This analysis extracts keywords (e.g., "spicy food") from the request and understands its intent.

[1125] Based on analyzed keywords, the system collects restaurant information, reviews, and reservation data from external sources (such as the Google Places API and Yelp API). Based on this collected information, data analysis is performed to generate a list of restaurants best suited to the user.

[1126] When a user selects a restaurant from the suggested list, the server arranges the reservation and order with the selected restaurant. Furthermore, if delivery is required, it also arranges the delivery. Once these arrangements are complete, the server uses Firebase Cloud Messaging to notify the user in real time of the delivery and order status.

[1127] 3. Specific examples

[1128] When a user enters "I want to eat spicy Indian food" into a smartphone app, the server extracts the keywords "spicy" and "Indian food," and gathers and suggests the most suitable restaurant information. If the user selects "Indian Restaurant A" from the suggestions, the order is confirmed and delivery is arranged. This allows the user to track the status of their order in real time until delivery is complete.

[1129] Example of a prompt

[1130] "I'd like to eat something spicy nearby."

[1131] "I'd like to order Indian food for delivery."

[1132] "What Asian dishes are popular today?"

[1133] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1134] Step 1:

[1135] The process by which the user enters a request.

[1136] Input: The user enters a request into the smartphone app saying, "I want to eat spicy Indian food."

[1137] Specific action: Enter text into the input form on the user's terminal and press the submit button.

[1138] Output: The user terminal sends this request data to the server.

[1139] Step 2:

[1140] A process that receives a request and performs natural language processing.

[1141] Input: Request data sent from the user's terminal.

[1142] Specific operation: The server receives the request and passes the request text to the natural language processing engine (spaCy).

[1143] Output: Keywords extracted by the natural language processing engine (e.g., "spicy", "Indian food") and intent are returned.

[1144] Step 3:

[1145] Information gathering process

[1146] Input: Keywords extracted by the natural language processing engine.

[1147] Specific operation: The server calls external information sources (such as the Google Places API and Yelp API) to collect restaurant information, reviews, and reservation status related to keywords.

[1148] Output: Collected restaurant information, customer reviews, and reservation status data.

[1149] Step 4:

[1150] Process to generate a list of suggestions

[1151] Input: Collected restaurant information, customer reviews, and reservation status data.

[1152] Specific operation: The server analyzes the collected data, selects the most suitable restaurants for the user, and generates a list of suggestions.

[1153] Output: Suggestion data listing multiple restaurant options.

[1154] Step 5:

[1155] Process to display a list of suggestions to the user.

[1156] Input: Proposal list data.

[1157] Specific operation: The server sends the suggestion list data to the user's terminal, and the terminal displays the list.

[1158] Output: A list of suggested restaurants displayed on the user's terminal.

[1159] Step 6:

[1160] The process of a user selecting a restaurant from suggestions.

[1161] Input: The displayed list of suggestions.

[1162] Specific action: The user selects a restaurant from the suggested list and presses the select button.

[1163] Output: The user terminal sends information about the selected restaurant to the server.

[1164] Step 7:

[1165] Processing to arrange reservations and orders.

[1166] Input: Information about the restaurant selected by the user.

[1167] Specific operation: The server makes reservations and orders for selected restaurants through an online reservation and ordering system (external API).

[1168] Output: Reservation confirmation and order processing completion data.

[1169] Step 8:

[1170] Process to notify the user of the completion of the reservation and order.

[1171] Input: Reservation confirmation and order processing completion data.

[1172] Specific operation: The server sends a notification to the user in real time via Firebase Cloud Messaging (FCM). The device receives the notification and displays it to the user.

[1173] Output: Reservation and order completion notifications displayed on the user's terminal.

[1174] Step 9:

[1175] A process to notify delivery status in real time.

[1176] Input: Delivery arrangement progress information.

[1177] Specific operation: The server updates the delivery status in real time based on progress information obtained from the delivery system and notifies the user using Firebase Cloud Messaging.

[1178] Output: Delivery status notification displayed in real time on the user's device.

[1179] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1180] Modes for carrying out the invention

[1181] This invention is a system that receives user requests, recognizes and analyzes the user's emotional state using an emotion engine, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that analysis. The program of this system consists of the following components, and its detailed processing is described below.

[1182] System components

[1183] 1. User terminal: Responsible for inputting requests and displaying results.

[1184] 2. Server: Responsible for request analysis, sentiment recognition, information gathering and analysis, suggestion generation, reservation arrangement, and result notification.

[1185] 3. External APIs: Provide restaurant information, reviews, reservation services, and transportation information.

[1186] 4. Emotion Engine: Extracts emotions from user requests and optimizes suggestions.

[1187] Explanation of program processing

[1188] 1. Receiving user requests

[1189] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo."

[1190] The terminal receives this request and sends it to the server. The server logs the received request.

[1191] 2. Intention analysis and sentiment recognition using natural language processing (NLP)

[1192] The server sends the received request to the natural language processing engine for analysis.

[1193] The natural language processing engine extracts keywords and intents from the request, such as "in Tokyo," "udon," and "delicious."

[1194] At the same time, the emotion engine recognizes the user's emotional state (e.g., joy, anticipation, fatigue) from the words and context included in the request.

[1195] The server retrieves these analysis results and sentiment data, and prepares the data for use in the next processing step.

[1196] 3. Information Gathering and Analysis

[1197] The server uses multiple external APIs to collect relevant information. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. Furthermore, it obtains the availability and waiting times for each restaurant from a reservation API.

[1198] The server performs analysis based on the collected data and generates a list of suggestions to present to the user. Here, the suggestions are optimized based on the results of the emotion engine's analysis. For example, if the user is feeling fatigued, the server prioritizes suggesting stores with short waiting times.

[1199] 4. Suggestions for users

[1200] The server sends the generated suggestion list to the terminal in JSON format. The terminal then displays the received suggestion list in an easy-to-understand format for the user.

[1201] For example, the server might suggest "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes." The system also optimizes the suggestions based on the user's emotional state.

[1202] 5. User Selection and Optimization

[1203] The user selects their preferred store from the displayed suggestions. For example, "Select Store A."

[1204] The terminal receives the user's selection and sends it to the server.

[1205] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. Specifically, it makes reservations at the selected establishments and, if necessary, arranges transportation (such as taxis).

[1206] 6. Confirmation and notification of reservations and arrangements

[1207] The server confirms the completion of the reservation and arrangement and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is scheduled to arrive in 15 minutes."

[1208] The server sends this notification to the device. The device receives the completion notification and provides the user with final confirmation information (e.g., push notification or SMS). The user can then confirm that the reservation and arrangements are complete.

[1209] Specific example

[1210] If a user requests "I want to eat delicious udon in Tokyo," the server will process it as follows:

[1211] The server analyzes the request and extracts keywords such as "Tokyo," "udon," and "delicious." At the same time, the emotion engine recognizes the user's emotional state from the request as "expectation."

[1212] The server collects target store information from an external API and analyzes customer reviews and seating availability. The emotion engine prioritizes and suggests highly-rated stores based on the emotion of "expectation."

[1213] The server generates a list of suggestions that include information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes," and presents them to the user.

[1214] The user selects store A from the suggested stores. The server makes a reservation for store A and arranges a taxi if necessary.

[1215] The server notifies the terminal of the reservation completion and arrangement details, and the terminal informs the user of this.

[1216] In this way, users can receive more personalized and optimal services that take their own emotional state into consideration.

[1217] The following describes the processing flow.

[1218] Step 1:

[1219] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo." The device receives this request and sends it to the server. The server logs the received request.

[1220] Step 2:

[1221] The server sends the received request to the natural language processing engine for analysis. The natural language processing engine extracts keywords and intent from the request, such as "in Tokyo," "udon," and "delicious." Simultaneously, the sentiment engine recognizes the user's emotional state from the words and context included in the request. For example, it extracts emotions such as "expectation" or "fatigue." The server obtains these analysis results and sentiment data and prepares the data for use in the next processing step.

[1222] Step 3:

[1223] The server collects relevant information using multiple external APIs. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. It also uses a reservation API to obtain the availability and waiting times for each restaurant. The server organizes and integrates this information to form recommendations for users.

[1224] Step 4:

[1225] The server generates a list of suggestions to present to the user based on the collected and organized information. In this process, it optimizes the suggestions using the results of the emotion engine's analysis. For example, if the user's emotional state is "fatigued," it prioritizes suggesting stores with short waiting times. Also, if the user expresses feelings of "anticipation," it prioritizes highly-rated stores.

[1226] Step 5:

[1227] The server sends the generated list of suggestions to the terminal in JSON format. The terminal then displays the received list of suggestions in an easy-to-understand format for the user. Specifically, suggestions are presented in the following format: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[1228] Step 6:

[1229] The user selects their preferred store from the displayed suggestions. For example, "Select Store A." The terminal receives the user's selection and sends it to the server.

[1230] Step 7:

[1231] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. Specifically, it calls the reservation API to make a reservation at the selected store. It also uses the taxi API to arrange transportation (such as a taxi) as needed. For example, it might use the reservation API to secure a reservation at store A and the taxi API to arrange transportation for the user.

[1232] Step 8:

[1233] The server confirms that the reservation and arrangements are complete and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes." The server then sends this notification message to the terminal.

[1234] Step 9:

[1235] The device receives notifications sent from the server and provides the user with final confirmation information. Specifically, users are notified via push notifications or SMS. This allows users to confirm that their reservation and arrangements are complete.

[1236] Through this series of steps, users can seamlessly receive more personalized and optimal services that even take their emotional state into consideration.

[1237] (Example 2)

[1238] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1239] Traditional information gathering and suggestion systems have a problem in that they provide simple information without considering the user's emotions, making it difficult to improve user satisfaction. Furthermore, there was a lack of methods to integrate information from multiple external sources and provide optimal suggestions based on the user's specific needs. As a result, it was difficult for users to obtain the necessary information quickly and accurately, leading to a decline in the quality of the user experience.

[1240] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1241] In this invention, the server includes means for receiving requests from users, means for analyzing requests using natural language processing and extracting relevant keywords and intentions, means for an emotion engine that analyzes the user's emotional state, means for collecting information from multiple external information sources, analyzing the acquired information and making suggestions to the user, means for optimizing the suggestions based on the emotional state, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary services based on the selection, and means for notifying the user of the completion of the reservations and arrangements. This enables more personalized and optimal information provision and service arrangements that take the user's emotions into consideration.

[1242] "Means of receiving requests" refers to devices or software that receive input from users and transfer it to a server.

[1243] "Natural language processing" is the process of analyzing incoming requests and extracting keywords and intent.

[1244] "Means for extracting relevant keywords and intent" refers to engines or software that identify important words and user intent from requests based on natural language processing.

[1245] An "emotional engine" refers to algorithms or software used to recognize and analyze a user's emotional state based on their request content and context.

[1246] "Means of collecting information, analyzing the acquired information, and making suggestions to users" refers to the process or engine for acquiring data from external sources, analyzing it, and creating appropriate suggestions for users.

[1247] "Methods for optimizing suggestions based on emotional state" refer to methods or software that take the user's emotional state into consideration and select the most appropriate suggestion from the collected information.

[1248] "Means for receiving user selections" refers to the interface or software used to receive a user's selection from the provided suggestions and communicate that selection to the server.

[1249] "Means for making reservations and arranging necessary services" refers to systems or software that automatically handle reservation procedures and the arrangement of related services based on the user's selection.

[1250] "Means of notifying users of the completion of reservations and arrangements" refers to notification systems or software that inform users when reservations or arrangements have been completed.

[1251] "External information sources" refer to external APIs and databases such as restaurant information, customer review information, reservation services, and transportation information.

[1252] A "system" is a set of hardware and software configurations that operate by integrating the above-mentioned means and processes.

[1253] Modes for carrying out the invention

[1254] This invention is a system that receives user requests, recognizes and analyzes the user's emotional state using an emotion engine, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that analysis. This system consists of the following components, and its detailed processing is described below.

[1255] System components

[1256] 1. User terminal: Responsible for inputting requests and displaying results.

[1257] 2. Server: Responsible for request analysis, sentiment recognition, information gathering and analysis, suggestion generation, reservation arrangement, and result notification.

[1258] 3. External information sources: Provide information on restaurants, customer reviews, reservation services, and transportation options.

[1259] 4. Emotion Engine: Extracts emotions from user requests and optimizes suggestions.

[1260] Specific operation of the system

[1261] Receiving user requests

[1262] When a user enters a request into their device and sends it, the device sends that request to the server. For example, if a user enters "I want to eat delicious udon in Tokyo," the device generates an HTTP request and sends it to the server. The server receives this request and logs it.

[1263] Intent analysis and emotion recognition using Natural Language Processing (NLP)

[1264] The server sends received requests to a natural language processing engine (e.g., SpaCy or Google Cloud NLP) for analysis. The NLP engine extracts keywords such as "Tokyo," "udon," and "delicious," analyzes the context, and understands the intent. An emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotional state (e.g., "expectation") from the words and context included in the request.

[1265] Information gathering and analysis

[1266] The server collects relevant information using multiple external sources (e.g., restaurant API, review API, reservation API). Specifically, it obtains a list of udon restaurants in Tokyo and basic information from the restaurant API, ratings and review counts for each restaurant from the review API, and availability and waiting times for restaurants from the reservation API. Based on this data, the server performs analysis and generates a list of suggestions to present to the user. Here, the suggestions are optimized based on the results of the emotion engine's analysis. For example, if the user is feeling tired, the server prioritizes suggesting restaurants with short waiting times.

[1267] Suggestions for users

[1268] The server encodes the generated suggestion list in JSON format and sends it to the terminal. The terminal parses the received data and displays it in the user interface. For example, suggestions such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes" may be displayed.

[1269] User selection and optimization

[1270] When a user selects a desired store from the device's suggestion list, the device sends that selection to the server. The server makes a reservation for the selected store and arranges transportation (e.g., a taxi) if necessary. Specifically, the server accesses a reservation API, makes a reservation, and uses a transportation API (e.g., Uber API) to arrange a taxi.

[1271] Reservation and arrangement confirmation and notification

[1272] The server retrieves information about completed reservations and arrangements and generates a notification for the user. For example, it might generate a notification such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and send it to the device. The device receives this notification and sends it to the user via push notification or SMS.

[1273] Examples of prompt statements

[1274] You can obtain a detailed explanation of the system by inputting the following prompt sentences into the generating AI model:

[1275] Please describe in detail the system's processing steps from the moment a user enters a request such as "I want to eat delicious udon in Tokyo," through which the emotion engine analyzes the user's emotions, collects and analyzes information from external sources, generates optimal suggestions, makes reservations and arrangements, and notifies the user.

[1276] This will enable a system that provides more personalized and optimal services, taking into account the user's emotional state.

[1277] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1278] Step 1: Receiving User Requests

[1279] The user enters a request into their device saying, "I want to eat delicious udon in Tokyo."

[1280] The terminal receives input, generates an HTTP request, and sends it to the server. The input is the user's text input, and the output is the data sent to the server.

[1281] The server receives this request at the API endpoint and records it in a log file or database. The input is an HTTP request from the terminal, and the output is the log entry.

[1282] Step 2: Intent analysis and emotion recognition using Natural Language Processing (NLP)

[1283] The server sends the received request to the natural language processing engine for analysis. The input is the user's request, and the output is the NLP analysis result.

[1284] A natural language processing engine (e.g., SpaCy or Google Cloud NLP) extracts keywords and intent from a request, such as "in Tokyo," "udon," and "delicious." The input is the text data of the request, and the output is the extracted keywords and intent.

[1285] An emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotional state (e.g., "expectation") from the words and context included in the request. The input is the text data of the request, and the output is the emotional state data.

[1286] Step 3: Information Gathering and Analysis

[1287] The server collects relevant information from multiple external sources. Specifically, it utilizes restaurant APIs, review APIs, and reservation APIs. The input consists of analyzed keywords and intent, while the output is the collected information.

[1288] This system retrieves a list of udon restaurants in Tokyo and their basic information from a restaurant API, and collects each restaurant's rating and review count from a review API. The input is the requested conditions, and the output is a list of restaurants and rating data.

[1289] This system uses a reservation API to retrieve seat availability and waiting times for each store. Input is store information, and output is seat availability and waiting time data.

[1290] The server integrates this data and generates a list of suggestions to present to the user, taking into account the results of the sentiment engine. Input is data from multiple APIs, and output is a list of suggestions.

[1291] Step 4: Proposal to the user

[1292] The server sends the generated proposal list to the terminal in JSON format. The input is the proposal list, and the output is the transmitted data.

[1293] The terminal analyzes the received data and displays it on the user interface. The input is a list of suggestions from the server, and the output is what is displayed to the user.

[1294] For example, "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes" will be displayed.

[1295] Step 5: User Selection and Optimization

[1296] The user selects their preferred store from the suggested stores. The input is the suggested list, and the output is the user's selection.

[1297] The terminal receives the user's selection and sends it to the server. The input is the user's selection, and the output is the data sent to the server.

[1298] The server makes reservations for the selected restaurant and arranges a taxi if necessary. Specifically, it accesses a reservation API, completes the reservation process, and uses a transportation API (e.g., Uber API) to arrange a taxi. Input is the user's selection information, and output is reservation and arrangement data.

[1299] Step 6: Confirmation and notification of booking / arrangement

[1300] The server confirms the completion of reservations and arrangements and organizes the information. Inputs are response data from the reservation API and the transportation API, and outputs are completion notifications.

[1301] The server generates a notification message such as "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and sends it to the terminal. The input is the booking status data, and the output is the notification data.

[1302] The device receives a completion notification and provides the user with final confirmation information. Specifically, this is done via push notification or SMS. The input is notification data from the server, and the output is the notification to the user.

[1303] The above is a detailed explanation of the system's program processing flow.

[1304] (Application Example 2)

[1305] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1306] In recent years, providing services tailored to individual user needs has become increasingly important. However, conventional request processing systems have been unable to consider users' emotional states when making suggestions, limiting the improvement of user satisfaction. Similarly, in food delivery services, the lack of optimal suggestions and procedures that align with users' emotions has been a problem.

[1307] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving requests from users, means for analyzing the requests using natural language processing and extracting relevant keywords and intentions, means for recognizing the user's emotional state from the requests using an emotion analysis engine, means for collecting information from multiple external information sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary services based on the selections, and means for notifying the user of the completion of the reservations and arrangements. This makes it possible to provide optimal services and food delivery procedures that take into account the user's emotional state.

[1308] "Means for receiving user requests" refers to interfaces and systems for receiving requests and questions entered by users through their devices.

[1309] "Means for analyzing using natural language processing and extracting relevant keywords and intents" refers to technologies that analyze user requests, understand the context and meaning of words, and find appropriate keywords and intents.

[1310] "A means of recognizing a user's emotional state from a request using an emotion analysis engine" refers to a technology that analyzes the emotional aspects of a user's request and identifies emotional states such as joy, anticipation, and fatigue.

[1311] "A means of collecting information from multiple external sources, analyzing the acquired information, and making suggestions to the user" refers to a technology that collects information from external sources through APIs and databases, analyzes it, and makes the most suitable suggestions to the user.

[1312] "Means for receiving user selections based on the aforementioned proposals" refers to an interface and system for presenting proposals to the user and receiving the user's selection from among them.

[1313] "Means for making reservations and arranging necessary services based on the aforementioned selection" refers to systems and technologies that automatically perform reservation procedures and arrange related services according to the user's selection.

[1314] "Means for notifying the user of the completion of the reservation and arrangement" refers to a system and interface for notifying the user of the completion information of the reservation and arrangement.

[1315] "Restaurant information" refers to basic information such as the restaurant's location, menu, business hours, and contact information.

[1316] "Rating information" refers to reviews and ratings of restaurants and services provided by users and reviewers.

[1317] A "delivery service" is a service that aims to deliver meals or goods ordered by users to a specified location.

[1318] "Transportation" includes means of enabling the movement of users and goods, such as taxis, buses, and sharing services.

[1319] This invention is a system that receives requests from users, recognizes the user's emotional state through sentiment analysis, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that. Specific embodiments for realizing this system will be described below.

[1320] System Configuration

[1321] This system consists of the following components:

[1322] 1. User terminal

[1323] User devices include smartphones and smart glasses. Users use these devices to enter requests and view and select suggested information.

[1324] 2. Server

[1325] The server receives requests from users and is responsible for their analysis, information gathering, suggestion generation, booking arrangements, and result notifications. The server interacts with the following engines and external APIs.

[1326] 3. Emotion Analysis Engine

[1327] The sentiment analysis engine recognizes the emotional state from the user's request. For example, if the user includes the phrase "I expect," the sentiment analysis engine recognizes the emotion "expectation."

[1328] 4. Natural Language Processing Engine

[1329] A natural language processing (NLP) engine analyzes a user's request and extracts relevant keywords and intent. For example, if the user enters "I want to eat delicious udon in Tokyo," the NLP engine will extract the keywords "Tokyo," "udon," and "delicious."

[1330] 5. External API

[1331] External APIs provide restaurant information, ratings, delivery services, and transportation options. For example, the Google Places API is used to retrieve information on nearby restaurants and to refer to reviews and ratings.

[1332] Explanation of the process

[1333] Request reception and analysis

[1334] The user enters a request via their smartphone or smart glasses and sends it to the server. The server then passes the request to a natural language processing engine, which analyzes the keywords and intent.

[1335] Recognition of emotions

[1336] The server inputs the analyzed request into its sentiment analysis engine to recognize the user's emotional state. For example, if the emotion "expectation" is extracted from the request, the server optimizes its suggestions based on that.

[1337] Information gathering and proposals

[1338] The server uses external APIs to collect relevant information. For example, it retrieves information about restaurants, ratings, and delivery status. It analyzes the collected information and generates optimal suggestions based on the user's emotional state.

[1339] User notifications and choices

[1340] The server notifies the user terminal of the generated suggestions and assists the user in the process of selecting from them. The user's selection information is sent to the server, and reservations and delivery arrangements are made.

[1341] Completion notification

[1342] The server confirms the completion of the reservation and arrangement and notifies the user of the result. The user's terminal receives the notification and displays the result to the user.

[1343] Specific example

[1344] If a user enters a request saying, "I want to eat delicious udon in Tokyo":

[1345] 1. The server analyzes the request using a natural language processing engine and extracts the keywords "Tokyo," "udon," and "delicious."

[1346] 2. The emotion analysis engine recognizes the emotion of "expectation."

[1347] 3. The server generates suggestions tailored to the user based on information obtained from external APIs.

[1348] 4. Notify users of the following suggestions: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes".

[1349] 5. The user selects store A, and the server makes the reservation and delivery arrangements.

[1350] 6. The server sends a notification to the user that the arrangement is complete, and the user confirms it.

[1351] Example of a prompt

[1352] "I want to eat delicious udon in Tokyo."

[1353] "I'd like sushi delivered for the dinner I'm looking forward to."

[1354] "I'd like to order a pizza for lunch, but I'd like to check if it's being delivered quickly."

[1355] Thus, the system based on the present invention enables the provision of optimal services that take user emotions into consideration.

[1356] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1357] Step 1:

[1358] The device receives a request from the user. This request is entered in text format. An example of such a request might be, "I want to eat delicious udon in Tokyo." The device then sends this request to the server.

[1359] Step 2:

[1360] The server sends the received request to the natural language processing engine for analysis. The NLP engine extracts keywords and intent from the request, such as "in Tokyo," "udon," and "delicious." The analysis results are returned to the server and used in the next processing step.

[1361] Step 3:

[1362] The server sends the analysis results to the sentiment analysis engine, which recognizes and analyzes the emotional state included in the request. The sentiment analysis engine extracts emotions such as "expectation" from the context and returns the results to the server. The server then prepares to optimize the suggestions based on this emotional state.

[1363] Step 4:

[1364] The server uses multiple external APIs to collect relevant information. For example, it retrieves a list and basic information of "udon restaurants in Tokyo" from a restaurant information API, and collects customer reviews and the number of reviews for each restaurant from a rating information API. This collected data is then integrated on the server side.

[1365] Step 5:

[1366] The server generates a suggestion list based on the collected information. Here, based on the emotional state "expectation" provided by the sentiment analysis engine, stores with high customer reviews are prioritized and included in the list. The generated suggestion list is sent to the terminal in JSON format.

[1367] Step 6:

[1368] The device displays a list of received suggestions to the user. Examples of suggestions displayed include: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[1369] Step 7:

[1370] The user makes a selection from the presented suggestions. For example, "Select store A." The terminal sends this selection to the server.

[1371] Step 8:

[1372] The server makes reservations and necessary arrangements based on the user's selection. Specifically, it makes reservations for the selected store via an online reservation API and arranges delivery methods as needed. The results of the reservations and arrangements are stored on the server.

[1373] Step 9:

[1374] The server confirms the completion of the reservation and arrangement and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. Delivery is expected to arrive in 15 minutes."

[1375] Step 10:

[1376] The server sends this final confirmation information to the device. The device receives a completion notification and provides the user with the final confirmation information via push notification or display. This allows the user to confirm the completion of their reservation and arrangements.

[1377] In this way, users can receive the optimal food delivery service tailored to their emotional state.

[1378] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1379] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1380] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1381] [Fourth Embodiment]

[1382] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1383] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1384] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1385] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1386] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1387] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1388] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1389] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1390] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1391] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1392] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1393] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1394] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1395] Modes for carrying out the invention

[1396] This invention is a system that handles everything from information gathering and proposals to reservations and necessary arrangements, primarily based on user requests. The program of this system mainly consists of the following components, and its detailed processing is described below.

[1397] System components

[1398] 1. User terminal: Responsible for inputting requests and displaying results.

[1399] 2. Server: Responsible for parsing requests, collecting and analyzing information, generating suggestions, arranging reservations, and notifying results.

[1400] 3. External APIs: Provide restaurant information, reviews, reservation services, and transportation information.

[1401] Explanation of program processing

[1402] 1. Receiving user requests

[1403] The user enters a request into their device and sends it. For example, they might enter, "I want to eat delicious udon in Tokyo."

[1404] The terminal receives the request and sends it to the server. The server logs the received request.

[1405] 2. Intent Analysis using Natural Language Processing (NLP)

[1406] The server passes the received request to the natural language processing engine for analysis.

[1407] The NLP engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the request and analyzes the user's intent.

[1408] The server retrieves the analysis results and prepares the data for subsequent processing.

[1409] 3. Information Gathering and Analysis

[1410] The server collects relevant information through external APIs. Specifically, it retrieves data from sources such as restaurant information, customer reviews, and reservation services.

[1411] Example: The server collects information on udon restaurants in Tokyo from a restaurant API and obtains ratings for each restaurant from a review API. It also obtains availability and waiting times for each restaurant from a reservation API.

[1412] The server organizes the collected information and creates a list to present to the user.

[1413] 4. Suggestions for users

[1414] The server generates multiple suggestions for the user based on the organized information.

[1415] For example, the server might suggest options like "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" or "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[1416] The device receives the suggestion and displays it to the user.

[1417] 5. User Selection and Optimization

[1418] The user makes a selection from the presented suggestions. For example, "Select store A."

[1419] The device sends the user's selection to the server.

[1420] The server receives the selection and begins the reservation and necessary arrangements. Specifically, it makes a reservation at the selected store and, if necessary, arranges transportation (such as a taxi).

[1421] 6. Confirmation and notification of reservations and arrangements

[1422] The server confirms the completion of the reservation and arrangements and organizes the information.

[1423] Example: The server creates a notification such as "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and sends it to the device.

[1424] The device receives a completion notification and provides the user with final confirmation information (e.g., push notification or SMS).

[1425] Specific example

[1426] If a user requests "I want to eat delicious udon in Tokyo," the server will process it as follows:

[1427] The server analyzes the request and extracts keywords such as "Tokyo area," "udon," and "delicious."

[1428] The server collects information on target stores from external APIs and analyzes customer reviews and seating availability.

[1429] The server suggests multiple store options to the user.

[1430] The user selects store A from the suggested stores.

[1431] The server makes a reservation for store A and arranges a taxi if necessary.

[1432] The server notifies the terminal of the reservation completion and arrangement details, and the terminal informs the user of this.

[1433] In this way, users can seamlessly complete everything from information gathering to booking and arrangements through a single platform.

[1434] The following describes the processing flow.

[1435] Step 1:

[1436] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo." The device receives this request and sends it to the server.

[1437] Step 2:

[1438] The server sends the received request to the natural language processing engine. The natural language processing engine analyzes the request and extracts keywords and the user's intent (e.g., "in Tokyo", "udon", "delicious"). The server retrieves the analysis results and prepares the data for use in the next processing step.

[1439] Step 3:

[1440] The server uses multiple external APIs to collect relevant information. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. Furthermore, it obtains the availability and waiting times for each restaurant from a reservation API.

[1441] Step 4:

[1442] The server performs analysis based on the collected data and generates a list of suggestions to present to the user. For example, it creates a list that includes information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes".

[1443] Step 5:

[1444] The server sends the generated suggestion list to the terminal in JSON format. The terminal then displays the received suggestion list in an easy-to-understand format for the user.

[1445] Step 6:

[1446] The user selects their preferred store from the displayed suggestions. For example, "Select Store A." The terminal receives the user's selection and sends it to the server.

[1447] Step 7:

[1448] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. The server uses the reservation API to make reservations for the selected stores and, if necessary, arranges transportation (e.g., taxis) via the taxi API.

[1449] Step 8:

[1450] The server confirms that the reservation and arrangements are complete and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes." The server then sends this notification message to the terminal.

[1451] Step 9:

[1452] The device receives notifications sent from the server and provides the user with final confirmation information (e.g., push notifications or SMS). This allows the user to confirm that the reservation and arrangements are complete.

[1453] This series of processes allows users to seamlessly navigate the entire process, from making a request and gathering information to selecting suggestions, booking and arranging, and finally confirming their booking.

[1454] (Example 1)

[1455] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1456] Traditional systems require users to use multiple platforms when making requests, resulting in a lack of consistency from information gathering to booking. Furthermore, the lack of coordination between platforms leads to significant time and effort. Additionally, the system lacks the ability to accurately understand user intent and provide appropriate suggestions, potentially resulting in lower user satisfaction.

[1457] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1458] In this invention, the server includes means for receiving requests from users, means for analyzing the requests using natural language processing and extracting relevant keywords and intentions, means for collecting information from multiple external sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary transportation based on the selections, means for notifying the user of the completion of the reservations and arrangements, and means for logging the requests, extracted keywords, and intentions. This enables users to consistently and seamlessly perform tasks from information gathering to reservation arrangements.

[1459] "Means for receiving user requests" refers to the means by which a user enters requests or demands into the system and initiates processing within the system.

[1460] A "natural language processing tool" is a means of analyzing a user's input request and extracting relevant keywords and intent from the text.

[1461] "Information gathering means" refers to the means of collecting relevant information from multiple external sources and converting it into a format usable within the system.

[1462] A "proposal generation method" is a means of generating appropriate suggestions for the user based on collected and analyzed information, and presenting the results to the user.

[1463] A "selection acceptance mechanism" is a means for a user to make a selection from the presented suggestions, accept that selection, and reflect it in the next process.

[1464] A "reservation method" refers to a means of reserving a desired store or service based on the user's choice.

[1465] "Means of arranging transportation" refers to the means of arranging the necessary means of travel in conjunction with a reservation.

[1466] "Notification means" refers to the means by which the system transmits information to inform the user that the reservation and arrangements have been completed.

[1467] A "logging method" is a means of recording user requests, extracted keywords, intentions, etc., within the system for later analysis and auditing.

[1468] This invention provides a system that allows users to seamlessly access a series of services. This system consistently handles information gathering, analysis, suggestions, reservations, and necessary arrangements, all centered around the user's request. Detailed embodiments of this system are described below.

[1469] Hardware and software to be used

[1470] 1. User terminal:

[1471] User devices can include smartphones, tablets, and personal computers. These devices receive requests from users and send them to the server. They also display suggestions and notifications to the user.

[1472] 2. Server:

[1473] The server oversees a series of processes, from analyzing user requests and gathering information to generating suggestions, arranging reservations, and sending notifications. Specifically, it uses the following software:

[1474] Natural Language Processing Engine (NLP): Utilizes tools such as the Google Cloud Natural Language API to understand user requests and extract relevant keywords and intents.

[1475] External APIs: Restaurant information, ratings, availability, reservation status, and transportation information are obtained via APIs such as the Yelp API and Google Places API.

[1476] Process Overview

[1477] 1. Receiving user requests:

[1478] The user enters a request through their device. For example, they might request, "I want to eat delicious udon in Tokyo." This request is sent from the device to the server.

[1479] 2. Intent analysis using natural language processing:

[1480] After receiving a request, the server passes the utterance to a natural language processing engine. The engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the request and analyzes the user's intent. The server retrieves the analysis results and prepares queries for the next data collection.

[1481] 3. Information gathering and analysis:

[1482] The server sends queries to external APIs to collect and analyze necessary information. Specifically, it uses the Yelp API to obtain a list of udon restaurants in Tokyo and the Google Places API to obtain information on the availability of seats and customer reviews for each restaurant.

[1483] Based on the information obtained, a list of suggestions for the user is generated.

[1484] 4. Suggestions for users:

[1485] The server generates several suggestions based on the collected information and sends them to the terminal. An example suggestion might be, "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes." The terminal then displays this to the user.

[1486] 5. User Selection and Optimization:

[1487] When a user makes a selection from a list of suggestions, for example, by choosing "Store A," the terminal sends the selection to the server. The server receives the selection and proceeds with the reservation process.

[1488] 6. Confirmation and notification of reservations and arrangements:

[1489] Once the server completes the reservation and necessary arrangements (e.g., taxi booking), it compiles the reservation confirmation information and sends it to the device. The device then notifies the user of the reservation completion details (e.g., via push notification or SMS).

[1490] Specific example

[1491] The specific processing flow when a user enters a request such as "I want to eat delicious udon in Tokyo" is as follows:

[1492] The server analyzes the request using an NLP engine and extracts keywords such as "Tokyo," "udon," and "delicious."

[1493] The server collects and analyzes information about the target stores from the Yelp API and Google Places API.

[1494] Based on the information collected by the server, it suggests potential stores to the user, such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes."

[1495] The user selects "Store A," the server makes a reservation for that store, and arranges a taxi if necessary.

[1496] The server notifies the user's terminal with reservation confirmation and taxi arrangement details.

[1497] Example of a prompt

[1498] Examples of prompts that users might enter include the following:

[1499] "I want to eat delicious udon in Tokyo."

[1500] "Can you recommend a good cafe for the weekend?"

[1501] "I'm looking for a yakiniku restaurant that's open tonight."

[1502] In this way, this system can efficiently and seamlessly handle everything from information gathering to arrangement in response to user requests.

[1503] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1504] Program processing steps

[1505] Step 1: Receiving User Requests

[1506] Specific actions:

[1507] The user enters the prompt "I want to eat delicious udon in Tokyo" into the terminal and sends it. The terminal sends this input to the server as an HTTP request. The server receives the HTTP request and logs its contents. The log includes the request content, a timestamp, and the user's IP address.

[1508] input:

[1509] User prompt (e.g., "I want to eat delicious udon in Tokyo")

[1510] output:

[1511] The prompt message received by the server, and the request content recorded in the log.

[1512] Step 2: Intent Analysis using Natural Language Processing

[1513] Specific actions:

[1514] The server sends the received prompt text to a natural language processing engine (e.g., Google Cloud Natural Language API). The NLP engine extracts keywords such as "in Tokyo," "udon," and "delicious" from the prompt text. The server retrieves the analysis results from the NLP engine and prepares queries containing these keywords for subsequent data collection.

[1515] input:

[1516] Received prompt message

[1517] output:

[1518] Extracted keywords (e.g., "Tokyo area", "udon", "delicious")

[1519] Step 3: Information Gathering and Analysis

[1520] Specific actions:

[1521] The server sends queries to external information source APIs using the extracted keywords. For example, it might retrieve a list of udon restaurants in Tokyo and their ratings via the Yelp API, and obtain the availability and waiting times for each restaurant via the Google Places API. The server receives this information, organizes and analyzes it, and generates a list to present to the user.

[1522] input:

[1523] Extracted keywords

[1524] output:

[1525] Store information and analyzed data (e.g., store name, customer reviews, seating availability, waiting time)

[1526] Step 4: Proposal to the user

[1527] Specific actions:

[1528] The server generates multiple suggestions based on the collected and analyzed store information. The generated suggestions are sent to the terminal in JSON format. The terminal parses the received JSON data and displays the suggestions in a user-friendly format. For example, it might display information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes."

[1529] input:

[1530] Organized and analyzed store information

[1531] output:

[1532] A list of suggestions for users (Example: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes")

[1533] Step 5: User Selection and Optimization

[1534] Specific actions:

[1535] The user makes a selection from a list of suggestions, for example, by selecting "Store A." The terminal sends the user's selection to the server. The server receives the selection and begins arranging a reservation for the selected store. Specifically, it sends a request to the store's reservation API and makes the reservation.

[1536] input:

[1537] User selection (e.g., "Store A")

[1538] output:

[1539] Submitting a booking request and booking confirmation information

[1540] Step 6: Confirmation and notification of booking / arrangement

[1541] Specific actions:

[1542] The server confirms the completion of the reservation and arrangement and organizes the information. The server sends the organized information to the terminal in JSON format. The terminal parses the received JSON data and notifies the user of the reservation and arrangement completion (e.g., push notification or SMS). The reservation confirmation information includes the reservation number, estimated taxi arrival time, etc.

[1543] input:

[1544] Reservation confirmation information

[1545] output:

[1546] Notifications and detailed information to the user (e.g., "Reservation complete. Reservation number 12345. Taxi is expected to arrive in 15 minutes.")

[1547] In this way, the system can efficiently and seamlessly handle everything from information gathering to arrangement in response to user requests.

[1548] (Application Example 1)

[1549] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1550] In recent years, there has been a growing demand for systems that allow users to easily order and reserve meals. However, current food delivery platforms require the use of multiple applications and services, making the process cumbersome. Therefore, there is a need to develop a system that allows users to handle everything from requests and orders to delivery arrangements and status notifications on a single platform.

[1551] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1552] In this invention, the server includes means for receiving requests from users, means for analyzing requests using natural language processing and extracting relevant keywords and intents, means for collecting information from multiple external sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations, orders, and arrangements for necessary services based on the selections, means for notifying the user of the completion of the reservations and arrangements, and means for notifying the user of delivery status and order status in real time. This enables users to seamlessly perform everything from making requests to ordering, making arrangements, and checking the status on a single platform.

[1553] "Means for receiving requests from users" refers to the part of a system that sends requests entered by users via devices such as smartphones and computers to a server and receives those requests.

[1554] "Natural language processing" is a technology that analyzes text data entered by users to extract keywords and intent.

[1555] "External information sources" refer to data sources provided from outside the system, such as restaurant information, customer reviews, reservation services, and delivery services.

[1556] "Information gathering means" refers to the part that has the function of acquiring and analyzing data from external information sources.

[1557] The "means of suggestion" refers to the function that provides users with multiple options based on the information collected and analyzed.

[1558] "Means of making reservations and arrangements" refers to the part of the system that has the functionality to make restaurant reservations and arrange delivery based on the options selected by the user.

[1559] "Notification means" refers to the function that informs users in real time about the completion of reservations and arrangements, as well as the delivery status.

[1560] "Restaurant information" refers to information about food and beverage establishments such as restaurants and cafes, and specifically includes the name of the establishment, address, business hours, menu, prices, and ratings.

[1561] "Word-of-mouth information" refers to feedback such as evaluations and impressions provided by users and customers.

[1562] "Ordering service" refers to the function or service that allows customers to order food from restaurants online.

[1563] "Delivery method" refers to the arrangements and means of transportation used to deliver the ordered meal to the location specified by the user.

[1564] This invention is a system that automatically suggests, reserves, orders, and arranges delivery of the most suitable restaurant based on a user's meal request. This system consists of a user terminal, a server, and multiple external information sources.

[1565] 1. System Program Overview

[1566] The server has the following functions:

[1567] Natural language processing engine (spaCy library)

[1568] Data collection from external sources (Google Places API, Yelp API, etc.)

[1569] Data analysis and proposal generation.

[1570] Reservation and order arrangement

[1571] Real-time notifications (Firebase Cloud Messaging)

[1572] User devices (primarily smartphones) have the following functions:

[1573] User Request Input Form

[1574] Display the proposal list

[1575] Receiving real-time notifications

[1576] 2. Program Processing

[1577] The server receives requests sent by users and analyzes them using a natural language processing engine. This analysis extracts keywords (e.g., "spicy food") from the request and understands its intent.

[1578] Based on analyzed keywords, the system collects restaurant information, reviews, and reservation data from external sources (such as the Google Places API and Yelp API). Based on this collected information, data analysis is performed to generate a list of restaurants best suited to the user.

[1579] When a user selects a restaurant from the suggested list, the server arranges the reservation and order with the selected restaurant. Furthermore, if delivery is required, it also arranges the delivery. Once these arrangements are complete, the server uses Firebase Cloud Messaging to notify the user in real time of the delivery and order status.

[1580] 3. Specific examples

[1581] When a user enters "I want to eat spicy Indian food" into a smartphone app, the server extracts the keywords "spicy" and "Indian food," and gathers and suggests the most suitable restaurant information. If the user selects "Indian Restaurant A" from the suggestions, the order is confirmed and delivery is arranged. This allows the user to track the status of their order in real time until delivery is complete.

[1582] Example of a prompt

[1583] "I'd like to eat something spicy nearby."

[1584] "I'd like to order Indian food for delivery."

[1585] "What Asian dishes are popular today?"

[1586] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1587] Step 1:

[1588] The process by which the user enters a request.

[1589] Input: The user enters a request into the smartphone app saying, "I want to eat spicy Indian food."

[1590] Specific action: Enter text into the input form on the user's terminal and press the submit button.

[1591] Output: The user terminal sends this request data to the server.

[1592] Step 2:

[1593] A process that receives a request and performs natural language processing.

[1594] Input: Request data sent from the user's terminal.

[1595] Specific operation: The server receives the request and passes the request text to the natural language processing engine (spaCy).

[1596] Output: Keywords extracted by the natural language processing engine (e.g., "spicy", "Indian food") and intent are returned.

[1597] Step 3:

[1598] Information gathering process

[1599] Input: Keywords extracted by the natural language processing engine.

[1600] Specific operation: The server calls external information sources (such as the Google Places API and Yelp API) to collect restaurant information, reviews, and reservation status related to keywords.

[1601] Output: Collected restaurant information, customer reviews, and reservation status data.

[1602] Step 4:

[1603] Process to generate a list of suggestions

[1604] Input: Collected restaurant information, customer reviews, and reservation status data.

[1605] Specific operation: The server analyzes the collected data, selects the most suitable restaurants for the user, and generates a list of suggestions.

[1606] Output: Suggestion data listing multiple restaurant options.

[1607] Step 5:

[1608] Process to display a list of suggestions to the user.

[1609] Input: Proposal list data.

[1610] Specific operation: The server sends the suggestion list data to the user's terminal, and the terminal displays the list.

[1611] Output: A list of suggested restaurants displayed on the user's terminal.

[1612] Step 6:

[1613] The process of a user selecting a restaurant from suggestions.

[1614] Input: The displayed list of suggestions.

[1615] Specific action: The user selects a restaurant from the suggested list and presses the select button.

[1616] Output: The user terminal sends information about the selected restaurant to the server.

[1617] Step 7:

[1618] Processing to arrange reservations and orders.

[1619] Input: Information about the restaurant selected by the user.

[1620] Specific operation: The server makes reservations and orders for selected restaurants through an online reservation and ordering system (external API).

[1621] Output: Reservation confirmation and order processing completion data.

[1622] Step 8:

[1623] Process to notify the user of the completion of the reservation and order.

[1624] Input: Reservation confirmation and order processing completion data.

[1625] Specific operation: The server sends a notification to the user in real time via Firebase Cloud Messaging (FCM). The device receives the notification and displays it to the user.

[1626] Output: Reservation and order completion notifications displayed on the user's terminal.

[1627] Step 9:

[1628] A process to notify delivery status in real time.

[1629] Input: Delivery arrangement progress information.

[1630] Specific operation: The server updates the delivery status in real time based on progress information obtained from the delivery system and notifies the user using Firebase Cloud Messaging.

[1631] Output: Delivery status notification displayed in real time on the user's device.

[1632] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1633] Modes for carrying out the invention

[1634] This invention is a system that receives user requests, recognizes and analyzes the user's emotional state using an emotion engine, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that analysis. The program of this system consists of the following components, and its detailed processing is described below.

[1635] System components

[1636] 1. User terminal: Responsible for inputting requests and displaying results.

[1637] 2. Server: Responsible for request analysis, sentiment recognition, information gathering and analysis, suggestion generation, reservation arrangement, and result notification.

[1638] 3. External APIs: Provide restaurant information, reviews, reservation services, and transportation information.

[1639] 4. Emotion Engine: Extracts emotions from user requests and optimizes suggestions.

[1640] Explanation of program processing

[1641] 1. Receiving user requests

[1642] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo."

[1643] The terminal receives this request and sends it to the server. The server logs the received request.

[1644] 2. Intention analysis and sentiment recognition using natural language processing (NLP)

[1645] The server sends the received request to the natural language processing engine for analysis.

[1646] The natural language processing engine extracts keywords and intents from the request, such as "in Tokyo," "udon," and "delicious."

[1647] At the same time, the emotion engine recognizes the user's emotional state (e.g., joy, anticipation, fatigue) from the words and context included in the request.

[1648] The server retrieves these analysis results and sentiment data, and prepares the data for use in the next processing step.

[1649] 3. Information Gathering and Analysis

[1650] The server uses multiple external APIs to collect relevant information. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. Furthermore, it obtains the availability and waiting times for each restaurant from a reservation API.

[1651] The server performs analysis based on the collected data and generates a list of suggestions to present to the user. Here, the suggestions are optimized based on the results of the emotion engine's analysis. For example, if the user is feeling fatigued, the server prioritizes suggesting stores with short waiting times.

[1652] 4. Suggestions for users

[1653] The server sends the generated suggestion list to the terminal in JSON format. The terminal then displays the received suggestion list in an easy-to-understand format for the user.

[1654] For example, the server might suggest "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes." The system also optimizes the suggestions based on the user's emotional state.

[1655] 5. User Selection and Optimization

[1656] The user selects their preferred store from the displayed suggestions. For example, "Select Store A."

[1657] The terminal receives the user's selection and sends it to the server.

[1658] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. Specifically, it makes reservations at the selected establishments and, if necessary, arranges transportation (such as taxis).

[1659] 6. Confirmation and notification of reservations and arrangements

[1660] The server confirms the completion of the reservation and arrangement and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is scheduled to arrive in 15 minutes."

[1661] The server sends this notification to the device. The device receives the completion notification and provides the user with final confirmation information (e.g., push notification or SMS). The user can then confirm that the reservation and arrangements are complete.

[1662] Specific example

[1663] If a user requests "I want to eat delicious udon in Tokyo," the server will process it as follows:

[1664] The server analyzes the request and extracts keywords such as "Tokyo," "udon," and "delicious." At the same time, the emotion engine recognizes the user's emotional state from the request as "expectation."

[1665] The server collects target store information from an external API and analyzes customer reviews and seating availability. The emotion engine prioritizes and suggests highly-rated stores based on the emotion of "expectation."

[1666] The server generates a list of suggestions that include information such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes," and presents them to the user.

[1667] The user selects store A from the suggested stores. The server makes a reservation for store A and arranges a taxi if necessary.

[1668] The server notifies the terminal of the reservation completion and arrangement details, and the terminal informs the user of this.

[1669] In this way, users can receive more personalized and optimal services that take their own emotional state into consideration.

[1670] The following describes the processing flow.

[1671] Step 1:

[1672] The user uses their device to enter and send a request. For example, they might enter, "I want to eat delicious udon in Tokyo." The device receives this request and sends it to the server. The server logs the received request.

[1673] Step 2:

[1674] The server sends the received request to the natural language processing engine for analysis. The natural language processing engine extracts keywords and intent from the request, such as "in Tokyo," "udon," and "delicious." Simultaneously, the sentiment engine recognizes the user's emotional state from the words and context included in the request. For example, it extracts emotions such as "expectation" or "fatigue." The server obtains these analysis results and sentiment data and prepares the data for use in the next processing step.

[1675] Step 3:

[1676] The server collects relevant information using multiple external APIs. Specifically, it obtains a list of udon restaurants in Tokyo and basic information from a restaurant API, and collects ratings and review counts for each restaurant from a review API. It also uses a reservation API to obtain the availability and waiting times for each restaurant. The server organizes and integrates this information to form recommendations for users.

[1677] Step 4:

[1678] The server generates a list of suggestions to present to the user based on the collected and organized information. In this process, it optimizes the suggestions using the results of the emotion engine's analysis. For example, if the user's emotional state is "fatigued," it prioritizes suggesting stores with short waiting times. Also, if the user expresses feelings of "anticipation," it prioritizes highly-rated stores.

[1679] Step 5:

[1680] The server sends the generated list of suggestions to the terminal in JSON format. The terminal then displays the received list of suggestions in an easy-to-understand format for the user. Specifically, suggestions are presented in the following format: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[1681] Step 6:

[1682] The user selects their preferred store from the displayed suggestions. For example, "Select Store A." The terminal receives the user's selection and sends it to the server.

[1683] Step 7:

[1684] The server receives the user's selection and begins arranging the relevant reservations and auxiliary services. Specifically, it calls the reservation API to make a reservation at the selected store. It also uses the taxi API to arrange transportation (such as a taxi) as needed. For example, it might use the reservation API to secure a reservation at store A and the taxi API to arrange transportation for the user.

[1685] Step 8:

[1686] The server confirms that the reservation and arrangements are complete and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes." The server then sends this notification message to the terminal.

[1687] Step 9:

[1688] The device receives notifications sent from the server and provides the user with final confirmation information. Specifically, users are notified via push notifications or SMS. This allows users to confirm that their reservation and arrangements are complete.

[1689] Through this series of steps, users can seamlessly receive more personalized and optimal services that even take their emotional state into consideration.

[1690] (Example 2)

[1691] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1692] Traditional information gathering and suggestion systems have a problem in that they provide simple information without considering the user's emotions, making it difficult to improve user satisfaction. Furthermore, there was a lack of methods to integrate information from multiple external sources and provide optimal suggestions based on the user's specific needs. As a result, it was difficult for users to obtain the necessary information quickly and accurately, leading to a decline in the quality of the user experience.

[1693] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1694] In this invention, the server includes means for receiving requests from users, means for analyzing requests using natural language processing and extracting relevant keywords and intentions, means for an emotion engine that analyzes the user's emotional state, means for collecting information from multiple external information sources, analyzing the acquired information and making suggestions to the user, means for optimizing the suggestions based on the emotional state, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary services based on the selection, and means for notifying the user of the completion of the reservations and arrangements. This enables more personalized and optimal information provision and service arrangements that take the user's emotions into consideration.

[1695] "Means of receiving requests" refers to devices or software that receive input from users and transfer it to a server.

[1696] "Natural language processing" is the process of analyzing incoming requests and extracting keywords and intent.

[1697] "Means for extracting relevant keywords and intent" refers to engines or software that identify important words and user intent from requests based on natural language processing.

[1698] An "emotional engine" refers to algorithms or software used to recognize and analyze a user's emotional state based on their request content and context.

[1699] "Means of collecting information, analyzing the acquired information, and making suggestions to users" refers to the process or engine for acquiring data from external sources, analyzing it, and creating appropriate suggestions for users.

[1700] "Methods for optimizing suggestions based on emotional state" refer to methods or software that take the user's emotional state into consideration and select the most appropriate suggestion from the collected information.

[1701] "Means for receiving user selections" refers to the interface or software used to receive a user's selection from the provided suggestions and communicate that selection to the server.

[1702] "Means for making reservations and arranging necessary services" refers to systems or software that automatically handle reservation procedures and the arrangement of related services based on the user's selection.

[1703] "Means of notifying users of the completion of reservations and arrangements" refers to notification systems or software that inform users when reservations or arrangements have been completed.

[1704] "External information sources" refer to external APIs and databases such as restaurant information, customer review information, reservation services, and transportation information.

[1705] A "system" is a set of hardware and software configurations that operate by integrating the above-mentioned means and processes.

[1706] Modes for carrying out the invention

[1707] This invention is a system that receives user requests, recognizes and analyzes the user's emotional state using an emotion engine, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that analysis. This system consists of the following components, and its detailed processing is described below.

[1708] System components

[1709] 1. User terminal: Responsible for inputting requests and displaying results.

[1710] 2. Server: Responsible for request analysis, sentiment recognition, information gathering and analysis, suggestion generation, reservation arrangement, and result notification.

[1711] 3. External information sources: Provide information on restaurants, customer reviews, reservation services, and transportation options.

[1712] 4. Emotion Engine: Extracts emotions from user requests and optimizes suggestions.

[1713] Specific operation of the system

[1714] Receiving user requests

[1715] When a user enters a request into their device and sends it, the device sends that request to the server. For example, if a user enters "I want to eat delicious udon in Tokyo," the device generates an HTTP request and sends it to the server. The server receives this request and logs it.

[1716] Intent analysis and emotion recognition using Natural Language Processing (NLP)

[1717] The server sends received requests to a natural language processing engine (e.g., SpaCy or Google Cloud NLP) for analysis. The NLP engine extracts keywords such as "Tokyo," "udon," and "delicious," analyzes the context, and understands the intent. An emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotional state (e.g., "expectation") from the words and context included in the request.

[1718] Information gathering and analysis

[1719] The server collects relevant information using multiple external sources (e.g., restaurant API, review API, reservation API). Specifically, it obtains a list of udon restaurants in Tokyo and basic information from the restaurant API, ratings and review counts for each restaurant from the review API, and availability and waiting times for restaurants from the reservation API. Based on this data, the server performs analysis and generates a list of suggestions to present to the user. Here, the suggestions are optimized based on the results of the emotion engine's analysis. For example, if the user is feeling tired, the server prioritizes suggesting restaurants with short waiting times.

[1720] Suggestions for users

[1721] The server encodes the generated suggestion list in JSON format and sends it to the terminal. The terminal parses the received data and displays it in the user interface. For example, suggestions such as "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes" may be displayed.

[1722] User selection and optimization

[1723] When a user selects a desired store from the device's suggestion list, the device sends that selection to the server. The server makes a reservation for the selected store and arranges transportation (e.g., a taxi) if necessary. Specifically, the server accesses a reservation API, makes a reservation, and uses a transportation API (e.g., Uber API) to arrange a taxi.

[1724] Reservation and arrangement confirmation and notification

[1725] The server retrieves information about completed reservations and arrangements and generates a notification for the user. For example, it might generate a notification such as, "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and send it to the device. The device receives this notification and sends it to the user via push notification or SMS.

[1726] Examples of prompt statements

[1727] You can obtain a detailed explanation of the system by inputting the following prompt sentences into the generating AI model:

[1728] Please describe in detail the system's processing steps from the moment a user enters a request such as "I want to eat delicious udon in Tokyo," through which the emotion engine analyzes the user's emotions, collects and analyzes information from external sources, generates optimal suggestions, makes reservations and arrangements, and notifies the user.

[1729] This will enable a system that provides more personalized and optimal services, taking into account the user's emotional state.

[1730] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1731] Step 1: Receiving User Requests

[1732] The user enters a request into their device saying, "I want to eat delicious udon in Tokyo."

[1733] The terminal receives input, generates an HTTP request, and sends it to the server. The input is the user's text input, and the output is the data sent to the server.

[1734] The server receives this request at the API endpoint and records it in a log file or database. The input is an HTTP request from the terminal, and the output is the log entry.

[1735] Step 2: Intent analysis and emotion recognition using Natural Language Processing (NLP)

[1736] The server sends the received request to the natural language processing engine for analysis. The input is the user's request, and the output is the NLP analysis result.

[1737] A natural language processing engine (e.g., SpaCy or Google Cloud NLP) extracts keywords and intent from a request, such as "in Tokyo," "udon," and "delicious." The input is the text data of the request, and the output is the extracted keywords and intent.

[1738] An emotion engine (e.g., IBM Watson Tone Analyzer) recognizes the user's emotional state (e.g., "expectation") from the words and context included in the request. The input is the text data of the request, and the output is the emotional state data.

[1739] Step 3: Information Gathering and Analysis

[1740] The server collects relevant information from multiple external sources. Specifically, it utilizes restaurant APIs, review APIs, and reservation APIs. The input consists of analyzed keywords and intent, while the output is the collected information.

[1741] This system retrieves a list of udon restaurants in Tokyo and their basic information from a restaurant API, and collects each restaurant's rating and review count from a review API. The input is the requested conditions, and the output is a list of restaurants and rating data.

[1742] This system uses a reservation API to retrieve seat availability and waiting times for each store. Input is store information, and output is seat availability and waiting time data.

[1743] The server integrates this data and generates a list of suggestions to present to the user, taking into account the results of the sentiment engine. Input is data from multiple APIs, and output is a list of suggestions.

[1744] Step 4: Proposal to the user

[1745] The server sends the generated proposal list to the terminal in JSON format. The input is the proposal list, and the output is the transmitted data.

[1746] The terminal analyzes the received data and displays it on the user interface. The input is a list of suggestions from the server, and the output is what is displayed to the user.

[1747] For example, "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes" will be displayed.

[1748] Step 5: User Selection and Optimization

[1749] The user selects their preferred store from the suggested stores. The input is the suggested list, and the output is the user's selection.

[1750] The terminal receives the user's selection and sends it to the server. The input is the user's selection, and the output is the data sent to the server.

[1751] The server makes reservations for the selected restaurant and arranges a taxi if necessary. Specifically, it accesses a reservation API, completes the reservation process, and uses a transportation API (e.g., Uber API) to arrange a taxi. Input is the user's selection information, and output is reservation and arrangement data.

[1752] Step 6: Confirmation and notification of booking / arrangement

[1753] The server confirms the completion of reservations and arrangements and organizes the information. Inputs are response data from the reservation API and the transportation API, and outputs are completion notifications.

[1754] The server generates a notification message such as "Reservation complete. Reservation number 12345. The taxi is expected to arrive in 15 minutes," and sends it to the terminal. The input is the booking status data, and the output is the notification data.

[1755] The device receives a completion notification and provides the user with final confirmation information. Specifically, this is done via push notification or SMS. The input is notification data from the server, and the output is the notification to the user.

[1756] The above is a detailed explanation of the system's program processing flow.

[1757] (Application Example 2)

[1758] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1759] In recent years, providing services tailored to individual user needs has become increasingly important. However, conventional request processing systems have been unable to consider users' emotional states when making suggestions, limiting the improvement of user satisfaction. Similarly, in food delivery services, the lack of optimal suggestions and procedures that align with users' emotions has been a problem.

[1760] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving requests from users, means for analyzing the requests using natural language processing and extracting relevant keywords and intentions, means for recognizing the user's emotional state from the requests using an emotion analysis engine, means for collecting information from multiple external information sources, analyzing the acquired information and making suggestions to the user, means for accepting selections from the user based on the suggestions, means for making reservations and arranging necessary services based on the selections, and means for notifying the user of the completion of the reservations and arrangements. This makes it possible to provide optimal services and food delivery procedures that take into account the user's emotional state.

[1761] "Means for receiving user requests" refers to interfaces and systems for receiving requests and questions entered by users through their devices.

[1762] "Means for analyzing using natural language processing and extracting relevant keywords and intents" refers to technologies that analyze user requests, understand the context and meaning of words, and find appropriate keywords and intents.

[1763] "A means of recognizing a user's emotional state from a request using an emotion analysis engine" refers to a technology that analyzes the emotional aspects of a user's request and identifies emotional states such as joy, anticipation, and fatigue.

[1764] "A means of collecting information from multiple external sources, analyzing the acquired information, and making suggestions to the user" refers to a technology that collects information from external sources through APIs and databases, analyzes it, and makes the most suitable suggestions to the user.

[1765] "Means for receiving user selections based on the aforementioned proposals" refers to an interface and system for presenting proposals to the user and receiving the user's selection from among them.

[1766] "Means for making reservations and arranging necessary services based on the aforementioned selection" refers to systems and technologies that automatically perform reservation procedures and arrange related services according to the user's selection.

[1767] "Means for notifying the user of the completion of the reservation and arrangement" refers to a system and interface for notifying the user of the completion information of the reservation and arrangement.

[1768] "Restaurant information" refers to basic information such as the restaurant's location, menu, business hours, and contact information.

[1769] "Rating information" refers to reviews and ratings of restaurants and services provided by users and reviewers.

[1770] A "delivery service" is a service that aims to deliver meals or goods ordered by users to a specified location.

[1771] "Transportation" includes means of enabling the movement of users and goods, such as taxis, buses, and sharing services.

[1772] This invention is a system that receives requests from users, recognizes the user's emotional state through sentiment analysis, and seamlessly performs information gathering, suggestions, reservations, and necessary arrangements based on that. Specific embodiments for realizing this system will be described below.

[1773] System Configuration

[1774] This system consists of the following components:

[1775] 1. User terminal

[1776] User devices include smartphones and smart glasses. Users use these devices to enter requests and view and select suggested information.

[1777] 2. Server

[1778] The server receives requests from users and is responsible for their analysis, information gathering, suggestion generation, booking arrangements, and result notifications. The server interacts with the following engines and external APIs.

[1779] 3. Emotion Analysis Engine

[1780] The sentiment analysis engine recognizes the emotional state from the user's request. For example, if the user includes the phrase "I expect," the sentiment analysis engine recognizes the emotion "expectation."

[1781] 4. Natural Language Processing Engine

[1782] A natural language processing (NLP) engine analyzes a user's request and extracts relevant keywords and intent. For example, if the user enters "I want to eat delicious udon in Tokyo," the NLP engine will extract the keywords "Tokyo," "udon," and "delicious."

[1783] 5. External API

[1784] External APIs provide restaurant information, ratings, delivery services, and transportation options. For example, the Google Places API is used to retrieve information on nearby restaurants and to refer to reviews and ratings.

[1785] Explanation of the process

[1786] Request reception and analysis

[1787] The user enters a request via their smartphone or smart glasses and sends it to the server. The server then passes the request to a natural language processing engine, which analyzes the keywords and intent.

[1788] Recognition of emotions

[1789] The server inputs the analyzed request into its sentiment analysis engine to recognize the user's emotional state. For example, if the emotion "expectation" is extracted from the request, the server optimizes its suggestions based on that.

[1790] Information gathering and proposals

[1791] The server uses external APIs to collect relevant information. For example, it retrieves information about restaurants, ratings, and delivery status. It analyzes the collected information and generates optimal suggestions based on the user's emotional state.

[1792] User notifications and choices

[1793] The server notifies the user terminal of the generated suggestions and assists the user in the process of selecting from them. The user's selection information is sent to the server, and reservations and delivery arrangements are made.

[1794] Completion notification

[1795] The server confirms the completion of the reservation and arrangement and notifies the user of the result. The user's terminal receives the notification and displays the result to the user.

[1796] Specific example

[1797] If a user enters a request saying, "I want to eat delicious udon in Tokyo":

[1798] 1. The server analyzes the request using a natural language processing engine and extracts the keywords "Tokyo," "udon," and "delicious."

[1799] 2. The emotion analysis engine recognizes the emotion of "expectation."

[1800] 3. The server generates suggestions tailored to the user based on information obtained from external APIs.

[1801] 4. Notify users of the following suggestions: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes".

[1802] 5. The user selects store A, and the server makes the reservation and delivery arrangements.

[1803] 6. The server sends a notification to the user that the arrangement is complete, and the user confirms it.

[1804] Example of a prompt

[1805] "I want to eat delicious udon in Tokyo."

[1806] "I'd like sushi delivered for the dinner I'm looking forward to."

[1807] "I'd like to order a pizza for lunch, but I'd like to check if it's being delivered quickly."

[1808] Thus, the system based on the present invention enables the provision of optimal services that take user emotions into consideration.

[1809] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1810] Step 1:

[1811] The device receives a request from the user. This request is entered in text format. An example of such a request might be, "I want to eat delicious udon in Tokyo." The device then sends this request to the server.

[1812] Step 2:

[1813] The server sends the received request to the natural language processing engine for analysis. The NLP engine extracts keywords and intent from the request, such as "in Tokyo," "udon," and "delicious." The analysis results are returned to the server and used in the next processing step.

[1814] Step 3:

[1815] The server sends the analysis results to the sentiment analysis engine, which recognizes and analyzes the emotional state included in the request. The sentiment analysis engine extracts emotions such as "expectation" from the context and returns the results to the server. The server then prepares to optimize the suggestions based on this emotional state.

[1816] Step 4:

[1817] The server uses multiple external APIs to collect relevant information. For example, it retrieves a list and basic information of "udon restaurants in Tokyo" from a restaurant information API, and collects customer reviews and the number of reviews for each restaurant from a rating information API. This collected data is then integrated on the server side.

[1818] Step 5:

[1819] The server generates a suggestion list based on the collected information. Here, based on the emotional state "expectation" provided by the sentiment analysis engine, stores with high customer reviews are prioritized and included in the list. The generated suggestion list is sent to the terminal in JSON format.

[1820] Step 6:

[1821] The device displays a list of received suggestions to the user. Examples of suggestions displayed include: "Store A: Customer review rating 4.5, average price 1000 yen, waiting time 10 minutes" and "Store B: Customer review rating 4.0, average price 900 yen, waiting time 5 minutes."

[1822] Step 7:

[1823] The user makes a selection from the presented suggestions. For example, "Select store A." The terminal sends this selection to the server.

[1824] Step 8:

[1825] The server makes reservations and necessary arrangements based on the user's selection. Specifically, it makes reservations for the selected store via an online reservation API and arranges delivery methods as needed. The results of the reservations and arrangements are stored on the server.

[1826] Step 9:

[1827] The server confirms the completion of the reservation and arrangement and organizes the information. For example, it generates a notification message such as, "Reservation complete. Reservation number 12345. Delivery is expected to arrive in 15 minutes."

[1828] Step 10:

[1829] The server sends this final confirmation information to the device. The device receives a completion notification and provides the user with the final confirmation information via push notification or display. This allows the user to confirm the completion of their reservation and arrangements.

[1830] In this way, users can receive the optimal food delivery service tailored to their emotional state.

[1831] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1832] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1833] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1834] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1835] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1836] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1837] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1838] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1839] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1840] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1841] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1842] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1843] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1844] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1845] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1846] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1847] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1848] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a ha...

Claims

1. A means of receiving requests from users, A means for analyzing the aforementioned request using natural language processing and extracting relevant keywords and intent, A means of collecting information from multiple external sources, analyzing the acquired information, and making suggestions to the user, A means of accepting user selections based on the above proposal, A means of making reservations and arranging necessary services based on the aforementioned selection, A system including means for notifying the user of the completion of the aforementioned reservation and arrangement.

2. The system according to claim 1, wherein the information gathering means collects information from restaurant information, word-of-mouth information, and reservation services.

3. The system according to claim 1, wherein the information gathering means also arranges the means of transportation necessary for making a reservation.

Images