System

Abstract

To provide a system which, even when unaccompanied by a tour conductor, makes it possible for a traveler to obtain necessary travel information and thereby mitigates the time and effort for traveling.SOLUTION: Provided is a system including: means for recognizing the emotional state of a user on the basis of the tone of voice and emotion data of the user; a chat generative AI means for providing travel information regarding a place that the user is planning to travel through interaction with the user; means for proposing meals and eating houses during travel at the site on the basis of questions from the user and the emotional state of the user; means for assisting the user in traveling on the basis of the travel information provided by the chat generative AI means; means for delivering real-time videos of the site on the basis of travel information provided by the chat generative AI means; and means for arranging for specialty products of the site on the basis of the travel information provided by the chat generative AI means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional travel services require a tour conductor to accompany the traveler, which results in increased labor costs. Furthermore, if a tour conductor is not present, the traveler must research local tourist attractions, transportation options, local know-how, and other information on their own, which is time-consuming and problematic. [Means for solving the problem]

[0005] The system of the present invention is equipped with a chat generation AI means that provides travel information through dialogue with users. This chat generation AI means extracts and explains travel information from the vast amount of information on the Internet. The system also supports users in traveling based on the travel information provided by the chat generation AI means, delivers real-time local video, and arranges for local specialties. This allows travelers to obtain the information they need without the need for a tour conductor, reducing the hassle of traveling. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention includes a chat generation AI means that provides travel information about places where the user is planning to travel through dialogue with the user. This chat generation AI means extracts and explains travel information from a vast amount of information on the Internet. For example, if a user asks, "Tell me about tourist attractions in Paris," the chat generation AI means will respond based on information on the Internet in the form of, "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[0029] "Example 2"

[0030] The system of the present invention also includes a means for supporting a user in traveling based on travel information provided by the chat generation AI means. For example, if a user asks, "Please tell me how to use the Paris subway," the chat generation AI means will respond by providing assistance with the user's travel in the form of, "For the Paris subway, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[0031] "Example 3"

[0032] Furthermore, the system of the present invention is equipped with a means for delivering real-time local video based on travel information provided by the chat generation AI means. For example, if a user asks, "Show me the current situation in Paris," the system will deliver real-time local video, allowing the user to obtain the latest local information.

[0033] "Example 4"

[0034] The system of the present invention also includes a means for arranging local specialties based on travel information provided by the chat generation AI means. For example, if a user asks, "I want to buy a local specialty from Paris," the system will arrange for the local specialty. This allows users to obtain local specialties without having to travel to the location.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: The user asks the system, "Tell me about tourist attractions in Paris."

[0038] Step 2: The chat generation AI in the system responds based on information on the Internet, such as "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[0039] "Example 2"

[0040] Step 1: The user asks the system, "How do I use the Paris subway?"

[0041] Step 2: The chat generation AI within the system responds by providing support for the journey, such as, "For the Paris Metro, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[0042] "Example 3"

[0043] Step 1: The user asks the system, "Show me the current situation in Paris." Step 2: The system delivers real-time video of the local area, allowing the user to get the latest information on the local area.

[0044] "Example 4"

[0045] Step 1: The user asks the system, "I want to buy a Parisian specialty."

[0046] Step 2: The system arranges for local specialties, allowing users to obtain the specialties without having to travel to the local area.

[0047] Example 1

[0048] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] Conventional travel information systems have had the problem of being unable to provide users with the information they need quickly and accurately. Furthermore, when users ask specific questions, it takes a lot of time and effort to get an appropriate answer. Furthermore, it is difficult to meet diverse travel-related needs, such as providing real-time information or arranging for local specialties.

[0050] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0051] In this invention, the server includes an information generation means for providing travel information through dialogue with a user, a means for supporting a user in traveling based on the travel information provided by the information generation means, a means for delivering real-time local video based on the travel information provided by the information generation means, a means for arranging local specialties based on the travel information provided by the information generation means, a means for receiving an inquiry from a user and sending it as a prompt to a generative AI model, a means for the generative AI model to search information on the Internet and generate an answer, and a means for returning the generated answer to the user. This allows users to quickly and accurately obtain travel information and makes it possible to meet a variety of needs, such as providing real-time information and arranging local specialties.

[0052] The "information generating means" is a means for providing travel information through dialogue with the user.

[0053] "Means for supporting travel" refers to means for supporting a user in planning and carrying out a travel based on travel information provided by an information generating means.

[0054] The "means for delivering real-time video" is a means for delivering local real-time video to users based on travel information provided by the information generating means.

[0055] The "means for arranging local specialties" is a means for arranging local specialties for the user based on the travel information provided by the information generating means.

[0056] "Means for receiving a query and sending it to the generative AI model as a prompt sentence" refers to means for receiving a query from a user and sending it to the generative AI model as a prompt sentence.

[0057] "Means by which the generative AI model searches for information on the Internet and generates an answer" refers to the means by which the generative AI model searches for information on the Internet based on a prompt sentence and generates an appropriate answer.

[0058] "Means for returning the generated answer to the user" refers to means for returning the answer generated by the generative AI model to the user.

[0059] This invention is a system for providing travel information through dialogue with a user. The system includes an information generating means, a means for supporting travel, a means for delivering real-time video, a means for arranging local specialties, a means for receiving an inquiry and sending it as a prompt to a generative AI model, a means for the generative AI model to search for information on the Internet and generate an answer, and a means for returning the generated answer to the user.

[0060] Hardware and software used

[0061] This system uses a server, a terminal, and a generative AI model. Specifically, the server is a high-performance cloud server, and the terminal is a device that provides the user interface, such as a smartphone or PC. The generative AI model uses an advanced natural language processing model such as OpenAI's GPT-4.

[0062] Data processing and calculation

[0063] The server receives a user's query and sends it to the generative AI model as a prompt. The generative AI model searches vast amounts of information on the Internet based on the prompt and generates an appropriate answer. The generated answer is then sent back to the user via the server.

[0064] Specific examples

[0065] Example 1

[0066] The user types "Tell me about tourist attractions in Paris" into the device. The device sends this query to the server. The server sends the prompt "Tell me about tourist attractions in Paris" to the generative AI model. The generative AI model searches information on the internet and generates the answer "Famous attractions include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The server sends this answer back to the device, which displays it to the user.

[0067] Example 2

[0068] The user types "What restaurants in Tokyo do you recommend?" into the device. The device sends this query to the server. The server sends the prompt "What restaurants in Tokyo do you recommend?" to the generative AI model. The generative AI model searches information on the internet and generates the answer "Popular sushi restaurants in Ginza and ramen restaurants in Shinjuku." The server sends this answer back to the device, which displays it to the user.

[0069] Prompt Sentence Examples

[0070] 1. "Tell me about the tourist attractions in Paris."

[0071] 2. "What are some recommended restaurants in Tokyo?"

[0072] 3. "What museums should I visit in New York?"

[0073] This allows users to quickly and accurately obtain travel information, and makes it possible to meet a variety of needs, such as providing real-time information and arranging for local specialties.

[0074] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0075] Step 1:

[0076] The user types a query into the terminal.

[0077] When a user wants to know travel information, they can type a question into the chat interface of the device, for example, "Tell me about tourist attractions in Paris." The typed query is temporarily stored in the device's internal memory.

[0078] Step 2:

[0079] The terminal sends a query to the server.

[0080] The terminal sends the query entered by the user to the server. At this time, the terminal converts the query content into an appropriate format and uses a communication protocol (e.g., HTTP POST request) to send it to the server. The input is the user's query, and the output is the data to be sent to the server.

[0081] Step 3:

[0082] The server receives the query and sends it as a prompt to the generative AI model.

[0083] The server analyzes the query received from the device and sends it to the generative AI model as a prompt. The prompt may be the user's question as is, or it may be preprocessed as needed. The input is the query data from the device, and the output is the prompt to the generative AI model.

[0084] Step 4:

[0085] A generative AI model searches the internet for information and generates answers.

[0086] A generative AI model searches for relevant data from the vast amount of information on the Internet based on a prompt and generates an answer. For example, in response to the prompt, "Tell me about tourist attractions in Paris," it generates the answer, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The input is the prompt, and the output is the generated answer.

[0087] Step 5:

[0088] The server receives the generated response and sends it to the terminal.

[0089] The server analyzes the response received from the generative AI model, converts it into an appropriate format, and sends it to the device. At this time, the server uses a communication protocol (e.g., HTTP response). The input is the response data from the generative AI model, and the output is the data sent to the device.

[0090] Step 6:

[0091] The device displays the answer to the user.

[0092] The terminal displays the answer received from the server to the user. The user can check the answer on the terminal screen. The input is the answer data from the server, and the output is the display data to the user.

[0093] (Application example 1)

[0094] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Conventional travel information systems have the problem that it takes a lot of time and effort for users to obtain information about travel destinations. In addition, they do not provide real-time information or a travel experience in a virtual environment, which results in low user satisfaction.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0097] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time local video based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, means for the user to select a travel destination in a virtual store and be provided with travel information in real time, and means for the user to obtain information on tourist attractions and recommended spots related to the travel destination in the virtual store. This allows the user to efficiently obtain travel information and enjoy a real-time travel experience in a virtual environment.

[0098] "User" means an individual or organization that uses the system to obtain travel information and engage in travel experiences within the virtual store.

[0099] "Dialogue" refers to the exchange of information between a user and a chat generation AI means.

[0100] "Travel information" refers to any information related to travel, such as tourist attractions, transportation, local know-how, and local specialties.

[0101] "Chat generation AI means" refers to the function of artificial intelligence that provides travel information through dialogue with users.

[0102] "Supporting means" refers to a function that supports users in traveling based on travel information provided by chat generation AI means.

[0103] "Means for delivering real-time video" refers to the function of providing users with local footage in real time.

[0104] "Means for arranging local specialties" refers to the function of making arrangements to provide local specialties and famous products to users.

[0105] "Virtual store" refers to a virtual store where users can select travel destinations and obtain travel information within a virtual space.

[0106] "Tourist attractions" refer to places and facilities worth visiting in a travel destination.

[0107] "Recommended spots" refer to places or facilities that are particularly recommended to visit at a travel destination.

[0108] The system for implementing this invention is mainly composed of chat generation AI means that provides travel information through dialogue with users. A specific embodiment of this system will be described below.

[0109] First, users access a virtual store using a device such as a smartphone or head-mounted display. Within the virtual store, users can select a travel destination and obtain information about that destination through chat generation AI.

[0110] The server uses OpenAI's API to implement a chat generation AI system. When a user enters a question about a travel destination, the question is generated as a prompt and sent to OpenAI's generative AI model. The generative AI model then extracts appropriate travel information from the vast amount of information on the Internet and provides it to the user.

[0111] For example, if the user types "Tell me about tourist attractions in Paris," the prompt text might look like this:

[0112] "Tell me about the tourist attractions in Paris"

[0113] This prompt is sent to a generative AI model, which then generates an answer such as, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." This answer is then displayed on the user's device.

[0114] In addition, the system is equipped with a means for streaming real-time video, allowing users to view real-time footage of the selected travel destination, giving them the experience of actually visiting the location.

[0115] The system also provides a means to arrange local specialties, allowing users to easily purchase the specialties they are interested in. This helps users gain a deeper understanding of the culture and specialties of the destination they are traveling to.

[0116] In this way, users can efficiently obtain travel information and enjoy real-time travel experiences in a virtual environment. The hardware used is a smartphone and a head-mounted display, and the software is Python and OpenAI API.

[0117] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0118] Step 1:

[0119] The user accesses the virtual store using a smartphone or head-mounted display. The user selects a travel destination and inputs a question to obtain travel information. The input data is the user's selected travel destination and the question.

[0120] Step 2:

[0121] The terminal receives the user's input and generates a prompt based on the question. The generated prompt might be something like, "Tell me about tourist attractions in Paris." This prompt becomes the input data for the next processing step.

[0122] Step 3:

[0123] The server sends the generated prompt to OpenAI's generative AI model, which then extracts relevant travel information from the vast amount of information on the Internet based on the prompt. This process takes the prompt as input and outputs text data containing travel information.

[0124] Step 4:

[0125] The server receives the travel information obtained from the generative AI model and sends it to the user's device. The user's device displays this travel information. For example, it displays information such as "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[0126] Step 5:

[0127] If the user wants more information, they enter a follow-up question. The device generates a new prompt and sends it to the server. The process repeats steps 2 through 4.

[0128] Step 6:

[0129] When a user wants to watch real-time video of a selected travel destination, the device sends the request to the server, which retrieves the local real-time video and delivers it to the user's device, where the user can watch the real-time video.

[0130] Step 7:

[0131] When a user wants to purchase a local specialty product, the device sends the request to the server, which then arranges for the product to be delivered to the user, and the user can then complete the purchase process on the device.

[0132] In this way, users can efficiently obtain travel information and enjoy real-time travel experiences in a virtual environment.

[0133] Example 2

[0134] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0135] Conventional travel information systems have struggled to provide users with the specific travel information they need quickly and accurately. Furthermore, users have had limited access to the information they need in real time while traveling, which often makes planning and executing trips inconvenient. Furthermore, they lacked features to enrich the travel experience, such as arranging local specialties and streaming real-time videos.

[0136] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0137] In this invention, the server includes a means for receiving a question entered by a user and inputting the question as a prompt sentence to the generative AI model, a means for the generative AI model to generate an answer to the question, and a means for displaying the generated answer to the user. This allows users to obtain quick and accurate answers to specific travel-related questions. It also provides functions to enrich the travel experience, such as streaming real-time videos and arranging for local specialties.

[0138] "User" means an individual or organization that uses the System to obtain travel information and plan and carry out travel.

[0139] "Chat generation AI means" refers to means that use artificial intelligence technology to provide travel information through dialogue with users.

[0140] "Travel information" refers to any information related to travel, such as tourist attractions, transportation, local know-how, and local specialties.

[0141] A "generative AI model" is an artificial intelligence model used to generate appropriate answers to user questions.

[0142] A "prompt" is text containing a user question or instruction that is input into a generative AI model.

[0143] "Real-time video" is video that delivers footage of the local situation in real time.

[0144] "Specialty products" are products or products that are produced in a particular region and are representative of that region.

[0145] A "server" is a computer system that receives questions from users, generates answers using a generative AI model, and provides them to users.

[0146] A "terminal" is a device through which a user inputs a question and receives and displays a response from the server.

[0147] MODE FOR CARRYING OUT THE INVENTION

[0148] This invention is a system in which, when a user inputs a travel-related question, a chat generation AI means generates an appropriate answer to the question, thereby supporting the user's travel. A specific embodiment of this system is described below.

[0149] System configuration

[0150] The system mainly consists of the following components:

[0151] 1. User Device

[0152] 2. Server

[0153] 3. Generative AI Models

[0154] User Device

[0155] The user terminal is a device that allows the user to input questions and receive and display answers from the server. Specifically, this applies to smartphones, tablets, PCs, etc. The user inputs travel-related questions into the input field of the terminal.

[0156] server

[0157] The server receives questions from users, generates answers to those questions using a generative AI model, and sends them to the user's device. The server performs the following processes:

[0158] 1. Receive a question from the user.

[0159] 2. The received question is input as a prompt to the generative AI model.

[0160] 3. Receive the answer generated by the generative AI model and send it to the user device.

[0161] Generative AI Models

[0162] A generative AI model is an artificial intelligence model used to generate appropriate answers to user questions. Specifically, generative AI models such as OpenAI's GPT-4 are used. This model has learned a large amount of travel information in advance and can provide quick and accurate answers to user questions.

[0163] Specific examples

[0164] Here's a concrete example of how it works:

[0165] Example 1: How to use the Paris subway

[0166] A user types in "How do I use the Paris subway?"

[0167] The terminal sends this question to the server.

[0168] The server receives the question and inputs the prompt sentence "Tell me how to use the Paris subway" into the generative AI model.

[0169] The generative AI model generates the answer, "On the Paris Metro, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[0170] The server sends the generated response to the terminal.

[0171] The device displays the answer to the user.

[0172] Example 2: New York tourist attractions

[0173] A user types, "What are some tourist attractions in New York?"

[0174] The terminal sends this question to the server.

[0175] The server receives the question and inputs the prompt sentence "Tell me about tourist attractions in New York" into the generative AI model.

[0176] The generative AI model generates the answer, "Tourist attractions in New York include the Statue of Liberty, Central Park, Times Square, and the Metropolitan Museum of Art."

[0177] The server sends the generated response to the terminal.

[0178] The device displays the answer to the user.

[0179] In this way, users can get fast and accurate answers to specific travel questions, and are also provided with features to enrich their travel experience, such as real-time video streaming and local specialty product arrangements.

[0180] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0181] Step 1:

[0182] The user enters a question.

[0183] A user enters a travel question into an input field on the device, for example, "How do I use the Paris subway?" The entered question is temporarily stored in the device's memory.

[0184] Step 2:

[0185] The terminal sends a question to the server.

[0186] The terminal transmits the question entered by the user to the server. At this time, the terminal transmits the question data to the server via the Internet. The input data is the user's question text. The output data is the question text transmitted to the server.

[0187] Step 3:

[0188] The server receives the question and inputs it as a prompt to the generative AI model.

[0189] The server receives the question sent from the terminal. The received question is input to the generative AI model as a prompt sentence. The input data is the user's question text. The output data is the prompt sentence input to the generative AI model.

[0190] Step 4:

[0191] A generative AI model generates answers to questions.

[0192] The generative AI model generates answers to questions based on the input prompt sentence. For example, it generates an answer such as, "For the Paris Metro, you buy a ticket at each station and insert it when passing through the ticket gate. You do not need to buy a new ticket when transferring." The input data is the prompt sentence. The output data is the generated answer text.

[0193] Step 5:

[0194] The server sends the generated response to the terminal.

[0195] The server receives the answer generated by the generative AI model and sends it to the device. At this time, the server converts the answer into an appropriate format before sending it. The input data is the generated answer text. The output data is the answer text sent to the device.

[0196] Step 6:

[0197] The terminal displays the answer to the user.

[0198] The terminal displays the answer received from the server to the user. The user can check the answer on the terminal screen. The input data is the answer text received from the server. The output data is the answer text displayed to the user.

[0199] (Application example 2)

[0200] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0201] Conventional travel support systems require users to spend a lot of time and effort to obtain information about their destination. It is also difficult to obtain appropriate information when ordering local food while traveling, resulting in low user convenience. This increases stress during travel and detracts from the travel experience.

[0202] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for assisting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time local video based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, chat generation AI means for providing food delivery information through dialogue with the user, means for assisting the user in ordering food based on the food delivery information provided by the chat generation AI means, and means for arranging local food based on the food delivery information provided by the chat generation AI means. This allows the user to quickly and accurately obtain information about their travel destination and easily order local food.

[0203] "User" means an individual or entity that uses the System to obtain travel information and food delivery information and to make travel and food orders.

[0204] "Dialogue" refers to the exchange of questions and answers between a user and a chat-generating AI means.

[0205] "Travel Information" refers to information necessary for users to plan and carry out their trips, such as tourist attractions, transportation options, and local know-how.

[0206] "Chat generation AI means" is an artificial intelligence means that provides travel information and food delivery information through dialogue with users.

[0207] "Supporting means" refers to means that support users to smoothly travel and order food based on information provided by chat generation AI means.

[0208] "Real-time video" refers to video that delivers footage of the local situation in real time.

[0209] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[0210] "Food Delivery Information" refers to information about food delivery in a particular region or city.

[0211] "Means for assisting in ordering food" refers to means for supporting users in ordering food based on food delivery information provided by chat generation AI means.

[0212] "Means for arranging local food" refers to a means for users to arrange local food based on food delivery information provided by the chat generation AI means.

[0213] The system for implementing the present invention includes a chat generation AI means for providing travel information and food delivery information through dialogue with a user. A specific embodiment of this system will be described below.

[0214] System Configuration

[0215] The system consists of the following main components:

[0216] 1. User device: The device used by the user, such as a smartphone or tablet.

[0217] 2. Server: A cloud server for running chat generation AI methods.

[0218] 3. Database: A database for storing travel information and food delivery information.

[0219] Hardware and software used

[0220] Hardware: smartphones, tablets, cloud servers

[0221] Software: Python, OpenAI GPT-3® API, database management system (e.g., MySQL®)

[0222] Data processing and calculation

[0223] 1. User terminal: The user inputs questions about travel information and food delivery information.

[0224] 2. Server: Receives the user's question and generates an appropriate answer using the OpenAI GPT-3 API.

[0225] 3. Database: Stores travel and food delivery information and provides it to the server as needed.

[0226] Processing flow

[0227] 1. A user uses their smartphone to enter a question about travel or food delivery information.

[0228] 2. The user terminal sends a query to the server.

[0229] 3. The server sends the question to the OpenAI GPT-3 API and generates an answer.

[0230] 4. The server sends the generated answer to the user terminal.

[0231] 5. The user device displays the answer to the user.

[0232] Specific examples

[0233] When a user asks, "What are the best pizza delivery options in New York?" the system works as follows:

[0234] 1. The user device sends a question to the server.

[0235] 2. The server sends the question to the OpenAI GPT-3 API and generates an answer.

[0236] 3. The server generates the answer, "A popular pizza delivery service in New York is Joe's Pizza. You can order online, and delivery takes about 30 minutes." and sends it to the user's device.

[0237] 4. The user device displays the answer to the user.

[0238] Prompt Sentence Examples

[0239] If a user asks, "What are the best sushi delivery options in Tokyo?", the following prompts might be used:

[0240] "A user asked a question about food delivery. Question: What's the best sushi delivery in Tokyo?"

[0241] answer:

[0242] In this way, users can quickly and accurately obtain information about their travel destinations and can easily order local food.

[0243] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0244] Step 1:

[0245] Users use their smartphones to input questions about travel information or food delivery information, and the input questions are captured as text data by an application on the user's device.

[0246] Step 2:

[0247] The user terminal transmits the acquired text data to the server. The transmitted data is text data containing the content of the user's question.

[0248] Step 3:

[0249] The server analyzes the received text data and generates a prompt to send to the OpenAI GPT-3 API. The prompt is generated in a format that includes the user's question.

[0250] Step 4:

[0251] The server sends the generated prompt to the OpenAI GPT-3 API. The data sent is text data containing the prompt.

[0252] Step 5:

[0253] The OpenAI GPT-3 API generates an appropriate answer based on the received prompt, which is then returned to the server as text data.

[0254] Step 6:

[0255] The server analyzes the answers received from the OpenAI GPT-3 API and converts them into a format suitable for sending to the user's device. The converted data is text data containing the answers to the user's questions.

[0256] Step 7:

[0257] The server then transmits the converted answer data to the user terminal. The transmitted data is text data containing the answer to the user's question.

[0258] Step 8:

[0259] The user terminal analyzes the received response data and displays it to the user. The displayed content is text data containing the response to the user's question.

[0260] In this way, users can quickly and accurately obtain information about their travel destinations and can easily order local food.

[0261] Example 3

[0262] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0263] Conventional travel information systems make it difficult for users to obtain the latest local information in real time or to easily arrange for local specialties. In addition, there was a lack of a centralized means of providing the information users needed when planning a trip, so users had to refer to multiple information sources, which was time-consuming.

[0264] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes: a chat generation AI means for providing travel information through dialogue with a user; a means for supporting a user in traveling based on the travel information provided by the chat generation AI means; a means for distributing real-time video of the local area based on the travel information provided by the chat generation AI means; a means for arranging local specialties based on the travel information provided by the chat generation AI means; a means for acquiring real-time video from a live camera of the local area based on the travel information provided by the chat generation AI means and distributing it to the user's terminal; and a means for linking with an online shop selling local specialties and processing the user's order based on the travel information provided by the chat generation AI means. This allows a user to obtain the latest local information in real time and easily arrange local specialties.

[0265] "User" means an individual or organization that uses the System to provide travel information and arrange local specialties.

[0266] "Chat generation AI means" refers to artificial intelligence technology for generating and providing travel information through dialogue with users.

[0267] "Travel Information" refers to information that users need when planning and carrying out their trips, such as tourist attractions, transportation options, and local know-how.

[0268] "Real-time video" refers to video data that delivers footage of the local situation in real time.

[0269] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[0270] A "live camera" refers to a camera device that captures images of a specific location in real time and distributes them over the Internet.

[0271] "Online Shop" refers to a website or platform that sells products over the Internet.

[0272] "Order processing means" refers to the systems and processes used to accept purchase requests from users, arrange for products, and process delivery.

[0273] The present invention provides a system that allows users to obtain travel information, watch real-time local video, and even arrange for local specialties. Specific embodiments of this system are described below.

[0274] System configuration

[0275] 1. User Device

[0276] Users access the system using devices such as smartphones and PCs, which have a browser and dedicated applications installed, and interact with the system through these.

[0277] 2. Server

[0278] The server receives user requests and generates appropriate travel information using a generative AI model. The server also acquires real-time video from local live cameras and distributes it to users' devices. The server also connects with online shops selling local specialties to process user orders.

[0279] 3. Generative AI Models

[0280] The generative AI model used is OpenAI's GPT-4, which generates prompts based on user requests and provides appropriate travel information.

[0281] Program processing

[0282] 1. Providing travel information

[0283] A user uses a device to send a request: "Show me what Paris looks like now."

[0284] The server receives this request and sends a prompt to the generative AI model, which is "Tell me what Paris is like now."

[0285] The generative AI model generates travel information based on the prompt sentence and returns it to the server.

[0286] The server transmits the generated travel information to the user's terminal.

[0287] 2. Real-time video distribution

[0288] The server obtains the URL of the local live camera based on the generated travel information.

[0289] The server accesses the live camera URL and retrieves the real-time video stream.

[0290] The server delivers the acquired real-time video to the user's device.

[0291] Users can watch real-time video on their devices.

[0292] 3. Arrangements for local specialties

[0293] The user uses the device to send a request saying, "I want to buy a local specialty of Paris."

[0294] The server receives this request and sends a prompt to the generative AI model: "What are some famous products of Paris?"

[0295] The generative AI model generates travel information based on the prompt sentence and returns it to the server.

[0296] Based on the generated travel information, the server calls the API of an online shop that sells local specialties and processes the user's order.

[0297] The server notifies the user that the order is complete.

[0298] Users can check the status of their orders on their terminal.

[0299] Specific examples

[0300] 1. Specific examples of real-time video streaming

[0301] User: "Show me what Paris looks like now."

[0302] Server: Receives user requests, generates travel information using GPT-4, and captures and distributes real-time video from live cameras in Paris.

[0303] User: Watch what's happening in Paris in real time on their device.

[0304] 2. Specific examples of arranging local specialties

[0305] User: "I want to buy some Parisian specialties."

[0306] Server: Receives user requests, generates travel information using GPT-4, and processes orders in collaboration with an e-commerce site selling Parisian specialties.

[0307] User: Purchase Parisian specialties on the device and have them delivered to their home.

[0308] The above is a specific embodiment for carrying out the present invention. This system allows users to obtain the latest local information in real time and easily arrange for local specialties. The flow of the specification process in Example 3 will be explained using Figure 15.

[0309] Step 1:

[0310] A user sends a request using a device. For example, the user opens a browser or a dedicated application, types "Show me the current state of Paris," and presses the send button. The input is a text request.

[0311] Step 2:

[0312] The server receives a request from the user. The server analyzes the data received as an HTTP request and extracts the request content. The input is the user's request data, and the output is the analyzed request content.

[0313] Step 3:

[0314] The server sends a prompt to the generative AI model. The server generates a prompt based on the request content and sends it to the generative AI model (e.g., GPT-4). The input is the parsed request content, and the output is the generated prompt. Specifically, the server generates the prompt, "Tell me what the current situation is in Paris."

[0315] Step 4:

[0316] The generative AI model generates travel information based on the prompt text. The generative AI model receives the prompt text and extracts and generates appropriate travel information from the vast amount of information on the Internet. The input is the prompt text, and the output is the generated travel information.

[0317] Step 5:

[0318] The server receives the generated travel information and sends it to the user's device. The server analyzes the travel information returned from the generative AI model and sends it to the user's device. The input is the generated travel information, and the output is the travel information sent to the user's device. Specifically, the server sends information such as "It's currently sunny in Paris, and the area around the Eiffel Tower is bustling with tourists."

[0319] Step 6:

[0320] The server retrieves the URL of the local live camera. Based on the generated travel information, the server calls an API to retrieve the URL of the local live camera. The input is the generated travel information, and the output is the URL of the live camera.

[0321] Step 7:

[0322] The server accesses the URL of the live camera and retrieves the real-time video stream. The server accesses the URL and retrieves the video data using the HTTP streaming protocol. The input is the URL of the live camera, and the output is the real-time video stream.

[0323] Step 8:

[0324] The server delivers the acquired real-time video to the user's device. The server sends the real-time video stream to the user's device, and the user watches the video using a browser or dedicated application. The input is the real-time video stream, and the output is the video delivered to the user's device.

[0325] Step 9:

[0326] The user watches real-time video on a device. The user watches the streamed real-time video through the device's browser or a dedicated application. The input is the streamed video, and the output is the user's viewing experience.

[0327] Step 10:

[0328] A user uses a terminal to request the delivery of local specialties. For example, the user inputs "I want to buy local specialties from Paris" and presses the send button. The input is a text request.

[0329] Step 11:

[0330] The server receives a request from the user and sends a prompt to the generative AI model. The server generates a prompt based on the request content and sends it to the generative AI model. The input is the parsed request content, and the output is the generated prompt. Specifically, the server generates the prompt, "Tell me what Paris is famous for."

[0331] Step 12:

[0332] The generative AI model generates travel information based on the prompt text. The generative AI model receives the prompt text and extracts and generates appropriate travel information from the vast amount of information on the Internet. The input is the prompt text, and the output is the generated travel information.

[0333] Step 13:

[0334] The server receives the generated travel information and calls the API of an online shop that sells local specialties. Based on the generated travel information, the server processes the user's order using the online shop's API. The input is the generated travel information, and the output is the result of order processing.

[0335] Step 14:

[0336] The server notifies the user that the order is complete. The server sends the order processing results to the user's terminal, and the user checks the order status. The input is the order processing results, and the output is the notification sent to the user's terminal.

[0337] Step 15:

[0338] The user checks the order status on the device. The user checks the order status through the device's browser or a dedicated application. The input is the sent notification, and the output is the user's confirmation result.

[0339] (Application example 3)

[0340] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0341] Conventional travel information systems make it difficult for users to grasp real-time local conditions and limit the means for easily purchasing local specialties. Furthermore, they have the problem of being unable to simultaneously experience real-time shopping in a virtual store. This limits users' travel and shopping experiences.

[0342] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0343] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time video of the local area based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, and means for purchasing local specialties while watching real-time video of the local area in a virtual store based on the travel information provided by the chat generation AI means. This enables the user to easily purchase local specialties in the virtual store while understanding the real-time situation of the local area.

[0344] "User" means an individual or corporation that uses the system to provide travel information or purchase local specialties.

[0345] "Dialogue" refers to the exchange of information between a user and a chat generation AI means.

[0346] "Travel information" refers to various information related to travel, such as tourist attractions, transportation options, and local know-how.

[0347] "Chat generation AI means" refers to an artificial intelligence system that provides travel information through dialogue with users.

[0348] "Real-time video" refers to video that delivers real-time footage of the current situation on-site.

[0349] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[0350] "Virtual Store" refers to a virtual store operated on the Internet where users can purchase products online.

[0351] "Viewing" refers to the act of a user watching a video or movie.

[0352] "Purchase" refers to the act of a user acquiring a product or service by paying money.

[0353] This invention is a system that allows users to purchase local specialties while watching real-time video of the local area in a virtual store. This system is realized using the following hardware and software.

[0354] Hardware

[0355] Smartphone

[0356] Smart Glasses

[0357] head-mounted display

[0358] server

[0359] software

[0360] Python

[0361] OpenCV (cv2)

[0362] ChatGPT(registered trademark) API

[0363] System configuration

[0364] 1. User Device:

[0365] Users access the system using smartphones, smart glasses, or head-mounted displays, which allow for user interaction and support real-time video viewing and local product purchases.

[0366] 2. Server:

[0367] The server hosts a chat generation AI system, provides travel information in response to user questions, and also handles real-time video streaming and local product ordering.

[0368] Processing flow

[0369] 1. User interaction:

[0370] When a user asks "Show me what Paris is like now" through their device, the server uses the ChatGPT API to parse the question and generate appropriate travel information.

[0371] 2. Real-time video streaming:

[0372] The server obtains the URL of the local real-time video based on the travel information and delivers it to the user's device, where the user can watch the real-time video.

[0373] 3. Arrangement of local specialties:

[0374] When a user asks, "I want to buy a local specialty from Paris," the server again uses the ChatGPT API to get information about the specialty and provides the user with a purchase link.

[0375] Specific examples

[0376] When a user launches the application on their smartphone and asks, "Show me what Paris is like now," the application will display a real-time video of Paris. If the user asks, "I want to buy some Parisian specialties," the application will display links to buy Parisian specialties.

[0377] Prompt Sentence Examples

[0378] User: Show me what Paris looks like now

[0379] System: Shows real-time video of Paris.

[0380] User: I want to buy some Parisian specialties.

[0381] System: Here are the links to buy Parisian specialties.

[0382] In this way, users can easily purchase local specialties while keeping track of the local situation in real time within the virtual store.

[0383] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0384] Step 1:

[0385] A user accesses the system using a terminal and enters questions to obtain travel information.

[0386] Input: User question (e.g., "Show me what Paris is like now")

[0387] Output: The user's question is sent to the server

[0388] Specific actions: The user uses a smartphone, smart glasses, or head-mounted display to launch the application and enter a question.

[0389] Step 2:

[0390] The server receives the user's question and parses it using the ChatGPT API.

[0391] Input: User question

[0392] Output: Travel information based on the parsed question

[0393] Specific operation: The server sends the user's query to the ChatGPT API and receives the travel information returned by the API.

[0394] Step 3:

[0395] The server obtains the URL of the local real-time video based on the travel information.

[0396] Input: Travel information based on the parsed question

[0397] Output: Real-time video URL

[0398] What happens: The server analyzes the trip information and sends a request to an external video streaming service to obtain the appropriate real-time video URL.

[0399] Step 4:

[0400] The server sends the URL of the real-time video it has acquired to the user's device.

[0401] Input: Real-time video URL

[0402] Output: URL of real-time video sent to user's device

[0403] Specific operation: The server sends the obtained URL to the user's device, allowing the user to watch the real-time video.

[0404] Step 5:

[0405] The user uses the device to watch real-time video.

[0406] Input: Real-time video URL

[0407] Output: Real-time video watched by the user

[0408] Specific operation: Users can play real-time video on their devices and check the local situation.

[0409] Step 6:

[0410] The user uses the terminal to input a question about the local specialty product they wish to purchase.

[0411] Input: User question (e.g., "I want to buy a local product from Paris")

[0412] Output: The user's question is sent to the server

[0413] How it works: The user uses a smartphone, smart glasses, or head-mounted display to input a question about the local specialty they wish to purchase.

[0414] Step 7:

[0415] The server receives the user's question and parses it using the ChatGPT API.

[0416] Input: User question

[0417] Output: Local product information based on the parsed question

[0418] Specific operation: The server sends the user's question to the ChatGPT API and receives the local product information returned by the API.

[0419] Step 8:

[0420] The server obtains a purchase link based on the specialty product information.

[0421] Input: Local product information based on the parsed question

[0422] Output: Link to purchase the specialty product

[0423] What happens: The server parses the product information and sends a request to an external shopping service to get the appropriate purchase link.

[0424] Step 9:

[0425] The server sends the acquired purchase link to the user's device.

[0426] Input: Link to purchase specialty products

[0427] Output: Purchase link sent to user's device

[0428] Specific operation: The server sends the acquired purchase link to the user's device, allowing the user to purchase the local specialty product.

[0429] Step 10:

[0430] The user uses the device to purchase local specialties.

[0431] Input: Link to purchase specialty products

[0432] Output: Specialty products purchased

[0433] Specific operation: The user clicks on the purchase link on their device and completes the purchase of the local specialty product.

[0434] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0435] "Example 1"

[0436] As one embodiment of the present invention, a system including a chat GPT means for providing travel information through dialogue with a user, a means for assisting a user in traveling based on the travel information provided by the chat generation AI means, a means for delivering real-time local video based on the travel information provided by the chat generation AI means, and a means for arranging local specialties based on the travel information provided by the chat generation AI means, is combined with an emotion engine that recognizes the user's emotions.

[0437] "Example 2"

[0438] Specifically, when a user asks, "Tell me about tourist attractions in Paris," the emotion engine recognizes the user's excitement from their tone of voice and facial expression, and responds accordingly by saying something like, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral. You'll have an exciting experience!"

[0439] "Example 3"

[0440] Furthermore, when a user asks, "How do I use the Paris subway?" the emotion engine recognizes the user's concerns and responds by providing support for their travels, such as, "For the Paris subway, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring. Please use the service with peace of mind."

[0441] "Example 4"

[0442] Furthermore, when a user asks, "Show me what's happening in Paris right now," the emotion engine recognizes the user's curiosity and delivers real-time video from the area, allowing users to get the latest information on the area.

[0443] "Example 5"

[0444] In addition, when a user says, "I want to buy a local specialty from Paris," the emotion engine recognizes the user's expectations and arranges for local specialties accordingly, allowing users to obtain local specialties without having to travel to the area.

[0445] The processing flow of each embodiment will be described below.

[0446] "Example 1"

[0447] Step 1: The user asks a travel-related question to the chat generation AI.

[0448] Step 2: The chat generation AI tool provides travel information in response to the user's questions.

[0449] Step 3: The emotion engine recognizes the user's emotion.

[0450] Step 4: Based on the recognition results of the emotion engine, the chat generation AI means provides travel information.

[0451] "Example 2"

[0452] Step 1: User asks, "Tell me about tourist attractions in Paris."

[0453] Step 2: The emotion engine recognizes excitement from the user's tone of voice and facial expression.

[0454] Step 3: Based on the recognition results of the emotion engine, the chat generation AI means responds, "The Eiffel Tower, the Louvre, Notre Dame Cathedral, etc. are famous. You will have an exciting experience!"

[0455] "Example 3"

[0456] Step 1: User asks, "How do I use the Paris Metro?"

[0457] Step 2: The emotion engine recognizes the user's anxiety.

[0458] Step 3: Based on the recognition results of the emotion engine, the chat generation AI means responds, "For the Paris Metro, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring. Please feel free to use the service."

[0459] "Example 4"

[0460] Step 1: User asks, "Show me what Paris looks like today."

[0461] Step 2: The emotion engine recognizes the user's curiosity.

[0462] Step 3: Based on the recognition results of the emotion engine, the chat generation AI means delivers real-time local video.

[0463] "Example 5"

[0464] Step 1: The user asks, "I want to buy a local product from Paris."

[0465] Step 2: The emotion engine recognizes the user's expectations.

[0466] Step 3: Based on the recognition results of the emotion engine, the chat generation AI means arranges local specialties.

[0467] Example 1

[0468] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0469] Conventional travel information systems have difficulty in quickly and accurately providing the information users need. They also face the challenge of being unable to provide a wide range of services in a unified manner, such as dialogue that takes into account the user's emotions, real-time local information, and arranging local specialties.

[0470] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0471] In this invention, the server includes a dialogue generation artificial intelligence means for providing travel information through dialogue with the user, a means for supporting the user in traveling based on the travel information provided by the dialogue generation artificial intelligence means, a means for delivering real-time local video based on the travel information provided by the dialogue generation artificial intelligence means, a means for arranging local specialties based on the travel information provided by the dialogue generation artificial intelligence means, and an emotion recognition engine for recognizing the user's emotions. This allows the user to not only quickly and accurately obtain the travel information they need, but also to receive a wide range of services in a unified manner, such as dialogue according to their emotions, real-time local information, and arrangements for local specialties.

[0472] "Dialogue generation type artificial intelligence means" refers to a means that uses artificial intelligence technology to provide travel information through dialogue with users.

[0473] "Travel information support means" refers to a means for supporting a user when planning and carrying out a trip based on travel information provided by a dialogue generation artificial intelligence means.

[0474] The "means for delivering real-time video" is a means for delivering real-time video of the local area to the user based on travel information provided by the dialogue generation artificial intelligence means.

[0475] The "means for arranging local specialties" is a means for arranging local specialties for the user based on travel information provided by the dialogue generation artificial intelligence means.

[0476] An "emotion recognition engine" is an engine that recognizes the user's emotions and provides dialogue and services accordingly.

[0477] MODE FOR CARRYING OUT THE INVENTION

[0478] This invention is a system that provides travel information through dialogue with a user, and includes a dialogue generation artificial intelligence means, a means for supporting travel information, a means for delivering real-time video, a means for arranging local specialties, and an emotion recognition engine.

[0479] 1. Generate the system program.

[0480] The server generates a program to provide travel information through dialogue with the user. This program has the function of collecting information on the Internet based on user input and providing appropriate travel information. Specifically, it uses a generative AI model (e.g., GPT-4).

[0481] 2. Explain the processing of the generated program in natural language.

[0482] When the server receives a question from a user, it uses a dialogue generation AI means to extract relevant travel information from the vast amount of information on the Internet. This AI means uses a generative AI model, for example. If a user types, "Tell me about tourist attractions in Paris," the server uses the generative AI model to generate an answer such as, "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[0483] Additionally, the system has the following additional features:

[0484] Travel support means: When a user plans a trip, assistance is provided in making travel arrangements and creating a schedule based on information provided by a dialogue generation AI means.

[0485] Real-time video distribution means: Distribute real-time video of tourist spots that users are interested in. For example, use the API of a video distribution service.

[0486] Local specialty product procurement method: Procure local specialty products that the user is interested in. For example, use an online shopping site.

[0487] Emotion recognition engine: Recognizes the user's emotions and adjusts the content and tone of the dialogue, for example, by using emotion recognition APIs.

[0488] 3. Add specific examples to your description.

[0489] When a user types "Tell me about tourist attractions in Paris," the device sends this question to the server. The server receives the question and inputs it as a prompt to the generative AI model. The generative AI model generates an answer, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral," and returns it to the server. The server sends this answer to the device, which displays it to the user.

[0490] Next, if the user types "I want to watch a real-time video of the Eiffel Tower," the device will send this request to the server. The server will call the video streaming service's API to search for "Eiffel Tower live stream" and send the link to the device. The device will then display the link to the user, allowing them to watch the real-time video.

[0491] Prompt Sentence Examples

[0492] "Tell me about the tourist attractions in Paris"

[0493] "Tell me some recommended sightseeing spots in Kyoto"

[0494] "I want to watch a real-time video of Kinkakuji Temple."

[0495] "What are some recommended restaurants near the Eiffel Tower?"

[0496] In this way, the server provides appropriate information in response to user questions, creating a multi-functional system to assist with travel planning.

[0497] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0498] Step 1:

[0499] The user enters a question.

[0500] The user inputs a question into the chat interface of the terminal, for example, "Tell me about tourist attractions in Paris." The input data is in text format.

[0501] Step 2:

[0502] The terminal sends a question to the server.

[0503] The terminal sends the user's input to the server. At this time, the input text data is sent to the server as an HTTP request. The input data is the user's question text, and the output data is the request to the server.

[0504] Step 3:

[0505] The server receives the question and inputs it as a prompt to the generative AI model.

[0506] The server analyzes the received question and inputs it as a prompt to the generative AI model. The prompt is "Tell me about tourist attractions in Paris." The input data is the user's question text, and the output data is the prompt to the generative AI model.

[0507] Step 4:

[0508] A generative AI model generates the answer.

[0509] The generative AI model generates an answer based on a prompt. For example, it generates the answer "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The input data is the prompt, and the output data is the generated answer text.

[0510] Step 5:

[0511] The server sends the generated response to the terminal.

[0512] The server sends the answer obtained from the generative AI model to the terminal. At this time, the answer is sent as an HTTP response. The input data is the generated answer text, and the output data is the response to the terminal.

[0513] Step 6:

[0514] The terminal displays the answer to the user.

[0515] The device displays the received answer on the chat interface. The user confirms the answer: "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The input data is the response from the server, and the output data is what is displayed to the user.

[0516] Step 7:

[0517] The user enters an additional request.

[0518] The user inputs an additional request, for example, "I want to see a real-time video of the Eiffel Tower." The input data is a text request.

[0519] Step 8:

[0520] The device sends an additional request to the server.

[0521] The terminal sends an additional request to the server. At this time, the input text data is sent to the server again as an HTTP request. The input data is the user's request text, and the output data is the request to the server.

[0522] Step 9:

[0523] The server processes any additional requests.

[0524] The server parses the additional request and takes appropriate action, for example calling a video streaming service's API to search for "Eiffel Tower live stream." The input data is the user's request text, and the output data is the API's response.

[0525] Step 10:

[0526] The server sends the processing result to the terminal.

[0527] The server sends the processing result to the device. For example, it sends data including a link to a live stream from a video streaming service. The input data is the response from the API, and the output data is the response to the device.

[0528] Step 11:

[0529] The terminal displays the processing results to the user.

[0530] The terminal displays the received processing results to the user. The user can click a link such as "Click here for a live stream of the Eiffel Tower" to watch the real-time video. The input data is the response from the server, and the output data is what is displayed to the user.

[0531] (Application example 1)

[0532] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0533] Conventional travel information systems have difficulty accurately providing users with the information they require, and do not provide personalized information that reflects the user's feelings. As a result, users are often dissatisfied and find it inconvenient when planning and executing their travel plans. Furthermore, there is a lack of means to provide real-time local videos and information on local specialties, making it difficult for users to obtain the latest local information.

[0534] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: a chat generation AI means for providing travel information through dialogue with the user; a means for assisting the user in traveling based on the travel information provided by the chat generation AI means; a means for delivering real-time local video based on the travel information provided by the chat generation AI means; a means for arranging local specialties based on the travel information provided by the chat generation AI means; a means including an emotion engine for recognizing the user's emotions; a means for personalizing the travel information based on the user's emotions recognized by the emotion engine; and a means for providing real-time video and local product information based on the user's emotions recognized by the emotion engine. This enables the provision of personalized travel information based on the user's emotions, thereby increasing user satisfaction. Furthermore, providing real-time local video and local product information allows the user to obtain the latest local information, making it easier to plan and execute travel plans.

[0535] "Chat generation AI means" is an artificial intelligence means that provides travel information through dialogue with users.

[0536] "Means for assisting in travel" refers to means for assisting users in planning and carrying out travel based on travel information provided by chat generation AI means.

[0537] "Means for delivering real-time video" refers to a means for delivering local real-time video to users based on travel information provided by chat generation AI means.

[0538] The "means of arranging local specialties" is a means of arranging local specialties for users based on travel information provided by the chat generation AI means.

[0539] An "emotion engine" is an engine that recognizes the user's emotions and provides appropriate information based on those emotions.

[0540] "Means for personalization" refers to means for individually adjusting the travel information provided based on the user's emotions recognized by the emotion engine.

[0541] The "means for providing real-time video and local specialty information" is a means for providing real-time video and local specialty information based on the user's emotions recognized by the emotion engine.

[0542] A system for implementing this invention includes a chat generation AI means for providing travel information through dialogue with a user, a means for assisting in travel, a means for delivering real-time local video, a means for arranging local specialties, an emotion engine for recognizing a user's emotions, a means for personalizing travel information based on emotions, and a means for providing real-time video and local specialty information.

[0543] System configuration

[0544] Hardware

[0545] Server: A central processing unit for running generative AI models and providing travel information.

[0546] User device: A device used by a user, such as a smartphone or head-mounted display (HMD).

[0547] Emotion recognition devices: Sensors such as cameras and microphones that recognize user emotions.

[0548] software

[0549] Generative AI model: Generates travel information in response to user questions using OpenAI APIs, etc.

[0550] Emotion recognition engine: Analyzes user emotions using libraries such as the EmotionRecognizer library.

[0551] Real-time video streaming API: An API for obtaining local real-time video.

[0552] Local specialty product ordering API: API for obtaining information on local specialty products.

[0553] Data processing and calculation

[0554] 1. User inquiry processing: When a user inquires about travel information through their device, the question is sent to the generative AI model, which then extracts relevant travel information from the vast amount of information on the Internet and provides it to the user.

[0555] 2. Emotion recognition: The emotion recognition device captures the user's facial expressions and voice, and the emotion recognition engine analyzes the data to recognize the user's emotions.

[0556] 3. Personalization: The generative AI model personalizes the travel information provided based on the user's perceived emotions. For example, if the user is excited, it will provide more active tourist attractions and event information.

[0557] 4. Providing real-time video and local product information: Based on user sentiment, we use real-time video streaming APIs and local product ordering APIs to provide the latest local information.

[0558] Specific examples

[0559] When a user uses a smartphone to ask, "Tell me about tourist attractions in Paris," the generative AI model responds, "Famous attractions include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." At the same time, an emotion recognition device captures the user's facial expression, and the emotion recognition engine recognizes that the user is excited. As a result, the system also provides URLs for real-time videos of Paris and information about local specialties.

[0560] Prompt Sentence Examples

[0561] "Trip Tip: Tell me about tourist attractions in Paris"

[0562] In this way, users can virtually travel and get real-time information about the local area.

[0563] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0564] Step 1:

[0565] A user uses a terminal to input a question about travel information. The input question is sent from the terminal to the server. The input data is the user's question, and the output data is the question sent to the server.

[0566] Step 2:

[0567] The server sends the received question to the generative AI model, which extracts appropriate travel information from the vast amount of information on the Internet. The input data is the user's question, and the output data is the travel information generated by the generative AI model.

[0568] Step 3:

[0569] The server transmits the travel information obtained from the generative AI model to the user terminal. The user terminal displays the received travel information to the user. The input data is the travel information generated by the generative AI model, and the output data is the travel information displayed on the user terminal.

[0570] Step 4:

[0571] The emotion recognition device captures the user's facial expressions and voice and sends the data to the server. The input data is the user's facial expressions and voice data, and the output data is the emotion data sent to the server.

[0572] Step 5:

[0573] The server uses an emotion recognition engine to analyze the received emotion data and recognize the user's emotion, where the input data is the emotion data and the output data is the recognized emotion information of the user.

[0574] Step 6:

[0575] The server personalizes the travel information provided by the generative AI model based on the recognized user emotions. The input data are the user's emotional information and the travel information generated by the generative AI model, and the output data is the personalized travel information.

[0576] Step 7:

[0577] The server transmits the personalized travel information to the user terminal. The user terminal displays the received personalized travel information to the user. The input data is the personalized travel information, and the output data is the personalized travel information displayed on the user terminal.

[0578] Step 8:

[0579] The server obtains the latest local information based on the user's emotions using a real-time video streaming API and a local specialty product ordering API. The input data is the user's emotion information, and the output data is the real-time video URL and the local specialty product information URL.

[0580] Step 9:

[0581] The server sends the acquired real-time video URL and local product information URL to the user terminal. The user terminal displays the received URL to the user. The input data is the real-time video URL and local product information URL, and the output data is the URL displayed on the user terminal.

[0582] Example 2

[0583] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0584] Conventional travel support systems have issues with the reliability and timeliness of information when users obtain travel information, and it is difficult to provide appropriate information that reflects the user's emotions. Furthermore, it is difficult to meet the diverse needs of users, such as real-time local information and arranging local specialties. This has resulted in users' travel experiences that are not fully satisfactory.

[0585] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0586] In this invention, the server includes a dialogue generation artificial intelligence means for providing travel information through dialogue with the user, a means for assisting the user in traveling based on the travel information provided by the dialogue generation artificial intelligence means, a means for delivering real-time video of the local area based on the travel information provided by the dialogue generation artificial intelligence means, a means for arranging local specialties based on the travel information provided by the dialogue generation artificial intelligence means, and a sentiment analysis means for analyzing the user's emotions and providing optimal answers based on the emotions. This allows the user to obtain reliable travel information in real time and receive appropriate information based on their emotions. Furthermore, the system can respond to a variety of needs, such as arranging real-time video of the local area and local specialties, thereby improving the user's travel experience.

[0587] "Dialogue generation artificial intelligence means" refers to artificial intelligence technology for providing travel information through dialogue with users.

[0588] "Travel information" refers to all travel-related information, such as tourist attractions, transportation options, and local know-how.

[0589] "Emotion analysis means" refers to technology that analyzes a user's emotions from their tone of voice, facial expressions, etc., and provides the most appropriate response based on those emotions.

[0590] "Real-time video distribution means" refers to technology for delivering real-time local video to users.

[0591] "Means for arranging local specialties" refers to the techniques used to arrange local specialties for users.

[0592] "Server" refers to a computer system that receives input from a user and provides appropriate information using dialogue generation artificial intelligence means and emotion analysis means.

[0593] "User" refers to an individual who uses this system to obtain travel information and plan and carry out a trip.

[0594] "Terminal" refers to the device through which a User accesses this System and inputs and receives information.

[0595] This invention is a system for users to acquire travel information and plan and execute a trip. This system includes an AI dialogue generation system, an emotion analysis system, a real-time video distribution system, and a local specialty product arrangement system.

[0596] The server receives input from the user and provides appropriate travel information using a dialogue generation AI method. Specifically, the server uses a generative AI model (e.g., GPT-4) to generate answers to the user's questions. When the user inputs a question such as "How do I use the Paris subway?" through their device, the device sends this input to the server. The server analyzes the question and inputs a prompt sentence into the generative AI model. The generative AI model generates an answer based on the input prompt sentence.

[0597] Furthermore, the server uses an emotion analysis means to analyze the user's tone of voice and facial expression. For example, if a user asks in an excited tone of voice, "Tell me about tourist attractions in Paris," the emotion analysis means will recognize the user's excitement. The server selects the optimal answer based on the user's emotion based on the answer generated by the generative AI model and the analysis results of the emotion analysis means.

[0598] The server sends the selected answer to the terminal, and the terminal displays the answer to the user, who can check the answer on the terminal screen.

[0599] For example, if a user types "How do I use the Paris subway?", the server will use the generative AI model to answer, "For the Paris subway, you buy a ticket at each station and insert it when passing through the ticket gate. You don't need to buy a new ticket when transferring." Similarly, if a user types "Tell me about tourist attractions in Paris," asking in an excited voice tone, the server will answer, "Famous attractions include the Eiffel Tower, the Louvre, and Notre Dame Cathedral. You'll have an exciting experience!"

[0600] An example of a prompt is as follows:

[0601] "How do I use the Paris subway?"

[0602] "Tell me about the tourist attractions in Paris"

[0603] This system allows users to obtain reliable travel information in real time, receive appropriate information tailored to their emotions, and can also meet a variety of needs, such as real-time local video and local product arrangements, thereby improving the user's travel experience.

[0604] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0605] Step 1:

[0606] The user enters a question.

[0607] The user inputs a travel-related question into the device, such as "How do I use the Paris subway?" The input data is stored in text format on the device.

[0608] Step 2:

[0609] The terminal sends a question to the server.

[0610] The terminal sends the question entered by the user to the server. At this time, the terminal sends the question content to the server as text data. The input data is the user's question text, and the output data is the text data sent to the server.

[0611] Step 3:

[0612] The server parses the question.

[0613] The server analyzes the received question, specifically using natural language processing techniques to understand the question and generate an appropriate prompt. The input data is the user's question text, and the output data is the generated prompt.

[0614] Step 4:

[0615] The server inputs a prompt sentence into the generative AI model.

[0616] The server inputs a prompt sentence to a generative AI model (e.g., GPT-4) based on the analyzed question. The input data is the generated prompt sentence, and the output data is the prompt sentence input to the generative AI model.

[0617] Step 5:

[0618] A generative AI model generates the answer.

[0619] The generative AI model generates an answer based on the input prompt. For example, it generates an answer such as, "For the Paris Metro, you buy a ticket at each station and insert it when passing through the ticket gate. You do not need to buy a new ticket when transferring." The input data is the prompt, and the output data is the generated answer text.

[0620] Step 6:

[0621] The server uses the emotion analysis means to analyze the user's emotions.

[0622] The server analyzes the user's emotions using emotion analysis means. It determines the user's emotional state based on the user's tone of voice and facial expression data sent from the terminal. The input data is the user's tone of voice and facial expression data, and the output data is the analyzed emotional information.

[0623] Step 7:

[0624] The server selects the best response based on the emotion.

[0625] The server selects the optimal answer based on the user's emotions based on the answers generated by the generative AI model and the analysis results of the emotion analysis means. For example, if the user is excited, it selects an answer that appeals to their emotions more. The input data is the generated answer text and analyzed emotion information, and the output data is the selected optimal answer text.

[0626] Step 8:

[0627] The server sends the response to the terminal.

[0628] The server sends the selected answer to the terminal. The answer is sent as text data. The input data is the selected optimal answer text, and the output data is the answer text sent to the terminal.

[0629] Step 9:

[0630] The terminal displays the answer to the user.

[0631] The terminal displays the answer received from the server to the user. The user can check the answer on the terminal screen. The input data is the answer text sent from the server, and the output data is the answer text displayed on the terminal screen.

[0632] (Application example 2)

[0633] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0634] Conventional travel information systems only provide static information in response to user questions, and have the problem of being unable to provide personalized information based on the user's emotions and interests. Furthermore, they are limited to travel information and lack support for choosing meals and restaurants while traveling. This often makes it difficult for users to choose meals while traveling.

[0635] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0636] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time local video based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, means for suggesting appropriate dishes and restaurants in response to user questions, and means for recognizing the user's emotions and suggesting special offers and recommended menus accordingly. This makes it possible to provide personalized information according to the user's emotions and interests, and also to provide support for choosing meals during the trip.

[0637] "Chat generation AI means" is an artificial intelligence technology that provides information in natural language through dialogue with users.

[0638] "Travel information" refers to all travel-related information, such as tourist attractions, transportation options, and local know-how.

[0639] "Means for recognizing user emotions" refers to technology that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[0640] "Special Offers" refer to exclusive discounts and perks that are tailored to a user's emotions and interests.

[0641] "Recommended menu" refers to food or restaurant menus suggested based on the user's preferences and feelings.

[0642] "Means for delivering real-time video" refers to technology that provides users with real-time footage of the local area.

[0643] "Means for arranging local specialties" refers to the technology for arranging local specialties in response to user requests.

[0644] "Means for suggesting dishes and restaurants" refers to technology that recommends appropriate dishes and restaurants based on user questions.

[0645] The system for implementing this invention includes a chat generation AI means for providing travel information through dialogue with the user, a means for recognizing the user's emotions, and a means for suggesting appropriate dishes and restaurants in response to the user's questions.

[0646] System configuration

[0647] The system consists of the following main components:

[0648] 1. Chat generation AI method: Provides travel information through dialogue with users. Uses OpenAI's API to generate answers in natural language to users' questions.

[0649] 2. Emotion Recognition: Recognizes emotions by analyzing the user's tone of voice and facial expressions. OpenCV and EmotionRecognizer are used to analyze emotions from images captured by the camera.

[0650] 3. Food and restaurant suggestions: Based on the user's questions and emotions, the app suggests appropriate food and restaurants. This also uses OpenAI's API to generate answers to the user's questions.

[0651] Hardware and Software

[0652] Hardware: Uses your smartphone's camera and microphone.

[0653] Software: OpenAI API, OpenCV, EmotionRecognizer.

[0654] Data processing and calculation

[0655] 1. Obtaining user's question: The user enters a question into their smartphone.

[0656] 2. Emotion Recognition: The smartphone camera captures the user's facial expressions and analyzes their emotions using EmotionRecognizer.

[0657] 3. Chat generation: Send the user's question and sentiment data to the OpenAI API to generate an appropriate answer.

[0658] 4. Show Suggestion: Show the generated answer to the user.

[0659] Specific examples

[0660] If a user asks, "I want to eat pizza," and the camera captures their facial expression and recognizes that they are excited, the AI ​​will suggest, "As a special offer, we're offering Margherita pizza at half price!"

[0661] Prompt Sentence Examples

[0662] "If users are excited, suggest special offers or menu recommendations."

[0663] User Question: I want pizza

[0664] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0665] Step 1:

[0666] The user types a question into their smartphone.

[0667] Input: User question (e.g. "I want pizza")

[0668] Output: User question text

[0669] Specific actions: The user opens the application on their smartphone, enters a question in the text box, and presses the send button.

[0670] Step 2:

[0671] The device captures the user's facial expressions using the smartphone camera.

[0672] Input: User's face image

[0673] Output: Captured face image data

[0674] Specific operation: The smartphone camera takes a picture of the user's face and acquires image data.

[0675] Step 3:

[0676] The device uses EmotionRecognizer to analyze the user's emotions from the captured facial image.

[0677] Input: Captured face image data

[0678] Output: User's emotion data (e.g. excited)

[0679] Specific operation: EmotionRecognizer analyzes facial images and determines the user's emotional state.

[0680] Step 4:

[0681] The device sends the user's question and emotion data to the OpenAI API, which generates an appropriate answer.

[0682] Input: User question text, user emotion data

[0683] Output: Generated answer text (e.g., "As a special offer, we're offering Margherita pizzas at half price!")

[0684] Specific operation: The device incorporates the user's question and emotion data into a prompt sentence and sends it to the OpenAI API. It receives the response from the API and generates an appropriate answer.

[0685] Step 5:

[0686] The terminal displays the generated answer to the user.

[0687] Input: Generated answer text

[0688] Output: Answers displayed on the user's smartphone screen

[0689] Specific operation: The device displays the generated answer text on the user's smartphone screen and notifies the user.

[0690] Example 3

[0691] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0692] Conventional travel information systems have had issues with providing users with the real-time information and local specialties they require, and are unable to respond appropriately to the user's emotions. As a result, users are unable to obtain the latest local information, which often causes inconvenience when planning and executing their trips. Furthermore, there is a lack of support that responds to the user's emotions, making it difficult to respond appropriately to their anxieties and expectations during their trip.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0694] In this invention, the server includes a generation AI means for providing travel information through dialogue with a user, a means for supporting a user in traveling based on the travel information provided by the generation AI means, a means for delivering real-time video of the local area based on the travel information provided by the generation AI means, a means for arranging local specialties based on the travel information provided by the generation AI means, an emotion recognition means for recognizing a user's emotion based on the travel information provided by the generation AI means and generating an appropriate response, a means for delivering real-time video of the local area in accordance with the user's emotion recognized by the emotion recognition means, and a means for arranging local specialties in accordance with the user's emotion recognized by the emotion recognition means. This allows a user to obtain real-time local information, making it easier to plan and carry out a trip, and enabling the user to receive appropriate support in accordance with their emotion.

[0695] "Generative AI means" refers to artificial intelligence technology that provides travel information through dialogue with users.

[0696] "Travel information" refers to information related to travel, such as local tourist attractions, transportation options, and local know-how.

[0697] "Real-time video distribution means" refers to technology that captures real-time video footage from the local area and distributes it to users.

[0698] "Specialty product arrangement method" refers to the technology of arranging local specialties based on user requests.

[0699] "Emotion recognition means" refers to technology that analyzes a user's emotions and generates an appropriate response accordingly.

[0700] "User" means an individual who uses the System to obtain travel information and plan or undertake a trip.

[0701] "Server" refers to a computer system that receives requests from users and provides appropriate information and services using generative AI means and emotion recognition means.

[0702] "Terminal" refers to the device (smartphone, PC, etc.) used by a user to access the system.

[0703] The system of the present invention provides travel information through dialogue with users, and has the ability to deliver real-time local videos and arrange local specialties in response to user requests. It also uses an emotion engine to recognize user emotions and respond appropriately accordingly.

[0704] Hardware and software used

[0705] Hardware: Servers, user devices (smartphones, PCs, etc.)

[0706] Software: Generative AI models (e.g., OpenAI's GPT-4), emotion engines, real-time video streaming systems, specialty product ordering systems

[0707] Data processing and calculation

[0708] 1. User Request Submission

[0709] The user uses a device (smartphone or PC) to send a request to the system.

[0710] Example: A user opens a chat app on their smartphone, types "Show me what's happening in Paris right now," and sends it.

[0711] 2. Server receives and analyzes the request

[0712] The server receives the request from the user and analyzes its contents.

[0713] Example: A server receives a request to "Show me what Paris is up to now," analyzes the request, and determines that it needs to provide a real-time video of Paris.

[0714] 3. Sending prompts to the generative AI model

[0715] Based on the request content analyzed by the server, a prompt is sent to the generative AI model.

[0716] Example: The server sends a prompt to a generative AI model (e.g., GPT-4) saying, "The user wants to see what Paris is currently like. Please provide a real-time video."

[0717] 4. Emotion Recognition by Emotion Engine

[0718] The server uses an emotion engine to recognize emotions from the user's request.

[0719] Example: An emotion engine recognizes "curiosity" in a user's request and determines what the user is interested in.

[0720] 5. Real-time video streaming or local specialty product arrangement

[0721] The server starts a real-time video distribution system or a local specialty product ordering system in response to a user request.

[0722] Examples:

[0723] Real-time video streaming: The server starts the real-time video streaming system, acquires live camera footage of Paris, and streams it to users.

[0724] Arranging local specialties: When a user requests "I want to buy a local specialty of Paris," the server launches the local specialties ordering system and arranges for the local specialty to be purchased.

[0725] 6. Sending Responses to Users

[0726] The server sends the acquired information and arrangement details to the user.

[0727] Examples:

[0728] Real-time video streaming: The server captures live camera footage of Paris and streams it to the user's smartphone, sending a message saying, "This is the current state of Paris."

[0729] Arrange local specialties: The server notifies the user that the local specialties have been arranged and sends a message saying, "Paris local specialties have been arranged. Please wait for them to arrive."

[0730] Prompt Sentence Examples

[0731] "Users want to see what's happening in Paris right now. Give them real-time video."

[0732] "A user wants to buy a Paris specialty. Can you help me with that?"

[0733] In this way, the system provides appropriate information and services in response to user requests. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0734] Step 1:

[0735] The user uses the device to submit a request.

[0736] Input: A user types into a terminal, "Show me what Paris looks like now."

[0737] Specific behavior: A user opens a chat app on their smartphone, types in a request, and presses the send button.

[0738] Output: The user's request is sent to the server.

[0739] Step 2:

[0740] The server receives and analyzes the user request.

[0741] Input: The request data sent by the user.

[0742] What happens: The server receives the request and analyzes the request using a text analysis algorithm.

[0743] Output: The request is determined to be "I need to provide a real-time video of Paris."

[0744] Step 3:

[0745] Based on the request content analyzed by the server, a prompt is sent to the generative AI model.

[0746] Input: The parsed request content.

[0747] What happens: The server sends a prompt to the generative AI model (e.g., GPT-4) saying, "The user wants to see what Paris is currently like. Please provide a real-time video."

[0748] Output: The generative AI model receives the prompt and is ready to generate an appropriate response.

[0749] Step 4:

[0750] The server uses an emotion engine to recognize emotions from the user's request.

[0751] Input: User request data.

[0752] What happens: The server launches the emotion engine, analyzes the request text, and recognizes the user's emotion.

[0753] Output: The emotion engine recognizes the user's "curiosity."

[0754] Step 5:

[0755] The server starts the real-time video distribution system in response to the user's request.

[0756] Input: User sentiment and request content recognized by the sentiment engine.

[0757] Specific operation: The server starts the real-time video distribution system and acquires live camera footage of Paris.

[0758] Output: Captured live camera video data.

[0759] Step 6:

[0760] The server sends the retrieved information to the user.

[0761] Input: Captured live camera footage data.

[0762] What it does: The server streams live camera footage to the user's device and sends a message saying, "This is what Paris looks like right now."

[0763] Output: A live camera image and a message will be displayed on the user's device.

[0764] In this way, the system provides appropriate information and services in response to user requests.

[0765] (Application example 3)

[0766] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0767] Conventional travel information systems make it difficult for users to obtain the latest local information in real time, and there are limited ways to easily arrange local specialties. Furthermore, they do not provide appropriate information based on the user's emotions, which reduces user satisfaction. To solve these issues, a system is needed that can deliver real-time video and arrange specialties in response to user requests, as well as recognize the user's emotions and provide appropriate information.

[0768] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0769] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time video of the local area based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, emotion recognition means for recognizing the user's emotion based on the travel information provided by the chat generation AI means and providing appropriate information, means for acquiring a URL of a real-time video in response to a user's request based on the travel information provided by the chat generation AI means, and means for acquiring information on local specialties in response to a user's request based on the travel information provided by the chat generation AI means. This allows the user to obtain the latest local information in real time and easily arrange local specialties, and further enables the provision of appropriate information in response to the user's emotion.

[0770] "Chat generation AI means" is an artificial intelligence means that provides travel information through dialogue with users.

[0771] "Travel information" refers to all travel-related information, such as tourist attractions, transportation options, and local know-how.

[0772] "Means for delivering real-time video" refers to a means for providing users with the latest local video footage in real time.

[0773] "Means for arranging local specialties" refers to the means for arranging local specialties in response to user requests.

[0774] An "emotion recognition means" is a means for recognizing a user's emotions and providing appropriate information accordingly.

[0775] "Means for obtaining a URL for a real-time video" refers to a means for obtaining a URL for a real-time video in response to a user request.

[0776] The "means for acquiring information on local specialties" refers to a means for acquiring information on local specialties in response to a user's request.

[0777] To implement this invention, the following system configuration and program are required. The system includes a chat generation AI means for providing travel information through dialogue with the user, a means for supporting the user in planning their trip based on the travel information, a means for delivering real-time local video, a means for arranging local specialties, an emotion recognition means for recognizing the user's emotions and providing appropriate information, a means for acquiring the URL of the real-time video, and a means for acquiring information on local specialties.

[0778] System Program

[0779] Hardware / Software used

[0780] EmotionEngine: An engine for recognizing user emotions. Uses emotion recognition API.

[0781] ChatAI: A generative chat AI that generates answers based on user requests. It uses generative AI models such as GPT-3.

[0782] VideoStreaming: A module for streaming local real-time video. It uses YouTube (registered trademark) Live API, etc.

[0783] ProductOrdering: A module for ordering local specialties. Uses mail order API.

[0784] Program processing explanation

[0785] The server first receives input from the user. The user's input text is passed to the Emotion Engine, which recognizes the user's emotions. The recognized emotions and the user's input text are then passed to ChatAI, which generates an appropriate answer.

[0786] For example, if a user says, "Show me the current state of Paris," the server uses the VideoStreaming module to obtain a URL for a real-time video of Paris and provides it to the user. If a user says, "I want to buy some Parisian specialties," the server uses the ProductOrdering module to obtain information about Parisian specialties and provides it to the user.

[0787] Specific examples

[0788] Prompt Sentence Examples

[0789] "Show me what Paris looks like now."

[0790] "I want to buy a specialty product of Paris."

[0791] "How do I use the Paris subway?"

[0792] This will allow users to obtain the latest local information in real time, easily arrange for local specialties, and provide appropriate information based on the user's emotions.

[0793] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0794] Step 1:

[0795] The user uses the terminal to input a request for travel information, for example, a prompt such as "Show me what Paris is like now" or "I want to buy some local Parisian products." The input data is sent to the server in text format.

[0796] Step 2:

[0797] The server passes the received user input text to the Emotion Engine. The Emotion Engine analyzes the input text and recognizes the user's emotion. For example, if a user inputs "Show me the current state of Paris," the Emotion Engine recognizes the user's curiosity. The input is the user's text, and the output is the recognized emotion.

[0798] Step 3:

[0799] The server passes the recognized emotion and the user's input text to ChatAI. ChatAI uses a generative AI model to generate an appropriate response based on the user's request. For example, in response to the request "Show me what Paris is like now," ChatAI generates a response such as "I'll show you what Paris is like now." The input is the user's text and the recognized emotion, and the output is the generated response.

[0800] Step 4:

[0801] The server calls the appropriate module based on the user's request. For example, if a user requests "Show me the current state of Paris," the server calls the VideoStreaming module and obtains the URL of a real-time video of Paris. The input is the user's request, and the output is the URL of the real-time video.

[0802] Step 5:

[0803] The server provides the URL of the acquired real-time video to the user. The user can then watch the real-time video through their device. The input is the URL of the real-time video, and the output is the video displayed on the user's device.

[0804] Step 6:

[0805] When a user requests "I want to buy a Parisian specialty," the server calls the ProductOrdering module to obtain information about Parisian specialty products. The input is the user's request, and the output is the specialty product information.

[0806] Step 7:

[0807] The server provides the acquired information about the local specialty to the user. The user can check the detailed information about the local specialty through their terminal and proceed with the purchase. The input is the information about the local specialty, and the output is the detailed information about the local specialty displayed on the user's terminal.

[0808] This allows users to obtain the latest local information in real time, easily arrange for local specialties, and also provides appropriate information according to the user's feelings.

[0809] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0810] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0811] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0812] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0813] [Second embodiment]

[0814] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0815] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0816] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0817] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0818] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0819] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0820] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0821] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0822] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0823] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0824] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0825] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0826] "Example 1"

[0827] The system of the present invention is equipped with a chat generation AI means that provides travel information through dialogue with the user. This chat generation AI means extracts and explains travel information from a vast amount of information on the Internet. For example, if a user asks, "Tell me about tourist attractions in Paris," the chat generation AI means will respond based on information on the Internet in the form of, "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[0828] "Example 2"

[0829] The system of the present invention also includes a means for supporting a user in traveling based on travel information provided by the chat generation AI means. For example, if a user asks, "Please tell me how to use the Paris subway," the chat generation AI means will respond by providing assistance with the user's travel in the form of, "For the Paris subway, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[0830] "Example 3"

[0831] Furthermore, the system of the present invention is equipped with a means for delivering real-time local video based on travel information provided by the chat generation AI means. For example, if a user asks, "Show me the current situation in Paris," the system will deliver real-time local video, allowing the user to obtain the latest local information.

[0832] "Example 4"

[0833] The system of the present invention also includes a means for arranging local specialties based on travel information provided by the chat generation AI means. For example, if a user asks, "I want to buy a local specialty from Paris," the system will arrange for the local specialty. This allows users to obtain local specialties without having to travel to the location.

[0834] The processing flow of each embodiment will be described below.

[0835] "Example 1"

[0836] Step 1: The user asks the system, "Tell me about tourist attractions in Paris."

[0837] Step 2: The chat generation AI in the system responds based on information on the Internet, such as "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[0838] "Example 2"

[0839] Step 1: The user asks the system, "How do I use the Paris subway?"

[0840] Step 2: The chat generation AI within the system responds by providing support for the journey, such as, "For the Paris Metro, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[0841] "Example 3"

[0842] Step 1: The user asks the system, "Show me the current situation in Paris." Step 2: The system delivers real-time video of the local area, allowing the user to get the latest information on the local area.

[0843] "Example 4"

[0844] Step 1: The user asks the system, "I want to buy a Parisian specialty."

[0845] Step 2: The system arranges for local specialties, allowing users to obtain the specialties without having to travel to the local area.

[0846] Example 1

[0847] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0848] Conventional travel information systems have had the problem of being unable to provide users with the information they need quickly and accurately. Furthermore, when users ask specific questions, it takes a lot of time and effort to get an appropriate answer. Furthermore, it is difficult to meet diverse travel-related needs, such as providing real-time information or arranging for local specialties.

[0849] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0850] In this invention, the server includes an information generation means for providing travel information through dialogue with a user, a means for supporting a user in traveling based on the travel information provided by the information generation means, a means for delivering real-time local video based on the travel information provided by the information generation means, a means for arranging local specialties based on the travel information provided by the information generation means, a means for receiving an inquiry from a user and sending it as a prompt to a generative AI model, a means for the generative AI model to search information on the Internet and generate an answer, and a means for returning the generated answer to the user. This allows users to quickly and accurately obtain travel information and makes it possible to meet a variety of needs, such as providing real-time information and arranging local specialties.

[0851] The "information generating means" is a means for providing travel information through dialogue with the user.

[0852] "Means for supporting travel" refers to means for supporting a user in planning and carrying out a travel based on travel information provided by an information generating means.

[0853] The "means for delivering real-time video" is a means for delivering local real-time video to users based on travel information provided by the information generating means.

[0854] The "means for arranging local specialties" is a means for arranging local specialties for the user based on the travel information provided by the information generating means.

[0855] "Means for receiving a query and sending it to the generative AI model as a prompt sentence" refers to means for receiving a query from a user and sending it to the generative AI model as a prompt sentence.

[0856] "Means by which the generative AI model searches for information on the Internet and generates an answer" refers to the means by which the generative AI model searches for information on the Internet based on a prompt sentence and generates an appropriate answer.

[0857] "Means for returning the generated answer to the user" refers to means for returning the answer generated by the generative AI model to the user.

[0858] This invention is a system for providing travel information through dialogue with a user. The system includes an information generating means, a means for supporting travel, a means for delivering real-time video, a means for arranging local specialties, a means for receiving an inquiry and sending it as a prompt to a generative AI model, a means for the generative AI model to search for information on the Internet and generate an answer, and a means for returning the generated answer to the user.

[0859] Hardware and software used

[0860] This system uses a server, a terminal, and a generative AI model. Specifically, the server is a high-performance cloud server, and the terminal is a device that provides the user interface, such as a smartphone or PC. The generative AI model uses an advanced natural language processing model such as OpenAI's GPT-4.

[0861] Data processing and calculation

[0862] The server receives a user's query and sends it to the generative AI model as a prompt. The generative AI model searches vast amounts of information on the Internet based on the prompt and generates an appropriate answer. The generated answer is then sent back to the user via the server.

[0863] Specific examples

[0864] Example 1

[0865] The user types "Tell me about tourist attractions in Paris" into the device. The device sends this query to the server. The server sends the prompt "Tell me about tourist attractions in Paris" to the generative AI model. The generative AI model searches information on the internet and generates the answer "Famous attractions include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The server sends this answer back to the device, which displays it to the user.

[0866] Example 2

[0867] The user types "What restaurants in Tokyo do you recommend?" into the device. The device sends this query to the server. The server sends the prompt "What restaurants in Tokyo do you recommend?" to the generative AI model. The generative AI model searches information on the internet and generates the answer "Popular sushi restaurants in Ginza and ramen restaurants in Shinjuku." The server sends this answer back to the device, which displays it to the user.

[0868] Prompt Sentence Examples

[0869] 1. "Tell me about the tourist attractions in Paris."

[0870] 2. "What are some recommended restaurants in Tokyo?"

[0871] 3. "What museums should I visit in New York?"

[0872] This allows users to quickly and accurately obtain travel information, and makes it possible to meet a variety of needs, such as providing real-time information and arranging for local specialties.

[0873] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0874] Step 1:

[0875] The user types a query into the terminal.

[0876] When a user wants to know travel information, they can type a question into the chat interface of the device, for example, "Tell me about tourist attractions in Paris." The typed query is temporarily stored in the device's internal memory.

[0877] Step 2:

[0878] The terminal sends a query to the server.

[0879] The terminal sends the query entered by the user to the server. At this time, the terminal converts the query content into an appropriate format and uses a communication protocol (e.g., HTTP POST request) to send it to the server. The input is the user's query, and the output is the data to be sent to the server.

[0880] Step 3:

[0881] The server receives the query and sends it as a prompt to the generative AI model.

[0882] The server analyzes the query received from the device and sends it to the generative AI model as a prompt. The prompt may be the user's question as is, or it may be preprocessed as needed. The input is the query data from the device, and the output is the prompt to the generative AI model.

[0883] Step 4:

[0884] A generative AI model searches the internet for information and generates answers.

[0885] A generative AI model searches for relevant data from the vast amount of information on the Internet based on a prompt and generates an answer. For example, in response to the prompt, "Tell me about tourist attractions in Paris," it generates the answer, "Famous attractions include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The input is the prompt, and the output is the generated answer.

[0886] Step 5:

[0887] The server receives the generated response and sends it to the terminal.

[0888] The server analyzes the response received from the generative AI model, converts it into an appropriate format, and sends it to the terminal. At this time, the server uses a communication protocol (e.g., HTTP response). The input is the response data from the generative AI model, and the output is the data sent to the terminal.

[0889] Step 6:

[0890] The device displays the answer to the user.

[0891] The terminal displays the answer received from the server to the user. The user can check the answer on the terminal screen. The input is the answer data from the server, and the output is the display data to the user.

[0892] (Application example 1)

[0893] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0894] Conventional travel information systems have the problem that it takes a lot of time and effort for users to obtain information about travel destinations. In addition, they do not provide real-time information or a travel experience in a virtual environment, which results in low user satisfaction.

[0895] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0896] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time local video based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, means for the user to select a travel destination in a virtual store and be provided with travel information in real time, and means for the user to obtain information on tourist attractions and recommended spots related to the travel destination in the virtual store. This allows the user to efficiently obtain travel information and enjoy a real-time travel experience in a virtual environment.

[0897] "User" means an individual or organization that uses the system to obtain travel information and engage in travel experiences within the virtual store.

[0898] "Dialogue" refers to the exchange of information between a user and a chat generation AI means.

[0899] "Travel information" refers to any information related to travel, such as tourist attractions, transportation, local know-how, and local specialties.

[0900] "Chat generation AI means" refers to the function of artificial intelligence that provides travel information through dialogue with users.

[0901] "Supporting means" refers to a function that supports users in traveling based on travel information provided by chat generation AI means.

[0902] "Means for delivering real-time video" refers to the function of providing users with local footage in real time.

[0903] "Means for arranging local specialties" refers to the function of making arrangements to provide local specialties and famous products to users.

[0904] "Virtual store" refers to a virtual store where users can select travel destinations and obtain travel information within a virtual space.

[0905] "Tourist attractions" refer to places and facilities worth visiting in a travel destination.

[0906] "Recommended spots" refer to places or facilities that are particularly recommended to visit at a travel destination.

[0907] The system for implementing this invention is mainly composed of chat generation AI means that provides travel information through dialogue with users. A specific embodiment of this system will be described below.

[0908] First, users access a virtual store using a device such as a smartphone or head-mounted display. Within the virtual store, users can select a travel destination and obtain information about that destination through chat generation AI.

[0909] The server uses OpenAI's API to implement a chat generation AI system. When a user enters a question about a travel destination, the question is generated as a prompt and sent to OpenAI's generative AI model. The generative AI model then extracts appropriate travel information from the vast amount of information on the Internet and provides it to the user.

[0910] For example, if the user types "Tell me about tourist attractions in Paris," the prompt text might look like this:

[0911] "Tell me about the tourist attractions in Paris"

[0912] This prompt is sent to a generative AI model, which then generates an answer such as, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." This answer is then displayed on the user's device.

[0913] In addition, the system is equipped with a means for streaming real-time video, allowing users to view real-time footage of the selected travel destination, giving them the experience of actually visiting the location.

[0914] The system also provides a means to arrange local specialties, allowing users to easily purchase the specialties they are interested in. This helps users gain a deeper understanding of the culture and specialties of the destination they are traveling to.

[0915] In this way, users can efficiently obtain travel information and enjoy real-time travel experiences in a virtual environment. The hardware used is a smartphone and a head-mounted display, and the software is Python and OpenAI API.

[0916] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0917] Step 1:

[0918] The user accesses the virtual store using a smartphone or head-mounted display. The user selects a travel destination and inputs a question to obtain travel information. The input data is the user's selected travel destination and the question.

[0919] Step 2:

[0920] The terminal receives the user's input and generates a prompt based on the question. The generated prompt might be something like, "Tell me about tourist attractions in Paris." This prompt becomes the input data for the next processing step.

[0921] Step 3:

[0922] The server sends the generated prompt to OpenAI's generative AI model, which then extracts relevant travel information from the vast amount of information on the Internet based on the prompt. This process takes the prompt as input and outputs text data containing travel information.

[0923] Step 4:

[0924] The server receives the travel information obtained from the generative AI model and sends it to the user's device. The user's device displays this travel information. For example, it displays information such as "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[0925] Step 5:

[0926] If the user wants more information, they enter a follow-up question. The device generates a new prompt and sends it to the server. The process repeats steps 2 through 4.

[0927] Step 6:

[0928] When a user wants to watch real-time video of a selected travel destination, the device sends the request to the server, which retrieves the local real-time video and delivers it to the user's device, where the user can watch the real-time video.

[0929] Step 7:

[0930] When a user wants to purchase a local specialty product, the device sends the request to the server, which then arranges for the product to be delivered to the user, and the user can then complete the purchase process on the device.

[0931] In this way, users can efficiently obtain travel information and enjoy real-time travel experiences in a virtual environment.

[0932] Example 2

[0933] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0934] Conventional travel information systems have struggled to provide users with the specific travel information they need quickly and accurately. Furthermore, users have had limited access to the information they need in real time while traveling, which often makes planning and executing trips inconvenient. Furthermore, they lacked features to enrich the travel experience, such as arranging local specialties and streaming real-time videos.

[0935] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0936] In this invention, the server includes a means for receiving a question entered by a user and inputting the question as a prompt sentence to the generative AI model, a means for the generative AI model to generate an answer to the question, and a means for displaying the generated answer to the user. This allows users to obtain quick and accurate answers to specific travel-related questions. It also provides functions to enrich the travel experience, such as streaming real-time videos and arranging for local specialties.

[0937] "User" means an individual or organization that uses the System to obtain travel information and plan and carry out travel.

[0938] "Chat generation AI means" refers to means that use artificial intelligence technology to provide travel information through dialogue with users.

[0939] "Travel information" refers to any information related to travel, such as tourist attractions, transportation, local know-how, and local specialties.

[0940] A "generative AI model" is an artificial intelligence model used to generate appropriate answers to user questions.

[0941] A "prompt" is text containing a user question or instruction that is input into a generative AI model.

[0942] "Real-time video" is video that delivers footage of the local situation in real time.

[0943] "Specialty products" are products or products that are produced in a particular region and are representative of that region.

[0944] A "server" is a computer system that receives questions from users, generates answers using a generative AI model, and provides them to users.

[0945] A "terminal" is a device through which a user inputs a question and receives and displays a response from the server.

[0946] MODE FOR CARRYING OUT THE INVENTION

[0947] This invention is a system in which, when a user inputs a travel-related question, a chat generation AI means generates an appropriate answer to the question, thereby supporting the user's travel. A specific embodiment of this system is described below.

[0948] System configuration

[0949] The system mainly consists of the following components:

[0950] 1. User Device

[0951] 2. Server

[0952] 3. Generative AI Models

[0953] User Device

[0954] The user terminal is a device that allows the user to input questions and receive and display answers from the server. Specifically, this applies to smartphones, tablets, PCs, etc. The user inputs travel-related questions into the input field of the terminal.

[0955] server

[0956] The server receives questions from users, generates answers to those questions using a generative AI model, and sends them to the user's device. The server performs the following processes:

[0957] 1. Receive a question from the user.

[0958] 2. The received question is input as a prompt to the generative AI model.

[0959] 3. Receive the answer generated by the generative AI model and send it to the user device.

[0960] Generative AI Models

[0961] A generative AI model is an artificial intelligence model used to generate appropriate answers to user questions. Specifically, generative AI models such as OpenAI's GPT-4 are used. This model has learned a large amount of travel information in advance and can provide quick and accurate answers to user questions.

[0962] Specific examples

[0963] Here's a concrete example of how it works:

[0964] Example 1: How to use the Paris subway

[0965] A user types in "How do I use the Paris subway?"

[0966] The terminal sends this question to the server.

[0967] The server receives the question and inputs the prompt sentence "Tell me how to use the Paris subway" into the generative AI model.

[0968] The generative AI model generates the answer, "On the Paris Metro, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[0969] The server sends the generated response to the terminal.

[0970] The device displays the answer to the user.

[0971] Example 2: New York tourist attractions

[0972] A user types, "What are some tourist attractions in New York?"

[0973] The terminal sends this question to the server.

[0974] The server receives the question and inputs the prompt sentence "Tell me about tourist attractions in New York" into the generative AI model.

[0975] The generative AI model generates the answer, "Tourist attractions in New York include the Statue of Liberty, Central Park, Times Square, and the Metropolitan Museum of Art."

[0976] The server sends the generated response to the terminal.

[0977] The device displays the answer to the user.

[0978] In this way, users can get fast and accurate answers to specific travel questions, and are also provided with features to enrich their travel experience, such as real-time video streaming and local specialty product ordering.

[0979] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0980] Step 1:

[0981] The user enters a question.

[0982] A user enters a travel question into an input field on the device, for example, "How do I use the Paris subway?" The entered question is temporarily stored in the device's memory.

[0983] Step 2:

[0984] The terminal sends a question to the server.

[0985] The terminal transmits the question entered by the user to the server. At this time, the terminal transmits the question data to the server via the Internet. The input data is the user's question text. The output data is the question text transmitted to the server.

[0986] Step 3:

[0987] The server receives the question and inputs it as a prompt to the generative AI model.

[0988] The server receives the question sent from the terminal. The received question is input to the generative AI model as a prompt sentence. The input data is the user's question text. The output data is the prompt sentence input to the generative AI model.

[0989] Step 4:

[0990] A generative AI model generates answers to questions.

[0991] The generative AI model generates answers to questions based on the input prompt sentence. For example, it generates an answer such as, "For the Paris Metro, you buy a ticket at each station and insert it when passing through the ticket gate. You do not need to buy a new ticket when transferring." The input data is the prompt sentence. The output data is the generated answer text.

[0992] Step 5:

[0993] The server sends the generated response to the terminal.

[0994] The server receives the answer generated by the generative AI model and sends it to the device. At this time, the server converts the answer into an appropriate format before sending it. The input data is the generated answer text. The output data is the answer text sent to the device.

[0995] Step 6:

[0996] The terminal displays the answer to the user.

[0997] The terminal displays the answer received from the server to the user. The user can check the answer on the terminal screen. The input data is the answer text received from the server. The output data is the answer text displayed to the user.

[0998] (Application example 2)

[0999] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1000] Conventional travel support systems require users to spend a lot of time and effort to obtain information about their destination. It is also difficult to obtain appropriate information when ordering local food while traveling, resulting in low user convenience. This increases stress during travel and detracts from the travel experience.

[1001] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for assisting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time local video based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, chat generation AI means for providing food delivery information through dialogue with the user, means for assisting the user in ordering food based on the food delivery information provided by the chat generation AI means, and means for arranging local food based on the food delivery information provided by the chat generation AI means. This allows the user to quickly and accurately obtain information about their travel destination and easily order local food.

[1002] "User" means an individual or entity that uses the System to obtain travel information and food delivery information and to make travel and food orders.

[1003] "Dialogue" refers to the exchange of questions and answers between a user and a chat-generating AI means.

[1004] "Travel Information" refers to information necessary for users to plan and carry out their trips, such as tourist attractions, transportation options, and local know-how.

[1005] "Chat generation AI means" is an artificial intelligence means that provides travel information and food delivery information through dialogue with users.

[1006] "Supporting means" refers to means that support users to smoothly travel and order food based on information provided by chat generation AI means.

[1007] "Real-time video" refers to video that delivers footage of the local situation in real time.

[1008] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[1009] "Food Delivery Information" refers to information about food delivery in a particular region or city.

[1010] "Means for assisting in ordering food" refers to means for supporting users in ordering food based on food delivery information provided by chat generation AI means.

[1011] "Means for arranging local food" refers to a means for users to arrange local food based on food delivery information provided by the chat generation AI means.

[1012] The system for implementing the present invention includes a chat generation AI means for providing travel information and food delivery information through dialogue with a user. A specific embodiment of this system will be described below.

[1013] System Configuration

[1014] The system consists of the following main components:

[1015] 1. User device: The device used by the user, such as a smartphone or tablet.

[1016] 2. Server: A cloud server for running chat generation AI methods.

[1017] 3. Database: A database for storing travel information and food delivery information.

[1018] Hardware and software used

[1019] Hardware: smartphones, tablets, cloud servers

[1020] Software: Python, OpenAI GPT-3 API, database management system (e.g., MySQL)

[1021] Data processing and calculation

[1022] 1. User terminal: The user inputs questions about travel information and food delivery information.

[1023] 2. Server: Receives the user's question and generates an appropriate answer using the OpenAI GPT-3 API.

[1024] 3. Database: Stores travel and food delivery information and provides it to the server as needed.

[1025] Processing flow

[1026] 1. A user uses their smartphone to enter a question about travel or food delivery information.

[1027] 2. The user terminal sends a query to the server.

[1028] 3. The server sends the question to the OpenAI GPT-3 API and generates an answer.

[1029] 4. The server sends the generated answer to the user terminal.

[1030] 5. The user device displays the answer to the user.

[1031] Specific examples

[1032] When a user asks, "What are the best pizza delivery options in New York?" the system works as follows:

[1033] 1. The user device sends a question to the server.

[1034] 2. The server sends the question to the OpenAI GPT-3 API and generates an answer.

[1035] 3. The server generates the answer, "A popular pizza delivery service in New York is Joe's Pizza. You can order online, and delivery takes about 30 minutes." and sends it to the user's device.

[1036] 4. The user device displays the answer to the user.

[1037] Prompt Sentence Examples

[1038] If a user asks, "What are the best sushi delivery options in Tokyo?", the following prompts might be used:

[1039] "A user asked a question about food delivery. Question: What's the best sushi delivery in Tokyo?"

[1040] answer:

[1041] In this way, users can quickly and accurately obtain information about their travel destinations and can easily order local food.

[1042] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1043] Step 1:

[1044] Users use their smartphones to input questions about travel information or food delivery information, and the input questions are captured as text data by an application on the user's device.

[1045] Step 2:

[1046] The user terminal transmits the acquired text data to the server. The transmitted data is text data containing the content of the user's question.

[1047] Step 3:

[1048] The server analyzes the received text data and generates a prompt to send to the OpenAI GPT-3 API. The prompt is generated in a format that includes the user's question.

[1049] Step 4:

[1050] The server sends the generated prompt to the OpenAI GPT-3 API. The data sent is text data containing the prompt.

[1051] Step 5:

[1052] The OpenAI GPT-3 API generates an appropriate answer based on the received prompt, which is then returned to the server as text data.

[1053] Step 6:

[1054] The server analyzes the answers received from the OpenAI GPT-3 API and converts them into a format suitable for sending to the user's device. The converted data is text data containing the answers to the user's questions.

[1055] Step 7:

[1056] The server then transmits the converted answer data to the user terminal. The transmitted data is text data containing the answer to the user's question.

[1057] Step 8:

[1058] The user terminal analyzes the received answer data and displays it to the user. The displayed content is text data containing the answer to the user's question.

[1059] In this way, users can quickly and accurately obtain information about their travel destinations and can easily order local food.

[1060] Example 3

[1061] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1062] Conventional travel information systems make it difficult for users to obtain the latest local information in real time or to easily arrange for local specialties. In addition, there was a lack of a centralized means of providing the information users needed when planning a trip, so users had to refer to multiple information sources, which was time-consuming.

[1063] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes: a chat generation AI means for providing travel information through dialogue with a user; a means for supporting a user in traveling based on the travel information provided by the chat generation AI means; a means for distributing real-time video of the local area based on the travel information provided by the chat generation AI means; a means for arranging local specialties based on the travel information provided by the chat generation AI means; a means for acquiring real-time video from a live camera of the local area based on the travel information provided by the chat generation AI means and distributing it to the user's terminal; and a means for linking with an online shop selling local specialties and processing the user's order based on the travel information provided by the chat generation AI means. This allows a user to obtain the latest local information in real time and easily arrange local specialties.

[1064] "User" means an individual or organization that uses the System to provide travel information and arrange local specialties.

[1065] "Chat generation AI means" refers to artificial intelligence technology for generating and providing travel information through dialogue with users.

[1066] "Travel Information" refers to information that users need when planning and carrying out their trips, such as tourist attractions, transportation options, and local know-how.

[1067] "Real-time video" refers to video data that delivers footage of the local situation in real time.

[1068] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[1069] A "live camera" refers to a camera device that captures images of a specific location in real time and distributes them over the Internet.

[1070] "Online Shop" refers to a website or platform that sells products over the Internet.

[1071] "Order processing means" refers to the systems and processes used to accept purchase requests from users, arrange for products, and process delivery.

[1072] The present invention provides a system that allows users to obtain travel information, watch real-time local video, and even arrange for local specialties. Specific embodiments of this system are described below.

[1073] System configuration

[1074] 1. User Device

[1075] Users access the system using devices such as smartphones and PCs, which have a browser and dedicated applications installed, and interact with the system through these.

[1076] 2. Server

[1077] The server receives user requests and generates appropriate travel information using a generative AI model. The server also acquires real-time video from local live cameras and distributes it to users' devices. The server also connects with online shops selling local specialties to process user orders.

[1078] 3. Generative AI Models

[1079] The generative AI model used is OpenAI's GPT-4, which generates prompts based on user requests and provides appropriate travel information.

[1080] Program processing

[1081] 1. Providing travel information

[1082] A user uses a device to send a request: "Show me what Paris looks like now."

[1083] The server receives this request and sends a prompt to the generative AI model, which is "Tell me what Paris is like now."

[1084] The generative AI model generates travel information based on the prompt sentence and returns it to the server.

[1085] The server transmits the generated travel information to the user's terminal.

[1086] 2. Real-time video distribution

[1087] The server obtains the URL of the local live camera based on the generated travel information.

[1088] The server accesses the live camera URL and retrieves the real-time video stream.

[1089] The server delivers the acquired real-time video to the user's device.

[1090] Users can watch real-time video on their devices.

[1091] 3. Arrangements for local specialties

[1092] The user uses the device to send a request saying, "I want to buy a local specialty of Paris."

[1093] The server receives this request and sends a prompt to the generative AI model: "What are some famous products of Paris?"

[1094] The generative AI model generates travel information based on the prompt sentence and returns it to the server.

[1095] Based on the generated travel information, the server calls the API of an online shop that sells local specialties and processes the user's order.

[1096] The server notifies the user that the order is complete.

[1097] Users can check the status of their orders on their terminal.

[1098] Specific examples

[1099] 1. Specific examples of real-time video streaming

[1100] User: "Show me what Paris looks like now."

[1101] Server: Receives user requests, generates travel information using GPT-4, and captures and distributes real-time video from live cameras in Paris.

[1102] User: Watch what's happening in Paris in real time on their device.

[1103] 2. Specific examples of arranging local specialties

[1104] User: "I want to buy some Parisian specialties."

[1105] Server: Receives user requests, generates travel information using GPT-4, and processes orders in collaboration with an e-commerce site selling Parisian specialties.

[1106] User: Purchase Parisian specialties on the device and have them delivered to their home.

[1107] The above is a specific embodiment for carrying out the present invention. This system allows users to obtain the latest local information in real time and easily arrange for local specialties. The flow of the specification process in Example 3 will be explained using Figure 15.

[1108] Step 1:

[1109] A user sends a request using a device. For example, the user opens a browser or a dedicated application, types "Show me the current state of Paris," and presses the send button. The input is a text request.

[1110] Step 2:

[1111] The server receives a request from the user. The server analyzes the data received as an HTTP request and extracts the request content. The input is the user's request data, and the output is the analyzed request content.

[1112] Step 3:

[1113] The server sends a prompt to the generative AI model. The server generates a prompt based on the request content and sends it to the generative AI model (e.g., GPT-4). The input is the parsed request content, and the output is the generated prompt. Specifically, the server generates the prompt, "Tell me what the current situation is in Paris."

[1114] Step 4:

[1115] The generative AI model generates travel information based on the prompt text. The generative AI model receives the prompt text and extracts and generates appropriate travel information from the vast amount of information on the Internet. The input is the prompt text, and the output is the generated travel information.

[1116] Step 5:

[1117] The server receives the generated travel information and sends it to the user's device. The server analyzes the travel information returned from the generative AI model and sends it to the user's device. The input is the generated travel information, and the output is the travel information sent to the user's device. Specifically, the server sends information such as "It's currently sunny in Paris, and the area around the Eiffel Tower is bustling with tourists."

[1118] Step 6:

[1119] The server retrieves the URL of the local live camera. Based on the generated travel information, the server calls an API to retrieve the URL of the local live camera. The input is the generated travel information, and the output is the URL of the live camera.

[1120] Step 7:

[1121] The server accesses the URL of the live camera and retrieves the real-time video stream. The server accesses the URL and retrieves the video data using the HTTP streaming protocol. The input is the URL of the live camera, and the output is the real-time video stream.

[1122] Step 8:

[1123] The server delivers the acquired real-time video to the user's device. The server sends the real-time video stream to the user's device, and the user watches the video using a browser or dedicated application. The input is the real-time video stream, and the output is the video delivered to the user's device.

[1124] Step 9:

[1125] The user watches real-time video on a device. The user watches the streamed real-time video through the device's browser or a dedicated application. The input is the streamed video, and the output is the user's viewing experience.

[1126] Step 10:

[1127] A user uses a terminal to request the delivery of local specialties. For example, the user inputs "I want to buy local specialties from Paris" and presses the send button. The input is a text request.

[1128] Step 11:

[1129] The server receives a request from the user and sends a prompt to the generative AI model. The server generates a prompt based on the request content and sends it to the generative AI model. The input is the parsed request content, and the output is the generated prompt. Specifically, the server generates the prompt, "Tell me what Paris is famous for."

[1130] Step 12:

[1131] The generative AI model generates travel information based on the prompt text. The generative AI model receives the prompt text and extracts and generates appropriate travel information from the vast amount of information on the Internet. The input is the prompt text, and the output is the generated travel information.

[1132] Step 13:

[1133] The server receives the generated travel information and calls the API of an online shop that sells local specialties. Based on the generated travel information, the server processes the user's order using the online shop's API. The input is the generated travel information, and the output is the result of order processing.

[1134] Step 14:

[1135] The server notifies the user that the order is complete. The server sends the order processing results to the user's terminal, and the user checks the order status. The input is the order processing results, and the output is the notification sent to the user's terminal.

[1136] Step 15:

[1137] The user checks the order status on the device. The user checks the order status through the device's browser or a dedicated application. The input is the sent notification, and the output is the user's confirmation result.

[1138] (Application example 3)

[1139] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1140] Conventional travel information systems make it difficult for users to grasp real-time local conditions and limit the means for easily purchasing local specialties. Furthermore, they have the problem of being unable to simultaneously experience real-time shopping in a virtual store. This limits users' travel and shopping experiences.

[1141] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1142] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time video of the local area based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, and means for purchasing local specialties while watching real-time video of the local area in a virtual store based on the travel information provided by the chat generation AI means. This enables the user to easily purchase local specialties in the virtual store while understanding the real-time situation of the local area.

[1143] "User" means an individual or corporation that uses the system to provide travel information or purchase local specialties.

[1144] "Dialogue" refers to the exchange of information between a user and a chat generation AI means.

[1145] "Travel information" refers to various information related to travel, such as tourist attractions, transportation options, and local know-how.

[1146] "Chat generation AI means" refers to an artificial intelligence system that provides travel information through dialogue with users.

[1147] "Real-time video" refers to video that delivers real-time footage of the current situation on-site.

[1148] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[1149] "Virtual Store" refers to a virtual store operated on the Internet where users can purchase products online.

[1150] "Viewing" refers to the act of a user watching a video or movie.

[1151] "Purchase" refers to the act of a user acquiring a product or service by paying money.

[1152] This invention is a system that allows users to purchase local specialties while watching real-time video of the local area in a virtual store. This system is realized using the following hardware and software.

[1153] Hardware

[1154] Smartphone

[1155] Smart Glasses

[1156] head-mounted display

[1157] server

[1158] software

[1159] Python

[1160] OpenCV (cv2)

[1161] ChatGPT API

[1162] System configuration

[1163] 1. User Device:

[1164] Users access the system using smartphones, smart glasses, or head-mounted displays, which allow for user interaction and support real-time video viewing and local product purchases.

[1165] 2. Server:

[1166] The server hosts a chat generation AI system, provides travel information in response to user questions, and also handles real-time video streaming and local product ordering.

[1167] Processing flow

[1168] 1. User interaction:

[1169] When a user asks "Show me what Paris is like now" through their device, the server uses the ChatGPT API to parse the question and generate appropriate travel information.

[1170] 2. Real-time video streaming:

[1171] The server obtains the URL of the local real-time video based on the travel information and delivers it to the user's device, where the user can watch the real-time video.

[1172] 3. Arrangement of local specialties:

[1173] When a user asks, "I want to buy a local specialty from Paris," the server again uses the ChatGPT API to get information about the specialty and provides the user with a purchase link.

[1174] Specific examples

[1175] When a user launches the application on their smartphone and asks, "Show me what Paris is like now," the application will display a real-time video of Paris. If the user asks, "I want to buy some Parisian specialties," the application will display links to buy Parisian specialties.

[1176] Prompt Sentence Examples

[1177] User: Show me what Paris looks like now.

[1178] System: Shows real-time video of Paris.

[1179] User: I want to buy some Parisian specialties.

[1180] System: Here are the links to buy Parisian specialties.

[1181] In this way, users can easily purchase local specialties while keeping track of the local situation in real time within the virtual store.

[1182] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1183] Step 1:

[1184] A user accesses the system using a terminal and enters questions to obtain travel information.

[1185] Input: User question (e.g., "Show me what Paris is like now")

[1186] Output: The user's question is sent to the server.

[1187] Specific actions: The user uses a smartphone, smart glasses, or head-mounted display to launch the application and enter a question.

[1188] Step 2:

[1189] The server receives the user's question and parses it using the ChatGPT API.

[1190] Input: User question

[1191] Output: Travel information based on the parsed question

[1192] Specific operation: The server sends the user's query to the ChatGPT API and receives the travel information returned by the API.

[1193] Step 3:

[1194] The server obtains the URL of the local real-time video based on the travel information.

[1195] Input: Travel information based on the parsed question

[1196] Output: Real-time video URL

[1197] What happens: The server analyzes the trip information and sends a request to an external video streaming service to obtain the appropriate real-time video URL.

[1198] Step 4:

[1199] The server sends the URL of the real-time video it has acquired to the user's device.

[1200] Input: Real-time video URL

[1201] Output: URL of real-time video sent to user's device

[1202] Specific operation: The server sends the obtained URL to the user's device, allowing the user to watch the real-time video.

[1203] Step 5:

[1204] The user uses the device to watch real-time video.

[1205] Input: Real-time video URL

[1206] Output: Real-time video watched by the user

[1207] Specific operation: Users can play real-time video on their devices and check the local situation.

[1208] Step 6:

[1209] The user uses the terminal to input a question about the local specialty product they wish to purchase.

[1210] Input: User question (e.g., "I want to buy a local product from Paris")

[1211] Output: The user's question is sent to the server

[1212] How it works: The user uses a smartphone, smart glasses, or head-mounted display to input a question about the local specialty they wish to purchase.

[1213] Step 7:

[1214] The server receives the user's question and parses it using the ChatGPT API.

[1215] Input: User question

[1216] Output: Local product information based on the parsed question

[1217] Specific operation: The server sends the user's question to the ChatGPT API and receives the local product information returned by the API.

[1218] Step 8:

[1219] The server obtains a purchase link based on the specialty product information.

[1220] Input: Local product information based on the parsed question

[1221] Output: Link to purchase the specialty product

[1222] What happens: The server parses the product information and sends a request to an external shopping service to get the appropriate purchase link.

[1223] Step 9:

[1224] The server sends the acquired purchase link to the user's device.

[1225] Input: Link to purchase specialty products

[1226] Output: Purchase link sent to user's device

[1227] Specific operation: The server sends the acquired purchase link to the user's device, allowing the user to purchase the local specialty product.

[1228] Step 10:

[1229] The user uses the device to purchase local specialties.

[1230] Input: Link to purchase specialty products

[1231] Output: Specialty products purchased

[1232] Specific operation: The user clicks on the purchase link on their device and completes the purchase of the local specialty product.

[1233] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1234] "Example 1"

[1235] One embodiment of the present invention is a system that includes a chat generation AI means that provides travel information through dialogue with a user, a means for assisting a user in traveling based on the travel information provided by the chat generation AI means, a means for delivering real-time local video based on the travel information provided by the chat generation AI means, and a means for arranging local specialties based on the travel information provided by the chat generation AI means, all of which are combined with an emotion engine that recognizes the user's emotions.

[1236] "Example 2"

[1237] Specifically, when a user asks, "Tell me about tourist attractions in Paris," the emotion engine recognizes the user's excitement from their tone of voice and facial expression, and responds accordingly by saying something like, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral. You'll have an exciting experience!"

[1238] "Example 3"

[1239] Furthermore, when a user asks, "How do I use the Paris subway?" the emotion engine recognizes the user's concerns and responds by providing support for their travels, such as, "For the Paris subway, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring. Please use the service with peace of mind."

[1240] "Example 4"

[1241] Furthermore, when a user asks, "Show me what's happening in Paris right now," the emotion engine recognizes the user's curiosity and delivers real-time video from the area, allowing users to get the latest information on the area.

[1242] "Example 5"

[1243] In addition, when a user says, "I want to buy a local specialty from Paris," the emotion engine recognizes the user's expectations and arranges for local specialties accordingly, allowing users to obtain local specialties without having to travel to the area.

[1244] The processing flow of each embodiment will be described below.

[1245] "Example 1"

[1246] Step 1: The user asks a travel-related question to the chat generation AI.

[1247] Step 2: The chat generation AI tool provides travel information in response to the user's questions.

[1248] Step 3: The emotion engine recognizes the user's emotion.

[1249] Step 4: Based on the recognition results of the emotion engine, the chat generation AI means provides travel information.

[1250] "Example 2"

[1251] Step 1: User asks, "Tell me about tourist attractions in Paris."

[1252] Step 2: The emotion engine recognizes excitement from the user's tone of voice and facial expression.

[1253] Step 3: Based on the recognition results of the emotion engine, the chat generation AI means responds, "The Eiffel Tower, the Louvre, Notre Dame Cathedral, etc. are famous. You will have an exciting experience!"

[1254] "Example 3"

[1255] Step 1: User asks, "How do I use the Paris Metro?"

[1256] Step 2: The emotion engine recognizes the user's anxiety.

[1257] Step 3: Based on the recognition results of the emotion engine, the chat generation AI means responds, "For the Paris Metro, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring. Please feel free to use the service."

[1258] "Example 4"

[1259] Step 1: User asks, "Show me what Paris looks like today."

[1260] Step 2: The emotion engine recognizes the user's curiosity.

[1261] Step 3: Based on the recognition results of the emotion engine, the chat generation AI means delivers real-time local video.

[1262] "Example 5"

[1263] Step 1: The user asks, "I want to buy a local product from Paris."

[1264] Step 2: The emotion engine recognizes the user's expectations.

[1265] Step 3: Based on the recognition results of the emotion engine, the chat generation AI means arranges local specialties.

[1266] Example 1

[1267] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1268] Conventional travel information systems have difficulty in quickly and accurately providing the information users need. They also face the challenge of being unable to provide a wide range of services in a unified manner, such as dialogue that takes into account the user's emotions, real-time local information, and arranging local specialties.

[1269] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1270] In this invention, the server includes a dialogue generation artificial intelligence means for providing travel information through dialogue with the user, a means for supporting the user in traveling based on the travel information provided by the dialogue generation artificial intelligence means, a means for delivering real-time local video based on the travel information provided by the dialogue generation artificial intelligence means, a means for arranging local specialties based on the travel information provided by the dialogue generation artificial intelligence means, and an emotion recognition engine for recognizing the user's emotions. This allows the user to not only quickly and accurately obtain the travel information they need, but also to receive a wide range of services in a unified manner, such as dialogue according to their emotions, real-time local information, and arrangements for local specialties.

[1271] "Dialogue generation type artificial intelligence means" refers to a means that uses artificial intelligence technology to provide travel information through dialogue with users.

[1272] "Travel information support means" refers to a means for supporting a user when planning and carrying out a trip based on travel information provided by a dialogue generation artificial intelligence means.

[1273] The "means for delivering real-time video" is a means for delivering real-time video of the local area to the user based on travel information provided by the dialogue generation artificial intelligence means.

[1274] The "means for arranging local specialties" is a means for arranging local specialties for the user based on travel information provided by the dialogue generation artificial intelligence means.

[1275] An "emotion recognition engine" is an engine that recognizes the user's emotions and provides dialogue and services accordingly.

[1276] MODE FOR CARRYING OUT THE INVENTION

[1277] This invention is a system that provides travel information through dialogue with a user, and includes a dialogue generation artificial intelligence means, a means for supporting travel information, a means for delivering real-time video, a means for arranging local specialties, and an emotion recognition engine.

[1278] 1. Generate the system program.

[1279] The server generates a program to provide travel information through dialogue with the user. This program has the function of collecting information on the Internet based on user input and providing appropriate travel information. Specifically, it uses a generative AI model (e.g., GPT-4).

[1280] 2. Explain the processing of the generated program in natural language.

[1281] When the server receives a question from a user, it uses a dialogue generation AI means to extract relevant travel information from the vast amount of information on the Internet. This AI means uses a generative AI model, for example. If a user types, "Tell me about tourist attractions in Paris," the server uses the generative AI model to generate an answer such as, "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[1282] Additionally, the system has the following additional features:

[1283] Travel support means: When a user plans a trip, assistance is provided in making travel arrangements and creating a schedule based on information provided by a dialogue generation AI means.

[1284] Real-time video distribution means: Distribute real-time video of tourist spots that users are interested in. For example, use the API of a video distribution service.

[1285] Local specialty product procurement method: Procure local specialty products that the user is interested in. For example, use an online shopping site.

[1286] Emotion recognition engine: Recognizes the user's emotions and adjusts the content and tone of the dialogue, for example, by using emotion recognition APIs.

[1287] 3. Add specific examples to your description.

[1288] When a user types "Tell me about tourist attractions in Paris," the device sends this question to the server. The server receives the question and inputs it as a prompt to the generative AI model. The generative AI model generates an answer, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral," and returns it to the server. The server sends this answer to the device, which displays it to the user.

[1289] Next, if the user types "I want to watch a real-time video of the Eiffel Tower," the device will send this request to the server. The server will call the video streaming service's API to search for "Eiffel Tower live stream" and send the link to the device. The device will then display the link to the user, allowing them to watch the real-time video.

[1290] Prompt Sentence Examples

[1291] "Tell me about the tourist attractions in Paris"

[1292] "Tell me some recommended sightseeing spots in Kyoto"

[1293] "I want to watch a real-time video of Kinkakuji Temple."

[1294] "What are some recommended restaurants near the Eiffel Tower?"

[1295] In this way, the server provides appropriate information in response to user questions, creating a multi-functional system to assist with travel planning.

[1296] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1297] Step 1:

[1298] The user enters a question.

[1299] The user inputs a question into the chat interface of the terminal, for example, "Tell me about tourist attractions in Paris." The input data is in text format.

[1300] Step 2:

[1301] The terminal sends a question to the server.

[1302] The terminal sends the user's input to the server. At this time, the input text data is sent to the server as an HTTP request. The input data is the user's question text, and the output data is the request to the server.

[1303] Step 3:

[1304] The server receives the question and inputs it as a prompt to the generative AI model.

[1305] The server analyzes the received question and inputs it as a prompt to the generative AI model. The prompt is "Tell me about tourist attractions in Paris." The input data is the user's question text, and the output data is the prompt to the generative AI model.

[1306] Step 4:

[1307] A generative AI model generates the answer.

[1308] The generative AI model generates an answer based on a prompt. For example, it generates the answer "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The input data is the prompt, and the output data is the generated answer text.

[1309] Step 5:

[1310] The server sends the generated response to the terminal.

[1311] The server sends the answer obtained from the generative AI model to the terminal. At this time, the answer is sent as an HTTP response. The input data is the generated answer text, and the output data is the response to the terminal.

[1312] Step 6:

[1313] The terminal displays the answer to the user.

[1314] The device displays the received answer on the chat interface. The user confirms the answer: "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The input data is the response from the server, and the output data is what is displayed to the user.

[1315] Step 7:

[1316] The user enters an additional request.

[1317] The user inputs an additional request, for example, "I want to see a real-time video of the Eiffel Tower." The input data is a text request.

[1318] Step 8:

[1319] The device sends an additional request to the server.

[1320] The terminal sends an additional request to the server. At this time, the input text data is sent to the server again as an HTTP request. The input data is the user's request text, and the output data is the request to the server.

[1321] Step 9:

[1322] The server processes any additional requests.

[1323] The server parses the additional request and takes appropriate action, for example calling a video streaming service's API to search for "Eiffel Tower live stream." The input data is the user's request text, and the output data is the API's response.

[1324] Step 10:

[1325] The server sends the processing result to the terminal.

[1326] The server sends the processing result to the device. For example, it sends data including a link to a live stream from a video streaming service. The input data is the response from the API, and the output data is the response to the device.

[1327] Step 11:

[1328] The terminal displays the processing results to the user.

[1329] The terminal displays the received processing results to the user. The user can click a link such as "Click here for a live stream of the Eiffel Tower" to watch the real-time video. The input data is the response from the server, and the output data is what is displayed to the user.

[1330] (Application example 1)

[1331] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1332] Conventional travel information systems have difficulty accurately providing users with the information they require, and do not provide personalized information that reflects their emotions. As a result, users are often left with low satisfaction and experience inconvenience when planning and executing their travel plans. Furthermore, there is a lack of means to provide real-time local videos and information on local specialties, making it difficult for users to obtain the latest local information.

[1333] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: a chat generation AI means for providing travel information through dialogue with the user; a means for assisting the user in traveling based on the travel information provided by the chat generation AI means; a means for delivering real-time local video based on the travel information provided by the chat generation AI means; a means for arranging local specialties based on the travel information provided by the chat generation AI means; a means including an emotion engine for recognizing the user's emotions; a means for personalizing the travel information based on the user's emotions recognized by the emotion engine; and a means for providing real-time video and local product information based on the user's emotions recognized by the emotion engine. This enables the provision of personalized travel information based on the user's emotions, thereby increasing user satisfaction. Furthermore, providing real-time local video and local product information allows the user to obtain the latest local information, making it easier to plan and execute travel plans.

[1334] "Chat generation AI means" is an artificial intelligence means that provides travel information through dialogue with users.

[1335] "Means for assisting in travel" refers to means for assisting users in planning and carrying out travel based on travel information provided by chat generation AI means.

[1336] "Means for delivering real-time video" refers to means for delivering local real-time video to users based on travel information provided by chat generation AI means.

[1337] The "means of arranging local specialties" is a means of arranging local specialties for users based on travel information provided by the chat generation AI means.

[1338] An "emotion engine" is an engine that recognizes the user's emotions and provides appropriate information based on those emotions.

[1339] "Means for personalization" refers to means for individually adjusting the travel information provided based on the user's emotions recognized by the emotion engine.

[1340] The "means for providing real-time video and local specialty information" is a means for providing real-time video and local specialty information based on the user's emotions recognized by the emotion engine.

[1341] A system for implementing this invention includes a chat generation AI means for providing travel information through dialogue with a user, a means for assisting in travel, a means for delivering real-time local video, a means for arranging local specialties, an emotion engine for recognizing a user's emotions, a means for personalizing travel information based on emotions, and a means for providing real-time video and local specialty information.

[1342] System configuration

[1343] Hardware

[1344] Server: A central processing unit for running generative AI models and providing travel information.

[1345] User device: A device used by a user, such as a smartphone or head-mounted display (HMD).

[1346] Emotion recognition devices: Sensors such as cameras and microphones that recognize user emotions.

[1347] software

[1348] Generative AI model: Generates travel information in response to user questions using OpenAI APIs, etc.

[1349] Emotion recognition engine: Analyzes user emotions using libraries such as the EmotionRecognizer library.

[1350] Real-time video streaming API: An API for obtaining local real-time video.

[1351] Local specialty product ordering API: API for obtaining information on local specialty products.

[1352] Data processing and calculation

[1353] 1. User inquiry processing: When a user inquires about travel information through their device, the question is sent to the generative AI model, which then extracts relevant travel information from the vast amount of information on the Internet and provides it to the user.

[1354] 2. Emotion recognition: The emotion recognition device captures the user's facial expressions and voice, and the emotion recognition engine analyzes the data to recognize the user's emotions.

[1355] 3. Personalization: The generative AI model personalizes the travel information provided based on the user's perceived emotions. For example, if the user is excited, it will provide more active tourist attractions and event information.

[1356] 4. Providing real-time video and local product information: Based on user sentiment, we use real-time video streaming APIs and local product ordering APIs to provide the latest local information.

[1357] Specific examples

[1358] When a user uses a smartphone to ask, "Tell me about tourist attractions in Paris," the generative AI model responds, "Famous attractions include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." At the same time, an emotion recognition device captures the user's facial expression, and the emotion recognition engine recognizes that the user is excited. As a result, the system also provides URLs for real-time videos of Paris and information about local specialties.

[1359] Prompt Sentence Examples

[1360] "Provide travel information: Tell me about tourist attractions in Paris."

[1361] In this way, users can virtually travel and get real-time information about the local area.

[1362] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1363] Step 1:

[1364] A user uses a terminal to input a question about travel information. The input question is sent from the terminal to the server. The input data is the user's question, and the output data is the question sent to the server.

[1365] Step 2:

[1366] The server sends the received question to the generative AI model, which extracts appropriate travel information from the vast amount of information on the Internet. The input data is the user's question, and the output data is the travel information generated by the generative AI model.

[1367] Step 3:

[1368] The server transmits the travel information obtained from the generative AI model to the user terminal. The user terminal displays the received travel information to the user. The input data is the travel information generated by the generative AI model, and the output data is the travel information displayed on the user terminal.

[1369] Step 4:

[1370] The emotion recognition device captures the user's facial expressions and voice and sends the data to the server. The input data is the user's facial expressions and voice data, and the output data is the emotion data sent to the server.

[1371] Step 5:

[1372] The server uses an emotion recognition engine to analyze the received emotion data and recognize the user's emotion, where the input data is the emotion data and the output data is the recognized emotion information of the user.

[1373] Step 6:

[1374] The server personalizes the travel information provided by the generative AI model based on the recognized user emotions. The input data are the user's emotional information and the travel information generated by the generative AI model, and the output data is the personalized travel information.

[1375] Step 7:

[1376] The server transmits the personalized travel information to the user terminal. The user terminal displays the received personalized travel information to the user. The input data is the personalized travel information, and the output data is the personalized travel information displayed on the user terminal.

[1377] Step 8:

[1378] The server obtains the latest local information based on the user's emotions using a real-time video streaming API and a local specialty product ordering API. The input data is the user's emotion information, and the output data is the real-time video URL and the local specialty product information URL.

[1379] Step 9:

[1380] The server sends the acquired real-time video URL and local product information URL to the user terminal. The user terminal displays the received URL to the user. The input data is the real-time video URL and local product information URL, and the output data is the URL displayed on the user terminal.

[1381] Example 2

[1382] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1383] Conventional travel support systems have issues with the reliability and timeliness of information when users obtain travel information, and it is difficult to provide appropriate information that reflects the user's emotions. Furthermore, it is difficult to meet the diverse needs of users, such as real-time local information and arranging local specialties. This has resulted in users' travel experiences that are not fully satisfactory.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1385] In this invention, the server includes a dialogue generation artificial intelligence means for providing travel information through dialogue with the user, a means for assisting the user in traveling based on the travel information provided by the dialogue generation artificial intelligence means, a means for delivering real-time video of the local area based on the travel information provided by the dialogue generation artificial intelligence means, a means for arranging local specialties based on the travel information provided by the dialogue generation artificial intelligence means, and a sentiment analysis means for analyzing the user's emotions and providing optimal answers based on the emotions. This allows the user to obtain reliable travel information in real time and receive appropriate information based on their emotions. Furthermore, the system can respond to a variety of needs, such as arranging real-time video of the local area and local specialties, thereby improving the user's travel experience.

[1386] "Dialogue generation artificial intelligence means" refers to artificial intelligence technology for providing travel information through dialogue with users.

[1387] "Travel information" refers to all travel-related information, such as tourist attractions, transportation options, and local know-how.

[1388] "Emotion analysis means" refers to technology that analyzes a user's emotions from their tone of voice, facial expressions, etc., and provides the most appropriate response based on those emotions.

[1389] "Real-time video distribution means" refers to technology for delivering real-time local video to users.

[1390] "Means for arranging local specialties" refers to the techniques used to arrange local specialties for users.

[1391] "Server" refers to a computer system that receives input from a user and provides appropriate information using dialogue generation artificial intelligence means and emotion analysis means.

[1392] "User" refers to an individual who uses this system to obtain travel information and plan and carry out a trip.

[1393] "Terminal" refers to the device through which a User accesses this System and inputs and receives information.

[1394] This invention is a system for users to acquire travel information and plan and execute a trip. This system includes an AI dialogue generation system, an emotion analysis system, a real-time video distribution system, and a local specialty product arrangement system.

[1395] The server receives input from the user and provides appropriate travel information using a dialogue generation AI method. Specifically, the server uses a generative AI model (e.g., GPT-4) to generate answers to the user's questions. When the user inputs a question such as "How do I use the Paris subway?" through their device, the device sends this input to the server. The server analyzes the question and inputs a prompt sentence into the generative AI model. The generative AI model generates an answer based on the input prompt sentence.

[1396] Furthermore, the server uses an emotion analysis means to analyze the user's tone of voice and facial expression. For example, if a user asks in an excited tone of voice, "Tell me about tourist attractions in Paris," the emotion analysis means will recognize the user's excitement. The server selects the optimal answer based on the user's emotion based on the answer generated by the generative AI model and the analysis results of the emotion analysis means.

[1397] The server sends the selected answer to the terminal, and the terminal displays the answer to the user, who can check the answer on the terminal screen.

[1398] For example, if a user types "How do I use the Paris subway?", the server will use the generative AI model to answer, "For the Paris subway, you buy a ticket at each station and insert it when passing through the ticket gate. You don't need to buy a new ticket when transferring." Similarly, if a user types "Tell me about tourist attractions in Paris," asking in an excited voice tone, the server will answer, "Famous attractions include the Eiffel Tower, the Louvre, and Notre Dame Cathedral. You'll have an exciting experience!"

[1399] An example of a prompt is as follows:

[1400] "How do I use the Paris subway?"

[1401] "Tell me about the tourist attractions in Paris."

[1402] This system allows users to obtain reliable travel information in real time, receive appropriate information tailored to their emotions, and can also meet a variety of needs, such as real-time local video and local product arrangements, thereby improving the user's travel experience.

[1403] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1404] Step 1:

[1405] The user enters a question.

[1406] The user inputs a travel-related question into the device, such as "How do I use the Paris subway?" The input data is stored in text format on the device.

[1407] Step 2:

[1408] The terminal sends a question to the server.

[1409] The terminal sends the question entered by the user to the server. At this time, the terminal sends the question content to the server as text data. The input data is the user's question text, and the output data is the text data sent to the server.

[1410] Step 3:

[1411] The server parses the question.

[1412] The server analyzes the received question, specifically using natural language processing techniques to understand the question and generate an appropriate prompt. The input data is the user's question text, and the output data is the generated prompt.

[1413] Step 4:

[1414] The server inputs a prompt sentence into the generative AI model.

[1415] The server inputs a prompt sentence to a generative AI model (e.g., GPT-4) based on the analyzed question. The input data is the generated prompt sentence, and the output data is the prompt sentence input to the generative AI model.

[1416] Step 5:

[1417] A generative AI model generates the answer.

[1418] The generative AI model generates an answer based on the input prompt. For example, it generates an answer such as, "For the Paris Metro, you buy a ticket at each station and insert it when passing through the ticket gate. You do not need to buy a new ticket when transferring." The input data is the prompt, and the output data is the generated answer text.

[1419] Step 6:

[1420] The server uses the emotion analysis means to analyze the user's emotions.

[1421] The server analyzes the user's emotions using emotion analysis means. It determines the user's emotional state based on the user's tone of voice and facial expression data sent from the terminal. The input data is the user's tone of voice and facial expression data, and the output data is the analyzed emotional information.

[1422] Step 7:

[1423] The server selects the best response based on the emotion.

[1424] The server selects the optimal answer based on the user's emotions based on the answers generated by the generative AI model and the analysis results of the emotion analysis means. For example, if the user is excited, it selects an answer that appeals to their emotions more. The input data is the generated answer text and analyzed emotion information, and the output data is the selected optimal answer text.

[1425] Step 8:

[1426] The server sends the response to the terminal.

[1427] The server sends the selected answer to the terminal. The answer is sent as text data. The input data is the selected optimal answer text, and the output data is the answer text sent to the terminal.

[1428] Step 9:

[1429] The terminal displays the answer to the user.

[1430] The terminal displays the answer received from the server to the user. The user can check the answer on the terminal screen. The input data is the answer text sent from the server, and the output data is the answer text displayed on the terminal screen.

[1431] (Application example 2)

[1432] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1433] Conventional travel information systems only provide static information in response to user questions, and have the problem of being unable to provide personalized information based on the user's emotions and interests. Furthermore, they are limited to travel information and lack support for choosing meals and restaurants while traveling. This often makes it difficult for users to choose meals while traveling.

[1434] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1435] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time local video based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, means for suggesting appropriate dishes and restaurants in response to user questions, and means for recognizing the user's emotions and suggesting special offers and recommended menus accordingly. This makes it possible to provide personalized information according to the user's emotions and interests, and also to provide support for choosing meals during the trip.

[1436] "Chat generation AI means" is an artificial intelligence technology that provides information in natural language through dialogue with users.

[1437] "Travel information" refers to all travel-related information, such as tourist attractions, transportation options, and local know-how.

[1438] "Means for recognizing user emotions" refers to technology that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[1439] "Special Offers" refer to exclusive discounts and perks that are tailored to a user's emotions and interests.

[1440] "Recommended menu" refers to food or restaurant menus suggested based on the user's preferences and feelings.

[1441] "Means for delivering real-time video" refers to technology that provides users with real-time footage of the local area.

[1442] "Means for arranging local specialties" refers to the technology for arranging local specialties in response to user requests.

[1443] "Means for suggesting dishes and restaurants" refers to technology that recommends appropriate dishes and restaurants based on user questions.

[1444] The system for implementing this invention includes a chat generation AI means for providing travel information through dialogue with the user, a means for recognizing the user's emotions, and a means for suggesting appropriate dishes and restaurants in response to the user's questions.

[1445] System configuration

[1446] The system consists of the following main components:

[1447] 1. Chat generation AI method: Provides travel information through dialogue with users. Uses OpenAI's API to generate answers in natural language to users' questions.

[1448] 2. Emotion Recognition: Recognizes emotions by analyzing the user's tone of voice and facial expressions. OpenCV and EmotionRecognizer are used to analyze emotions from images captured by the camera.

[1449] 3. Food and restaurant suggestions: Based on the user's questions and emotions, the app suggests appropriate food and restaurants. This also uses OpenAI's API to generate answers to the user's questions.

[1450] Hardware and Software

[1451] Hardware: Uses your smartphone's camera and microphone.

[1452] Software: OpenAI API, OpenCV, EmotionRecognizer.

[1453] Data processing and calculation

[1454] 1. Obtaining user's question: The user enters a question into their smartphone.

[1455] 2. Emotion Recognition: The smartphone camera captures the user's facial expressions and analyzes their emotions using EmotionRecognizer.

[1456] 3. Chat generation: Send the user's question and sentiment data to the OpenAI API to generate an appropriate answer.

[1457] 4. Show Suggestion: Show the generated answer to the user.

[1458] Specific examples

[1459] If a user asks, "I want to eat pizza," and the camera captures their facial expression and recognizes that they are excited, the AI ​​will suggest, "As a special offer, we're offering Margherita pizza at half price!"

[1460] Prompt Sentence Examples

[1461] If the user is excited, suggest a special offer or menu recommendation. User asks: I want pizza.

[1462] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1463] Step 1:

[1464] The user types a question into their smartphone.

[1465] Input: User question (e.g. "I want pizza")

[1466] Output: User question text

[1467] Specific actions: The user opens the application on their smartphone, enters a question in the text box, and presses the send button.

[1468] Step 2:

[1469] The device captures the user's facial expressions using the smartphone camera.

[1470] Input: User's face image

[1471] Output: Captured face image data

[1472] Specific operation: The smartphone camera takes a picture of the user's face and acquires image data.

[1473] Step 3:

[1474] The device uses EmotionRecognizer to analyze the user's emotions from the captured facial image.

[1475] Input: Captured face image data

[1476] Output: User's emotion data (e.g. excited)

[1477] Specific operation: EmotionRecognizer analyzes facial images and determines the user's emotional state.

[1478] Step 4:

[1479] The device sends the user's question and emotion data to the OpenAI API, which generates an appropriate answer.

[1480] Input: User question text, user emotion data

[1481] Output: Generated answer text (e.g., "As a special offer, we're offering Margherita pizzas at half price!")

[1482] Specific operation: The device incorporates the user's question and emotion data into a prompt sentence and sends it to the OpenAI API. It receives the response from the API and generates an appropriate answer.

[1483] Step 5:

[1484] The terminal displays the generated answer to the user.

[1485] Input: Generated answer text

[1486] Output: Answers displayed on the user's smartphone screen

[1487] Specific operation: The device displays the generated answer text on the user's smartphone screen and notifies the user.

[1488] Example 3

[1489] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1490] Conventional travel information systems have had issues with providing users with the real-time information and local specialties they require, and are unable to respond appropriately to the user's emotions. As a result, users are unable to obtain the latest local information, which often causes inconvenience when planning and executing their trips. Furthermore, there is a lack of support that responds to the user's emotions, making it difficult to respond appropriately to their anxieties and expectations during their trip.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1492] In this invention, the server includes a generation AI means for providing travel information through dialogue with a user, a means for supporting a user in traveling based on the travel information provided by the generation AI means, a means for delivering real-time video of the local area based on the travel information provided by the generation AI means, a means for arranging local specialties based on the travel information provided by the generation AI means, an emotion recognition means for recognizing a user's emotion based on the travel information provided by the generation AI means and generating an appropriate response, a means for delivering real-time video of the local area in accordance with the user's emotion recognized by the emotion recognition means, and a means for arranging local specialties in accordance with the user's emotion recognized by the emotion recognition means. This allows a user to obtain real-time local information, making it easier to plan and carry out a trip, and enabling the user to receive appropriate support in accordance with their emotion.

[1493] "Generative AI means" refers to artificial intelligence technology that provides travel information through dialogue with users.

[1494] "Travel information" refers to information related to travel, such as local tourist attractions, transportation options, and local know-how.

[1495] "Real-time video distribution means" refers to technology that captures real-time video footage from the local area and distributes it to users.

[1496] "Specialty product arrangement method" refers to the technology of arranging local specialties based on user requests.

[1497] "Emotion recognition means" refers to technology that analyzes a user's emotions and generates an appropriate response accordingly.

[1498] "User" means an individual who uses the System to obtain travel information and plan or undertake a trip.

[1499] "Server" refers to a computer system that receives requests from users and provides appropriate information and services using generative AI means and emotion recognition means.

[1500] "Terminal" refers to the device (smartphone, PC, etc.) used by a user to access the system.

[1501] The system of the present invention provides travel information through dialogue with users, and has the ability to deliver real-time local videos and arrange local specialties in response to user requests. It also uses an emotion engine to recognize user emotions and respond appropriately accordingly.

[1502] Hardware and software used

[1503] Hardware: Servers, user devices (smartphones, PCs, etc.)

[1504] Software: Generative AI models (e.g., OpenAI's GPT-4), emotion engines, real-time video streaming systems, specialty product ordering systems

[1505] Data processing and calculation

[1506] 1. User Request Submission

[1507] The user uses a device (smartphone or PC) to send a request to the system.

[1508] Example: A user opens a chat app on their smartphone, types "Show me what's happening in Paris right now," and sends it.

[1509] 2. Server receives and analyzes the request

[1510] The server receives the request from the user and analyzes its contents.

[1511] Example: A server receives a request to "Show me what Paris is up to now," analyzes the request, and determines that it needs to provide a real-time video of Paris.

[1512] 3. Sending prompts to the generative AI model

[1513] Based on the request content analyzed by the server, a prompt is sent to the generative AI model.

[1514] Example: The server sends a prompt to a generative AI model (e.g., GPT-4) saying, "The user wants to see what Paris is currently like. Please provide a real-time video."

[1515] 4. Emotion Recognition by Emotion Engine

[1516] The server uses an emotion engine to recognize emotions from the user's request.

[1517] Example: An emotion engine recognizes "curiosity" in a user's request and determines what the user is interested in.

[1518] 5. Real-time video streaming or local specialty product arrangement

[1519] The server starts a real-time video distribution system or a local specialty product ordering system in response to a user request.

[1520] Examples:

[1521] Real-time video streaming: The server starts the real-time video streaming system, acquires live camera footage of Paris, and streams it to users.

[1522] Arranging local specialties: When a user requests "I want to buy a local specialty of Paris," the server launches the local specialties ordering system and arranges for the local specialty to be purchased.

[1523] 6. Sending Responses to Users

[1524] The server sends the acquired information and arrangement details to the user.

[1525] Examples:

[1526] Real-time video streaming: The server captures live camera footage of Paris and streams it to the user's smartphone, sending a message saying, "This is the current state of Paris."

[1527] Arrange local specialties: The server notifies the user that the local specialties have been arranged and sends a message saying, "Paris local specialties have been arranged. Please wait for them to arrive."

[1528] Prompt Sentence Examples

[1529] "Users want to see what's happening in Paris right now. Give them real-time video."

[1530] "A user wants to buy a Paris specialty. Can you help me with that?"

[1531] In this way, the system provides appropriate information and services in response to user requests. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1532] Step 1:

[1533] The user uses the device to submit a request.

[1534] Input: A user types into a terminal, "Show me what Paris looks like now."

[1535] Specific behavior: A user opens a chat app on their smartphone, types in a request, and presses the send button.

[1536] Output: The user's request is sent to the server.

[1537] Step 2:

[1538] The server receives and analyzes the user request.

[1539] Input: The request data sent by the user.

[1540] What happens: The server receives the request and analyzes the request using a text analysis algorithm.

[1541] Output: The request is determined to be "I need to provide a real-time video of Paris."

[1542] Step 3:

[1543] Based on the request content analyzed by the server, a prompt is sent to the generative AI model.

[1544] Input: The parsed request content.

[1545] What happens: The server sends a prompt to the generative AI model (e.g., GPT-4) saying, "The user wants to see what Paris is currently like. Please provide a real-time video."

[1546] Output: The generative AI model receives the prompt and is ready to generate an appropriate response.

[1547] Step 4:

[1548] The server uses an emotion engine to recognize emotions from the user's request.

[1549] Input: User request data.

[1550] What happens: The server launches the emotion engine, analyzes the request text, and recognizes the user's emotion.

[1551] Output: The emotion engine recognizes the user's "curiosity."

[1552] Step 5:

[1553] The server starts the real-time video distribution system in response to the user's request.

[1554] Input: User sentiment and request content recognized by the sentiment engine.

[1555] Specific operation: The server starts the real-time video distribution system and acquires live camera footage of Paris.

[1556] Output: Captured live camera video data.

[1557] Step 6:

[1558] The server sends the retrieved information to the user.

[1559] Input: Captured live camera footage data.

[1560] What it does: The server streams live camera footage to the user's device and sends a message saying, "This is what Paris looks like right now."

[1561] Output: A live camera image and a message will be displayed on the user's device.

[1562] In this way, the system provides appropriate information and services in response to user requests.

[1563] (Application example 3)

[1564] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1565] Conventional travel information systems make it difficult for users to obtain the latest local information in real time, and there are limited ways to easily arrange local specialties. Furthermore, they do not provide appropriate information based on the user's emotions, which reduces user satisfaction. To solve these issues, a system is needed that can deliver real-time video and arrange specialties in response to user requests, as well as recognize the user's emotions and provide appropriate information.

[1566] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1567] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time video of the local area based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, emotion recognition means for recognizing the user's emotion based on the travel information provided by the chat generation AI means and providing appropriate information, means for acquiring a URL of a real-time video in response to a user's request based on the travel information provided by the chat generation AI means, and means for acquiring information on local specialties in response to a user's request based on the travel information provided by the chat generation AI means. This allows the user to obtain the latest local information in real time and easily arrange local specialties, and further enables the provision of appropriate information in response to the user's emotion.

[1568] "Chat generation AI means" is an artificial intelligence means that provides travel information through dialogue with users.

[1569] "Travel information" refers to all travel-related information, such as tourist attractions, transportation options, and local know-how.

[1570] "Means for delivering real-time video" refers to a means for providing users with the latest local video footage in real time.

[1571] "Means for arranging local specialties" refers to the means for arranging local specialties in response to user requests.

[1572] An "emotion recognition means" is a means for recognizing a user's emotions and providing appropriate information accordingly.

[1573] "Means for obtaining a URL for a real-time video" refers to a means for obtaining a URL for a real-time video in response to a user request.

[1574] The "means for acquiring information on local specialties" refers to a means for acquiring information on local specialties in response to a user's request.

[1575] To implement this invention, the following system configuration and program are required. The system includes a chat generation AI means for providing travel information through dialogue with the user, a means for supporting the user in planning their trip based on the travel information, a means for delivering real-time local video, a means for arranging local specialties, an emotion recognition means for recognizing the user's emotions and providing appropriate information, a means for acquiring the URL of the real-time video, and a means for acquiring information on local specialties.

[1576] System Program

[1577] Hardware / Software used

[1578] EmotionEngine: An engine for recognizing user emotions. Uses emotion recognition API.

[1579] ChatAI: A generative chat AI that generates answers based on user requests. It uses generative AI models such as GPT-3.

[1580] VideoStreaming: A module for streaming local real-time video. It uses the YouTube Live API, etc.

[1581] ProductOrdering: A module for ordering local specialties. Uses mail order API.

[1582] Program processing explanation

[1583] The server first receives input from the user. The user's input text is passed to the Emotion Engine, which recognizes the user's emotions. The recognized emotions and the user's input text are then passed to ChatAI, which generates an appropriate answer.

[1584] For example, if a user says, "Show me the current state of Paris," the server uses the VideoStreaming module to obtain a URL for a real-time video of Paris and provides it to the user. If a user says, "I want to buy some Parisian specialties," the server uses the ProductOrdering module to obtain information about Parisian specialties and provides it to the user.

[1585] Specific examples

[1586] Prompt Sentence Examples

[1587] "Show me what Paris looks like now."

[1588] "I want to buy a specialty product of Paris."

[1589] "How do I use the Paris subway?"

[1590] This will allow users to obtain the latest local information in real time, easily arrange for local specialties, and provide appropriate information based on the user's emotions.

[1591] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1592] Step 1:

[1593] The user uses the terminal to input a request for travel information, for example, a prompt such as "Show me what Paris is like now" or "I want to buy some local Parisian products." The input data is sent to the server in text format.

[1594] Step 2:

[1595] The server passes the received user input text to the Emotion Engine. The Emotion Engine analyzes the input text and recognizes the user's emotion. For example, if a user inputs "Show me the current state of Paris," the Emotion Engine recognizes the user's curiosity. The input is the user's text, and the output is the recognized emotion.

[1596] Step 3:

[1597] The server passes the recognized emotion and the user's input text to ChatAI. ChatAI uses a generative AI model to generate an appropriate response based on the user's request. For example, in response to the request "Show me what Paris is like now," ChatAI generates a response such as "I'll show you what Paris is like now." The input is the user's text and the recognized emotion, and the output is the generated response.

[1598] Step 4:

[1599] The server calls the appropriate module based on the user's request. For example, if a user requests "Show me the current state of Paris," the server calls the VideoStreaming module and obtains the URL of a real-time video of Paris. The input is the user's request, and the output is the URL of the real-time video.

[1600] Step 5:

[1601] The server provides the URL of the acquired real-time video to the user. The user can then watch the real-time video through their device. The input is the URL of the real-time video, and the output is the video displayed on the user's device.

[1602] Step 6:

[1603] When a user requests "I want to buy a Parisian specialty," the server calls the ProductOrdering module to obtain information about Parisian specialty products. The input is the user's request, and the output is the specialty product information.

[1604] Step 7:

[1605] The server provides the acquired information about the local specialty to the user. The user can check the detailed information about the local specialty through their terminal and proceed with the purchase. The input is the information about the local specialty, and the output is the detailed information about the local specialty displayed on the user's terminal.

[1606] This allows users to obtain the latest local information in real time, easily arrange for local specialties, and also provides appropriate information according to the user's feelings.

[1607] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1608] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1609] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1610] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1611] [Third embodiment]

[1612] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1613] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1614] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1615] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1616] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1617] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1618] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1619] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1620] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1621] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1622] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1623] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1624] "Example 1"

[1625] The system of the present invention is equipped with a chat generation AI means that provides travel information through dialogue with the user. This chat generation AI means extracts and explains travel information from a vast amount of information on the Internet. For example, if a user asks, "Tell me about tourist attractions in Paris," the chat generation AI means will respond based on information on the Internet in the form of, "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[1626] "Example 2"

[1627] The system of the present invention also includes a means for supporting a user in traveling based on travel information provided by the chat generation AI means. For example, if a user asks, "Please tell me how to use the Paris subway," the chat generation AI means will respond by providing assistance with the user's travel in the form of, "For the Paris subway, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[1628] "Example 3"

[1629] Furthermore, the system of the present invention is equipped with a means for delivering real-time local video based on travel information provided by the chat generation AI means. For example, if a user asks, "Show me the current situation in Paris," the system will deliver real-time local video, allowing the user to obtain the latest local information.

[1630] "Example 4"

[1631] The system of the present invention also includes a means for arranging local specialties based on travel information provided by the chat generation AI means. For example, if a user asks, "I want to buy a local specialty from Paris," the system will arrange for the local specialty. This allows users to obtain local specialties without having to travel to the location.

[1632] The processing flow of each embodiment will be described below.

[1633] "Example 1"

[1634] Step 1: The user asks the system, "Tell me about tourist attractions in Paris."

[1635] Step 2: The chat generation AI in the system responds based on information on the Internet, such as "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[1636] "Example 2"

[1637] Step 1: The user asks the system, "How do I use the Paris subway?"

[1638] Step 2: The chat generation AI within the system responds by providing support for the journey, such as, "For the Paris Metro, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[1639] "Example 3"

[1640] Step 1: The user asks the system, "Show me the current situation in Paris." Step 2: The system delivers real-time video of the local area, allowing the user to get the latest information on the local area.

[1641] "Example 4"

[1642] Step 1: The user asks the system, "I want to buy a Parisian specialty."

[1643] Step 2: The system arranges for local specialties, allowing users to obtain the specialties without having to travel to the local area.

[1644] Example 1

[1645] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1646] Conventional travel information systems have had the problem of being unable to provide users with the information they need quickly and accurately. Furthermore, when users ask specific questions, it takes a lot of time and effort to get an appropriate answer. Furthermore, it is difficult to meet diverse travel-related needs, such as providing real-time information or arranging for local specialties.

[1647] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1648] In this invention, the server includes an information generation means for providing travel information through dialogue with a user, a means for supporting a user in traveling based on the travel information provided by the information generation means, a means for delivering real-time local video based on the travel information provided by the information generation means, a means for arranging local specialties based on the travel information provided by the information generation means, a means for receiving an inquiry from a user and sending it as a prompt to a generative AI model, a means for the generative AI model to search information on the Internet and generate an answer, and a means for returning the generated answer to the user. This allows users to quickly and accurately obtain travel information and makes it possible to meet a variety of needs, such as providing real-time information and arranging local specialties.

[1649] The "information generating means" is a means for providing travel information through dialogue with the user.

[1650] "Means for supporting travel" refers to means for supporting a user in planning and carrying out a travel based on travel information provided by an information generating means.

[1651] The "means for delivering real-time video" is a means for delivering local real-time video to users based on travel information provided by the information generating means.

[1652] The "means for arranging local specialties" is a means for arranging local specialties for the user based on the travel information provided by the information generating means.

[1653] "Means for receiving a query and sending it to the generative AI model as a prompt sentence" refers to means for receiving a query from a user and sending it to the generative AI model as a prompt sentence.

[1654] "Means by which the generative AI model searches for information on the Internet and generates an answer" refers to the means by which the generative AI model searches for information on the Internet based on a prompt sentence and generates an appropriate answer.

[1655] "Means for returning the generated answer to the user" refers to means for returning the answer generated by the generative AI model to the user.

[1656] This invention is a system for providing travel information through dialogue with a user. The system includes an information generating means, a means for supporting travel, a means for delivering real-time video, a means for arranging local specialties, a means for receiving an inquiry and sending it as a prompt to a generative AI model, a means for the generative AI model to search for information on the Internet and generate an answer, and a means for returning the generated answer to the user.

[1657] Hardware and software used

[1658] This system uses a server, a terminal, and a generative AI model. Specifically, the server is a high-performance cloud server, and the terminal is a device that provides the user interface, such as a smartphone or PC. The generative AI model uses an advanced natural language processing model such as OpenAI's GPT-4.

[1659] Data processing and calculation

[1660] The server receives a user's query and sends it to the generative AI model as a prompt. The generative AI model searches vast amounts of information on the Internet based on the prompt and generates an appropriate answer. The generated answer is then sent back to the user via the server.

[1661] Specific examples

[1662] Example 1

[1663] The user types "Tell me about tourist attractions in Paris" into the device. The device sends this query to the server. The server sends the prompt "Tell me about tourist attractions in Paris" to the generative AI model. The generative AI model searches information on the internet and generates the answer "Famous attractions include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The server sends this answer back to the device, which displays it to the user.

[1664] Example 2

[1665] The user types "What restaurants in Tokyo do you recommend?" into the device. The device sends this query to the server. The server sends the prompt "What restaurants in Tokyo do you recommend?" to the generative AI model. The generative AI model searches information on the internet and generates the answer "Popular sushi restaurants in Ginza and ramen restaurants in Shinjuku." The server sends this answer back to the device, which displays it to the user.

[1666] Prompt Sentence Examples

[1667] 1. "Tell me about the tourist attractions in Paris."

[1668] 2. "What are some recommended restaurants in Tokyo?"

[1669] 3. "What museums should I visit in New York?"

[1670] This allows users to quickly and accurately obtain travel information, and makes it possible to meet a variety of needs, such as providing real-time information and arranging for local specialties.

[1671] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1672] Step 1:

[1673] The user types a query into the terminal.

[1674] When a user wants to know travel information, they can type a question into the chat interface of the device, for example, "Tell me about tourist attractions in Paris." The typed query is temporarily stored in the device's internal memory.

[1675] Step 2:

[1676] The terminal sends a query to the server.

[1677] The terminal sends the query entered by the user to the server. At this time, the terminal converts the query content into an appropriate format and uses a communication protocol (e.g., HTTP POST request) to send it to the server. The input is the user's query, and the output is the data to be sent to the server.

[1678] Step 3:

[1679] The server receives the query and sends it as a prompt to the generative AI model.

[1680] The server analyzes the query received from the device and sends it to the generative AI model as a prompt. The prompt may be the user's question as is, or it may be preprocessed as needed. The input is the query data from the device, and the output is the prompt to the generative AI model.

[1681] Step 4:

[1682] A generative AI model searches the internet for information and generates answers.

[1683] A generative AI model searches for relevant data from the vast amount of information on the Internet based on a prompt and generates an answer. For example, in response to the prompt, "Tell me about tourist attractions in Paris," it generates the answer, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." The input is the prompt, and the output is the generated answer.

[1684] Step 5:

[1685] The server receives the generated response and sends it to the terminal.

[1686] The server analyzes the response received from the generative AI model, converts it into an appropriate format, and sends it to the device. At this time, the server uses a communication protocol (e.g., HTTP response). The input is the response data from the generative AI model, and the output is the data sent to the device.

[1687] Step 6:

[1688] The device displays the answer to the user.

[1689] The terminal displays the answer received from the server to the user. The user can check the answer on the terminal screen. The input is the answer data from the server, and the output is the display data to the user.

[1690] (Application example 1)

[1691] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1692] Conventional travel information systems have the problem that it takes a lot of time and effort for users to obtain information about travel destinations. In addition, they do not provide real-time information or a travel experience in a virtual environment, which results in low user satisfaction.

[1693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1694] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time local video based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, means for the user to select a travel destination in a virtual store and be provided with travel information in real time, and means for the user to obtain information on tourist attractions and recommended spots related to the travel destination in the virtual store. This allows the user to efficiently obtain travel information and enjoy a real-time travel experience in a virtual environment.

[1695] "User" means an individual or organization that uses the system to obtain travel information and engage in travel experiences within the virtual store.

[1696] "Dialogue" refers to the exchange of information between a user and a chat generation AI means.

[1697] "Travel information" refers to any information related to travel, such as tourist attractions, transportation, local know-how, and local specialties.

[1698] "Chat generation AI means" refers to the function of artificial intelligence that provides travel information through dialogue with users.

[1699] "Supporting means" refers to a function that supports users in traveling based on travel information provided by chat generation AI means.

[1700] "Means for delivering real-time video" refers to the function of providing users with local footage in real time.

[1701] "Means for arranging local specialties" refers to the function of making arrangements to provide local specialties and famous products to users.

[1702] "Virtual store" refers to a virtual store where users can select travel destinations and obtain travel information within a virtual space.

[1703] "Tourist attractions" refer to places and facilities worth visiting in a travel destination.

[1704] "Recommended spots" refer to places or facilities that are particularly recommended to visit at a travel destination.

[1705] The system for implementing this invention is mainly composed of chat generation AI means that provides travel information through dialogue with users. A specific embodiment of this system will be described below.

[1706] First, users access a virtual store using a device such as a smartphone or head-mounted display. Within the virtual store, users can select a travel destination and obtain information about that destination through chat generation AI.

[1707] The server uses OpenAI's API to implement a chat generation AI system. When a user enters a question about a travel destination, the question is generated as a prompt and sent to OpenAI's generative AI model. The generative AI model then extracts appropriate travel information from the vast amount of information on the Internet and provides it to the user.

[1708] For example, if the user types "Tell me about tourist attractions in Paris," the prompt text might look like this:

[1709] "Tell me about the tourist attractions in Paris."

[1710] This prompt is sent to a generative AI model, which then generates an answer such as, "Famous places include the Eiffel Tower, the Louvre, and Notre Dame Cathedral." This answer is then displayed on the user's device.

[1711] In addition, the system is equipped with a means for streaming real-time video, allowing users to view real-time footage of the selected travel destination, giving them the experience of actually visiting the location.

[1712] The system also provides a means to arrange local specialties, allowing users to easily purchase the specialties they are interested in. This helps users gain a deeper understanding of the culture and specialties of the destination they are traveling to.

[1713] In this way, users can efficiently obtain travel information and enjoy real-time travel experiences in a virtual environment. The hardware used is a smartphone and a head-mounted display, and the software is Python and OpenAI API.

[1714] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1715] Step 1:

[1716] The user accesses the virtual store using a smartphone or head-mounted display. The user selects a travel destination and inputs a question to obtain travel information. The input data is the user's selected travel destination and the question.

[1717] Step 2:

[1718] The terminal receives the user's input and generates a prompt based on the question. The generated prompt might be something like, "Tell me about tourist attractions in Paris." This prompt becomes the input data for the next processing step.

[1719] Step 3:

[1720] The server sends the generated prompt to OpenAI's generative AI model, which then extracts relevant travel information from the vast amount of information on the Internet based on the prompt. This process takes the prompt as input and outputs text data containing travel information.

[1721] Step 4:

[1722] The server receives the travel information obtained from the generative AI model and sends it to the user's device. The user's device displays this travel information. For example, it displays information such as "Famous places include the Eiffel Tower, the Louvre Museum, and Notre Dame Cathedral."

[1723] Step 5:

[1724] If the user wants more information, they enter a follow-up question. The device generates a new prompt and sends it to the server. The process repeats steps 2 through 4.

[1725] Step 6:

[1726] When a user wants to watch real-time video of a selected travel destination, the device sends the request to the server, which retrieves the local real-time video and delivers it to the user's device, where the user can watch the real-time video.

[1727] Step 7:

[1728] When a user wants to purchase a local specialty product, the device sends the request to the server, which then arranges for the product to be delivered to the user, and the user can then complete the purchase process on the device.

[1729] In this way, users can efficiently obtain travel information and enjoy real-time travel experiences in a virtual environment.

[1730] Example 2

[1731] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1732] Conventional travel information systems have struggled to provide users with the specific travel information they need quickly and accurately. Furthermore, users have had limited access to the information they need in real time while traveling, which often makes planning and executing trips inconvenient. Furthermore, they lacked features to enrich the travel experience, such as arranging local specialties and streaming real-time videos.

[1733] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1734] In this invention, the server includes a means for receiving a question entered by a user and inputting the question as a prompt sentence to the generative AI model, a means for the generative AI model to generate an answer to the question, and a means for displaying the generated answer to the user. This allows users to obtain quick and accurate answers to specific travel-related questions. It also provides functions to enrich the travel experience, such as streaming real-time videos and arranging for local specialties.

[1735] "User" means an individual or organization that uses the System to obtain travel information and plan and carry out travel.

[1736] "Chat generation AI means" refers to means that use artificial intelligence technology to provide travel information through dialogue with users.

[1737] "Travel information" refers to any information related to travel, such as tourist attractions, transportation, local know-how, and local specialties.

[1738] A "generative AI model" is an artificial intelligence model used to generate appropriate answers to user questions.

[1739] A "prompt" is text containing a user question or instruction that is input into a generative AI model.

[1740] "Real-time video" is video that delivers footage of the local situation in real time.

[1741] "Specialty products" are products or products that are produced in a particular region and are representative of that region.

[1742] A "server" is a computer system that receives questions from users, generates answers using a generative AI model, and provides them to users.

[1743] A "terminal" is a device through which a user inputs a question and receives and displays a response from the server.

[1744] MODE FOR CARRYING OUT THE INVENTION

[1745] This invention is a system in which, when a user inputs a travel-related question, a chat generation AI means generates an appropriate answer to the question, thereby supporting the user's travel. A specific embodiment of this system is described below.

[1746] System configuration

[1747] The system mainly consists of the following components:

[1748] 1. User Device

[1749] 2. Server

[1750] 3. Generative AI Models

[1751] User Device

[1752] The user terminal is a device that allows the user to input questions and receive and display answers from the server. Specifically, this applies to smartphones, tablets, PCs, etc. The user inputs travel-related questions into the input field of the terminal.

[1753] server

[1754] The server receives questions from users, generates answers to those questions using a generative AI model, and sends them to the user's device. The server performs the following processes:

[1755] 1. Receive a question from the user.

[1756] 2. The received question is input as a prompt to the generative AI model.

[1757] 3. Receive the answer generated by the generative AI model and send it to the user device.

[1758] Generative AI Models

[1759] A generative AI model is an artificial intelligence model used to generate appropriate answers to user questions. Specifically, generative AI models such as OpenAI's GPT-4 are used. This model has learned a large amount of travel information in advance and can provide quick and accurate answers to user questions.

[1760] Specific examples

[1761] Here's a concrete example of how it works:

[1762] Example 1: How to use the Paris subway

[1763] A user types in "How do I use the Paris subway?"

[1764] The terminal sends this question to the server.

[1765] The server receives the question and inputs the prompt sentence "Tell me how to use the Paris subway" into the generative AI model.

[1766] The generative AI model generates the answer, "On the Paris Metro, you purchase a ticket at each station and insert it when passing through the ticket gate. You do not need to purchase a new ticket when transferring."

[1767] The server sends the generated response to the terminal.

[1768] The device displays the answer to the user.

[1769] Example 2: New York tourist attractions

[1770] A user types, "What are some tourist attractions in New York?"

[1771] The terminal sends this question to the server.

[1772] The server receives the question and inputs the prompt sentence "Tell me about tourist attractions in New York" into the generative AI model.

[1773] The generative AI model generates the answer, "Tourist attractions in New York include the Statue of Liberty, Central Park, Times Square, and the Metropolitan Museum of Art."

[1774] The server sends the generated response to the terminal.

[1775] The device displays the answer to the user.

[1776] In this way, users can get fast and accurate answers to specific travel questions, and are also provided with features to enrich their travel experience, such as real-time video streaming and local specialty product arrangements.

[1777] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1778] Step 1:

[1779] The user enters a question.

[1780] A user enters a travel question into an input field on the device, for example, "How do I use the Paris subway?" The entered question is temporarily stored in the device's memory.

[1781] Step 2:

[1782] The terminal sends a question to the server.

[1783] The terminal transmits the question entered by the user to the server. At this time, the terminal transmits the question data to the server via the Internet. The input data is the user's question text. The output data is the question text transmitted to the server.

[1784] Step 3:

[1785] The server receives the question and inputs it as a prompt to the generative AI model.

[1786] The server receives the question sent from the terminal. The received question is input to the generative AI model as a prompt sentence. The input data is the user's question text. The output data is the prompt sentence input to the generative AI model.

[1787] Step 4:

[1788] A generative AI model generates answers to questions.

[1789] The generative AI model generates answers to questions based on the input prompt sentence. For example, it generates an answer such as, "For the Paris Metro, you buy a ticket at each station and insert it when passing through the ticket gate. You do not need to buy a new ticket when transferring." The input data is the prompt sentence. The output data is the generated answer text.

[1790] Step 5:

[1791] The server sends the generated response to the terminal.

[1792] The server receives the answer generated by the generative AI model and sends it to the device. At this time, the server converts the answer into an appropriate format before sending it. The input data is the generated answer text. The output data is the answer text sent to the device.

[1793] Step 6:

[1794] The terminal displays the answer to the user.

[1795] The terminal displays the answer received from the server to the user. The user can check the answer on the terminal screen. The input data is the answer text received from the server. The output data is the answer text displayed to the user.

[1796] (Application example 2)

[1797] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1798] Conventional travel support systems require users to spend a lot of time and effort to obtain information about their destination. It is also difficult to obtain appropriate information when ordering local food while traveling, resulting in low user convenience. This increases stress during travel and detracts from the travel experience.

[1799] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for assisting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time local video based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, chat generation AI means for providing food delivery information through dialogue with the user, means for assisting the user in ordering food based on the food delivery information provided by the chat generation AI means, and means for arranging local food based on the food delivery information provided by the chat generation AI means. This allows the user to quickly and accurately obtain information about their travel destination and easily order local food.

[1800] "User" means an individual or entity that uses the System to obtain travel information and food delivery information and to make travel and food orders.

[1801] "Dialogue" refers to the exchange of questions and answers between a user and a chat-generating AI means.

[1802] "Travel Information" refers to information necessary for users to plan and carry out their trips, such as tourist attractions, transportation options, and local know-how.

[1803] "Chat generation AI means" is an artificial intelligence means that provides travel information and food delivery information through dialogue with users.

[1804] "Supporting means" refers to means that support users to smoothly travel and order food based on information provided by chat generation AI means.

[1805] "Real-time video" refers to video that delivers footage of the local situation in real time.

[1806] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[1807] "Food Delivery Information" refers to information about food delivery in a particular region or city.

[1808] "Means for assisting in ordering food" refers to means for supporting users in ordering food based on food delivery information provided by chat generation AI means.

[1809] "Means for arranging local food" refers to a means for users to arrange local food based on food delivery information provided by the chat generation AI means.

[1810] The system for implementing the present invention includes a chat generation AI means for providing travel information and food delivery information through dialogue with a user. A specific embodiment of this system will be described below.

[1811] System Configuration

[1812] The system consists of the following main components:

[1813] 1. User device: The device used by the user, such as a smartphone or tablet.

[1814] 2. Server: A cloud server for running chat generation AI methods.

[1815] 3. Database: A database for storing travel information and food delivery information.

[1816] Hardware and software used

[1817] Hardware: smartphones, tablets, cloud servers

[1818] Software: Python, OpenAI GPT-3 API, database management system (e.g., MySQL)

[1819] Data processing and calculation

[1820] 1. User terminal: The user inputs questions about travel information and food delivery information.

[1821] 2. Server: Receives the user's question and generates an appropriate answer using the OpenAI GPT-3 API.

[1822] 3. Database: Stores travel and food delivery information and provides it to the server as needed.

[1823] Processing flow

[1824] 1. A user uses their smartphone to enter a question about travel or food delivery information.

[1825] 2. The user terminal sends a query to the server.

[1826] 3. The server sends the question to the OpenAI GPT-3 API and generates an answer.

[1827] 4. The server sends the generated answer to the user terminal.

[1828] 5. The user device displays the answer to the user.

[1829] Specific examples

[1830] When a user asks, "What are the best pizza delivery options in New York?" the system works as follows:

[1831] 1. The user device sends a question to the server.

[1832] 2. The server sends the question to the OpenAI GPT-3 API and generates an answer.

[1833] 3. The server generates the answer, "A popular pizza delivery service in New York is Joe's Pizza. You can order online, and delivery takes about 30 minutes." and sends it to the user's device.

[1834] 4. The user device displays the answer to the user.

[1835] Prompt Sentence Examples

[1836] If a user asks, "What are the best sushi delivery options in Tokyo?", the following prompts might be used:

[1837] "A user asked a question about food delivery. Question: What's the best sushi delivery in Tokyo?"

[1838] answer:

[1839] In this way, users can quickly and accurately obtain information about their travel destinations and can easily order local food.

[1840] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1841] Step 1:

[1842] Users use their smartphones to input questions about travel information or food delivery information, and the input questions are captured as text data by an application on the user's device.

[1843] Step 2:

[1844] The user terminal transmits the acquired text data to the server. The transmitted data is text data containing the content of the user's question.

[1845] Step 3:

[1846] The server analyzes the received text data and generates a prompt to send to the OpenAI GPT-3 API. The prompt is generated in a format that includes the user's question.

[1847] Step 4:

[1848] The server sends the generated prompt to the OpenAI GPT-3 API. The data sent is text data containing the prompt.

[1849] Step 5:

[1850] The OpenAI GPT-3 API generates an appropriate answer based on the received prompt, which is then returned to the server as text data.

[1851] Step 6:

[1852] The server analyzes the answers received from the OpenAI GPT-3 API and converts them into a format suitable for sending to the user's device. The converted data is text data containing the answers to the user's questions.

[1853] Step 7:

[1854] The server then transmits the converted answer data to the user terminal. The transmitted data is text data containing the answer to the user's question.

[1855] Step 8:

[1856] The user terminal analyzes the received response data and displays it to the user. The displayed content is text data containing the response to the user's question.

[1857] In this way, users can quickly and accurately obtain information about their travel destinations and can easily order local food.

[1858] Example 3

[1859] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1860] Conventional travel information systems make it difficult for users to obtain the latest local information in real time or to easily arrange for local specialties. In addition, there was a lack of a centralized means of providing the information users needed when planning a trip, so users had to refer to multiple information sources, which was time-consuming.

[1861] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes: a chat generation AI means for providing travel information through dialogue with a user; a means for supporting a user in traveling based on the travel information provided by the chat generation AI means; a means for distributing real-time video of the local area based on the travel information provided by the chat generation AI means; a means for arranging local specialties based on the travel information provided by the chat generation AI means; a means for acquiring real-time video from a live camera of the local area based on the travel information provided by the chat generation AI means and distributing it to the user's terminal; and a means for linking with an online shop selling local specialties and processing the user's order based on the travel information provided by the chat generation AI means. This allows a user to obtain the latest local information in real time and easily arrange local specialties.

[1862] "User" means an individual or organization that uses the System to provide travel information and arrange local specialties.

[1863] "Chat generation AI means" refers to artificial intelligence technology for generating and providing travel information through dialogue with users.

[1864] "Travel Information" refers to information that users need when planning and carrying out their trips, such as tourist attractions, transportation options, and local know-how.

[1865] "Real-time video" refers to video data that delivers footage of the local situation in real time.

[1866] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[1867] A "live camera" refers to a camera device that captures images of a specific location in real time and distributes them over the Internet.

[1868] "Online Shop" refers to a website or platform that sells products over the Internet.

[1869] "Order processing means" refers to the systems and processes used to accept purchase requests from users, arrange for products, and process delivery.

[1870] The present invention provides a system that allows users to obtain travel information, watch real-time local video, and even arrange for local specialties. Specific embodiments of this system are described below.

[1871] System configuration

[1872] 1. User Device

[1873] Users access the system using devices such as smartphones and PCs, which have a browser and dedicated applications installed, and interact with the system through these.

[1874] 2. Server

[1875] The server receives user requests and generates appropriate travel information using a generative AI model. The server also acquires real-time video from local live cameras and distributes it to users' devices. The server also connects with online shops selling local specialties to process user orders.

[1876] 3. Generative AI Models

[1877] The generative AI model used is OpenAI's GPT-4, which generates prompts based on user requests and provides appropriate travel information.

[1878] Program processing

[1879] 1. Providing travel information

[1880] A user uses a device to send a request: "Show me what Paris looks like now."

[1881] The server receives this request and sends a prompt to the generative AI model, which is "Tell me what Paris is like now."

[1882] The generative AI model generates travel information based on the prompt sentence and returns it to the server.

[1883] The server transmits the generated travel information to the user's terminal.

[1884] 2. Real-time video distribution

[1885] The server obtains the URL of the local live camera based on the generated travel information.

[1886] The server accesses the live camera URL and retrieves the real-time video stream.

[1887] The server delivers the acquired real-time video to the user's device.

[1888] Users can watch real-time video on their devices.

[1889] 3. Arrangements for local specialties

[1890] The user uses the device to send a request saying, "I want to buy a local specialty of Paris."

[1891] The server receives this request and sends a prompt to the generative AI model: "What are some famous products of Paris?"

[1892] The generative AI model generates travel information based on the prompt sentence and returns it to the server.

[1893] Based on the generated travel information, the server calls the API of an online shop that sells local specialties and processes the user's order.

[1894] The server notifies the user that the order is complete.

[1895] Users can check the status of their orders on their terminal.

[1896] Specific examples

[1897] 1. Specific examples of real-time video streaming

[1898] User: "Show me what Paris looks like now."

[1899] Server: Receives user requests, generates travel information using GPT-4, and captures and distributes real-time video from live cameras in Paris.

[1900] User: Watch what's happening in Paris in real time on their device.

[1901] 2. Specific examples of arranging local specialties

[1902] User: "I want to buy some Parisian specialties."

[1903] Server: Receives user requests, generates travel information using GPT-4, and processes orders in collaboration with an e-commerce site selling Parisian specialties.

[1904] User: Purchase Parisian specialties on the device and have them delivered to their home.

[1905] The above is a specific embodiment for carrying out the present invention. This system allows users to obtain the latest local information in real time and easily arrange for local specialties. The flow of the specification process in Example 3 will be explained using Figure 15.

[1906] Step 1:

[1907] A user sends a request using a device. For example, the user opens a browser or a dedicated application, types "Show me the current state of Paris," and presses the send button. The input is a text request.

[1908] Step 2:

[1909] The server receives a request from the user. The server analyzes the data received as an HTTP request and extracts the request content. The input is the user's request data, and the output is the analyzed request content.

[1910] Step 3:

[1911] The server sends a prompt to the generative AI model. The server generates a prompt based on the request content and sends it to the generative AI model (e.g., GPT-4). The input is the parsed request content, and the output is the generated prompt. Specifically, the server generates the prompt, "Tell me what the current situation is in Paris."

[1912] Step 4:

[1913] The generative AI model generates travel information based on the prompt text. The generative AI model receives the prompt text and extracts and generates appropriate travel information from the vast amount of information on the Internet. The input is the prompt text, and the output is the generated travel information.

[1914] Step 5:

[1915] The server receives the generated travel information and sends it to the user's device. The server analyzes the travel information returned from the generative AI model and sends it to the user's device. The input is the generated travel information, and the output is the travel information sent to the user's device. Specifically, the server sends information such as "It's currently sunny in Paris, and the area around the Eiffel Tower is bustling with tourists."

[1916] Step 6:

[1917] The server retrieves the URL of the local live camera. Based on the generated travel information, the server calls an API to retrieve the URL of the local live camera. The input is the generated travel information, and the output is the URL of the live camera.

[1918] Step 7:

[1919] The server accesses the URL of the live camera and retrieves the real-time video stream. The server accesses the URL and retrieves the video data using the HTTP streaming protocol. The input is the URL of the live camera, and the output is the real-time video stream.

[1920] Step 8:

[1921] The server delivers the acquired real-time video to the user's device. The server sends the real-time video stream to the user's device, and the user watches the video using a browser or dedicated application. The input is the real-time video stream, and the output is the video delivered to the user's device.

[1922] Step 9:

[1923] The user watches real-time video on a device. The user watches the streamed real-time video through the device's browser or a dedicated application. The input is the streamed video, and the output is the user's viewing experience.

[1924] Step 10:

[1925] A user uses a terminal to request the delivery of local specialties. For example, the user inputs "I want to buy local specialties from Paris" and presses the send button. The input is a text request.

[1926] Step 11:

[1927] The server receives a request from the user and sends a prompt to the generative AI model. The server generates a prompt based on the request content and sends it to the generative AI model. The input is the parsed request content, and the output is the generated prompt. Specifically, the server generates the prompt, "Tell me what Paris is famous for."

[1928] Step 12:

[1929] The generative AI model generates travel information based on the prompt text. The generative AI model receives the prompt text and extracts and generates appropriate travel information from the vast amount of information on the Internet. The input is the prompt text, and the output is the generated travel information.

[1930] Step 13:

[1931] The server receives the generated travel information and calls the API of an online shop that sells local specialties. Based on the generated travel information, the server processes the user's order using the online shop's API. The input is the generated travel information, and the output is the result of order processing.

[1932] Step 14:

[1933] The server notifies the user that the order is complete. The server sends the order processing results to the user's terminal, and the user checks the order status. The input is the order processing results, and the output is the notification sent to the user's terminal.

[1934] Step 15:

[1935] The user checks the order status on the device. The user checks the order status through the device's browser or a dedicated application. The input is the sent notification, and the output is the user's confirmation result.

[1936] (Application example 3)

[1937] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1938] Conventional travel information systems make it difficult for users to grasp real-time local conditions and limit the means for easily purchasing local specialties. Furthermore, they have the problem of being unable to simultaneously experience real-time shopping in a virtual store. This limits users' travel and shopping experiences.

[1939] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1940] In this invention, the server includes chat generation AI means for providing travel information through dialogue with the user, means for supporting the user in traveling based on the travel information provided by the chat generation AI means, means for delivering real-time video of the local area based on the travel information provided by the chat generation AI means, means for arranging local specialties based on the travel information provided by the chat generation AI means, and means for purchasing local specialties while watching real-time video of the local area in a virtual store based on the travel information provided by the chat generation AI means. This enables the user to easily purchase local specialties in the virtual store while understanding the real-time situation of the local area.

[1941] "User" means an individual or corporation that uses the system to provide travel information or purchase local specialties.

[1942] "Dialogue" refers to the exchange of information between a user and a chat generation AI means.

[1943] "Travel information" refers to various information related to travel, such as tourist attractions, transportation options, and local know-how.

[1944] "Chat generation AI means" refers to an artificial intelligence system that provides travel information through dialogue with users.

[1945] "Real-time video" refers to video that delivers real-time footage of the current situation on-site.

[1946] "Specialty products" refer to special products that are produced in a particular region and represent that region.

[1947] "Virtual Store" refers to a virtual store operated on the Internet where users can purchase products online.

[1948] "Viewing" refers to the act of a user watching a video or movie.

[1949] "Purchase" refers to the act of a user acquiring a product or service by paying money.

[1950] This invention is a system that allows users to purchase local specialties while watching real-time video of the local area in a virtual store. This system is realized using the following hardware and software.

[1951] Hardware

[1952] Smartphone

[1953] Smart Glasses

[1954] head-mounted display

[1955] server

[1956] software

[1957] Python

[1958] OpenCV (cv2)

[1959] ChatGPT API

[1960] System configuration

[1961] 1. User Device:

[1962] Users access the system using smartphones, smart glasses, or head-mounted displays, which allow for user interaction and support real-time video viewing and local product purchases.

[1963] 2. Server:

[1964] The server hosts a chat generation AI system, provides travel information in response to user questions, and also handles real-time video streaming and local product ordering.

[1965] Processing flow

[1966] 1. User interaction:

[1967] When a user asks "Show me what Paris is like now" through their device, the server uses the ChatGPT API to parse the question and generate appropr...

Claims

1. A means for recognizing a user's emotional state based on the user's tone of voice and facial expression data; A chat generation AI means for providing travel information for a place where the user is planning to travel through a dialogue with the user; A means for suggesting local travel meals and restaurants based on user questions and the user's perceived emotional state; A means for supporting a user in traveling based on travel information provided by the chat generation AI means; A means for delivering local real-time video based on travel information provided by the chat generation AI means; A means for arranging local specialty products based on travel information provided by the chat generation AI means; A system including:

2. The means of supporting travel include support for travel arrangements and schedule creation, support for selecting meals or restaurants during the trip, and support for how to use transportation; The system of claim 1 .

3. The chat generation AI means provides local tourist attractions, transportation, and local know-how information as travel information through dialogue with the user. The system of claim 1 .

Images